Commit graph

29375 commits

Author SHA1 Message Date
Joey Hess
2fb6fd7434
Merge branch 'master' into tor 2016-12-07 14:32:25 -04:00
Joey Hess
ad5ef51040
more p2p progress meters
Display progress meter on send and receive from remote.

Added a new hGetMetered that can read an exact number of bytes (or
less), updating a meter as it goes.

This commit was sponsored by Andreas on Patreon.
2016-12-07 14:25:01 -04:00
Joey Hess
f3a3dc14ec
add section on security 2016-12-07 13:58:10 -04:00
Joey Hess
83ea1cec86
update progress meter when sending to p2p remote
This commit was sponsored by Thom May on Patreon.
2016-12-07 13:37:35 -04:00
Joey Hess
7c245b2180
update 2016-12-07 12:48:24 -04:00
Joey Hess
757d36f8ca
validate peer uuid each time we talk to it
In case the repo on the peer changes uuid (eg by a new repo being moved
into place).

Also, added some warning messages when unable to communicate with a
peer.

This commit was sponsored by Anthony DeRobertis on Patreon.
2016-12-07 12:39:28 -04:00
Joey Hess
0d9a11625c
remote uuid discovery in p2p --link
This also tests that we can connect to the peer.

This commit was sponsored by Jeff Goeke-Smith on Patreon.
2016-12-07 12:38:21 -04:00
Joey Hess
729e864ad0
add todo list for remaining tor stuff 2016-12-07 12:11:38 -04:00
Joey Hess
528c7bee24
fix up some commands 2016-12-07 12:00:27 -04:00
Joey Hess
f2354ddef5
devbog 2016-12-06 17:08:39 -04:00
Joey Hess
60f4b1cf36
PAKE 2016-12-06 16:55:53 -04:00
Joey Hess
bb5168e894
need to auth with the peer 2016-12-06 15:50:02 -04:00
Joey Hess
f744bd5391
refactor 2016-12-06 15:43:03 -04:00
Joey Hess
26a53fb4a5
finish implementation of Remote.P2P (untested)
Not tested at all, but it just might work.
Only known problem is that progress is not updated when storing to a P2P
remote.

This commit was sponsored by Nick Daly on Patreon.
2016-12-06 15:09:04 -04:00
Joey Hess
2bd2e0880c
added StoreContentTo
This is needed in addition to StoreContent, because retrieveKeyFile can
be used to retrieve to different destination files, not only the tmp
file for a key.

This commit was sponsored by Ole-Morten Duesund on Patreon.
2016-12-06 15:05:44 -04:00
Joey Hess
b29088b8dc
stub Remote.P2P
Similar to GCrypt remotes, P2P remotes have an url, so Remote.Git has to
separate them out and handle them, passing off to Remote.P2P.

This commit was sponsored by Ignacio on Patreon.
2016-12-06 12:27:58 -04:00
0xloem@0bd8a79a57e4f0dcade8fc81d162c37eae4d6730
7a91c4952b 2016-12-06 14:59:40 +00:00
0xloem@0bd8a79a57e4f0dcade8fc81d162c37eae4d6730
3dc64fe867 2016-12-06 13:56:13 +00:00
0xloem@0bd8a79a57e4f0dcade8fc81d162c37eae4d6730
89ff5bece0 2016-12-06 13:03:51 +00:00
0xloem@0bd8a79a57e4f0dcade8fc81d162c37eae4d6730
b8e27383fd 2016-12-06 13:01:51 +00:00
alpernebbi
7222ed2b20 2016-12-05 21:55:55 +00:00
alpernebbi
0e1c2ff1a4 Added a comment: UTF-8 problems in some other commands 2016-12-05 20:46:07 +00:00
alpernebbi
3e0176054b 2016-12-05 19:34:55 +00:00
Joey Hess
f61508aed4
add: Stage modified non-large files when running in indirect mode.
(This was already done in v6 mode and direct mode.)
2016-12-05 14:10:21 -04:00
Joey Hess
e9c44536b2
fix formatting
mdwn2man gets confused as `command` spanning multiple lines..
2016-12-05 13:52:40 -04:00
Joey Hess
82d01f5619
rekey: Added --batch mode.
Would have liked to make the Parser parse the file and key pairs, but it
seems that optparse-applicative is unable to handle eg:

	many ((,) <$> argument <*> argument)

This commit was sponsored by Thomas Hochstein on Patreon.
2016-12-05 12:55:50 -04:00
Joey Hess
e65c31e56b
changelog 2016-12-05 12:16:35 -04:00
Joey Hess
6246c4a6db
minor style 2016-12-05 12:16:07 -04:00
Joey Hess
b0978b0196
Merge kite:tmp/git-annex 2016-12-05 12:15:48 -04:00
Joey Hess
702c671c92
cleanup 2016-12-05 12:13:25 -04:00
Joey Hess
93852dd7e8
rmurl: --batch
* rmurl: Multiple pairs of files and urls can be provided on the
  command line.
* rmurl: Added --batch mode.

This commit was sponsored by Trenton Cronholm on Patreon.
2016-12-05 12:10:07 -04:00
Daniel Brooks
24317be646 git-annex fromkey now takes multiple pairs of keys and filenames
It also still reads from stdin when none are specified.
2016-12-05 09:59:20 -05:00
https://me.yahoo.com/a/EbvxpTI_xP9Aod7Mg4cwGhgjrCrdM5s-#7c0f4
e0be5b354a "fixed" by reading --help 2016-12-04 16:20:00 +00:00
https://me.yahoo.com/a/EbvxpTI_xP9Aod7Mg4cwGhgjrCrdM5s-#7c0f4
8cb409150c added forgotten author tag 2016-12-04 15:54:32 +00:00
https://me.yahoo.com/a/EbvxpTI_xP9Aod7Mg4cwGhgjrCrdM5s-#7c0f4
3b7a51ea0e initial report 2016-12-04 15:52:29 +00:00
marek@33e8ba4fbc201af14a2badcc0656024401f5c916
19b1c5bf59 2016-12-04 15:28:41 +00:00
justin.lebar@7a36fcafc322d9a381e89f08ab6289033c6dde91
f69a79fc70 Added a comment 2016-12-04 04:30:38 +00:00
Joey Hess
6d39a30d79
use list 2016-12-03 16:52:18 -04:00
Joey Hess
1a4f9660c8
update 2016-12-03 16:51:16 -04:00
Joey Hess
4866dd33c7
Merge branch 'master' of ssh://git-annex.branchable.com 2016-12-03 16:32:42 -04:00
Joey Hess
ae0f60b628
break out curent list of names into its own file so I can auto-add from patreon
and some misc updates
2016-12-03 16:30:48 -04:00
lorenzo@a77fdbe826df1c175cbd30ed050312ccf7b7fc7b
67bc5c2e35 2016-12-03 09:04:45 +00:00
Joey Hess
a8c868c2e1
plumb assicated files through P2P protocol for updating transfer logs
ReadContent can't update the log, since it reads lazily. This part of
the P2P monad will need to be rethought.

Associated files are heavily sanitized when received from a peer;
they could be an exploit vector.

This commit was sponsored by Jochen Bartl on Patreon.
2016-12-02 16:42:54 -04:00
Joey Hess
b16a1cee4b
plumb peer uuid through to runLocal
This will allow updating transfer logs with the uuid.
2016-12-02 15:39:49 -04:00
Joey Hess
71ddb10699
initial implementation of P2P.Annex runner
Untested, and it does not yet update transfer logs.

Verifying transferred content is modeled on git-annex-shell recvkey.
In a direct mode or annex.thin repository, content can change while it's
being transferred. So, verification is always done, even if annex.verify
would normally prevent it.

Note that a WORM or URL key could change in a way the verification
doesn't catch. That can happen in git-annex-shell recvkey too. We don't
worry about it, because those key backends don't guarantee preservation
of data. (Which is to say, I worried about it, and then convinced myself
again it was ok.)
2016-12-02 14:54:33 -04:00
Joey Hess
c29f2e262a
catch non-IO exceptions too 2016-12-02 14:16:50 -04:00
Joey Hess
881274d021
make remote-daemon able to send and receive objects over tor
Each worker thread needs to run in the Annex monad, but the
remote-daemon's liftAnnex can only run 1 action at a time. Used
Annex.Concurrent to deal with that.

P2P.Annex is incomplete as of yet.
2016-12-02 13:52:43 -04:00
Joey Hess
7b7afbbedc
improve Local monad 2016-12-02 13:47:42 -04:00
Joey Hess
15dc63d47f
make sure that the specified number of bytes of DATA are always sent
It's possible, in direct or thin mode, that an object file gets
truncated or appended to as it's being sent. This would break the
protocol badly, so make sure never to send too many bytes, and to
close the protocol connection if too few bytes are available.
2016-12-02 13:45:45 -04:00
https://openid.stackexchange.com/user/3ee5cf54-f022-4a71-8666-3c2b5ee231dd
1a1bbeb659 2016-12-01 18:54:57 +00:00