Commit graph

8590 commits

Author SHA1 Message Date
Joey Hess
58d1b2510c
Merge branch 'master' of ssh://git-annex.branchable.com 2018-11-12 13:26:10 -04:00
yarikoptic
9d47fcac49 Added a comment 2018-11-12 17:25:43 +00:00
Joey Hess
5ce078da92
bash completion fix
Fix bash completion of "git annex" to propertly handle files with spaces
and other problem characters. (Completion of "git-annex" already did.)

This commit was sponsored by Jake Vosloo on Patreon.
2018-11-12 13:23:05 -04:00
andrew
34d77ecdfd 2018-11-12 16:51:12 +00:00
yarikoptic
8f8b746f67 Added a comment: reply on the issue 2018-11-12 16:48:08 +00:00
andrew
b3d6b2dad9 Added a comment 2018-11-12 16:37:44 +00:00
andrew
4184878218 2018-11-12 16:36:26 +00:00
Joey Hess
d01c7c173c
response 2018-11-12 12:10:40 -04:00
Joey Hess
5bb2c72b6d
followup 2018-11-12 12:03:39 -04:00
Joey Hess
911f75232b
followup 2018-11-12 11:49:44 -04:00
Joey Hess
398bac4530
not dup 2018-11-12 11:23:56 -04:00
xelxebar
cd5c53ca60 2018-11-12 03:28:38 +00:00
andrew
2429763490 Added a comment 2018-11-12 03:17:48 +00:00
andrew
8853f6b13c 2018-11-11 21:39:56 +00:00
andrew
87efebb590 Added a comment 2018-11-11 21:35:26 +00:00
andrew
97453224ed 2018-11-11 21:33:37 +00:00
yarikoptic
b1a5ea5f53 original double complain on get -J 2018-11-08 18:05:56 +00:00
yarikoptic
1011d957aa Added a comment 2018-11-06 20:01:08 +00:00
Joey Hess
6ecd55a9fa
Fixed some other potential hangs in the P2P protocol
Finishes the start made in 983c9d5a53, by
handling the case where `transfer` fails for some other reason, and so the
ReadContent callback does not get run. I don't know of a case where
`transfer` does fail other than the locking dealt with in that commit, but
it's good to have a guarantee.

StoreContent and StoreContentTo had a similar problem.
Things like `getViaTmp` may decide not to run the transfer action.
And `transfer` could certianly fail, if another transfer of the same
object was in progress. (Or a different object when annex.pidlock is set.)

If the transfer action was not run, the content of the object would
not all get consumed, and so would get interpreted as protocol commands,
which would not go well.

My approach to fixing all of these things is to set a TVar only
once all the data in the transfer is known to have been read/written.
This way the internals of `transfer`, `getViaTmp` etc don't matter.

So in ReadContent, it checks if the transfer completed.
If not, as long as it didn't throw an exception, send empty and Invalid
data to the callback. On an exception the state of the protocol is unknown
so it has to raise ProtoFailureException and close the connection,
same as before.

In StoreContent, if the transfer did not complete
some portion of the DATA has been read, so the protocol is in an unknown
state and it has to close the conection as well.

(The ProtoFailureMessage used here matches the one in Annex.Transfer, which
is the most likely reason. Not ideal to duplicate it..)

StoreContent did not ever close the protocol connection before. So this is
a protocol change, but only in an exceptional circumstance, and it's not
going to break anything, because clients already need to deal with the
connection breaking at any point.

The way this new behavior looks (here origin has annex.pidlock = true so will
only accept one upload to it at a time):

git annex copy --to origin -J2
copy x (to origin...) ok
copy y (to origin...)
  Lost connection (fd:25: hGetChar: end of file)

This work is supported by the NIH-funded NICEMAN (ReproNim TR&D3) project.
2018-11-06 14:52:32 -04:00
Joey Hess
9adc0b3417
Merge branch 'master' of ssh://git-annex.branchable.com 2018-11-06 13:01:00 -04:00
Joey Hess
983c9d5a53
git-annex-shell: fix transfer hang
Fix hang when transferring the same objects to two different clients at the
same time. (Or when annex.pidlock is used, two different objects to the
same or different clients.)

Could also potentially occur if a client was downloading an object and
somehow lost connection but that git-annex-shell was still running and
holding the transfer lock.

This does not guarantee that, if `transfer` fails for some other reason,
a DATA response will be made.

This work is supported by the NIH-funded NICEMAN (ReproNim TR&D3) project.
2018-11-06 13:00:37 -04:00
yarikoptic
68adc08502 initial report on failing to get a file after it actually got to tmp/ fully 2018-11-06 03:43:40 +00:00
Joey Hess
0a0242224d
full analysis 2018-11-05 18:25:55 -04:00
Joey Hess
44769d9285
Merge branch 'master' of ssh://git-annex.branchable.com 2018-11-05 17:13:16 -04:00
Joey Hess
14075afbd7
updated analysis 2018-11-05 17:12:19 -04:00
yarikoptic
0f0a6336cd Added a comment 2018-11-05 21:01:35 +00:00
yarikoptic
2003b42efb Added a comment 2018-11-05 20:53:48 +00:00
Joey Hess
c31ea81ee9
pidlock 2018-11-05 15:37:46 -04:00
Joey Hess
3c50fa130c
Merge branch 'master' of ssh://git-annex.branchable.com 2018-11-05 14:39:18 -04:00
Joey Hess
3ef13f128f
followup on bug triage 2018-11-05 14:38:32 -04:00
Joey Hess
fd128e5a3f
code review and response 2018-11-05 14:28:02 -04:00
marvin@3296bf3c446430c3b2ebc32b5c784ee976620847
4d11214908 Added a comment 2018-11-05 18:22:49 +00:00
yarikoptic
aabfdca800 Added a comment 2018-11-05 17:31:46 +00:00
Joey Hess
5f213d45b1
Merge branch 'master' of ssh://git-annex.branchable.com 2018-11-05 12:34:54 -04:00
Joey Hess
0cb6d14383
followup 2018-11-05 12:34:27 -04:00
yarikoptic
13f9e79df1 removed 2018-11-05 16:03:47 +00:00
yarikoptic
02c74a0dfd Added a comment 2018-11-05 16:00:53 +00:00
yarikoptic
eb218d4e59 Added a comment 2018-11-05 15:57:17 +00:00
Joey Hess
42f1ea72de
followup 2018-11-05 11:47:58 -04:00
Joey Hess
cb69b77625
fix link 2018-11-05 11:39:04 -04:00
anarcat
16cbeb6aff Added a comment: oops 2018-11-03 19:52:48 +00:00
spwhitton
f09f846fea Added a comment 2018-11-03 19:41:32 +00:00
anarcat
4241de01c8 p2pstdio doesn't exist in stretch and buster doesn't degrade gracefully 2018-11-03 19:30:29 +00:00
yarikoptic
c6cb372a9d Added a comment: tests fail - can't build to test the change 2018-11-02 21:51:16 +00:00
yarikoptic
4d24b712b0 Added a comment 2018-11-02 19:58:06 +00:00
Joey Hess
0c05b92b7c
Merge branch 'master' of ssh://git-annex.branchable.com 2018-11-02 13:45:57 -04:00
Joey Hess
c371a8b10e
followup 2018-11-02 13:42:57 -04:00
andrew
af385e2d34 Added a comment 2018-11-01 12:15:35 +00:00
andrew
937d9d545a Added a comment 2018-11-01 12:02:59 +00:00
yarikoptic
0753ad46c2 It was "user error", things work now properlyish 2018-11-01 03:53:02 +00:00