mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix negative DATA when 1 node of a cluster has a partial transfer

Browse source
This commit is contained in:
Joey Hess 2024-07-30 11:42:17 -04:00
parent 43b8d96d8a
commit 1632beaf70
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
2 changed files with 11 additions and 17 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
P2P/Proxy.hs
View file

@ -563,7 +563,9 @@ proxyRequest proxydone proxyparams requestcomplete requestmessage protoerrhandle
rs <- forMC (proxyConcurrencyConfig proxyparams) remotes $ \r@(remoteside, remoteoffset) -> rs <- forMC (proxyConcurrencyConfig proxyparams) remotes $ \r@(remoteside, remoteoffset) ->
runRemoteSideOrSkipFailed remoteside $ do runRemoteSideOrSkipFailed remoteside $ do
net $ sendMessage $ DATA $ Len $ net $ sendMessage $ DATA $ Len $
totallen - remoteoffset if remoteoffset > totallen
then 0
else totallen - remoteoffset
return r return r
protoerrhandler (send (catMaybes rs) minoffset) $ protoerrhandler (send (catMaybes rs) minoffset) $
client $ net $ receiveBytes (Len datalen) nullMeterUpdate client $ net $ receiveBytes (Len datalen) nullMeterUpdate

24
doc/todo/git-annex_proxies.mdwn
View file

@ -18,27 +18,19 @@ Joey has received funding to work on this.
Planned schedule of work: Planned schedule of work:
* June: git-annex proxies and clusters * June: git-annex proxies and clusters
* July, part 1: p2p protocol over http * July: p2p protocol over http
* July, part 2: git-annex proxy support for exporttree * August, part 1: git-annex proxy support for exporttree
* August: balanced preferred content * August, part 2: balanced preferred content
* September: streaming through proxy to special remotes (especially S3) * September, part 1: balanced preferred content continued
* October: proving behavior of balanced preferred content with proxies * September, part 2: streaming through proxy to special remotes (especially S3)
* October, part 1: streaming through proxy continued
* October, part 2: proving behavior of balanced preferred content with proxies
[[!tag projects/openneuro]] [[!tag projects/openneuro]]
## work notes ## work notes
* When part of a file has been sent to a cluster via the http server, * release time
the transfer interrupted, and another node is added to the cluster,
and the transfer of the file performed again, there is a failure
sending to the node that had an incomplete copy. It fails like this:
//home/joey/tmp/bench/c3/.git/annex/tmp/SHA256E-s1048576000--09d7e19983a65682138fa5944f135e4fc593330c2693c41d22cc7881443d6060: withBinaryFile: illegal operation
git-annex: transfer already in progress, or unable to take transfer lock
p2pstdio: 1 failed
When using ssh and not the http server, the node that had the incomplete
copy also doesn't get the file, altough no error is displayed.
## items deferred until later for p2p protocol over http ## items deferred until later for p2p protocol over http