Commit graph

125 commits

Author SHA1 Message Date
Joey Hess
cc1da2d516
http p2p proxy is now largely working 2024-07-26 10:44:10 -04:00
Joey Hess
96ad0ccc5b
wip 2024-07-25 15:39:57 -04:00
Joey Hess
b13c2407af
p2phttp drop supports checking proof timestamps
At this point the p2phttp implementation is fully complete!
2024-07-25 10:11:09 -04:00
Joey Hess
f5624a69e3
expire lock after 10 minutes initially
Once keeplocked is called, the lock will expire at the end
of that call. But if keeplocked never gets called, this avoids
the lock persisting forever.
2024-07-24 14:25:40 -04:00
Joey Hess
97836aafba
Remote.Git lockContent works with annex+http urls 2024-07-24 13:42:57 -04:00
Joey Hess
9fa9678585
Remote.Git removeKey works with annex+http urls
Does not yet handle drop proof lock timestamp checking.
2024-07-24 12:33:26 -04:00
Joey Hess
fd3bdb2300
update 2024-07-24 12:19:53 -04:00
Joey Hess
0d81d1ee2f
update 2024-07-24 12:18:51 -04:00
Joey Hess
cfdb80cd05
progress meter for p2phttp storeKey 2024-07-24 12:14:56 -04:00
Joey Hess
0280e2dd5e
update 2024-07-24 11:13:37 -04:00
Joey Hess
10f2c23fd7
fix slowloris timeout in hashing resume of download of large file
Hash the data that is already present in the file before connecting to
the http server.
2024-07-24 11:03:59 -04:00
Joey Hess
7bd616e169
Remote.Git retrieveKeyFile works with annex+http urls
This includes a bugfix to serveGet, it hung at the end.
2024-07-24 10:28:44 -04:00
Joey Hess
48657405c6
cache credentials for p2phttp in memory 2024-07-23 18:45:02 -04:00
Joey Hess
b89c784a9b
use git credential when p2phttp needs auth 2024-07-23 18:11:15 -04:00
Joey Hess
73ffb58456
p2phttp support https 2024-07-23 15:37:36 -04:00
Joey Hess
b7454f1eeb
protocol version fallback on 404
and prettified errors
2024-07-23 14:58:49 -04:00
Joey Hess
75b1d50b99
add remoteAnnexP2PHttpUrl to RemoveGitConfig
This is always parsed, when building without servant, a Baseurl is not
generated, and users of it will need to fail.
2024-07-23 09:57:01 -04:00
Joey Hess
758cff0fde
update 2024-07-22 20:59:45 -04:00
Joey Hess
9984252ab5
P2P protocol is finalized 2024-07-22 19:50:08 -04:00
Joey Hess
e979e85bff
make serveKeepLocked check auth just to be safe 2024-07-22 19:15:52 -04:00
Joey Hess
f5dd7a8bc0
implemented serveLockContent (untested) 2024-07-22 17:38:42 -04:00
Joey Hess
b697c6b9da
fix TMVar left full crash affecting servePutOffset
Problem is that whatever is reading from the TMVar may not have read
from it yet before the client writes the next thing to it.
2024-07-22 15:48:46 -04:00
Joey Hess
3069e28dd8
implemented servePutOffset and clientPutOffset
But, it's buggy: the server hangs without processing the VALIDITY,
and I can't seem to work out why. As far as I can see, storefile
is getting as far as running the validitycheck, which is supposed to
read that, but never does.

This is especially strange because what seems like the same protocol
doesn't hang when servePut runs it. This made me think that it needed
to use inAnnexWorker to be more like servePut, but that didn't help.

Another small problem with this is that it does create an empty
.git/annex/tmp/ file for the key. Since this will usually be used in
combination with servePut, that doesn't seem worth worrying about much.
2024-07-22 15:04:10 -04:00
Joey Hess
b240a11b79
clientPut seeking to offset 2024-07-22 12:50:21 -04:00
Joey Hess
a01426b713
avoid padding in servePut
This means that when the client sends a truncated data to indicate
invalidity, DATA is not passed the full expected data. That leaves the
P2P connection in a state where it cannot be reused. While so far, they
are not reused, they will be later when proxies are supported. So, have
to close the P2P connection in this situation.
2024-07-22 12:30:30 -04:00
Joey Hess
efa0efdc44
avoid padding in clientPut
Instead truncate when necessary to indicate invalid content was sent.
Very similar to how serveGet handles it.
2024-07-22 11:47:24 -04:00
Joey Hess
72d0769ca5
avoid padding content in serveGet
Always truncate instead. The padding risked something not noticing the
content was bad and getting a file that was corrupted in a novel way
with the padding "X" at the end. A truncated file is better.
2024-07-22 11:19:52 -04:00
Joey Hess
4826a3745d
servePut and clientPut implementation
Made the data-length header required even for v0. This simplifies the
implementation, and doesn't preclude extra verification being done for
v0.

The connectionWaitVar is an ugly hack. In servePut, nothing waits
on the waitvar, and I could not find a good way to make anything wait on
it.
2024-07-22 10:27:44 -04:00
Joey Hess
97a2d0e4fb
use worker pool in withLocalP2PConnections
This allows multiple clients to be handled at the same time.
2024-07-11 14:37:52 -04:00
Joey Hess
14e0f778b7
simplify 2024-07-11 11:50:44 -04:00
Joey Hess
2228d56db3
serveGet invalidation 2024-07-11 11:42:32 -04:00
Joey Hess
3b37b9e53f
fix serveGet hang
This came down to SendBytes waiting on the waitv. Nothing ever filled
it.

Only Annex.Proxy needs the waitv, and it handles filling it. So make it
optional.
2024-07-11 07:46:52 -04:00
Joey Hess
8cb1332407
update 2024-07-10 16:10:08 -04:00
Joey Hess
b5b3d8cde2
update 2024-07-09 14:30:50 -04:00
Joey Hess
751b8e0baf
implemented serveCheckPresent
Still need a way to run Proto though
2024-07-09 09:08:42 -04:00
Joey Hess
3f402a20a8
implement Locker 2024-07-08 21:00:10 -04:00
Joey Hess
838169ee86
status 2024-07-07 16:16:11 -04:00
Joey Hess
40306d3fcf
finalizing HTTP P2p protocol some more
Added v2-v0 endpoints. These are tedious, but will be needed in order to
use the HTTP protocol to proxy to repositories with older git-annex,
where git-annex-shell will be speaking an older version of the protocol.

Changed GET to use 422 when the content is not present. 404 is needed to
detect when a protocol version is not supported.
2024-07-05 15:34:58 -04:00
Joey Hess
0bfdc57d25
update 2024-07-04 15:18:06 -04:00
Joey Hess
f69661ab65
status 2024-07-03 17:04:12 -04:00
Joey Hess
6a95eb08ce
status 2024-07-03 15:01:34 -04:00
Joey Hess
badcb502a4
todo 2024-07-03 13:15:09 -04:00
Joey Hess
24d63e8c8e
update 2024-07-02 18:04:29 -04:00
Joey Hess
b2a24a1669
update 2024-07-02 16:16:37 -04:00
Joey Hess
fbc4d549f3
reorder 2024-07-01 11:44:54 -04:00
Joey Hess
8db30323b0
update 2024-07-01 11:38:29 -04:00
Joey Hess
1e1584d34b
toc 2024-07-01 11:37:12 -04:00
Joey Hess
d9e66f7754
update 2024-07-01 11:33:07 -04:00
Joey Hess
f58a5f577d
update 2024-07-01 11:29:04 -04:00
Joey Hess
fa5e7463eb
fix display when proxied GET yields ERROR
The error message is not displayed to the use, but this mirrors the
behavior when a regular get from a special remote fails. At least now
there is not a protocol error.
2024-07-01 11:19:02 -04:00