mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

Merge branch 'master' into httpproto

Browse source
This commit is contained in:
Joey Hess 2024-07-07 12:08:24 -04:00
commit 4133063ab1
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
4 changed files with 105 additions and 16 deletions
Download patch file Download diff file Expand all files Collapse all files

8
doc/design/p2p_protocol.mdwn
View file

@ -66,7 +66,7 @@ Now both client and server should use version 1.
## Cluster cycle prevention
In protocol version 2, immediately after VERSION, the
In protocol version 2 and above, immediately after VERSION, the
client can send an additional message that is used to
prevent cycles when accessing clusters.
@ -135,9 +135,9 @@ The server responds with either SUCCESS or FAILURE.
Note that if the content was not present, SUCCESS will be returned.
In protocol version 2, the server can optionally reply with SUCCESS-PLUS
or FAILURE-PLUS. Each has a subsequent list of UUIDs of repositories
that the content was removed from.
In protocol version 2 and above, the server can optionally reply with
SUCCESS-PLUS or FAILURE-PLUS. Each has a subsequent list of UUIDs of
repositories that the content was removed from.
## Removing content before a specified time

89
doc/design/p2p_protocol_over_http/draft1.mdwn
View file

@ -30,9 +30,7 @@ may want git-annex to use HTTP in eg a LAN.
## protocol version
Each request in the protocol is versioned. The versions correspond
to P2P protocol versions, but for simplicity, the minimum version supported
over HTTP is version 3. Every implementation of the HTTP protocol must
support version 3.
to P2P protocol versions.
The protocol version comes before the request. Eg: `/git-annex/v3/put`
@ -65,6 +63,8 @@ Any request may also optionally include these parameters:
This parameter can be given multiple times to list several cluster
gateway UUIDs.
This parameter is only available for v3 and above.
[Internally, git-annex can use these common parameters, plus the protocol
version, to create a P2P session. The P2P session is driven through
the AUTH, VERSION, and BYPASS messages, leaving the session ready to
@ -79,9 +79,11 @@ It is not part of the P2P protocol per se, but is provided to let
other clients than git-annex easily download the content of keys from the
http server.
This behaves the same as `GET /git-annex/v3/key/$key`, although its
This behaves almost the same as `GET /git-annex/v3/key/$key`, although its
behavior may change in later versions.
When the key is not present on the server, this returns a 404 Not Found.
### GET /git-annex/v3/key/$key
Get the content of a key from the server.
@ -130,7 +132,21 @@ X-git-annex-data-length header indicated, then the data is invalid and
should not be used. This can happen when eg, the data was being sent from
an unlocked annexed file, which got modified while it was being sent.
When the content is not present, the server will respond with 404.
When the content is not present, the server will respond with
422 Unprocessable Content.
### GET /git-annex/v2/key/$key
Identical to v3.
### GET /git-annex/v1/key/$key
Identical to v3.
### GET /git-annex/v0/key/$key
Same as v3, except there is no X-git-annex-data-length header.
Additional checking client-side will be required to validate the data.
### POST /git-annex/v3/checkpresent
@ -148,6 +164,18 @@ The body of the request is empty.
The server responds with a JSON object with a "present" field that is true
if the key is present, or false if it is not present.
### POST /git-annex/v2/checkpresent
Identical to v3.
### POST /git-annex/v1/checkpresent
Identical to v3.
### POST /git-annex/v0/checkpresent
Identical to v3.
### POST /git-annex/v3/lockcontent
Locks the content of a key on the server, preventing it from being removed.
@ -172,6 +200,18 @@ If the client disconnects without sending "UNLOCKCONTENT", or the web
server gets shut down before it can receive that, the content will remain
locked for at least 10 minutes from when the server sent "SUCCESS".
### POST /git-annex/v2/lockcontent
Identical to v3.
### POST /git-annex/v1/lockcontent
Identical to v3.
### POST /git-annex/v0/lockcontent
Identical to v3.
### POST /git-annex/v3/remove
Remove a key's content from the server.
@ -196,6 +236,18 @@ If the server does not allow removing the key due to a policy
(eg due to being read-only or append-only), it will respond with a JSON
object with an "error" field that has an error message as its value.
### POST /git-annex/v2/remove
Identical to v3.
### POST /git-annex/v1/remove
Same as v3, except the JSON will not include "plusuuids".
### POST /git-annex/v0/remove
Identival to v1.
## POST /git-annex/v3/remove-before
Remove a key's content from the server, but only before a specified time.
@ -281,6 +333,19 @@ If the server does not allow storing the key due eg to a policy
invalid, or because it ran out of disk space, it will respond with a
JSON object with an "error" field that has an error message as its value.
### POST /git-annex/v2/put
Identical to v3.
### POST /git-annex/v1/put
Same as v3, except the JSON will not include "plusuuids".
### POST /git-annex/v0/put
Same as v1, except there is no X-git-annex-data-length header.
Additional checking client-side will be required to validate the data.
### POST /git-annex/v3/putoffset
Asks the server what `offset` can be used in a `put` of a key.
@ -301,6 +366,12 @@ The body of the request is empty.
The server responds with a JSON object with an "offset" field that
is the largest allowable offset.
If the server already has the content of the key, it will respond with a
JSON object with an "alreadyhave" field that is set to true. This JSON
object may also have a field "plusuuids" that lists
the UUIDs of other repositories where the content is stored, in addition to
the serveruuid.
If the server does not allow storing the key due to a policy
(eg due to being read-only or append-only), it will respond with a JSON
object with an "error" field that has an error message as its value.
@ -311,6 +382,14 @@ part way through a `PUT`, a synthetic empty `DATA` followed by `INVALID`
will be used to get the P2P protocol back into a state where it will accept
any request.]
### POST /git-annex/v2/putoffset
Identical to v3.
### POST /git-annex/v1/putoffset
Same as v3, except the JSON will not include "plusuuids".
## parts of P2P protocol that are not supported over HTTP
`NOTIFYCHANGE` is not supported, but it would be possible to extend

9
doc/forum/Questions_about_borg_special_remotes/comment_3_a036be41663d81a79fe47e9021208169._comment Normal file
View file

@ -0,0 +1,9 @@
[[!comment format=mdwn
username="git-annex@4a0625db6ced1ac00744697d5bac41393bcde646"
nickname="git-annex"
avatar="http://cdn.libravatar.org/avatar/d9fd8db33996d9a1da6b31045c43accf"
subject="comment 3"
date="2024-07-05T10:22:45Z"
content="""
Is there a way to configure a certain BORG_PASSCOMMAND to always be used for a given remote. In general, is there a way to set environment variables per remote?
"""]]

15
doc/todo/git-annex_proxies.mdwn
View file

@ -28,19 +28,20 @@ Planned schedule of work:
## work notes
* websockets or something else for LOCKCONTENT over http?
* Next step: Ready to begin implementing in servant. I have a file
`servant.hs` in the httpproto branch that works through some of the
bytestring streaming issues.
* Should integrate into an existing web server similarly to git
smart http server. May as well also support it as a standalone
webserver.
* endpoint versioning should include v1 and v0 to support connections
proxied from older clients
* Perhaps: Support cgi program that proxies over to a webserver
speaking the http protocol.
## completed items for July's work on p2p protocol over http
* addressed [[doc/todo/P2P_locking_connection_drop_safety]]
* finalized HTTP P2P protocol draft 1,
[[design/p2p_protocol_over_http/draft1]]
## items deferred until later for [[design/passthrough_proxy]]
* Check annex.diskreserve when proxying for special remotes