7546e686a2
Simplified the protocol by removing END-ASYNC. There's a STM crash when a non-async protocol message is sent, which needs to be fixed.
123 lines
4 KiB
Markdown
123 lines
4 KiB
Markdown
(This is a draft and not implemented yet.)
|
|
|
|
This is an appendix to the [[external_special_remote_protocol]].
|
|
|
|
[[!toc]]
|
|
|
|
## introduction
|
|
|
|
Normally, an external special remote can only be used to do one thing at a
|
|
time. When git-annex has concurrency enabled, it will start up multiple
|
|
processes for the same external special remote.
|
|
|
|
This extension lets a single external special remote process handle
|
|
multiple concurrent requests, which can be useful if multiple processes
|
|
would use too many resources, or if it can be better coordinated using a
|
|
single process.
|
|
|
|
## protocol overview
|
|
|
|
As usual, the protocol starts by the external special remote sending
|
|
the version of the protocol it's using.
|
|
|
|
VERSION 1
|
|
|
|
This extension is negotiated by git-annex sending an `EXTENSIONS` message
|
|
that includes `ASYNC`, and the external special remote responding in kind.
|
|
|
|
EXTENSIONS INFO ASYNC
|
|
EXTENSIONS ASYNC
|
|
|
|
From this point forward, *everything* that the external special remote
|
|
has to be wrapped in the async protocol. Messages git-annex sends are
|
|
unchanged.
|
|
|
|
Generally the first message git-annex sends will be PREPARE.
|
|
|
|
PREPARE
|
|
|
|
Rather than just responding PREPARE-SUCCESS, it has to be wrapped
|
|
in the async protocol:
|
|
|
|
RESULT-ASYNC PREPARE-SUCCESS
|
|
|
|
Suppose git-annex wants to make some transfers. So it sends:
|
|
|
|
TRANSFER RETRIEVE Key1 file1
|
|
|
|
The special remote should respond with an unique identifier for this
|
|
async job that it's going to start. The identifier can
|
|
be anything you want to use, but an incrementing number is a
|
|
reasonable choice. (The Key itself is not a good choice, because git-annex
|
|
could make different requests involving the same Key.)
|
|
|
|
START-ASYNC 1
|
|
|
|
Once that's sent, git-annex can send its next request immediately,
|
|
while that transfer is still running. For example, it might request a
|
|
second transfer, and the special remote can reply when it's started that
|
|
transfer too:
|
|
|
|
TRANSFER RETRIEVE 2 file2
|
|
START-ASYNC 2
|
|
|
|
When the special remote sends a message, such as PROGRESS, it has to
|
|
wrap it in ASYNC, to specify the job identifier.
|
|
|
|
ASYNC 1 PROGRESS 10
|
|
ASYNC 2 PROGRESS 500
|
|
ASYNC 1 PROGRESS 20
|
|
|
|
This can also be used to query git-annex for some information.
|
|
The reply to a query is eventually sent back wrapped in `REPLY-ASYNC`.
|
|
|
|
ASYNC 1 GETCONFIG url
|
|
TRANSFER RETRIEVE 3 file3
|
|
REPLY-ASYNC 1 VALUE http://example.com/
|
|
|
|
Once a transfer is done, the special remote indicates this by
|
|
wrapping the usual `TRANSFER-SUCCESS` or
|
|
`TRANSFER-FAILURE` message in `ASYNC`.
|
|
|
|
ASYNC 2 TRANSFER-SUCCESS RETRIEVE Key2
|
|
ASYNC Key1 PROGRESS 100
|
|
ASYNC 1 TRANSFER-SUCCESS RETRIEVE Key1
|
|
|
|
Not only transfers, but everything the special remote sends to git-annex
|
|
has to be wrapped in the async protocol.
|
|
|
|
CHECKPRESENT Key3
|
|
START-ASYNC 3
|
|
CHECKPRESENT Key4
|
|
START-ASYNC 4
|
|
ASYNC 3 CHECKPRESENT-SUCCESS Key3
|
|
REMOVE Key3
|
|
ASYNC 4 CHECKPRESENT-FAILURE Key4
|
|
START-ASYNC 5
|
|
ASYNC 5 REMOVE-SUCCESS Key3
|
|
|
|
## added messages
|
|
|
|
Here's the details about the additions to the protocol.
|
|
|
|
* `START-ASYNC JobId`
|
|
This (or `RESULT-ASYNC` must be sent in response to all requests
|
|
git-annex sends after `EXTENSIONS` has been used to negotiate the
|
|
async protocol.
|
|
The JobId is a unique value, typically an incrementing number.
|
|
This does not need to be sent immediately after git-annex sends a request;
|
|
other messages can be sent in between. But the next START-ASYNC git-annex sees
|
|
after sending a request tells it the JobId that will be used for that request.
|
|
* `ASYNC JobId Msg`
|
|
All the usual protocol messages that are sent by the external special
|
|
remote must be wrapped in this, to specify which job the message relates
|
|
to.
|
|
* `RESULT-ASYNC ResultMsg`
|
|
This is the same as sending `START-ASYNC` immediately followed by
|
|
`ASYNC` with a result message. This is often used to respond to
|
|
`PREPARE` other things that are trivial or just don't need to be handled
|
|
async.
|
|
* `REPLY-ASYNC JobId Reply`
|
|
Sent by git-annex when an `ASYNC` requested a reply.
|
|
Note that this may not be the next message received from
|
|
git-annex immediately after sending an `ASYNC` request.
|