This is an appendix to the [[external_special_remote_protocol]].

[[!toc]]

## introduction

Normally, an external special remote can only be used to do one thing at a
time, and when git-annex has concurrency enabled, it will start up multiple
processes for the same external special remote. 

The `ASYNC` extension lets a single external special remote process handle
multiple concurrent jobs, which can be useful if multiple processes
would use too many resources, or if it can be better coordinated using a
single process.

## protocol overview

As usual, the protocol starts by the external special remote sending
the version of the protocol it's using.
	
	VERSION 1

This extension is negotiated by git-annex sending an `EXTENSIONS` message
that includes `ASYNC`, and the external special remote responding in kind.

	EXTENSIONS INFO ASYNC
	EXTENSIONS ASYNC

(Older versions of git-annex will not include `ASYNC` in their extensions
list. To support them, it's a good idea for the external special remote to
fall back to using the regular protocol.)

Once the extension is negotiated, messages in the protocol are
tagged with a job number, by prefixing them with "J n".

As usual, the first message git-annex sends is generally PREPARE,
which gets tagged with a job number:

	J 1 PREPARE

Rather than just responding PREPARE-SUCCESS, the job number has to be
included in the reply:

	J 1 PREPARE-SUCCESS

Suppose git-annex wants to make some transfers. It can request several
at the same time, using different job numbers:

	J 1 TRANSFER RETRIEVE Key1 file1
	J 2 TRANSFER RETRIEVE Key2 file2

The special remote can now perform both transfers at the same time.
If it sends PROGRESS messages for these transfers, they have to be tagged
with the job numbers:
	
	J 1 PROGRESS 10
	J 2 PROGRESS 500
	J 1 PROGRESS 20

The special remote can also send messages that query git-annex for some
information. These messages and the reply will also be tagged with a job
number.

	J 1 GETCONFIG url
	J 3 RETRIEVE Key3 file3
	J 1 VALUE http://example.com/

One transfers are done, the special remote sends `TRANSFER-SUCCESS` tagged
with the job number.

	J 2 TRANSFER-SUCCESS RETRIEVE Key2
	J 1 PROGRESS 100
	J 1 TRANSFER-SUCCESS RETRIEVE Key1

Lots of different jobs can be requested at the same time.

	J 4 CHECKPRESENT Key3
	J 5 CHECKPRESENT Key4
	J 6 REMOVE Key5
	J 4 CHECKPRESENT-SUCCESS Key3
	J 6 REMOVE-SUCCESS Key5
	J 5 CHECKPRESENT-FAILURE Key4

## notes

There will be one job number for each thread that git-annex runs
concurrently, so around the same number as the -J value, although in some
cases git-annex does more concurrent operations than the -J value.

`PREPARE` is sent only once per run of a special remote
program, and despite being tagged with a job number, it should prepare the
special remote to run that and any other jobs.

`ERROR` should not be tagged with a job number if either git-annex
or the special remote needs to send it.

`VERSION`, `EXTENSIONS` and `ERROR` are the only protocol messages
that do not get tagged with a job number.