git-annex/doc/todo/serialize_Messages_for_transferkeys.mdwn

For [[todo/more_extensive_retries_to_mask_transient_failures]],
multiple transferkeys processes need to be run. So all console IO needs to
be serialized and sent back to the main git-annex process from them,
to avoid concurrent output.

Using --json and --json-progress and --json-error-messages is pretty close
to what would be needed. That handles progress of transfers, and overall
success/failure. 

Using --json-error-messages makes most error messages be output in json.
However, that uses an error-messages array in the json object,
which error messages are added to as an action runs, and it's only sent at
the end. So a problem for eg, a relay of stderr output from a command
(eg ssh password prompt), or for anything that displays a warning that
should be displayed before the transfer is complete.

Since the existing json is not a perfect fit, it might be better to use a
custom protocol, implemented as a new output type. Then anything in
Messages and child modules that looks at output types will support it.
(Although perhaps some of that stuff is not used by remotes.)

A few notes on implementing that:

* Messages.Progress.mkOutputHandler, which uses mkStderrEmitter,
  outputs to stderr directly no matter the output type currently.
  It would need to be changed to support the new output type.
  (And probably should for concurrent output mode too actually!)

  > It's true, this is not concurrent output safe. However, that's already
  > the case, and output to stderr doesn't affect the piping of serialized
  > messages on stdout. So, punted on this.

* So does warningIO, though it's only used in a couple of remotes
  and rarely. It would be good to find a way to eliminate it.

  > Eliminated except for one call in a non-relevant code path.

* Messages.prompt. Which is used by remotes, and would need to 
  communicate over the pipe to the parent git-annex bidirectionally.
  Eg, send a message saying the parent needs to prepare for prompt,
  wait for it to reply saying it has, and then send a message when the
  prompting is done. (Note that the parent would need to detect if the child
  process crashed to avoid being locked waiting for the prompt.)

  > Done.

[[done]]