Hi!

I'm wondering how clever git-annex is at parallelizing different
remotes with different bandwidth. For example here I have a `origin`
remote that is on the local LAN (so it's fast) and an `offsite-annex`
remote that's, well, offsite, so it's much slower. If I run `git annex
sync --content -J2`, it *looks* like it indiscriminately starts
copying over files to either host without too much though:

    $ git annex sync --content -J2
    [...]
    copy 2019/01/29/montreal/DSCF8148.JPG (to origin...) ok
    copy 2019/01/29/montreal/DSCF8148.JPG (checking offsite-annex...) (to offsite-annex...) ok
    copy 2019/01/29/montreal/DSCF8147.RAF (checking offsite-annex...) (to offsite-annex...) 
    copy 2019/01/29/montreal/DSCF8148.RAF (to origin...) ok
    copy 2019/01/29/montreal/DSCF8148.RAF (checking offsite-annex...) (to offsite-annex...) 

The interactivity of this doesn't show well here, but what happens is
this, in order:

 1. a first JPG is copied to origin and offsite-annex in parallel
    (good)

 2. the origin (local) JPG transfer completes, a RAF file transfer
    gets started to offsite-annex (not so great - a best strategy
    would be to continue copying files to the local remote, as it's
    fast and its bandwidth is now unused)

 3. the offsite-annex JPG transfer completes, a RAF transfer starts to
    origin (good)

 4. that transfer completes, the same file is now copied to offsite
    (again, not so great - local remote is now unused)

What I think git-annex should be doing is try, as much as possible to
saturate the different network links represented by the different
remotes. In my case, files should be transfered on the local LAN as
soon as possible: a single thread should be busy with that `origin`
remote as long as files are missing there, while the other thread can
slowly trickle files to `offsite`. Only when `origin` is full should
both threads work on the `offsite` one. A simple heuristic would be
"is there a thread already busy with that remote? if yes, see if
another remote needs a file transfered that is not busy".

Am I analysing this correctly? Is this a bug? or feature request?
:) -- [[anarcat]]
