58 lines
2.6 KiB
Text
58 lines
2.6 KiB
Text
|
Webapp needs a simple speed control knob, especially to avoid saturating
|
||
|
|
bandwidth on uploads.
|
||
|
|
|
||
|
|
We have basic throttling support in git-annex for rsync,
|
||
|
|
but none for any special remotes. A good first step would be to expose
|
||
|
|
this in the webapp, and ensure that `git-annex-shell` also honors it when
|
||
|
|
sending/receiving data.
|
||
|
|
|
||
|
|
We actually need two speed controls, one for upload and one for download.
|
||
|
|
|
||
|
|
It is probably not necessary to throttle git push/pull operations, as the
|
||
|
|
data transfers tend to be small. Only throttling file transfers is
|
||
|
|
essential.
|
||
|
|
|
||
|
|
## possibility: trickle
|
||
|
|
|
||
|
|
Since `git-annex transferkeys` is a separate process, one easy option would
|
||
|
|
be to run it inside `trickle`. If the user changes the bandwidth limits,
|
||
|
|
it could kill the transfer and restart it with different trickle options.
|
||
|
|
|
||
|
|
Problem: Not all special remotes support resuming transfers, so this is
|
||
|
|
suboptimal. (So too are the pause/resume buttons, when using those
|
||
|
|
remotes!)
|
||
|
|
|
||
|
|
`trickle` is available for OSX as well as Linux and BSDs.
|
||
|
|
<http://hints.macworld.com/comment.php?mode=view&cid=65362>
|
||
|
|
<http://mac.softpedia.com/get/Network-Admin/Trickle.shtml>
|
||
|
|
It is probably not easily available for Android, as it uses `LD_PRELOAD`.
|
||
|
|
|
||
|
|
## possibility: built in IO limiting
|
||
|
|
|
||
|
|
A cleaner method would be to do the limiting inside git-annex. We already
|
||
|
|
have metered file IO. It should be possible to make the meter not only report
|
||
|
|
on the transfer speed, but detect when it's going too fast, and delay. This
|
||
|
|
will delay the file streaming through the special remote's transfer code,
|
||
|
|
so should work for a variety of special remotes. (Not for rsync or bup
|
||
|
|
or git-annex-shell though, so those need to be handled separately.)
|
||
|
|
|
||
|
|
Should work well for uploads at least. I don't know how well it would work
|
||
|
|
for throttling downloads; the sender may just keep sending data and the
|
||
|
|
data buffer before it gets to the IO meter. Maybe once the buffers fill the
|
||
|
|
OS would have the TCP throttled down. Needs investigation; trickle claims
|
||
|
|
to throttle downloads.
|
||
|
|
|
||
|
|
## communications channels
|
||
|
|
|
||
|
|
There would need to be a communication channel for the assistant to tell
|
||
|
|
`git annex transferkeys` when the rate limit has changed. It could for
|
||
|
|
example send it a SIGUSR1, and then leave it up to the process to reload
|
||
|
|
the git config. Inside the IO meter, we could have an MVar that contains
|
||
|
|
the current throttle value, so the IO meter could check it each time it's
|
||
|
|
called and adjust its throttling appropriately.
|
||
|
|
|
||
|
|
Ideally, the assistant could also communicate in the same way with
|
||
|
|
`git-annex-shell` to tell it when the limit has changed. Since
|
||
|
|
`git-annex-shell` uses rsync, it would need to abort the transfer, and rely
|
||
|
|
on the other side retrying to start it up with the new limit.
|