9e11699c76
Most remotes have meters in their implementations of retrieveKeyFile already. Simply hooking these up to the transfer log makes that information available. Easy peasy. This is particularly valuable information for encrypted remotes, which otherwise bypass the assistant's polling of temp files, and so don't have good progress bars yet. Still some work to do here (see progressbars.mdwn changes), but this is entirely an improvement from the lack of progress bars for encrypted downloads.
63 lines
2.5 KiB
Markdown
63 lines
2.5 KiB
Markdown
Currently, git-annex takes a very lazy approch to displaying
|
|
progress into. It just lets rsync or whatever display the progress
|
|
for it, in the terminal.
|
|
|
|
Something better is needed for the [[webapp]]. There needs to be a
|
|
way for the web app to know what the current progress is of all transfers.
|
|
|
|
This is one of those potentially hidden but time consuming problems.
|
|
|
|
## downloads
|
|
|
|
* Watch temp file as it's coming in and use its size.
|
|
Can either poll every .5 seconds or so to check file size, or
|
|
could use inotify. **done**
|
|
* When easily available, remotes call the MeterUpdate callback as uploads
|
|
progress. **done**
|
|
|
|
* TODO a bad interaction can happen between the TransferPoller and the
|
|
TransferWatcher when downloading from an encrypted remote. If
|
|
a partially transferred file exists already, in the gitAnnexTmpLocation
|
|
of the (un-encrypted) key, the TransferPoller will trust it to have
|
|
the right size of the content downloaded. This will stomp, every 0.5
|
|
seconds, over the updates to the size that the TransferWatcher is seeing
|
|
in the transfer log files.
|
|
|
|
We still need the TransferPoller for the remotes that don't have
|
|
download meters. This includes git, web, bup, and hook.
|
|
|
|
Need to teach the TransferPoller to detect when transfer logs for downloads
|
|
have file size info, and use it, rather than looking at the temp file.
|
|
The question is, how to do this efficiently? It could just poll the
|
|
transfer log every time, and if size is nonzero, ignore the temp file.
|
|
This would work, but it would require a lot more work than the simple
|
|
statting of the file it does now. And this runs every 0.5 seconds.
|
|
|
|
I could try to convert all remotes I care about to having progress
|
|
for downloads. But converting the web special remote will be hard..
|
|
|
|
I think perhaps the best solution is to make the TransferWatcher also watch
|
|
the temp files. Then if one changes, it can get its new size. If a
|
|
transfer info file changes, it can get the size from there.
|
|
|
|
## uploads
|
|
|
|
Each individual remote type needs to implement its own support for calling
|
|
the MeterUpdate callback as the upload progresses.
|
|
|
|
* git: **done**
|
|
* rsync: **done**
|
|
* directory: **done**
|
|
* web: Not applicable; does not upload
|
|
* webdav: **done**
|
|
* S3: **done**
|
|
* glacier: **done**
|
|
* bup: TODO
|
|
* hook: Would require the hook interface to somehow do this, which seems
|
|
too complicated. So skipping.
|
|
|
|
## communication
|
|
|
|
It may be worth using a better communication channel than files on disk for
|
|
the transfer progress. Shared memory could be used, or http posts to the
|
|
webapp.
|