assistant unused file handling

Make sanity checker run git annex unused daily, and queue up transfers
of unused files to any remotes that will have them. The transfer retrying
code works for us here, so eg when a backup disk remote is plugged in,
any transfers to it are done. Once the unused files reach a remote,
they'll be removed locally as unwanted.

If the setup does not cause unused files to go to a remote, they'll pile
up, and the sanity checker detects this using some heuristics that are
pretty good -- 1000 unused files, or 10% of disk used by unused files,
or more disk wasted by unused files than is left free. Once it detects
this, it pops up an alert in the webapp, with a button to take action.

TODO: Webapp UI to configure this, and also the ability to launch an
immediate cleanup of all unused files.

This commit was sponsored by Simon Michael.
This commit is contained in:
Joey Hess 2014-01-22 22:48:56 -04:00
parent 85aae97b63
commit 3da0064657
15 changed files with 188 additions and 14 deletions

9
debian/changelog vendored
View file

@ -18,6 +18,15 @@ git-annex (5.20140118) UNRELEASED; urgency=medium
preferred content expressions.
* Client, transfer, incremental backup, and archive repositories
now want to get content that does not yet have enough copies.
* Client, transfer, and source repositories now do not want to retain
unused file contents.
* assistant: Checks daily for unused file contents, and when possible
moves them to a repository (such as a backup repository) that
wants to retain them.
* assistant: annex.expireunused can be configured to cause unused
file contents to be deleted after some period of time.
* webapp: Nudge user to see if they want to expire old unused file
contents when a lot of them seem to be piling up in the repository.
* repair: Check git version at run time.
* assistant: Run the periodic git gc in batch mode.