mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

assistant unused file handling

Browse source 
Make sanity checker run git annex unused daily, and queue up transfers
of unused files to any remotes that will have them. The transfer retrying
code works for us here, so eg when a backup disk remote is plugged in,
any transfers to it are done. Once the unused files reach a remote,
they'll be removed locally as unwanted.

If the setup does not cause unused files to go to a remote, they'll pile
up, and the sanity checker detects this using some heuristics that are
pretty good -- 1000 unused files, or 10% of disk used by unused files,
or more disk wasted by unused files than is left free. Once it detects
this, it pops up an alert in the webapp, with a button to take action.

TODO: Webapp UI to configure this, and also the ability to launch an
immediate cleanup of all unused files.

This commit was sponsored by Simon Michael.
This commit is contained in:
Joey Hess 2014-01-22 22:48:56 -04:00
parent 85aae97b63
commit 3da0064657
15 changed files with 188 additions and 14 deletions
Download patch file Download diff file Expand all files Collapse all files

7
doc/todo/Limit_file_revision_history.mdwn
View file

@ -51,12 +51,13 @@ Finally, how to specify a feature request for git-annex?
> `and (not unused)`. Transfer repositories too, because typically
> only client repos connect to them, and so otherwise unused files
> would build up there. Backup repos would want unused files. I
> think that archive repos would too.
> think that archive repos would too. **done**
> * Make the assistant check for unused files periodically. Exactly
> how often may need to be tuned, but once per day seems reasonable
> for most repos. Note that the assistant could also notice on the
> fly when files are removed and mark their keys as unused if that was
> the last associated file. (Only currently possible in direct mode.)
> **done**
> * After scanning for unused files, it makes sense for the
> assistant to queue transfers of unused files to any remotes that
> do want them (eg, backup remotes). If the files can successfully be
@ -75,7 +76,9 @@ Finally, how to specify a feature request for git-annex?
> is not set, and there is some significant quantity of unused files
> (eg, more than 1000, or more than 1 gb, or more than the amount of
> remaining free disk space),
> it can pop up a webapp alert asking to configure it.
> it can pop up a webapp alert asking to configure it. **done**
> * Webapp interface to configure annex.expireunused. Reasonable values
> are no expiring, or any number of days.
>
> This does not cover every use case that was requested.
> But I don't see a cheap way to ensure it keeps eg the past 10 versions of