assistant unused file handling

Make sanity checker run git annex unused daily, and queue up transfers
of unused files to any remotes that will have them. The transfer retrying
code works for us here, so eg when a backup disk remote is plugged in,
any transfers to it are done. Once the unused files reach a remote,
they'll be removed locally as unwanted.

If the setup does not cause unused files to go to a remote, they'll pile
up, and the sanity checker detects this using some heuristics that are
pretty good -- 1000 unused files, or 10% of disk used by unused files,
or more disk wasted by unused files than is left free. Once it detects
this, it pops up an alert in the webapp, with a button to take action.

TODO: Webapp UI to configure this, and also the ability to launch an
immediate cleanup of all unused files.

This commit was sponsored by Simon Michael.
This commit is contained in:
Joey Hess 2014-01-22 22:48:56 -04:00
parent 85aae97b63
commit 3da0064657
15 changed files with 188 additions and 14 deletions

View file

@ -1,6 +1,6 @@
{- git-annex configuration
-
- Copyright 2012 Joey Hess <joey@kitenet.net>
- Copyright 2012-2014 Joey Hess <joey@kitenet.net>
-
- Licensed under the GNU GPL version 3 or higher.
-}
@ -20,6 +20,7 @@ import Config.Cost
import Types.Distribution
import Types.Availability
import Types.NumCopies
import Utility.HumanTime
{- Main git-annex settings. Each setting corresponds to a git-config key
- such as annex.foo -}
@ -46,6 +47,7 @@ data GitConfig = GitConfig
, annexLargeFiles :: Maybe String
, annexFsckNudge :: Bool
, annexAutoUpgrade :: AutoUpgrade
, annexExpireUnused :: Maybe Duration
, coreSymlinks :: Bool
, gcryptId :: Maybe String
}
@ -75,6 +77,7 @@ extractGitConfig r = GitConfig
, annexLargeFiles = getmaybe (annex "largefiles")
, annexFsckNudge = getbool (annex "fscknudge") True
, annexAutoUpgrade = toAutoUpgrade $ getmaybe (annex "autoupgrade")
, annexExpireUnused = parseDuration =<< getmaybe (annex "expireunused")
, coreSymlinks = getbool "core.symlinks" True
, gcryptId = getmaybe "core.gcrypt-id"
}