cache remotes via annex-speculate-present

Added remote.name.annex-speculate-present config that can be used to
make cache remotes.

Implemented it in Remote.keyPossibilities, which is used by the
get/move/copy/mirror commands, and nothing else. This way, things like
whereis will not show content that's speculatively present.

The assistant and sync --content were not using Remote.keyPossibilities,
and were changed to use it.

The efficiency hit should be small; Remote.keyPossibilities is only
used before transferring a file, which is the expensive operation.
And, it's only doing one lookup of the remoteList and a very cheap
filter over it.

Note that, git-annex still updates the location log when copying content
to a remote with annex-speculate-present set. In this case, the location
tracking will indicate that content is present in the remote. This may
not be wanted for caches, or may not be a real problem for them. TBD.

This commit was supported by the NSF-funded DataLad project.
This commit is contained in:
Joey Hess 2018-08-01 14:22:52 -04:00
parent 2884637cab
commit fd5a392006
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
7 changed files with 114 additions and 4 deletions

View file

@ -92,7 +92,7 @@ queueTransfersMatching matching reason schedule k f direction
filter (\r -> not (inset s r || Remote.readonly r))
(syncDataRemotes st)
where
locs = S.fromList <$> Remote.keyLocations k
locs = S.fromList . map Remote.uuid <$> Remote.keyPossibilities k
inset s r = S.member (Remote.uuid r) s
gentransfer r = Transfer
{ transferDirection = direction