git-annex/Logs
Joey Hess 6821ba8dab
sync: use log to track adjusted branch needs updating
Speeds up sync in an adjusted branch by avoiding re-adjusting the branch
unncessarily, particularly when it is adjusted with --hide-missing or
--unlock-present.

When there are a lot of files, that was the majority of the time of a
--no-content sync.

Uses a log file, which is updated when content presence changes. This
adds a little bit of overhead to every file get/drop when on such an
adjusted branch. The overhead is minimal for get of any size of file,
but might be noticable for drop in some cases. It seems like a reasonable
trade-off. It would be possible to update the log file only at the end, but
then it would not happen if the command is interrupted.

When not in an adjusted branch, there should be no additional overhead.
(getCurrentBranch is an MVar read, and it avoids the MVar read of
getGitConfig.)

Note that this does not deal with situations such as:
git checkout master, git-annex get, git checkout adjusted branch,
git-annex sync. The sync won't know that the adjusted branch needs to be
updated. Dealing with that would add overhead to operation in non-adjusted
branches, which I don't like. Also, there are other situations like having
two adjusted branches that both need to be updated like this, and switching
between them and sync not updating.

This does mean a behavior change to sync, since it did previously deal
with those situations. But, the documentation did not say that it did.
The man pages only talk about sync updating the adjusted branch after
it transfers content.

I did consider making sync keep track of content it transferred (and
dropped) and only update the adjusted branch then, not to catch up to other
changes made previously. That would perform better. But it seemed rather
hard to implement, and also it would have problems with races with a
concurrent get/drop, which this implementation avoids.

And it seemed pretty likely someone had gotten used to get/drop followed by
sync updating the branch. It seems much less likely someone is switching
branches, doing get/drop, and then switching back and expecting sync to update
the branch.

Re-running git-annex adjust still does a full re-adjusting of the branch,
for anyone who needs that.

Sponsored-by: Leon Schuermann on Patreon
2023-06-08 14:35:41 -04:00
..
Chunk
ContentIdentifier
Difference
Export Apply codespell -w throughout 2023-03-17 15:14:58 -04:00
MetaData
PreferredContent
Presence split out appending to journal from writing, high level only 2022-07-18 13:22:50 -04:00
Remote
SingleValue
Trust
Activity.hs
AdjustedBranchUpdate.hs sync: use log to track adjusted branch needs updating 2023-06-08 14:35:41 -04:00
Chunk.hs Apply codespell -w throughout 2023-03-17 15:14:58 -04:00
Config.hs
ContentIdentifier.hs
Difference.hs
Export.hs Apply codespell -w throughout 2023-03-17 15:14:58 -04:00
File.hs fix deadlock in restagePointerFiles 2022-12-08 14:36:11 -04:00
FsckResults.hs
Group.hs
Import.hs implement importChanges optimisaton (not used yet) 2023-05-31 16:01:34 -04:00
Line.hs
Location.hs
MapLog.hs
MetaData.hs
Multicast.hs
NumCopies.hs cache negative lookups of global numcopies and mincopies 2023-06-06 14:43:25 -04:00
PreferredContent.hs
Presence.hs split out appending to journal from writing, high level only 2022-07-18 13:22:50 -04:00
Remote.hs
RemoteState.hs
Restage.hs fix deadlock in restagePointerFiles 2022-12-08 14:36:11 -04:00
Schedule.hs
SingleValue.hs
Smudge.hs add a finalizer to streamLogFile 2022-09-23 13:49:01 -04:00
Transfer.hs rename Git.Filename to Git.Quote 2023-04-12 17:22:03 -04:00
Transitions.hs
Trust.hs info: Added --autoenable option 2022-06-01 14:20:38 -04:00
Unused.hs Apply codespell -w throughout 2023-03-17 15:14:58 -04:00
Upgrade.hs add annex.dbdir (WIP) 2022-08-11 16:58:53 -04:00
UUID.hs one-way escaping of newlines in uuid.log 2023-03-13 14:19:32 -04:00
UUIDBased.hs Apply codespell -w throughout 2023-03-17 15:14:58 -04:00
View.hs understand adjusted view branch names 2023-02-27 14:57:58 -04:00
Web.hs add annex.alwayscompact 2022-07-18 16:39:19 -04:00