CI to automate build of cargo lockfiles on different Alpine releases for git-annex aports https://gitlab.alpinelinux.org/alpine/aports/-/tree/master/community/git-annex
Find a file
Joey Hess 6821ba8dab
sync: use log to track adjusted branch needs updating
Speeds up sync in an adjusted branch by avoiding re-adjusting the branch
unncessarily, particularly when it is adjusted with --hide-missing or
--unlock-present.

When there are a lot of files, that was the majority of the time of a
--no-content sync.

Uses a log file, which is updated when content presence changes. This
adds a little bit of overhead to every file get/drop when on such an
adjusted branch. The overhead is minimal for get of any size of file,
but might be noticable for drop in some cases. It seems like a reasonable
trade-off. It would be possible to update the log file only at the end, but
then it would not happen if the command is interrupted.

When not in an adjusted branch, there should be no additional overhead.
(getCurrentBranch is an MVar read, and it avoids the MVar read of
getGitConfig.)

Note that this does not deal with situations such as:
git checkout master, git-annex get, git checkout adjusted branch,
git-annex sync. The sync won't know that the adjusted branch needs to be
updated. Dealing with that would add overhead to operation in non-adjusted
branches, which I don't like. Also, there are other situations like having
two adjusted branches that both need to be updated like this, and switching
between them and sync not updating.

This does mean a behavior change to sync, since it did previously deal
with those situations. But, the documentation did not say that it did.
The man pages only talk about sync updating the adjusted branch after
it transfers content.

I did consider making sync keep track of content it transferred (and
dropped) and only update the adjusted branch then, not to catch up to other
changes made previously. That would perform better. But it seemed rather
hard to implement, and also it would have problems with races with a
concurrent get/drop, which this implementation avoids.

And it seemed pretty likely someone had gotten used to get/drop followed by
sync updating the branch. It seems much less likely someone is switching
branches, doing get/drop, and then switching back and expecting sync to update
the branch.

Re-running git-annex adjust still does a full re-adjusting of the branch,
for anyone who needs that.

Sponsored-by: Leon Schuermann on Patreon
2023-06-08 14:35:41 -04:00
Annex sync: use log to track adjusted branch needs updating 2023-06-08 14:35:41 -04:00
Assistant filter out control characters in all other Messages 2023-04-11 12:58:01 -04:00
Backend filter out control characters in all other Messages 2023-04-11 12:58:01 -04:00
Build document how to include= a path with a space in it 2023-05-15 16:02:06 -04:00
CmdLine Fix bug in -z handling of trailing NUL in input 2023-05-19 14:34:02 -04:00
Command sync content with import remotes 2023-06-01 18:52:19 -04:00
Config use status --ignore-submodules in configureSmudgeFilter 2022-12-20 16:02:42 -04:00
Database speed up very first tree import by 25% 2023-06-02 13:30:30 -04:00
debian add m68k to arch lists for webapp build deps 2023-02-23 14:26:20 -04:00
doc update 2023-06-07 11:32:53 -04:00
Git implement importChanges optimisaton (not used yet) 2023-05-31 16:01:34 -04:00
Limit new matching options --want-get-by and --want-drop-by 2022-07-28 13:26:03 -04:00
Logs sync: use log to track adjusted branch needs updating 2023-06-08 14:35:41 -04:00
Messages factor out maybeAddJSONField 2023-05-08 16:15:41 -04:00
P2P filter out control characters in error messages 2023-04-10 13:50:51 -04:00
Remote speed up very first tree import by 25% 2023-06-02 13:30:30 -04:00
RemoteDaemon filter out control characters in error messages 2023-04-10 13:50:51 -04:00
standalone Revert "Removed the vendored git-lfs and the GitLfs build flag" 2023-01-04 17:33:29 -04:00
static Revert "remove newlines from static js and css" 2014-06-13 02:20:39 -04:00
templates Apply codespell -w throughout 2023-03-17 15:14:58 -04:00
Test add test summary with number of parts and time 2023-05-18 11:20:27 -04:00
Types implement importChanges optimisaton (not used yet) 2023-05-31 16:01:34 -04:00
Upgrade filter out control characters in all other Messages 2023-04-11 12:58:01 -04:00
Utility build git trees using ContentIdentifier to speed up import 2023-05-31 12:46:54 -04:00
.appveyor.yml update appveyor config 2023-03-22 01:02:26 -04:00
.codespellrc keyspell ignore tmp directory 2023-03-17 15:21:31 -04:00
.ghci turn of PackageImports in cabal file 2022-02-25 13:16:36 -04:00
.gitattributes update changelog location 2016-08-22 23:54:11 -04:00
.gitignore ignore git-annex-build-deps files from dpkg-genbuilddeps 2023-01-29 11:29:51 -04:00
.mailmap alias for Yaroslav @ yahoo to mailmap 2015-09-09 12:38:59 -04:00
Annex.hs cache negative lookups of global numcopies and mincopies 2023-06-06 14:43:25 -04:00
Assistant.hs assistant: Fix a crash on startup by avoiding using forkProcess 2021-05-12 15:08:03 -04:00
Backend.hs filter out control characters in warning messages 2023-04-10 15:55:44 -04:00
bash-completion.bash bash completion fix 2018-11-12 13:23:05 -04:00
Benchmark.hs --size-limit exit 101 2021-06-04 16:43:47 -04:00
BuildFlags.hs show when build with Benchmark build flag 2022-08-29 14:55:58 -04:00
BuildInfo.hs update licenses from GPL to AGPL 2019-03-13 15:48:14 -04:00
CHANGELOG sync: use log to track adjusted branch needs updating 2023-06-08 14:35:41 -04:00
CmdLine.hs avoid flushing keys db queue after each Annex action 2022-10-12 14:12:23 -04:00
Command.hs uninit: Support --json and --json-error-messages 2023-05-11 13:43:02 -04:00
Common.hs Windows: Support long filenames in more (possibly all) of the code 2023-03-01 15:55:58 -04:00
Config.hs added an optional cost= configuration to all special remotes 2023-01-12 13:42:28 -04:00
COPYRIGHT general copyright year bump 2023-01-09 15:05:27 -04:00
Creds.hs filter out control characters in all other Messages 2023-04-11 12:58:01 -04:00
Crypto.hs filter out control characters in error messages 2023-04-10 13:50:51 -04:00
git-annex.cabal sync: use log to track adjusted branch needs updating 2023-06-08 14:35:41 -04:00
git-annex.hs sanitize control characters in main thread fatal exceptions 2023-04-12 14:21:53 -04:00
git-union-merge.hs update licenses from GPL to AGPL 2019-03-13 15:48:14 -04:00
Git.hs filter out control characters in error messages 2023-04-10 13:50:51 -04:00
Key.hs convert Key to ShortByteString 2021-10-05 20:20:08 -04:00
Limit.hs filter out control characters in error messages 2023-04-10 13:50:51 -04:00
Logs.hs filter-branch working aside from some edge cases 2021-05-17 13:24:58 -04:00
Makefile clean up cabal.project.local~* 2023-04-11 13:18:31 -04:00
Messages.hs factor out maybeAddJSONField 2023-05-08 16:15:41 -04:00
NEWS Fix ambigous typos 2023-03-17 15:14:47 -04:00
README improve description 2022-01-06 12:24:27 -04:00
Remote.hs filter out control characters in all other Messages 2023-04-11 12:58:01 -04:00
Setup.hs Added git-remote-tor-annex, which allows git pull and push to the tor hidden service. 2016-11-21 17:27:38 -04:00
stack.yaml Revert "stack.yaml: Update to lts-19.33 and aws-0.24" 2023-02-20 15:15:06 -04:00
Test.hs improve test descriptions 2023-05-30 16:11:29 -04:00
Types.hs plumb VerifyConfig into retrieveKeyFile 2021-08-17 12:43:13 -04:00
Upgrade.hs handle upgrading repositories initialized with --version=9 2022-09-26 12:59:51 -04:00

git-annex allows managing large files with git, without storing the file
contents in git. It can sync, backup, and archive your data, offline
and online. Checksums and encryption keep your data safe and secure. Bring
the power and distributed nature of git to bear on your large files with
git-annex.

For documentation, see doc/ or <https://git-annex.branchable.com/>