CI to automate build of cargo lockfiles on different Alpine releases for git-annex aports
https://gitlab.alpinelinux.org/alpine/aports/-/tree/master/community/git-annex
c2e46f4707
So that eg, addurl of several large files that take time to download will update the index for each file, rather than deferring the index updates to the end. In cases like an add of many smallish files, where a new file is being added every few seconds. In that case, the queue will still build up a lot of changes which are flushed at once, for best performance. Since the default queue size is 10240, often it only gets flushed once at the end, same as before. (Notice that updateQueue updated _lastchanged when adding a new item to the queue without flushing it; that is necessary to avoid it flushing the queue every 5 minutes in this case.) But, when it takes more than a 5 minutes to add a file, the overhead of updating the index immediately is probably small, so do it after each file. This avoids git-annex potentially taking a very very long time indeed to stage newly added files, which can be annoying to the user who would like to get on with doing something with the files it's already added, eg using git mv to rename them to a better name. This is only likely to cause a problem if it takes say, 30 seconds to update the index; doing an extra 30 seconds of work after every 5 minute file add would be less optimal. Normally, updating the index takes significantly less time than that. On a SSD with 100k files it takes less than 1 second, and the index write time is bound by disk read and write so is not too much worse on a hard drive. So I hope this will not impact users, although if it does turn out to, the time limit could be made configurable. A perhaps better way to do it would be to have a background worker thread that wakes up every 60 seconds or so and flushes the queue. That is made somewhat difficult because the queue can contain Annex actions and so this would add a new source of concurrency issues. So I'm trying to avoid that approach if possible. Sponsored-by: Erik Bjäreholt on Patreon |
||
|---|---|---|
| Annex | ||
| Assistant | ||
| Backend | ||
| Build | ||
| CmdLine | ||
| Command | ||
| Config | ||
| Database | ||
| debian | ||
| doc | ||
| Git | ||
| Limit | ||
| Logs | ||
| Messages | ||
| P2P | ||
| Remote | ||
| RemoteDaemon | ||
| standalone | ||
| static | ||
| templates | ||
| Test | ||
| Types | ||
| Upgrade | ||
| Utility | ||
| .ghci | ||
| .gitattributes | ||
| .gitignore | ||
| .mailmap | ||
| Annex.hs | ||
| Assistant.hs | ||
| Backend.hs | ||
| bash-completion.bash | ||
| Benchmark.hs | ||
| BuildFlags.hs | ||
| BuildInfo.hs | ||
| CHANGELOG | ||
| CmdLine.hs | ||
| Command.hs | ||
| Common.hs | ||
| Config.hs | ||
| COPYRIGHT | ||
| Creds.hs | ||
| Crypto.hs | ||
| git-annex.cabal | ||
| git-annex.hs | ||
| git-union-merge.hs | ||
| Git.hs | ||
| Key.hs | ||
| Limit.hs | ||
| Logs.hs | ||
| Makefile | ||
| Messages.hs | ||
| NEWS | ||
| README | ||
| Remote.hs | ||
| Setup.hs | ||
| stack.yaml | ||
| Test.hs | ||
| Types.hs | ||
| Upgrade.hs | ||
git-annex allows managing files with git, without checking the file contents into git. While that may seem paradoxical, it is useful when dealing with files larger than git can currently easily handle, whether due to limitations in memory, checksumming time, or disk space. For documentation, see doc/ or <https://git-annex.branchable.com/>