CI to automate build of cargo lockfiles on different Alpine releases for git-annex aports
https://gitlab.alpinelinux.org/alpine/aports/-/tree/master/community/git-annex
68a49adcda
After a false start, I found a fairly non-intrusive way to deal with it. Although it only handles transfers -- there may be issues with eg concurrent dropping of the same key, or other operations. There is no added overhead when -J is not used, other than an added inAnnex check. When -J is used, it has to maintain and check a small Set, which should be negligible overhead. It could output some message saying that the transfer is being done by another thread. Or it could even display the same progress info for both files that are being downloaded since they have the same content. But I opted to keep it simple, since this is rather an edge case, so it just doesn't say anything about the transfer of the file until the other thread finishes. Since the deferred transfer action still runs, actions that do more than transfer content will still get a chance to do their other work. (An example of something that needs to do such other work is P2P.Annex, where the download always needs to receive the content from the peer.) And, if the first thread fails to complete a transfer, the second thread can resume it. But, this unfortunately means that there's a risk of redundant work being done to transfer a key that just got transferred. That's not ideal, but should never cause breakage; the same thing can occur when running two separate git-annex processes. The get/move/copy/mirror --from commands had extra inAnnex checks added, inside the download actions. Without those checks, the first thread downloaded the content, and then the second thread woke up and downloaded the same content redundantly. move/copy/mirror --to is left doing redundant uploads for now. It would need a second checkPresent of the remote inside the upload to avoid them, which would be expensive. A better way to avoid redundant work needs to be found.. This commit was supported by the NSF-funded DataLad project. |
||
|---|---|---|
| Annex | ||
| Assistant | ||
| Backend | ||
| Build | ||
| CmdLine | ||
| Command | ||
| Config | ||
| Database | ||
| debian | ||
| doc | ||
| Git | ||
| Limit | ||
| Logs | ||
| Messages | ||
| P2P | ||
| Remote | ||
| RemoteDaemon | ||
| standalone | ||
| static | ||
| templates | ||
| Types | ||
| Upgrade | ||
| Utility | ||
| .ghci | ||
| .gitattributes | ||
| .gitignore | ||
| .mailmap | ||
| Annex.hs | ||
| Assistant.hs | ||
| Backend.hs | ||
| bash-completion.bash | ||
| build.bat | ||
| BuildInfo.hs | ||
| CHANGELOG | ||
| CmdLine.hs | ||
| Command.hs | ||
| Common.hs | ||
| Config.hs | ||
| COPYRIGHT | ||
| Creds.hs | ||
| Crypto.hs | ||
| ghci | ||
| git-annex.cabal | ||
| git-annex.hs | ||
| git-union-merge.hs | ||
| Git.hs | ||
| Jenkinsfile | ||
| Key.hs | ||
| Limit.hs | ||
| Logs.hs | ||
| Makefile | ||
| Messages.hs | ||
| NEWS | ||
| README | ||
| Remote.hs | ||
| Setup.hs | ||
| stack.yaml | ||
| Test.hs | ||
| Types.hs | ||
| Upgrade.hs | ||
git-annex allows managing files with git, without checking the file contents into git. While that may seem paradoxical, it is useful when dealing with files larger than git can currently easily handle, whether due to limitations in memory, checksumming time, or disk space. For documentation, see doc/ or <https://git-annex.branchable.com/>