CI to automate build of cargo lockfiles on different Alpine releases for git-annex aports https://gitlab.alpinelinux.org/alpine/aports/-/tree/master/community/git-annex
Find a file
Joey Hess c20358b671
incremental verify for byteRetriever special remotes
Several special remotes verify content while it is being retrieved,
avoiding a separate checksum pass. They are: S3, bup, ddar, and
gcrypt (with a local repository).

Not done when using chunking, yet.

Complicated by Retriever needing to change to be polymorphic. Which in turn
meant RankNTypes is needed, and also needed some code changes. The
change in Remote.External does not change behavior at all but avoids
the type checking failing because of a "rigid, skolem type" which
"would escape its scope". So I refactored slightly to make the type
checker's job easier there.

Unfortunately, directory uses fileRetriever (except when chunked),
so it is not amoung the improved ones. Fixing that would need a way for
FileRetriever to return a Verification. But, since the file retrieved
may be encrypted or chunked, it would be extra work to always
incrementally checksum the file while retrieving it. Hm.

Some other special remotes use fileRetriever, and so don't get incremental
verification, but could be converted to byteRetriever later. One is
GitLFS, which uses downloadConduit, which writes to the file, so could
verify as it goes. Other special remotes like web could too, but don't
use Remote.Helper.Special and so will need to be addressed separately.

Sponsored-by: Dartmouth College's DANDI project
2021-08-11 14:20:38 -04:00
Annex simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
Assistant simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
Backend simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
Build don't use changelog version in commit message 2021-08-09 12:31:48 -04:00
CmdLine simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
Command simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
Config
Database simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
debian
doc Added a comment: sorry 2021-08-10 16:28:33 +00:00
Git simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
Limit
Logs simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
Messages
P2P simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
Remote incremental verify for byteRetriever special remotes 2021-08-11 14:20:38 -04:00
RemoteDaemon
standalone
static
templates
Test simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
Types incremental verify for byteRetriever special remotes 2021-08-11 14:20:38 -04:00
Upgrade simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
Utility simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
.ghci
.gitattributes
.gitignore
.mailmap
Annex.hs simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
Assistant.hs
Backend.hs move verifyKeyContent to Annex.Verify 2021-07-27 14:07:23 -04:00
bash-completion.bash
Benchmark.hs
BuildFlags.hs
BuildInfo.hs
CHANGELOG incremental verify for byteRetriever special remotes 2021-08-11 14:20:38 -04:00
CmdLine.hs
Command.hs
Common.hs
Config.hs simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
COPYRIGHT
Creds.hs
Crypto.hs
git-annex.cabal releasing package git-annex version 8.20210803 2021-08-03 12:20:45 -04:00
git-annex.hs
git-union-merge.hs
Git.hs
Key.hs simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
Limit.hs
Logs.hs
Makefile
Messages.hs simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
NEWS
README
Remote.hs simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
Setup.hs
stack.yaml
Test.hs sync, merge: Added --allow-unrelated-histories option 2021-07-19 12:14:26 -04:00
Types.hs
Upgrade.hs

git-annex allows managing files with git, without checking the file
contents into git. While that may seem paradoxical, it is useful when
dealing with files larger than git can currently easily handle, whether due
to limitations in memory, checksumming time, or disk space.

For documentation, see doc/ or <https://git-annex.branchable.com/>