git-annex/Remote
Joey Hess dadbb510f6
incremental hashing for fileRetriever
It uses tailVerify to hash the file while it's being written.

This is able to sometimes avoid a separate checksum step. Although
if the file gets written quickly enough, tailVerify may not see it
get created before the write finishes, and the checksum still happens.

Testing with the directory special remote, incremental checksumming did
not happen. But then I disabled the copy CoW probing, and it did work.
What's going on with that is the CoW probe creates an empty file on
failure, then deletes it, and then the file is created again. tailVerify
will open the first, empty file, and so fails to read the content that
gets written to the file that replaces it.

The directory special remote really ought to be able to avoid needing to
use tailVerify, and while other special remotes could do things that
cause similar problems, they probably don't. And if they do, it just
means the checksum doesn't get done incrementally.

Sponsored-by: Dartmouth College's DANDI project
2021-08-13 15:43:29 -04:00
..
Directory incremental hashing for fileRetriever 2021-08-13 15:43:29 -04:00
External Added GETGITREMOTENAME to extenal special remote protocol 2021-01-26 12:42:47 -04:00
Helper incremental hashing for fileRetriever 2021-08-13 15:43:29 -04:00
List auto-init autoenable=yes 2020-05-27 12:40:35 -04:00
Rsync add newtypes for QuickCheck to avoid LANG=C issues 2020-11-09 20:21:18 -04:00
WebDAV fix exporting when the file is in the top of the repo 2021-03-16 14:17:29 -04:00
Adb.hs incremental verify for byteRetriever special remotes 2021-08-11 14:20:38 -04:00
BitTorrent.hs Tahoe: Avoid verifying hash after download, since tahoe does sufficient verification itself 2021-02-09 13:42:16 -04:00
Borg.hs simplify and speed up Utility.FileSystemEncoding 2021-08-11 12:13:31 -04:00
Bup.hs incremental verify for byteRetriever special remotes 2021-08-11 14:20:38 -04:00
Ddar.hs incremental verify for byteRetriever special remotes 2021-08-11 14:20:38 -04:00
Directory.hs incremental verify for byteRetriever special remotes 2021-08-11 14:20:38 -04:00
External.hs incremental verify for byteRetriever special remotes 2021-08-11 14:20:38 -04:00
GCrypt.hs incremental hashing for fileRetriever 2021-08-13 15:43:29 -04:00
Git.hs propagate debugging into remote's Annex monad 2021-07-26 11:40:51 -04:00
GitLFS.hs incremental verify for byteRetriever special remotes 2021-08-11 14:20:38 -04:00
Glacier.hs incremental verify for byteRetriever special remotes 2021-08-11 14:20:38 -04:00
Hook.hs incremental verify for byteRetriever special remotes 2021-08-11 14:20:38 -04:00
HttpAlso.hs remove "checking remotename" message 2021-04-27 13:05:27 -04:00
List.hs start borg special remote 2020-12-18 16:03:51 -04:00
P2P.hs incremental checksum on download from ssh or p2p 2021-02-09 17:03:27 -04:00
Rsync.hs simplify 2021-08-09 15:17:48 -04:00
S3.hs incremental verify for byteRetriever special remotes 2021-08-11 14:20:38 -04:00
Tahoe.hs Tahoe: Avoid verifying hash after download, since tahoe does sufficient verification itself 2021-02-09 13:42:16 -04:00
Web.hs remove "checking remotename" message 2021-04-27 13:05:27 -04:00
WebDAV.hs incremental verify for byteRetriever special remotes 2021-08-11 14:20:38 -04:00