fsck: avoid redundant checksum when transfer is Verified

When downloading content from a remote, if the content is able to be
verified during the transfer, skip checksumming it a second time.

Note that in this case, the fsck output does not include "(checksum)"
which it does when the checksumming is done separately from the download.

This commit was sponsored by Brock Spratlen on Patreon.
This commit is contained in:
Joey Hess 2021-04-14 13:22:54 -04:00
parent 5ee14db037
commit 5783a8d081
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
3 changed files with 41 additions and 8 deletions

View file

@ -0,0 +1,28 @@
[[!comment format=mdwn
username="joey"
subject="""comment 1"""
date="2021-04-14T17:07:50Z"
content="""
Only some remotes support checksums in-flight; this recently includes
downloads from other git-annex repositories over ssh. Progress
on that front is being tracked at
<https://git-annex.branchable.com/todo/OPT__58_____34__bundle__34___get_+_check___40__of_checksum__41___in_a_single_operation/>
Most special remotes can't yet, but that should change eventually
for at least some of them.
I've made fsck notice when content was able to be verified as part of a
transfer, and avoid a redundant checksum of them.
What I've not done, and don't think I will be able to, is make the file
not be written to disk by fsck in that case. Since the `retrieveKeyFile`
interface is explicitly about writing to a file on disk, it would take ether
a whole separate interface being implemented for all remotes that avoids
writing to the file when they can checksum in flight, or it would need
some change to the `retrieveKeyFile` interface to do the same.
Neither seems worth the complication to implement just to reduce disk IO in
this particular case. And it seems likely that, for files that fit in
memory, it never actually reaches disk before it's deleted. Also if this is
a concern for you, you can I guess avoid fscking remotes too frequently or
use a less fragile medium?
"""]]