fsck: avoid redundant checksum when transfer is Verified
When downloading content from a remote, if the content is able to be verified during the transfer, skip checksumming it a second time. Note that in this case, the fsck output does not include "(checksum)" which it does when the checksumming is done separately from the download. This commit was sponsored by Brock Spratlen on Patreon.
This commit is contained in:
parent
5ee14db037
commit
5783a8d081
3 changed files with 41 additions and 8 deletions
|
@ -0,0 +1,28 @@
|
|||
[[!comment format=mdwn
|
||||
username="joey"
|
||||
subject="""comment 1"""
|
||||
date="2021-04-14T17:07:50Z"
|
||||
content="""
|
||||
Only some remotes support checksums in-flight; this recently includes
|
||||
downloads from other git-annex repositories over ssh. Progress
|
||||
on that front is being tracked at
|
||||
<https://git-annex.branchable.com/todo/OPT__58_____34__bundle__34___get_+_check___40__of_checksum__41___in_a_single_operation/>
|
||||
Most special remotes can't yet, but that should change eventually
|
||||
for at least some of them.
|
||||
|
||||
I've made fsck notice when content was able to be verified as part of a
|
||||
transfer, and avoid a redundant checksum of them.
|
||||
|
||||
What I've not done, and don't think I will be able to, is make the file
|
||||
not be written to disk by fsck in that case. Since the `retrieveKeyFile`
|
||||
interface is explicitly about writing to a file on disk, it would take ether
|
||||
a whole separate interface being implemented for all remotes that avoids
|
||||
writing to the file when they can checksum in flight, or it would need
|
||||
some change to the `retrieveKeyFile` interface to do the same.
|
||||
|
||||
Neither seems worth the complication to implement just to reduce disk IO in
|
||||
this particular case. And it seems likely that, for files that fit in
|
||||
memory, it never actually reaches disk before it's deleted. Also if this is
|
||||
a concern for you, you can I guess avoid fscking remotes too frequently or
|
||||
use a less fragile medium?
|
||||
"""]]
|
Loading…
Add table
Add a link
Reference in a new issue