77de20c925
Tagging todos that seem to have a plan ready as confirmed. Also closed some old ones for various reasons. Including several that turn out to be addressed by newer features. Also opened a new todo about git-annex-config needing a criteria to add new configs to it.
4 lines
984 B
Markdown
4 lines
984 B
Markdown
Would it be hard to add a variantion to checksumming [[backends]], that would change how the checksum is computed: instead of computing it on the whole file, it would first be computed on file chunks of given size, and then the final checksum computed on the concatenation of the chunk checksums? You'd add a new [[key field|internals/key_format]], say cNNNNN, specifying the chunking size (the last chunk might be shorter). Then (1) for large files, checksum computation could be parallelized (there could be a config option specifying the default chunk size for newly added files); (2) I often have large files on a remote, for which I have md5 for each chunk, but not for the full file; this would enable me to register the location of these fies with git-annex without downloading them, while still using a checksum-based key.
|
|
|
|
> Closing, because [[external_backends]] is implemented, so you should be
|
|
> able to roll your own backend for your use case here. [[done]] --[[Joey]]
|