From 3bb8b6269953053578f19d4a8aecbff8ce84da00 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Wed, 26 Jun 2019 11:23:41 -0400 Subject: [PATCH] comments --- ..._9d73d6dbfabe3d58a41b661825bf3dcb._comment | 61 +++++++++++++++++++ ..._352bc3eb419184a4ec70ddaffd78968e._comment | 9 +++ 2 files changed, 70 insertions(+) create mode 100644 doc/todo/external_backends/comment_1_9d73d6dbfabe3d58a41b661825bf3dcb._comment create mode 100644 doc/todo/key_checksum_from_chunk_checksums/comment_3_352bc3eb419184a4ec70ddaffd78968e._comment diff --git a/doc/todo/external_backends/comment_1_9d73d6dbfabe3d58a41b661825bf3dcb._comment b/doc/todo/external_backends/comment_1_9d73d6dbfabe3d58a41b661825bf3dcb._comment new file mode 100644 index 0000000000..f59787ea0a --- /dev/null +++ b/doc/todo/external_backends/comment_1_9d73d6dbfabe3d58a41b661825bf3dcb._comment @@ -0,0 +1,61 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 1""" + date="2019-06-26T14:54:01Z" + content=""" +The main operations on keys are: + +* generate +* verify +* is stable (always refers to the same data, used only to determine if + chunks can be used when storing the content on a special remote) +* is verifiable with a hash (for annex.security.allow-unverified-downloads) +* is cryptographically secure hash + +Note that the last two of those are not currently part of the Backend +object but would need to be moved to it to implement external backends. + +Now, if a external backend is not installed or is broken, how would those +operations failing be handled? + +* generate: Not a problem, git-annex already falls back to another backend + if a backend fails to generate a key. +* verify: This would need to return False. Since verification + is done after download in the default configuration, this would make + the downloaded content be thrown away. Seems likely it would result in + repeated downloads of the same content when git-annex is run + repeatedly, or when using the assistant. + fsck and reinject would also fail, so if a repo contained content using + an external backend and the program broken, fsck could move all the + content to .git/annex/bad/ +* is verifiable with a hash: Affects downloads same as verify. +* is cryptographically secure hash: Similar to verify, but also affects + uploads when annex.securehashesonly is set. + +So I think that all these operations except for generate would need to have +their type changed from Bool to Maybe Bool, and all code that uses them +handle the case where the operation was unable to be performed. + +For fsck, it might need to start checking files it had previously moved to +bad, and move them back into the annex if they now verify due to the +external backend having gotten fixed. + +For downloads, if it can't verify, it might move to some holding location, +and avoid re-downloading objects that are already located there. This would +add more problems though; what if all the objects in that holding location +use too much disk space? Should drop also drop them? + +Or the external backend could be tested somehow before starting a download +(or a fsck) and skip its key if it's not installed or broken. But such a test +can't catch every possible breakage. It's hard to see how such a test +can do more than checking that it can start the program and that the +program seems to be speaking the right protocol version. + +I suppose we could say that, if a external backend works to that extent but +then breaks, the resulting bad behavior is its fault and not really the +concern of git-annex. Or I could rationalize, that an external special +remote can break in ways that eg, prevent content being moved to it, so it +piles up in the local repo and uses too much disk space, and that this new +potential breakage is not much worse -- though it seems the scenarios where +it would have an adverse affect are likely to be more common. +"""]] diff --git a/doc/todo/key_checksum_from_chunk_checksums/comment_3_352bc3eb419184a4ec70ddaffd78968e._comment b/doc/todo/key_checksum_from_chunk_checksums/comment_3_352bc3eb419184a4ec70ddaffd78968e._comment new file mode 100644 index 0000000000..53209da86b --- /dev/null +++ b/doc/todo/key_checksum_from_chunk_checksums/comment_3_352bc3eb419184a4ec70ddaffd78968e._comment @@ -0,0 +1,9 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 3""" + date="2019-06-26T14:51:32Z" + content=""" +That seems like an unusual use case that would be unncessary complication +to add to git-annex, but that [[external_backends]] could be used to +implenent as needed. +"""]]