VURL verifyKeyContent True when there are no recorded equivkeys

Avoid verification error when addurl --verifiable is used with an url
claimed by a special remote other than the web.

Backend/VURL.hs

@@ -31,8 +31,12 @@ backendVURL = Backend
 			-- Normally there will always be an key
 			-- recorded when a VURL's content is available,
 			-- because downloading the content from the web in
-			-- the first place records one.
+			-- the first place records one. But, when the
-			[] -> return False
+			-- content is downloaded from some other special
+			-- remote that claims an url, that might not be the
+			-- case. So, default to True when no key is
+			-- recorded.
+			[] -> return True
 			eks -> do
 				let check ek = getbackend ek >>= \case
 					Nothing -> pure False

CHANGELOG

@@ -3,6 +3,8 @@ git-annex (10.20250103) UNRELEASED; urgency=medium
   * Improve handing of ssh connection problems during 
     remote annex.uuid discovery.
   * log: Support --key, as well as --branch and --unused.
+  * Avoid verification error when addurl --verifiable is used
+    with an url claimed by a special remote other than the web.
 
  -- Joey Hess <id@joeyh.name>  Fri, 03 Jan 2025 14:30:38 -0400
 

doc/bugs/VURL_verification_failure_on_first_download.mdwn

@@ -1,3 +1,5 @@
+[[!meta title="VURL equivilant keys not populated when addurl is used with special remotes other than web"]]
+
 ### Please describe the problem.
 
 With an external special remote that handles a custom URL scheme, I receive a "Verification of content failed" on the first `git annex get` of a file (i.e. when git-annex cannot know a checksum for the file, yet).

doc/bugs/VURL_verification_failure_on_first_download/comment_1_fe5525d4c03fdafc28dc23ed5ee3fab8._comment

@@ -0,0 +1,49 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 1"""
+ date="2025-01-03T19:52:28Z"
+ content="""
+Looking at [[!commit 55bf01b78888b410a8ac07e834ed7104ffa1f4d0]] it talks
+about something like this:
+
+	When downloading a VURL from the web, make sure that the equivilant key
+	log is populated.
+
+	...
+
+	Downloads of VURL keys from other special remotes that claim urls,
+	like bittorrent for example, does not popilate the equivilant key log.
+	So for now, no checksum verification will be done for those.
+
+I think at the time, I just punted on the question of how to register the
+equivilant key when downloading a VURL from other special remotes.
+
+Thing is, git-annex just calls retrieveKeyFile at some point. At that
+point, it doesn't know if it's retrieving the key from the web or some
+other special remote that claims urls.
+
+It would not do for a `git-annex get` that happens to get the VURL key from
+eg, a directory special remote to say hey, we've not populated the
+equivilant key log for this VURL yet, let's trust what we downloaded from
+the remote is good and populate it now.
+
+So that's why the code went into the web special remote. It seems to me that to
+move it out of the web special remote, there would need to be a way for
+git-annex to check if a given special remote claims the url associated with a
+VURL. Eg, look up the key's url(s) and check claimUrl.
+
+Hmm, if it did that, there would still be a problem that retrieveKeyFile can
+try to verify the content it downloads, using verifyKeyContent. But when no
+equivilant key is yet registered for the VURL, retrieveKeyFile would find that
+anything it downloaded failed to verify. So it would fail. Chicken and egg problem.
+
+But well, it would be easy enough to make this not be treated as a failure.
+In Backend.VURL, verifyKeyContent returns False when there are no equivilant
+keys. It could just return true.
+
+That seems like a necessary first step. Just making that change will solve your
+problem to the extent that it's ok for no verification of the content of VURLs
+to be done when they originate with your special remote.
+
+So I've done that. Whether it makes sense to do the rest I am not yet sure..
+"""]]

doc/git-annex-addurl.mdwn

@@ -50,12 +50,14 @@ be used to get better filenames.
   verified with a checksum when it is transferred between git-annex
   repositories, as well as by things like `git-annex fsck`.
 
-  When used with --relaxed, content from the web will always be accepted,
+  When used with --relaxed, content from the web special remote will
-  even if it has changed, and the checksum recorded for later verification.
+  always be accepted, even if it has changed, and the checksum recorded
+  for later verification.
 
   When used with --fast, the checksum is recorded the first time the
-  content is downloaded from the web. Once a checksum has been recorded,
+  content is downloaded from the web special remote. Once a checksum has
-  subsequent downloads from the web must have the same checksum.
+  been recorded, subsequent downloads from the web special remote 
+  must have the same checksum.
 
   When addurl was used without this option before, the file it added 
   can be converted to be verifiable by migrating it to the VURL backend.

doc/todo/migration_to_VURL_by_default/comment_2_83b03035f1cae24180038f412e1886f2._comment

@@ -0,0 +1,10 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 2"""
+ date="2025-01-03T21:17:31Z"
+ content="""
+I think that is no longer a blocker for this becoming default, it no longer
+causes a verification failure, instead the content originating with such a
+special remote for a VURL is effectively treated the same as for an URL,
+and just doesn't get verified in transit later on.
+"""]]