restore old registerurl location tracking behavior

registerurl: When an url is claimed by a special remote other than the web,
update location tracking for that special remote.

registerurl's behavior was changed in commit
451171b7c1, apparently accidentially to not
update location tracking except for the web.

This makes registerurl followed by unregisterurl not be a no-op, when the
url happens to be claimed by a remote other than the web. It is a noop when
the url is unclaimed except by the web. I don't like the inconsistency,
and wish that registerurl and unregisterurl never updated location
tracking, which would be more in keeping with them being plumbing.

But there is the fact that it used to behave this way, and also it was
inconsistent that it updated location tracking for the web but not for
other remotes, unlike addurl. And there's an argument that the user might
not know what remote to expect to claim an url, so would be considerably in
the dark when using registerurl. (Although they have to know what content
gets downloaded, since they specify a key..)

Sponsored-By: the NIH-funded NICEMAN (ReproNim TR&D3) project

CHANGELOG

@@ -7,6 +7,9 @@ git-annex (10.20230330) UNRELEASED; urgency=medium
     of --unlock-present and --hide-missing adjusted branches.
   * Support user.useConfigOnly git config.
   * registerurl, unregisterurl: Added --remote option.
+  * registerurl: When an url is claimed by a special remote other than the
+    web, update location tracking for that special remote.
+    (This was the behavior before version 6.20181011)
 
  -- Joey Hess <id@joeyh.name>  Fri, 31 Mar 2023 12:48:54 -0400
 

Command/RegisterUrl.hs

@@ -1,6 +1,6 @@
 {- git-annex command
  -
- - Copyright 2015-2022 Joey Hess <id@joeyh.name>
+ - Copyright 2015-2023 Joey Hess <id@joeyh.name>
  -
  - Licensed under the GNU AGPL version 3 or higher.
  -}
@@ -9,6 +9,7 @@ module Command.RegisterUrl where
 
 import Command
 import Logs.Web
+import Logs.Location
 import Command.FromKey (keyOpt, keyOpt')
 import qualified Remote
 import Annex.UUID
@@ -33,12 +34,12 @@ optParser desc = RegisterUrlOptions
 
 seek :: RegisterUrlOptions -> CommandSeek
 seek o = case (batchOption o, keyUrlPairs o) of
-	(Batch fmt, _) -> seekBatch setUrlPresent o fmt
+	(Batch fmt, _) -> seekBatch registerUrl o fmt
 	-- older way of enabling batch input, does not support BatchNull
-	(NoBatch, []) -> seekBatch setUrlPresent o (BatchFormat BatchLine (BatchKeys False))
-	(NoBatch, ps) -> commandAction (start setUrlPresent o ps)
+	(NoBatch, []) -> seekBatch registerUrl o (BatchFormat BatchLine (BatchKeys False))
+	(NoBatch, ps) -> commandAction (start registerUrl o ps)
 
-seekBatch :: (Key -> URLString -> Annex ()) -> RegisterUrlOptions -> BatchFormat -> CommandSeek
+seekBatch :: (Remote -> Key -> URLString -> Annex ()) -> RegisterUrlOptions -> BatchFormat -> CommandSeek
 seekBatch a o fmt = batchOnly Nothing (keyUrlPairs o) $
 	batchInput fmt (pure . parsebatch) $
 		batchCommandAction . start' a o
@@ -51,20 +52,20 @@ seekBatch a o fmt = batchOnly Nothing (keyUrlPairs o) $
 				Left e -> Left e
 				Right k -> Right (k, u)
 
-start :: (Key -> URLString -> Annex ()) -> RegisterUrlOptions -> [String] -> CommandStart
+start :: (Remote -> Key -> URLString -> Annex ()) -> RegisterUrlOptions -> [String] -> CommandStart
 start a o (keyname:url:[]) = start' a o (si, (keyOpt keyname, url))
   where
 	si = SeekInput [keyname, url]
 start _ _ _ = giveup "specify a key and an url"
 
-start' :: (Key -> URLString -> Annex ()) -> RegisterUrlOptions -> (SeekInput, (Key, URLString)) -> CommandStart
+start' :: (Remote -> Key -> URLString -> Annex ()) -> RegisterUrlOptions -> (SeekInput, (Key, URLString)) -> CommandStart
 start' a o (si, (key, url)) =
 	starting "registerurl" ai si $
 		perform a o key url
   where
 	ai = ActionItemOther (Just url)
 
-perform :: (Key -> URLString -> Annex ()) -> RegisterUrlOptions -> Key -> URLString -> CommandPerform
+perform :: (Remote -> Key -> URLString -> Annex ()) -> RegisterUrlOptions -> Key -> URLString -> CommandPerform
 perform a o key url = do
 	needremote <- maybe (pure Nothing) (Just <$$> getParsed) (remoteOption o)
 	r <- case needremote of
@@ -75,5 +76,15 @@ perform a o key url = do
 			showNote $ "The url " ++ url ++ " is claimed by remote " ++ Remote.name r
 			next $ return False
 		_ -> do
-			a key (setDownloader' url r)
+			a r key (setDownloader' url r)
 			next $ return True
+
+registerUrl :: Remote -> Key -> String -> Annex ()
+registerUrl remote key url = do
+	setUrlPresent key url
+	-- setUrlPresent only updates location tracking when the url
+	-- does not have an OtherDownloader, but this command needs to do
+	-- it for urls claimed by other remotes as well.
+	case snd (getDownloader url) of
+		OtherDownloader -> logChange key (Remote.uuid remote) InfoPresent
+		_ -> return ()

Command/RmUrl.hs

@@ -62,3 +62,7 @@ cleanup url key = do
 	forM_ [minBound..maxBound] $ \dl -> 
 		setUrlMissing key (setDownloader url dl)
 	return True
+	-- Unlike addurl, this does not update location tracking
+	-- for remotes other than the web special remote. Doing so with
+	-- a remote that git-annex can drop content from would rather
+	-- unexpectedly leave content stranded on that remote.

Command/UnregisterUrl.hs

@@ -24,11 +24,15 @@ seek o = case (batchOption o, keyUrlPairs o) of
 	(Batch fmt, _) -> seekBatch unregisterUrl o fmt
 	(NoBatch, ps) -> commandAction (start unregisterUrl o ps)
 
-unregisterUrl :: Key -> String -> Annex ()
-unregisterUrl key url = do
+unregisterUrl :: Remote -> Key -> String -> Annex ()
+unregisterUrl _remote key url = do
 	-- Remove the url no matter what downloader;
 	-- registerurl can set OtherDownloader, and this should also
 	-- be able to remove urls added by addurl, which may use
 	-- YoutubeDownloader.
 	forM_ [minBound..maxBound] $ \dl ->
 		setUrlMissing key (setDownloader url dl)
+	-- Unlike unregisterurl, this does not update location tracking
+	-- for remotes other than the web special remote. Doing so with
+	-- a remote that git-annex can drop content from would rather
+	-- unexpectedly leave content stranded on that remote.

doc/bugs/registerurl_does_not_register_if_external_remote.mdwn

@@ -102,3 +102,5 @@ so - both keys have urls, but only 123.dat one is associated with datalad specia
 
 [[!meta author=yoh]]
 [[!tag projects/repronim]]
+
+> [[fixed|done]] --[[Joey]]

doc/bugs/registerurl_does_not_register_if_external_remote/comment_8_542bd923122ba24fe9cb14d36b97f15d._comment

@@ -0,0 +1,14 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 8"""
+ date="2023-04-05T21:00:04Z"
+ content="""
+Guess I'll come down on the side of restoring old behavior which was
+changed w/o warning (and without the new behavior ever being documented).
+
+And on the side of user experience showing the current behavior is surprising.
+
+The future users who get surprised by the resulting inconsistency 
+of unregisterurl not unsetting location tracking will just have to
+live with it.. Sigh.
+"""]]

doc/git-annex-registerurl.mdwn

@@ -17,8 +17,8 @@ Normally the key is a git-annex formatted key. However, to make it easier
 to use this to add urls, if the key cannot be parsed as a key, and is a
 valid url, an URL key is constructed from the url.
 
-Registering an url makes content be treated as being present in the web
-special remote, unless some other special remote claims the url.
+Registering an url also makes git-annex treat the key as present in the
+special remote that claims it. (Usually the web special remote.)
 
 # OPTIONS
 

doc/git-annex-rmurl.mdwn

@@ -11,7 +11,9 @@ git annex rmurl `[file url ..]`
 Record that the file is no longer available at the url.
 
 Removing the last web url will make git-annex no longer treat content as being
-present in the web special remote.
+present in the web special remote. If some other special remote
+claims the url, unregistering the url will not update presence information
+for it, because the content may still be present on the remote.
 
 # OPTIONS
 

doc/git-annex-unregisterurl.mdwn

@@ -11,12 +11,14 @@ git annex unregisterurl `[key url]`
 This plumbing-level command can be used to unregister urls when keys can
 no longer be downloaded from them.
 
-Unregistering a key's last web url will make git-annex no longer treat content
-as being present in the web special remote.
-
 Normally the key is a git-annex formatted key. However, if the key cannot be
 parsed as a key, and is a valid url, an URL key is constructed from the url.
 
+Unregistering a key's last web url will make git-annex no longer treat content
+as being present in the web special remote. If some other special remote
+claims the url, unregistering the url will not update presence information
+for it, because the content may still be present on the remote.
+
 # OPTIONS
 
 * `--remote=name|uuid`