Added WHEREIS to external special remote protocol.

Remote/External.hs

@@ -60,7 +60,7 @@ gen r u c gc = do
 			, removeKey = removeKeyDummy
 			, checkPresent = checkPresentDummy
 			, checkPresentCheap = False
-			, whereisKey = Nothing
+			, whereisKey = Just $ whereis external
 			, remoteFsck = Nothing
 			, repairRepo = Nothing
 			, config = c
@@ -144,6 +144,13 @@ checkKey external k = either error id <$> go
 				| k' == k -> Just $ return $ Left errmsg
 			_ -> Nothing
 
+whereis :: External -> Key -> Annex [String]
+whereis external k = handleRequest external (WHEREIS k) Nothing $ \resp -> case resp of
+	WHEREIS_SUCCESS s -> Just $ return [s]
+	WHEREIS_FAILURE -> Just $ return []
+	UNSUPPORTED_REQUEST -> Just $ return []
+	_ -> Nothing
+
 safely :: Annex Bool -> Annex Bool
 safely a = go =<< tryNonAsync a
   where

Remote/External/Types.hs

@@ -97,6 +97,7 @@ data Request
 	| TRANSFER Direction Key FilePath
 	| CHECKPRESENT Key
 	| REMOVE Key
+	| WHEREIS Key
 	deriving (Show)
 
 -- Does PREPARE need to have been sent before this request?
@@ -120,6 +121,7 @@ instance Proto.Sendable Request where
 		]
 	formatMessage (CHECKPRESENT key) = [ "CHECKPRESENT", Proto.serialize key ]
 	formatMessage (REMOVE key) = [ "REMOVE", Proto.serialize key ]
+	formatMessage (WHEREIS key) = [ "WHEREIS", Proto.serialize key ]
 
 -- Responses the external remote can make to requests.
 data Response
@@ -141,6 +143,8 @@ data Response
 	| CHECKURL_CONTENTS Size FilePath
 	| CHECKURL_MULTI [(URLString, Size, FilePath)]
 	| CHECKURL_FAILURE ErrorMsg
+	| WHEREIS_SUCCESS String
+	| WHEREIS_FAILURE
 	| UNSUPPORTED_REQUEST
 	deriving (Show)
 
@@ -163,6 +167,8 @@ instance Proto.Receivable Response where
 	parseCommand "CHECKURL-CONTENTS" = Proto.parse2 CHECKURL_CONTENTS
 	parseCommand "CHECKURL-MULTI" = Proto.parse1 CHECKURL_MULTI
 	parseCommand "CHECKURL-FAILURE" = Proto.parse1 CHECKURL_FAILURE
+	parseCommand "WHEREIS-SUCCESS" = Just . WHEREIS_SUCCESS
+	parseCommand "WHEREIS-FAILURE" = Proto.parse0 WHEREIS_FAILURE
 	parseCommand "UNSUPPORTED-REQUEST" = Proto.parse0 UNSUPPORTED_REQUEST
 	parseCommand _ = Proto.parseFail
 

debian/changelog

@@ -4,6 +4,7 @@ git-annex (5.20150813) UNRELEASED; urgency=medium
   * Sped up downloads of files from ssh remotes, reducing the
     non-data-transfer overhead 6x.
   * --debug is passed along to git-annex-shell when git-annex is in debug mode.
+  * Added WHEREIS to external special remote protocol.
 
  -- Joey Hess <id@joeyh.name>  Wed, 12 Aug 2015 14:31:01 -0400
 

doc/bugs/External_special_remote_SETURLPRESENT_doesn__39__t_seem_to_work/comment_3_6e856db0eb3bb3e7674649857d53cc11._comment

@@ -0,0 +1,24 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 3"""
+ date="2015-08-13T20:47:28Z"
+ content="""
+Needing enableremote is a trifle annoying, but I don't see a way to avoid
+it and it's symmetric with needing to add a git remote to a repo before
+accessing it. And of course, the user has to install your external special
+remote's implementation too. Which is perhaps more annoying in this use case
+where the file is only being retrieved with a dumb http call in the end.
+
+Instead of the current approach, I could have had special remotes
+use SETURLPRESENT to record the public urls for keys. Then git-annex
+would have something that notices if a special remote is not enabled, but
+supports public urls, and perhaps auto-enables the special remote in
+readonly mode, not using the normal implementation of the special remote,
+but a standin implementation that just uses the public urls. That seems a
+little complicated, especially with the auto-enabling, and it bloats the
+git-annex branch with public urls. IIRC those are the reasons I decided not
+to go that route.
+
+I've added WHEREIS to the protocol. It is only used for `whereis`
+display.
+"""]]

doc/design/external_special_remote_protocol.mdwn

@@ -133,6 +133,15 @@ replying with `UNSUPPORTED-REQUEST` is acceptable.
   Asks the remote to check if the url's content can currently be downloaded
   (without downloading it). The remote replies with one of `CHECKURL-FAILURE`,
   `CHECKURL-CONTENTS`, or `CHECKURL-MULTI`.
+* `WHEREIS Key`
+  Asks the remote to provide any information it can about ways to access
+  the content of a key stored in it, such as eg, public urls.
+  This will be displayed to the user by eg, `git annex whereis`. The remote
+  replies with `WHEREIS-SUCCESS` or `WHEREIS-FAILURE`.  
+  Note that users expect `git annex whereis` to run fast, without eg,
+  network access.  
+  This is not needed when `SETURIPRESENT` is used, since such uris are
+  automatically displayed by `git annex whereis`.  
 
 More optional requests may be added, without changing the protocol version,
 so if an unknown request is seen, reply with `UNSUPPORTED-REQUEST`.
@@ -193,6 +202,12 @@ while it's handling a request.
   can contain spaces.
 * `CHECKURL-FAILURE`  
   Indicates that the requested url could not be accessed.
+* `WHEREIS-SUCCESS String`  
+  Indicates a location of a key. Typically an url, the string can
+  be anything that it makes sense to display to the user about content
+  stored in the special remote.
+* `WHEREIS-FAILURE`  
+  Indicates that no location is known for a key.
 * `UNSUPPORTED-REQUEST`  
   Indicates that the special remote does not know how to handle a request.
 
@@ -281,11 +296,9 @@ in control.
   Records that the key can no longer be downloaded from the specified
   URL.
 * `SETURIPRESENT Key Uri`  
-  Records a special URI where the Key can be downloaded from.  
+  Records an URI where the Key can be downloaded from.  
   For example, "ipfs:ADDRESS" is used for the ipfs special remote;
-  its CLAIMURL handler checks for such URIS and claims them. Setting
+  its CLAIMURL handler checks for such URIS and claims them.
-  it present as an URI makes `git annex whereis` display the URI
-  as belonging to the special remote.
 * `SETURIMISSING Key Uri`  
   Records that the key can no longer be downloaded from the specified
   URI.