git-remote-annex: support importrree=yes remotes

When exporttree=yes is also set. Probably it would also be possible to
support ones with only importtree=yes, by enabling exporttree=yes for
the remote only when using git-remote-annex, but let's keep this
simple... I'm not sure what gets recorded in .git/annex/ state
differently in the two cases that might cause a problem when doing that.

Note that the full annex:: urls generated and displayed for such a
remote omit the importree=yes. Which is ok, cloning from such an url
uses an exporttree=remote, but the git-annex branch doesn't get written
by this program, so once the real config is available from the git-annex
branch, it will still function as an importree=yes remote.

CmdLine/GitRemoteAnnex.hs

@@ -22,7 +22,6 @@ import qualified Git.Remote
 import qualified Git.Remote.Remove
 import qualified Git.Version
 import qualified Annex.SpecialRemote as SpecialRemote
-import qualified Annex.SpecialRemote.Config as SpecialRemote
 import qualified Annex.Branch
 import qualified Annex.BranchState
 import qualified Types.Remote as Remote
@@ -576,7 +575,7 @@ getEnabledSpecialRemoteByName remotename =
 			| unparsedRemoteConfig (Remote.config rmt) == mempty ->
 				return Nothing
 			| otherwise -> 
-				maybe (return (Just rmt)) giveup
+				maybe (Just <$> importTreeWorkAround rmt) giveup
 					(checkSpecialRemoteProblems rmt)
 
 checkSpecialRemoteProblems :: Remote -> Maybe String
@@ -586,9 +585,6 @@ checkSpecialRemoteProblems rmt
 	| Remote.thirdPartyPopulated (Remote.remotetype rmt) =
 		Just $ "Cannot use this thirdparty-populated special"
 			++ " remote as a git remote."
-	| importTree (Remote.config rmt) = 
-		Just $ "Using importtree=yes special remotes as git remotes"
-			++ " is not yet supported."
 	| parseEncryptionMethod (unparsedRemoteConfig (Remote.config rmt)) /= Right NoneEncryption
 		&& not (remoteAnnexAllowEncryptedGitRepo (Remote.gitconfig rmt)) =
 			Just $ "Using an encrypted special remote as a git"
@@ -600,6 +596,27 @@ checkSpecialRemoteProblems rmt
   where
 	ConfigKey allowencryptedgitrepo = remoteAnnexConfig rmt "allow-encrypted-gitrepo"
 
+-- Using importTree remotes needs the content identifier database to be
+-- populated, but it is not when cloning, and cannot be updated when
+-- pushing since git-annex branch updates by this program are prevented.
+--
+-- So, generate instead a version of the remote that uses exportTree actions,
+-- which do not need content identifiers. Since Remote.Helper.exportImport
+-- replaces the exportActions in exportActionsForImport with ones that use
+-- import actions, have to instantiate a new remote with a modified config.
+importTreeWorkAround :: Remote -> Annex Remote
+importTreeWorkAround rmt
+	| not (importTree (Remote.config rmt)) = pure rmt
+	| not (exportTree (Remote.config rmt)) = giveup "Using special remotes with importtree=yes but without exporttree=yes as git remotes is not supported."
+	| otherwise = do
+		m <- Logs.Remote.remoteConfigMap
+		r <- Remote.getRepo rmt
+		remoteGen' adjustconfig m (Remote.remotetype rmt) r >>= \case
+			Just rmt' -> return rmt'
+			Nothing -> giveup "Failed to use importtree=yes remote."
+  where
+	adjustconfig = M.delete importTreeField
+
 -- Downloads the Manifest when present in the remote. When not present,
 -- returns an empty Manifest.
 downloadManifestWhenPresent :: Remote -> Annex Manifest

Remote/List.hs

@@ -87,12 +87,20 @@ remoteList' autoinit = do
 
 {- Generates a Remote. -}
 remoteGen :: M.Map UUID RemoteConfig -> RemoteType -> Git.Repo -> Annex (Maybe Remote)
-remoteGen m t g = do
+remoteGen = remoteGen' id
+
+remoteGen'
+	:: (RemoteConfig -> RemoteConfig)
+	-> M.Map UUID RemoteConfig
+	-> RemoteType
+	-> Git.Repo
+	-> Annex (Maybe Remote)
+remoteGen' adjustconfig m t g = do
 	u <- getRepoUUID g
 	gc <- Annex.getRemoteGitConfig g
 	let cu = fromMaybe u $ remoteAnnexConfigUUID gc
 	let rs = RemoteStateHandle cu
-	let c = fromMaybe M.empty $ M.lookup cu m
+	let c = adjustconfig (fromMaybe M.empty $ M.lookup cu m)
 	generate t g u c gc rs >>= \case
 		Nothing -> return Nothing
 		Just r -> Just <$> adjustExportImport (adjustReadOnly (addHooks r)) rs

doc/todo/git-remote-annex.mdwn

@@ -15,10 +15,6 @@ This is implememented and working. Remaining todo list for it:
   not display any useful error message. git fetch does, but it seems
   git clone eats git-remote-annex stderr.
 
-* Cloning from an annex:: url with importtree=yes doesn't work
-  (with or without exporttree=yes). This is because the ContentIdentifier
-  db is not populated. It should be possible to work around this.
-
 * datalad-annex supports cloning from the web special remote,
   using an url that contains the result of pushing to eg, a directory
   special remote.