fix problems with upgrade of local remotes

Upgrade other repos than the current one by running git-annex upgrade
inside them, which avoids problems with upgrade code making assumptions
that the cwd will be inside the repo being upgraded.

In particular, this fixes a problem where upgrading a v7 repo to v8 caused
an ugly git error message.

I actually could not find a way to make Upgrade.V7 work properly
without changing directory to the remote. Once I got git ls-files to work,
the git cat-file failed because :path can only be used in the current git
repo.

CHANGELOG

@@ -4,6 +4,11 @@ git-annex (8.20200227) UNRELEASED; urgency=medium
     GETCONFIG to query values like "name". (Introduced in version 7.20200202.7.)
   * Fix bug that caused unlocked annexed dotfiles to be added to git by the
     smudge filter when annex.dotfiles was not set.
+  * Upgrade other repos than the current one by running git-annex upgrade
+    inside them, which avoids problems with upgrade code making assumptions
+    that the cwd will be inside the repo being upgraded. In particular,
+    this fixes a problem where upgrading a v7 repo to v8 caused an ugly
+    git error message.
   * Improve behavior when a directory git-annex is writing to gets
     unmounted. Previously it could in some cases re-create the mount point
     and directory tree, and even write object contents to the wrong disk.

Command/Upgrade.hs

@@ -1,6 +1,6 @@
 {- git-annex command
  -
- - Copyright 2011 Joey Hess <id@joeyh.name>
+ - Copyright 2011-2020 Joey Hess <id@joeyh.name>
  -
  - Licensed under the GNU AGPL version 3 or higher.
  -}
@@ -20,13 +20,28 @@ cmd = dontCheck repoExists $
 	noDaemonRunning $
 	-- ^ avoid upgrading repo out from under daemon
 	command "upgrade" SectionMaintenance "upgrade repository"
-		paramNothing (withParams seek)
+		paramNothing (seek <$$> optParser)
 
-seek :: CmdParams -> CommandSeek
+data UpgradeOptions = UpgradeOptions
-seek = withNothing (commandAction start)
+	{ autoOnly :: Bool
+	}
 
-start :: CommandStart
+optParser :: CmdParamsDesc -> Parser UpgradeOptions
-start = starting "upgrade" (ActionItemOther Nothing) $ do
+optParser _ = UpgradeOptions
+	<$> switch
+		( long "autoonly"
+		<> help "only do automatic upgrades"
+		)
+
+seek :: UpgradeOptions -> CommandSeek
+seek o = commandAction (start o)
+
+start :: UpgradeOptions -> CommandStart
+start (UpgradeOptions { autoOnly = True }) = do
+	starting "upgrade" (ActionItemOther Nothing) $ do
+	getVersion >>= maybe noop checkUpgrade
+	next $ return True
+start _ = starting "upgrade" (ActionItemOther Nothing) $ do
 	whenM (isNothing <$> getVersion) $ do
 		initialize Nothing Nothing
 	r <- upgrade False latestVersion

Upgrade.hs

@@ -1,6 +1,6 @@
 {- git-annex upgrade support
  -
- - Copyright 2010-2019 Joey Hess <id@joeyh.name>
+ - Copyright 2010-2020 Joey Hess <id@joeyh.name>
  -
  - Licensed under the GNU AGPL version 3 or higher.
  -}
@@ -12,6 +12,8 @@ module Upgrade where
 import Annex.Common
 import qualified Annex
 import qualified Git
+import Config
+import Config.Files
 import Annex.Version
 import Types.RepoVersion
 #ifndef mingw32_HOST_OS
@@ -61,18 +63,26 @@ needsUpgrade v
 upgrade :: Bool -> RepoVersion -> Annex Bool
 upgrade automatic destversion = do
 	upgraded <- go =<< getVersion
-	when upgraded $
+	when upgraded
-		setVersion destversion
+		postupgrade
 	return upgraded
   where
 	go (Just v)
 		| v >= destversion = return True
-		| otherwise = ifM (up v)
+		| otherwise = ifM upgradingRemote
+			( upgraderemote
+			, ifM (up v)
 				( go (Just (RepoVersion (fromRepoVersion v + 1)))
 				, return False
 				)
+			)
 	go _ = return True
 
+	postupgrade = ifM upgradingRemote
+		( reloadConfig
+		, setVersion destversion
+		)
+
 #ifndef mingw32_HOST_OS
 	up (RepoVersion 0) = Upgrade.V0.upgrade
 	up (RepoVersion 1) = Upgrade.V1.upgrade
@@ -87,3 +97,19 @@ upgrade automatic destversion = do
 	up (RepoVersion 6) = Upgrade.V6.upgrade automatic
 	up (RepoVersion 7) = Upgrade.V7.upgrade automatic
 	up _ = return True
+
+	-- Upgrade local remotes by running git-annex upgrade in them.
+	-- This avoids complicating the upgrade code by needing to handle
+	-- upgrading a git repo other than the current repo.
+	upgraderemote = do
+		rp <- fromRawFilePath <$> fromRepo Git.repoPath
+		cmd <- liftIO readProgramFile
+		liftIO $ boolSystem' cmd
+			[ Param "upgrade"
+			, Param "--quiet"
+			, Param "--autoonly"
+			]
+			(\p -> p { cwd = Just rp })
+
+upgradingRemote :: Annex Bool
+upgradingRemote = isJust <$> fromRepo Git.remoteName

doc/bugs/v7_upgrade_of_local_clone_bug.mdwn

@@ -5,10 +5,17 @@ a repo that has it as a remote, this is displayed:
 
 This happens because git ls-files is run to list the files of the clone.
 But, it has some strange behavior when relative paths are used. Result is
-it always fails. This also causes the keys database of the clone to not get
+it always fails. 
-repopulated after being deleted for the upgrade. That's not a fatal problem
+
-because git-annex is always prepared for the keys database being out of
+This also causes the keys database of the clone to not get
-date, but it could result in considerably more work being done later.
+repopulated after being deleted for the upgrade. 
+That's not a fatal problem because git-annex is always prepared for the
+keys database being out of date, but it could result in considerably more
+work being done later.
+
+Also, the associated files does not get repopulated when it fails like
+that. That could cause worse problems. I have not tried to see what happens
+when the repo that fails to be upgraded has unlocked files.
 
 I also found some v1 upgrade code that does the same thing, and presumably
 also has the problem, although there are probably no v1 repos left. 
@@ -22,3 +29,11 @@ that looks like it would have then problem. That would affect upgrading
 from a V5 direct mode repo to V6.
 
 --[[Joey]
+
+> Fixed all such upgrade bugs by changing how local remotes are upgraded,
+> running git-annex upgrade inside the remote.
+> 
+> Also, guarded all git ls-files calls with a check that it's not being
+> run on a local remote, just in case there are other such bugs.
+> 
+> [[done]] --[[Joey]]

doc/git-annex-upgrade.mdwn

@@ -8,7 +8,7 @@ git annex upgrade
 
 # DESCRIPTION
 
-Upgrades the repository.
+Upgrades the repository to the latest version.
 
 Each git-annex repository has an annex.version in its git configuration,
 that indicates the repository version. When an old repository version
@@ -24,6 +24,13 @@ was only used by its author. It's expected that git-annex will always
 support upgrading from all past repository versions -- this is necessary to
 allow archives to be taken offline for years and later used.
 
+# OPTIONS
+
+* --autoonly
+
+  Only do whatever automatic upgrade can be done, don't necessarily
+  upgrade to the latest version. This is used internally by git-annex.
+
 # SEE ALSO
 
 [[git-annex]](1)