proxy: for all your direct mode repository munging needs

This allows bypassing the direct mode guard in a safe way to do all sorts
of things including git revert, git mv, git checkout ...

This commit was sponsored by the WikiMedia Foundation.

Annex/Direct.hs

@@ -225,9 +225,17 @@ mergeDirectCommit allowff old branch commitmode = do
   where
 	canff = maybe (return False) (\o -> inRepo $ Git.Branch.fastForwardable o branch) old
 
-{- Cleans up after a direct mode merge. The merge must have been staged
- - in the index. Uses diff-index to compare the staged changes with
- - the tree before the merge, and applies those changes to the work tree.
+mergeDirectCleanup :: FilePath -> Git.Ref -> Annex ()
+mergeDirectCleanup d oldref = do
+	updateWorkTree d oldref
+	liftIO $ removeDirectoryRecursive d
+
+{- Updates the direct mode work tree to reflect the changes staged in the
+ - index by a git command, that was run in a temporary work tree.
+ -
+ - Uses diff-index to compare the staged changes with provided ref
+ - which should be the tree before the merge, and applies those
+ - changes to the work tree.
  -
  - There are really only two types of changes: An old item can be deleted,
  - or a new item added. Two passes are made, first deleting and then
@@ -236,8 +244,8 @@ mergeDirectCommit allowff old branch commitmode = do
  - order, but we cannot add the directory until the file with the
  - same name is removed.)
  -}
-mergeDirectCleanup :: FilePath -> Git.Ref -> Annex ()
-mergeDirectCleanup d oldref = do
+updateWorkTree :: FilePath -> Git.Ref -> Annex ()
+updateWorkTree d oldref = do
 	(items, cleanup) <- inRepo $ DiffTree.diffIndex oldref
 	makeabs <- flip fromTopFilePath <$> gitRepo
 	let fsitems = zip (map (makeabs . DiffTree.file) items) items
@@ -246,7 +254,6 @@ mergeDirectCleanup d oldref = do
 	forM_ fsitems $
 		go makeabs DiffTree.dstsha DiffTree.dstmode movein movein_raw
 	void $ liftIO cleanup
-	liftIO $ removeDirectoryRecursive d
   where
 	go makeabs getsha getmode a araw (f, item)
 		| getsha item == nullSha = noop

CmdLine/GitAnnex.hs

@@ -83,6 +83,7 @@ import qualified Command.Direct
 import qualified Command.Indirect
 import qualified Command.Upgrade
 import qualified Command.Forget
+import qualified Command.Proxy
 import qualified Command.Version
 import qualified Command.Help
 #ifdef WITH_ASSISTANT
@@ -175,6 +176,7 @@ cmds = concat
 	, Command.Indirect.cmd
 	, Command.Upgrade.cmd
 	, Command.Forget.cmd
+	, Command.Proxy.cmd
 	, Command.Version.cmd
 	, Command.Help.cmd
 #ifdef WITH_ASSISTANT

Command/Proxy.hs

@@ -0,0 +1,48 @@
+{- git-annex command
+ -
+ - Copyright 2014 Joey Hess <joey@kitenet.net>
+ -
+ - Licensed under the GNU GPL version 3 or higher.
+ -}
+
+module Command.Proxy where
+
+import Common.Annex
+import Command
+import Config
+import Utility.Tmp
+import Utility.Env
+import Annex.Direct
+import qualified Git.Branch
+import qualified Git.Sha
+
+cmd :: [Command]
+cmd = [notBareRepo $
+	command "proxy" ("-- git command") seek
+		SectionCommon "safely bypass direct mode guard"]
+
+seek :: CommandSeek
+seek ("--":ps) = withWords start ps
+seek ps = withWords start ps
+
+start :: [String] -> CommandStart
+start [] = error "Did not specify command to run."
+start (c:ps) = liftIO . exitWith =<< ifM isDirect
+	( do
+		g <- gitRepo
+		withTmpDirIn (gitAnnexTmpMiscDir g) "proxy" go
+	, liftIO $ safeSystem c (map Param ps)
+	)
+  where
+	go tmp = do
+		oldref <- fromMaybe Git.Sha.emptyTree
+			<$> inRepo Git.Branch.currentSha
+		liftIO $ print oldref
+		exitcode <- liftIO $ proxy tmp
+		mergeDirectCleanup tmp oldref
+		return exitcode
+	proxy tmp = do
+		usetmp <- Just . addEntry "GIT_WORK_TREE" tmp  <$> getEnvironment
+		unlessM (boolSystemEnv "git" [Param "checkout", Param "--", Param "."] usetmp) $
+			error "Failed to set up proxy work tree."
+		safeSystemEnv c (map Param ps) usetmp

Git/Branch.hs

@@ -43,6 +43,12 @@ currentUnsafe r = parse . firstLine
 		| null l = Nothing
 		| otherwise = Just $ Git.Ref l
 
+currentSha :: Repo -> IO (Maybe Git.Sha)
+currentSha r = go =<< current r
+  where
+	go Nothing = return Nothing
+	go (Just ref) = Git.Ref.sha ref r
+
 {- Checks if the second branch has any commits not present on the first
  - branch. -}
 changed :: Branch -> Branch -> Repo -> IO Bool

debian/changelog

@@ -3,6 +3,9 @@ git-annex (5.20141126) UNRELEASED; urgency=medium
   * pre-commit: Block partial commit of unlocked annexed file, since
     that left a typechange staged in index due to some infelicity of git's
     handling of partial commits.
+  * proxy: New command for direct mode repositories, allows bypassing
+    the direct mode guard in a safe way to do all sorts of things
+    including git revert, git mv, git checkout ...
   * Debian package is now maintained by Gergely Nagy.
 
  -- Joey Hess <joeyh@debian.org>  Mon, 10 Nov 2014 15:31:55 -0400

doc/direct_mode.mdwn

@@ -3,8 +3,9 @@ git, and in turn point at the content of large files that is stored in
 `.git/annex/objects/`. Direct mode gets rid of the symlinks.
 
 The advantage of direct mode is that you can access files directly,
-including modifying them. The disadvantage is that most regular git
-commands cannot be used in a direct mode repository.
+including modifying them. The disadvantage is that mant regular git
+commands cannot be used in a direct mode repository, since they don't
+understand how to update its working tree.
 
 Normally, git-annex repositories start off in indirect mode. With some
 exceptions:
@@ -82,11 +83,29 @@ There are still lots of git commands you can use in direct mode. For
 example, you can run `git log` on files, run `git push`, `git fetch`,
 `git config`, `git remote add` etc.
 
+## proxing git commands in direct mode
+
+For those times when you really need to run a command like `git revert
+HEAD` in a direct mode repository, git-annex has the ability to proxy
+the command to work in direct mode.
+
+For example:
+
+	git annex proxy -- git revert HEAD
+
+	git annex proxy -- git checkout HEAD^^
+
+	git annex proxy -- git mv mydir newname
+
+This works by setting up a temporary work tree, letting the git
+command run on that work tree, and then updating the real work
+tree to reflect any changes staged or committed by the git command,
+with appropriate handling of the direct mode files.
+
 ## forcing git to use the work tree in direct mode
 
 This is for experts only. You can lose data doing this, or check enormous
 files directly into your git repository, and it's your fault if you do!
-Also, there should be no good reason to need to do this, ever.
 
 Ok, with the warnings out of the way, all you need to do to make any
 git command access the work tree in direct mode is pass it 

doc/git-annex.mdwn

@@ -282,6 +282,25 @@ subdirectories).
   are on a video hosting site, and the video is downloaded. This allows
   importing e.g., youtube playlists.
 
+* `proxy -- git cmd [options]`
+
+  Only useful in a direct mode repository, this runs the specified git
+  command with a temporary work tree, and updates the working tree to
+  reflect any changes staged or committed by the git command.
+  
+  For example, to revert the most recent change that was committed
+  to the repository:
+
+    git annex proxy -- git revert HEAD
+
+  To check out a past version of the repository:
+
+    git annex proxy -- git checkout HEAD^^
+
+  To rename a directory:
+
+    git annex proxy -- git mv mydir newname
+
 * `watch`
 
   Watches for changes to files in the current directory and its subdirectories,
@@ -499,9 +518,9 @@ subdirectories).
 
   As part of the switch to direct mode, any changed files will be committed.
 
-  Note that git commands that operate on the work tree are often unsafe to
-  use in direct mode repositories, and can result in data loss or other
-  bad behavior.
+  Note that git commands that operate on the work tree will refuse to
+  run in direct mode repositories. Use `git annex proxy` to safely run such
+  commands.
 
 * `indirect`
 

doc/todo/direct_mode_undo.mdwn

@@ -37,6 +37,10 @@ better to make it use a separate work tree, but the same .git directory?
 Then step #3 would instead update the direct mode work tree to refect
 the new HEAD, and step #4 would not be needed.
 
+> This is done.. But, I think an undo command would also be good
+> to do, as a nicer user interface that can integrate well with a file
+> manager. --[[Joey]]
+
 ## git annex undo
 
 I don't want to recapitulate all of the git commands in git-annex for