restage: New git-annex command, handles restaging unlocked files

This is much easier and less failure-prone than having the user run
git update-index --refresh themselves.

Sponsored-by: Dartmouth College's DANDI project

Annex/Link.hs

@@ -316,7 +316,7 @@ unableToRestage mf = unwords
 	, "This is only a cosmetic problem affecting git status; git add,"
 	, "git commit, etc won't be affected."
 	, "To fix the git status display, you can run:"
-	, "git update-index -q --refresh " ++ fromMaybe "<file>" mf
+	, "git-annex restage"
 	]
 
 {- Parses a symlink target or a pointer file to a Key.

CHANGELOG

@@ -31,6 +31,7 @@ git-annex (10.20220823) UNRELEASED; urgency=medium
   * Improved handling of --time-limit when combined with -J
   * Fix updating git index file after getting an unlocked file 
     when annex.stalldetection is set.
+  * restage: New git-annex command, handles restaging unlocked files.
   * test: Added --test-with-git-config option.
 
  -- Joey Hess <id@joeyh.name>  Mon, 29 Aug 2022 15:03:04 -0400

CmdLine/GitAnnex.hs

@@ -1,6 +1,6 @@
 {- git-annex main program
  -
- - Copyright 2010-2021 Joey Hess <id@joeyh.name>
+ - Copyright 2010-2022 Joey Hess <id@joeyh.name>
  -
  - Licensed under the GNU AGPL version 3 or higher.
  -}
@@ -114,6 +114,7 @@ import qualified Command.Proxy
 import qualified Command.DiffDriver
 import qualified Command.Smudge
 import qualified Command.FilterProcess
+import qualified Command.Restage
 import qualified Command.Undo
 import qualified Command.Version
 import qualified Command.RemoteDaemon
@@ -228,6 +229,7 @@ cmds testoptparser testrunner mkbenchmarkgenerator = map addGitAnnexCommonOption
 	, Command.DiffDriver.cmd
 	, Command.Smudge.cmd
 	, Command.FilterProcess.cmd
+	, Command.Restage.cmd
 	, Command.Undo.cmd
 	, Command.Version.cmd
 	, Command.RemoteDaemon.cmd

Command/Restage.hs

@@ -0,0 +1,25 @@
+{- git-annex command
+ -
+ - Copyright 2022 Joey Hess <id@joeyh.name>
+ -
+ - Licensed under the GNU AGPL version 3 or higher.
+ -}
+
+module Command.Restage where
+
+import Command
+import qualified Annex
+import Annex.Link
+
+cmd :: Command
+cmd = command "restage" SectionPlumbing 
+	"estages unlocked files in the git index"
+	paramNothing (withParams seek)
+
+seek :: CmdParams -> CommandSeek
+seek = withNothing (commandAction start)
+
+start :: CommandStart
+start = starting "restage" (ActionItemOther Nothing) (SeekInput []) $ do
+	restagePointerFiles =<< Annex.gitRepo
+	next $ return True

Command/VCycle.hs

@@ -26,7 +26,7 @@ start ::CommandStart
 start = go =<< currentView
   where
 	go Nothing = giveup "Not in a view."
-	go (Just v) = starting "vcycle" (ActionItemOther Nothing) (SeekInput [])$ do
+	go (Just v) = starting "vcycle" (ActionItemOther Nothing) (SeekInput []) $ do
 		let v' = v { viewComponents = vcycle [] (viewComponents v) }
 		if v == v'
 			then do

doc/bugs/reports_file___34__modified__34___whenever_it_is_not.mdwn

@@ -91,4 +91,6 @@ I think I get it after I `annex move` and then `annex get` that file back. Just
 [[!meta author=yoh]]
 [[!tag projects/dandi]]
 
-> [[!meta title="annex.stalldetection prevents git-annex get from restaging unlocked files"]]
+[[!meta title="annex.stalldetection prevents git-annex get from restaging unlocked files"]]
+
+> [[fixed|done]] --[[Joey]]

doc/bugs/reports_file___34__modified__34___whenever_it_is_not/comment_16_4b8a360c38dfa4c8d22e3a96fbace0e5._comment

@@ -0,0 +1,33 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""status update"""
+ date="2022-09-23T19:57:38Z"
+ content="""
+I've implemented the log file. The stalled transferrer case is now handled.
+This bug is fixed.
+
+As to a few other cases I considered in comments upthread:
+
+When a get/drop was interrupted before it could restage, 
+the next get/drop will cause the necessary restaging for the 
+interrupted process to happen. However, this doesn't help if there's
+nothing left to get/drop. Should git-annex always run restagePointerFiles
+on shutdown? That would make any git-annex command handle the restaging.
+But it doesn't seem right for query commands to do potentially a lot of
+work to handle this case. Anyway, I don't think this needs to be dealt
+with in this bug report.
+
+When multiple processes try to restage at the same time, one will
+restage everything that all of them logged. The others will still display a
+warning to the user that they couldn't restage. It would be hard to avoid
+displaying that warning, since it does need to warn when it was
+unable to restage because git has the index locked at the time. Anyway,
+I think it's ok to display the message despite the files having been
+restaged, because it's the same as a later git-annex process handling the
+restaging. (It does seem like two transferrers belonging to the same parent
+could collide in this way, and one display the warning, which isn't great..)
+
+I also implemented a "git-annex restage" command that
+is an easier way to restage in the cases where git-annex is not able
+to do it itself.
+"""]]

doc/git-annex-restage.mdwn

@@ -0,0 +1,34 @@
+# NAME
+
+git-annex restage - restages unlocked files in the git index
+
+# SYNOPSIS
+
+git annex restage
+
+# DESCRIPTION
+
+Since getting or dropping an unlocked file modifies the file in the work
+tree, git needs to be told that the modification does not change the
+content that it has recorded (the annex pointer). Restaging the file
+accomplishes that.
+
+You do not normally need to run this command, because usually git-annex
+is able to restage unlocked files itself. There are some situations
+where git-annex needs to restage a file, but the git index is locked,
+and so it cannot. It will then display a warning suggesting you run this
+command.
+
+It's safe to run this command even after you have made a modification to an
+unlocked file.
+
+# SEE ALSO
+
+[[git-annex]](1)
+[[git-annex-smudge]](1)
+
+# AUTHOR
+
+Joey Hess <id@joeyh.name>
+
+Warning: Automatically converted into a man page by mdwn2man. Edit with care.

doc/git-annex.mdwn

@@ -708,8 +708,8 @@ content from the key-value store.
 * `smudge`
 
   This command lets git-annex be used as a git filter driver, allowing
-  annexed files in the git repository to be unlocked at all times, instead
-  of being symlinks.
+  annexed files in the git repository to be unlocked regular files instead
+  of symlinks.
 
   See [[git-annex-smudge]](1) for details.
 
@@ -720,6 +720,12 @@ content from the key-value store.
 
   See [[git-annex-filter-process]](1) for details.
 
+* `restage`
+  
+  Restages unlocked files in the git index.
+
+  See [[git-annex-restage]](1) for details.
+
 * `findref [ref]`
 
   Lists files in a git ref. (deprecated)

git-annex.cabal

@@ -775,6 +775,7 @@ Executable git-annex
     Command.Repair
     Command.Required
     Command.ResolveMerge
+    Command.Restage
     Command.RmUrl
     Command.Schedule
     Command.Semitrust