annex.gitaddtoannex configuration

Added annex.gitaddtoannex configuration. Setting it to false prevents
git add from usually adding files to the annex.
(Unless the file was annexed before, or a renamed annexed file is detected.)

Currently left at true; some users are encouraging it be set to false.

CHANGELOG

@@ -5,6 +5,12 @@ git-annex (7.20191018) UNRELEASED; urgency=medium
     when running a command.
   * sync: Fix crash when there are submodules and an adjusted branch is
     checked out.
+  * Made git add smarter about renamed annexed files. It can tell when an
+    annexed file was renamed, and will add it to the annex, and not to git,
+    unless annex.largefiles tells it to do otherwise.
+  * Added annex.gitaddtoannex configuration. Setting it to false prevents
+    git add from usually adding files to the annex. 
+    (Unless the file was annexed before, or a renamed annexed file is detected.)
 
  -- Joey Hess <id@joeyh.name>  Mon, 21 Oct 2019 11:01:06 -0400
 

Command/Smudge.hs

@@ -20,6 +20,7 @@ import qualified Git.BuildVersion
 import Git.FilePath
 import qualified Git
 import qualified Git.Ref
+import qualified Annex
 import Backend
 import Utility.Metered
 import Annex.InodeSentinal
@@ -145,10 +146,12 @@ clean file = do
 		filepath <- liftIO $ absPath file
 		return $ not $ dirContains repopath filepath
 
--- New files are annexed as configured by annex.largefiles.
+-- New files are annexed as configured by annex.gitaddtoannex and 
+-- annex.largefiles.
 --
--- If annex.largefiles is not configured for a file, some heuristics are
--- used to avoid bad behavior:
+-- If annex.gitaddtoannex is false, or it's true and annex.largefiles
+-- is not configured for a file, some heuristics are used to avoid bad
+-- behavior:
 --
 -- When the file's inode is the same as one that was used for annexed
 -- content before, annex it. This handles cases such as renaming an
@@ -158,11 +161,16 @@ clean file = do
 -- Otherwise, if the index already contains the file, preserve its
 -- annexed/not annexed state. This prevents accidental conversions.
 shouldAnnex :: FilePath -> Maybe Key -> Annex Bool
-shouldAnnex file moldkey = do
-	matcher <- largeFilesMatcher
-	checkFileMatcher' matcher file whenempty
+shouldAnnex file moldkey = ifM (annexGitAddToAnnex <$> Annex.getGitConfig)
+	( checkmatcher (checkheuristics checkwasingit)
+	, checkheuristics (pure False)
+	)
   where
-	whenempty = case moldkey of
+	checkmatcher def = do
+		matcher <- largeFilesMatcher
+		checkFileMatcher' matcher file def
+	
+	checkheuristics def = case moldkey of
 		Just _ -> return True
 		Nothing -> do
 			isknown <- withTSDelta (liftIO . genInodeCache file) >>= \case
@@ -170,7 +178,11 @@ shouldAnnex file moldkey = do
 				Just ic -> Database.Keys.isInodeKnown ic =<< sentinalStatus
 			if isknown
 				then return True
-				else isNothing <$> catObjectMetaData (Git.Ref.fileRef file)
+				else def
+	
+	checkwasingit = case moldkey of
+		Just _ -> return False
+		Nothing -> isNothing <$> catObjectMetaData (Git.Ref.fileRef file)
 
 emitPointer :: Key -> IO ()
 emitPointer = S.putStr . formatPointer

Types/GitConfig.hs

@@ -78,6 +78,7 @@ data GitConfig = GitConfig
 	, annexAriaTorrentOptions :: [String]
 	, annexCrippledFileSystem :: Bool
 	, annexLargeFiles :: Maybe String
+	, annexGitAddToAnnex :: Bool
 	, annexAddSmallFiles :: Bool
 	, annexFsckNudge :: Bool
 	, annexAutoUpgrade :: AutoUpgrade
@@ -147,6 +148,7 @@ extractGitConfig r = GitConfig
 	, annexAriaTorrentOptions = getwords (annex "aria-torrent-options")
 	, annexCrippledFileSystem = getbool (annex "crippledfilesystem") False
 	, annexLargeFiles = getmaybe (annex "largefiles")
+	, annexGitAddToAnnex = getbool (annex "gitaddtoannex") True
 	, annexAddSmallFiles = getbool (annex "addsmallfiles") True
 	, annexFsckNudge = getbool (annex "fscknudge") True
 	, annexAutoUpgrade = toAutoUpgrade $ getmaybe (annex "autoupgrade")

doc/forum/lets_discuss_git_add_behavior/comment_31_9dd20cc1b27e58fe0273a9d24c13b928._comment

@@ -0,0 +1,15 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 31"""
+ date="2019-10-23T19:20:25Z"
+ content="""
+This will get the behavior you seek, once you have upgraded to current
+master:
+
+	git config annex.gitaddtoannex false
+
+It could be made to default to false, TBD. If someone wants to make a patch
+to all the documentation that currently talks about using git add and git
+commit -a to annex files, to document the mooted new behavior, that would
+be helpful.
+"""]]

doc/git-annex.mdwn

@@ -897,6 +897,20 @@ Like other git commands, git-annex is configured via `.git/config`.
 
   See <https://git-annex.branchable.com/tips/largefiles> for details.
 
+* `annex.gitaddtoannex`
+
+  This controls the behavior of `git add`. If you want `git add` to
+  add files to the annex (either all files, or the files matched
+  by your annex.largefiles configuration), set it to true.
+
+  To make `git add` add files to git but not to the annex, set it to false.
+  Note that `git add` will still add files to the annex in a couple of
+  situations. When an annexed file has been modified, it makes sense to add
+  the new version to the annex too. When an annexed file has been renamed
+  to a new name, it should remain annexed.
+
+  The default is currently true.
+
 * `annex.addsmallfiles`
 
   Controls whether small files (not matching annex.largefiles)

doc/todo/inode_based_clean_filter_for_less_surprising_git_add.mdwn

@@ -42,3 +42,7 @@ databases.
 Some filesystems don't have stable inodes etc, but all that is already
 handled by the InodeCache machinery, so I think this could work pretty
 well. --[[Joey]]
+
+> [[done]] although the sql database is used in a horrible way by the
+> current implementation, which is probably very slow in some situations,
+> so [[sqlite_database_improvements]] are now really needed. --[[Joey]]

doc/todo/separate_annex.largefiles.git-add_and_annex.largefiles.git-annex-add_settings.mdwn

@@ -3,3 +3,5 @@ Could there be separate `annex.git-add.largefiles` and `annex.git-annex-add.larg
 Reason: to prevent `git add` from inadvertently adding annexed files in unlocked form, I set `* annex.largefiles=nothing` at repo root; but then, `git annex add` won't annex anything either, unless specifically asked.   I want to use `git add` to add files to git only (since it can't add them to git-annex in locked form), and to use `git annex add` to add files to either git or annex based on `annex.git-annex-add.largefiles` setting.
 
 Related: [[forum/lets_discuss_git_add_behavior]]
+
+> [[wontfix|done]] --[[Joey]]

doc/todo/separate_annex.largefiles.git-add_and_annex.largefiles.git-annex-add_settings/comment_3_6f720c44e6c5ad8b9df6c2003b1b1c40._comment

@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 3"""
+ date="2019-10-23T19:25:21Z"
+ content="""
+I suspect that anyone who would have wanted this will instead be happy
+with setting the new annex.gitaddtoannex to false.
+
+If someone does have a good reason to want git add and git annex add to
+have different opinions about what constitutes a large file, please reopen
+the todo with a justification.
+"""]]