add: Consistently treat files in a dotdir as dotfiles, even when ran inside that dotdir

Assistant and smudge also updated.

This does add a small amount of extra work, getting the TopFilePath.
Not enough to be concerned by.

Also improve documentation to make clear that files inside dotdirs are
treated as dotfiles.

Sponsored-by: Eve on Patreon

Assistant/Threads/Committer.hs

@@ -45,6 +45,7 @@ import qualified Git.Branch
 import Utility.Tuple
 import Utility.Metered
 import qualified Utility.RawFilePath as R
+import Git.FilePath
 
 import Data.Time.Clock
 import qualified Data.Set as S
@@ -319,15 +320,19 @@ handleAdds lockdowndir havelsof largefilematcher annexdotfiles delayadd cs = ret
 					(LinkChange (Just key))
 	
 	checksmall change
-		| not annexdotfiles && dotfile f =
+		| not annexdotfiles = do
-			return (Right change)
+			topfile <- liftAnnex $ 
-		| otherwise =
+				getTopFilePath <$> inRepo (toTopFilePath f)
-			ifM (liftAnnex $ checkFileMatcher NoLiveUpdate largefilematcher f)
+			if dotfile topfile
-				( return (Left change)
+				then return (Right change)
-				, return (Right change)
+				else checkmatcher
-				)
+		| otherwise = checkmatcher
 	  where
 		f = toRawFilePath (changeFile change)
+		checkmatcher = ifM (liftAnnex $ checkFileMatcher NoLiveUpdate largefilematcher f)
+			( return (Left change)
+			, return (Right change)
+			)
 
 	addsmall [] = noop
 	addsmall toadd = liftAnnex $ void $ tryIO $

CHANGELOG

@@ -3,6 +3,8 @@ git-annex (10.20241032) UNRELEASED; urgency=medium
   * git-remote-annex: Fix a reversion introduced in version 10.20241031
     that broke cloning from a special remote.
   * vpop: Only update state after successful checkout.
+  * add: Consistently treat files in a dotdir as dotfiles, even
+    when ran inside that dotdir.
 
  -- Joey Hess <id@joeyh.name>  Mon, 11 Nov 2024 12:26:00 -0400
 

Command/Add.hs

@@ -94,17 +94,19 @@ seek' o = do
 	addunlockedmatcher <- addUnlockedMatcher
 	annexdotfiles <- getGitConfigVal annexDotFiles 
 	let gofile includingsmall (si, file) = case largeFilesOverride o of
-		Nothing -> ifM (pure (annexdotfiles || not (dotfile file))
+		Nothing -> do
-			<&&> (checkFileMatcher NoLiveUpdate largematcher file 
+			topfile <- getTopFilePath <$> inRepo (toTopFilePath file)
-			<||> Annex.getRead Annex.force))
+			ifM (pure (annexdotfiles || not (dotfile topfile))
-			( start dr si file addunlockedmatcher
+				<&&> (checkFileMatcher NoLiveUpdate largematcher file 
-			, if includingsmall
+				<||> Annex.getRead Annex.force))
-				then ifM (annexAddSmallFiles <$> Annex.getGitConfig)
+				( start dr si file addunlockedmatcher
-					( startSmall dr si file
+				, if includingsmall
-					, stop
+					then ifM (annexAddSmallFiles <$> Annex.getGitConfig)
-					)
+						( startSmall dr si file
-				else stop
+						, stop
-			)
+						)
+					else stop
+				)
 		Just True -> start dr si file addunlockedmatcher
 		Just False -> startSmallOverridden dr si file
 	case batchOption o of

Command/Smudge.hs

@@ -239,12 +239,14 @@ shouldAnnex file indexmeta moldkey = do
 		, checkunchanged checkwasannexed
 		)
   where
-	checkmatcher d
+	checkmatcher d = do
-		| dotfile file = ifM (getGitConfigVal annexDotFiles)
+		topfile <- getTopFilePath <$> inRepo (toTopFilePath file)
-			( go
+		if dotfile topfile
-			, d
+			then ifM (getGitConfigVal annexDotFiles)
-			)
+				( go
-		| otherwise = go
+				, d
+				)
+			else go
 	  where
 		go = do
 			matcher <- largeFilesMatcher

doc/bugs/add__58___inconsistently_treats_files_in_dotdirs_as_dotfiles.mdwn

@@ -101,3 +101,7 @@ go-to solution for “want something versioned, but can't store the
 contents themselves (too big, too sensitive, etc.)?”. Furthermore, git-annex
 documentation in general is excellent. But that is also why I'm stumped that
 the manual is so silent on this point.
+
+> [[fixed|done]] by resolving inconsistent behavior. Also improved
+> documentation to be clear that dot directories are treated same as
+> dotfiles.

doc/bugs/add__58___inconsistently_treats_files_in_dotdirs_as_dotfiles/comment_5_c9206d1a0c74149df970d44025160d89._comment

@@ -0,0 +1,21 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 5"""
+ date="2024-11-13T17:04:59Z"
+ content="""
+> Why can't git-annex just handle the .git folder differently and for all others just annex or not as set in the largefile rules?
+
+Because creating a .gitignore followed by `git-annex add` would then blow the
+user's foot off. And this would be a very common foot-shooting opportunity,
+and .gitignore is only the perhaps most common trigger for it.
+
+Files in dot directories are generally less common, outside of course of
+.git and $HOME. Which is the only reason I'm willing to consider changing
+the dotfiles handling to not include those.
+
+But, .config/ seems to me to perfectly match what dotfiles *are*, which is
+files that are configuration that are named with a name starting with a
+dot in order to keep them from cluttering up `ls`. Just because in your use
+case you don't want to check those into git as dotfiles does not seem like
+a good argument for git-annex to not treat them as dotfiles by default.
+"""]]

doc/bugs/add__58___inconsistently_treats_files_in_dotdirs_as_dotfiles/comment_6_eb82a22ffe512bf0d6f2e7841ce022f0._comment

@@ -0,0 +1,9 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 6"""
+ date="2024-11-13T17:14:47Z"
+ content="""
+Revisiting this, it seems best to fix the inconsistent behavior by 
+having git-annex get the path to the file relative to the top of the git
+repository, and check if there's a dot directory in the path.
+"""]]

doc/git-annex-add.mdwn

@@ -18,10 +18,10 @@ git has been configured to ignore will be silently skipped.
 If annex.largefiles is configured (in git config, gitattributes, or
 git-annex config), and does not match a file, `git annex add` will behave
 the same as `git add` and add the non-large file directly to the git
-repository, instead of to the annex. (By default dotfiles are assumed to
+repository, instead of to the annex. (By default dotfiles and the contents
-not be large, and are added directly to git, but annex.dotfiles can be
+of dotdirs) are assumed to not be large, and are added directly to git, but
-configured to annex those too.) See the git-annex manpage for documentation
+annex.dotfiles can be configured to annex those too.) See the git-annex
-of these and other configuration settings.
+manpage for documentation of these and other configuration settings.
 
 By default, large files are added to the annex in locked form, which
 prevents further modification of their content until

doc/git-annex-config.mdwn

@@ -81,8 +81,8 @@ looks for these.
   
   This configures the behavior of both git-annex and git when adding
   files to the repository. By default, `git-annex add` adds all files
-  to the annex (except dotfiles), and `git add` adds files to git
+  to the annex (except dotfiles and files in dotdirs), and 
-  (unless they were added to the annex previously).
+  `git add` adds files to git (unless they were added to the annex previously).
   When annex.largefiles is configured, both
   `git annex add` and `git add` will add matching large files to the
   annex, and the other files to git.
@@ -95,11 +95,12 @@ looks for these.
   attributes in `.gitattributes` files, or by `git config`.
 
 * `annex.dotfiles`
-
+  
-  Normally, dotfiles are assumed to be files like .gitignore,
+  Normally, dotfiles and files inside dotdirs are assumed to be
-  whose content should always be part of the git repository, so 
+  configuration files like .gitignore, whose content should always
-  they will not be added to the annex. Setting annex.dotfiles to true
+  be part of the git repository, so they will not be added to the annex.
-  makes dotfiles be added to the annex the same as any other file. 
+  Setting annex.dotfiles to true makes these files be added to the
+  annex the same as any other file. 
 
   This sets a default, which can be overridden by annex.dotfiles
   in `git config`.

doc/git-annex.mdwn

@@ -974,8 +974,8 @@ repository, using [[git-annex-config]]. See its man page for a list.)
 
   This configures the behavior of both git-annex and git when adding
   files to the repository. By default, `git-annex add` adds all files
-  to the annex (except dotfiles), and `git add` adds files to git
+  to the annex (except dotfiles and files in dotdirs), and `git add`
-  (unless they were added to the annex previously).
+  adds files to git (unless they were added to the annex previously).
   When annex.largefiles is configured, both
   `git annex add` and `git add` will add matching large files to the
   annex, and the other files to git.
@@ -986,10 +986,11 @@ repository, using [[git-annex-config]]. See its man page for a list.)
 
 * `annex.dotfiles`
 
-  Normally, dotfiles are assumed to be files like .gitignore,
+  Normally, dotfiles and files inside dotdirs are assumed to be
-  whose content should always be part of the git repository, so 
+  configuration files like .gitignore, whose content should always
-  they will not be added to the annex. Setting annex.dotfiles to true
+  be part of the git repository, so they will not be added to the annex.
-  makes dotfiles be added to the annex the same as any other file. 
+  Setting annex.dotfiles to true makes these files be added to the
+  annex the same as any other file. 
 
   To annex only some dotfiles, set this and configure annex.largefiles
   to match the ones you want. For example, to match only dotfiles ending