only stage regular files from the journal

git-annex only writes regular files there, but other things may drop junk
like empty .DAV directories around the tree. And trying to hash such things
can have weird and hard to understand effects. So it seems best to do a
small amount of work in statting the journal file to make sure it's a
regular file.

Sponsored-by: Jack Hill on Patreon

Annex/Branch.hs

@@ -49,6 +49,7 @@ import Data.ByteString.Builder
 import Control.Concurrent (threadDelay)
 import Control.Concurrent.MVar
 import qualified System.FilePath.ByteString as P
+import System.PosixCompat.Files (isRegularFile)
 
 import Annex.Common hiding (append)
 import Types.BranchState
@@ -726,13 +727,14 @@ stageJournal jl commitindex = withIndex $ withOtherTmp $ \tmpdir -> do
 	genstream dir h jh jlogh streamer = readDirectory jh >>= \case
 		Nothing -> return ()
 		Just file -> do
-			unless (dirCruft file) $ do
+			let path = dir P.</> toRawFilePath file
-				let path = dir P.</> toRawFilePath file
+			unless (dirCruft file) $ whenM (isfile path) $ do
 				sha <- Git.HashObject.hashFile h path
 				hPutStrLn jlogh file
 				streamer $ Git.UpdateIndex.updateIndexLine
 					sha TreeFile (asTopFilePath $ fileJournal $ toRawFilePath file)
 			genstream dir h jh jlogh streamer
+	isfile file = isRegularFile <$> R.getFileStatus file
 	-- Clean up the staged files, as listed in the temp log file.
 	-- The temp file is used to avoid needing to buffer all the
 	-- filenames in memory.

CHANGELOG

@@ -1,3 +1,9 @@
+git-annex (10.20230927) UNRELEASED; urgency=medium
+
+  * Ignore directories and other unusual files in .git/annex/journal/
+
+ -- Joey Hess <id@joeyh.name>  Tue, 10 Oct 2023 13:17:31 -0400
+
 git-annex (10.20230926) upstream; urgency=medium
 
   * Fix more breakage caused by git's fix for CVE-2022-24765, this time

doc/bugs/encountering_fatal_error_on_auto_annex_upgrade.mdwn

@@ -93,3 +93,4 @@ git-annex: fd:17: Data.ByteString.hGetLine: end of file
 
 Yes, it's one of my favourite opensource tools.
 
+> [[fixed|done]] --[[Joey]]

doc/bugs/encountering_fatal_error_on_auto_annex_upgrade/comment_5_28d35ef30f76ec326363d86ab2948ad3._comment

@@ -0,0 +1,32 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 5"""
+ date="2023-10-10T17:00:37Z"
+ content="""
+Reproduced as follows:
+
+	joey@darkstar:~/tmp/bench>git init  --bare dav
+	Initialized empty Git repository in /home/joey/tmp/bench/dav/
+	joey@darkstar:~/tmp/bench>cd dav
+	joey@darkstar:~/tmp/bench/dav>git-annex init --version=9
+	init  ok
+	(recording state in git...)
+	joey@darkstar:~/tmp/bench/dav>for s in $(find -type d); do mkdir $s/.DAV;done
+	joey@darkstar:~/tmp/bench/dav>git-annex init --version=9
+	init  fatal: Unable to add (null) to database
+
+So it's these empty directories indeed. (Empty .DAV files don't cause this.)
+
+In particular, it's any empty directory in .git/annex/journal. Which is
+supposed to only contain files that git-annex wrote there. Staging the journal
+is why git hash-object gets involved.
+
+	mkdir .DAV
+	echo .DAV | git hash-object -w --stdin-paths
+	fatal: Unable to add .DAV to database
+
+Still unclear why git ends up with "(null)" in the error message.
+
+While it will slow git-annex down a tiny bit to check if it's a regular file,
+it seems better for git-annex to be robust against this kind of pollution.
+"""]]