narrow the race where a file gets modified before update-index

Check just before running update-index if the worktree file's content is
still the same, don't update it when it's been modified. This narrows
the race window a lot, from possibly minutes or hours, to seconds or
less.

(Use replaceFile so that the worktree update happens atomically,
allowing the InodeCache of the new worktree file to itself be gathered
w/o any other race.)

This doesn't eliminate the race; it can still occur in the window before
update-index runs. When annex.queue is large, a lot of files will be
statted by the checks, and so the window may still be large enough to be a
problem.

When only a few files are being processed, the window is as small as it
is in the race where a modification gets overwritten by git-annex when
it updates the worktree. Or maybe as small as whatever race git
checkout/pull/merge may have when the worktree gets modified during it.
Still, I've kept a todo about this race.

This commit was supported by the NSF-funded DataLad project.
This commit is contained in:
Joey Hess 2018-08-16 15:15:20 -04:00
parent 6a445dc086
commit 82a239675f
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
5 changed files with 52 additions and 29 deletions

View file

@ -25,6 +25,8 @@ import Git.FilePath
import Annex.HashObject
import Utility.FileMode
import Utility.FileSystemEncoding
import Utility.InodeCache
import Annex.InodeSentinal
import qualified Data.ByteString.Lazy as L
@ -153,9 +155,11 @@ newtype Restage = Restage Bool
-
- This uses the git queue, so the update is not performed immediately,
- and this can be run multiple times cheaply.
-
- The InodeCache is for the worktree file.
-}
restagePointerFile :: Restage -> FilePath -> Annex ()
restagePointerFile (Restage False) f = toplevelWarning True $ unwords
restagePointerFile :: Restage -> FilePath -> InodeCache -> Annex ()
restagePointerFile (Restage False) f _ = toplevelWarning True $ unwords
[ "git status will show " ++ f
, "to be modified, since its content availability has changed."
, "This is only a cosmetic problem affecting git status; git add,"
@ -163,8 +167,18 @@ restagePointerFile (Restage False) f = toplevelWarning True $ unwords
, "To fix the git status display, you can run:"
, "git update-index -q --refresh " ++ f
]
restagePointerFile (Restage True) f =
Annex.Queue.addCommand "update-index" [Param "-q", Param "--refresh"] [f]
restagePointerFile (Restage True) f orig = withTSDelta $ \tsd ->
Annex.Queue.addCommandCond "update-index" [Param "-q", Param "--refresh"]
[(f, check tsd)]
where
-- If the file gets modified before update-index runs,
-- it would stage the modified file, which would be surprising
-- behavior. So check for modifications and avoid running
-- update-index on the file. This does not close the race, but it
-- makes the window as narrow as possible.
check tsd = genInodeCache f tsd >>= return . \case
Nothing -> False
Just new -> compareStrong orig new
{- Parses a symlink target or a pointer file to a Key.
- Only looks at the first line, as pointer files can have subsequent