fsck --from remote -J concurrency bug

fsck --from remote: Fix a concurrency bug that could make it incorrectly
detect that content in the remote is corrupt, and remove it, resulting in
data loss.
This commit is contained in:
Joey Hess 2020-02-14 14:52:15 -04:00
parent 399319ccbc
commit 352963690a
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
4 changed files with 23 additions and 2 deletions

View file

@ -1,5 +1,8 @@
git-annex (7.20200205) UNRELEASED; urgency=medium
* fsck --from remote: Fix a concurrency bug that could make it incorrectly
detect that content in the remote is corrupt, and remove it, resulting in
data loss.
* When git-annex is built with a ssh that does not support ssh connection
caching, default annex.sshcaching to false, but let the user override it.
* Improve warning messages further when ssh connection caching cannot

View file

@ -1,6 +1,6 @@
{- git-annex command
-
- Copyright 2010-2019 Joey Hess <id@joeyh.name>
- Copyright 2010-2020 Joey Hess <id@joeyh.name>
-
- Licensed under the GNU AGPL version 3 or higher.
-}
@ -161,6 +161,11 @@ performRemote key afile backend numcopies remote =
]
ai = mkActionItem (key, afile)
withtmp a = do
-- Put it in the gitAnnexTmpObjectDir since that's on a
-- filesystem where object temp files are normally
-- stored. The pid prevents multiple fsck processes
-- contending over the same file. (Multiple threads cannot,
-- because OnlyActionOn is used.)
pid <- liftIO getPID
t <- fromRepo gitAnnexTmpObjectDir
createAnnexDirectory t
@ -541,7 +546,7 @@ badContentRemote remote localcopy key = do
runFsck :: Incremental -> ActionItem -> Key -> Annex Bool -> CommandStart
runFsck inc ai key a = stopUnless (needFsck inc key) $
starting "fsck" ai $ do
starting "fsck" (OnlyActionOn key ai) $ do
ok <- a
when ok $
recordFsckTime inc key

View file

@ -145,3 +145,5 @@ whereis file2 (1 copy)
5d9b0df2-000b-4273-bc4a-fb3b9d8319bd -- testrepo [here]
ok
"""]]
> [[fixed|done]] --[[Joey]]

View file

@ -0,0 +1,11 @@
[[!comment format=mdwn
username="joey"
subject="""comment 1"""
date="2020-02-14T18:49:07Z"
content="""
Ugh, I think this could potentially result in data loss. Not when using bup,
but other special remotes.
I've fixed it in git and will think about moving the date of the next
release up.
"""]]