Avoid any access to keys database in v5 mode repositories, which are not supposed to use that database.

This commit is contained in:
Joey Hess 2016-07-19 12:12:19 -04:00
parent 50e63f75d1
commit 2619019630
Failed to extract signature
4 changed files with 47 additions and 5 deletions

View file

@ -55,6 +55,9 @@ versionSupportsUnlockedPointers = go <$> getVersion
versionSupportsAdjustedBranch :: Annex Bool versionSupportsAdjustedBranch :: Annex Bool
versionSupportsAdjustedBranch = versionSupportsUnlockedPointers versionSupportsAdjustedBranch = versionSupportsUnlockedPointers
versionUsesKeysDatabase :: Annex Bool
versionUsesKeysDatabase = versionSupportsUnlockedPointers
setVersion :: Version -> Annex () setVersion :: Version -> Annex ()
setVersion = setConfig versionField setVersion = setConfig versionField

View file

@ -28,6 +28,8 @@ git-annex (6.20160614) UNRELEASED; urgency=medium
* webapp: Escape unusual characters in ssh hostnames when generating * webapp: Escape unusual characters in ssh hostnames when generating
mangled hostnames. This allows IPv6 addresses to be used on filesystems mangled hostnames. This allows IPv6 addresses to be used on filesystems
not supporting : in filenames. not supporting : in filenames.
* Avoid any access to keys database in v5 mode repositories, which
are not supposed to use that database.
-- Joey Hess <id@joeyh.name> Mon, 13 Jun 2016 21:52:24 -0400 -- Joey Hess <id@joeyh.name> Mon, 13 Jun 2016 21:52:24 -0400

View file

@ -27,6 +27,7 @@ import Database.Keys.Handle
import qualified Database.Queue as H import qualified Database.Queue as H
import Annex.Locations import Annex.Locations
import Annex.Common hiding (delete) import Annex.Common hiding (delete)
import Annex.Version (versionUsesKeysDatabase)
import qualified Annex import qualified Annex
import Annex.Perms import Annex.Perms
import Annex.LockFile import Annex.LockFile
@ -53,9 +54,12 @@ import Database.Esqueleto hiding (Key)
- Any queued writes will be flushed before the read. - Any queued writes will be flushed before the read.
-} -}
runReader :: Monoid v => (SQL.ReadHandle -> Annex v) -> Annex v runReader :: Monoid v => (SQL.ReadHandle -> Annex v) -> Annex v
runReader a = do runReader a = ifM versionUsesKeysDatabase
h <- getDbHandle ( do
withDbState h go h <- getDbHandle
withDbState h go
, return mempty
)
where where
go DbUnavailable = return (mempty, DbUnavailable) go DbUnavailable = return (mempty, DbUnavailable)
go st@(DbOpen qh) = do go st@(DbOpen qh) = do
@ -77,7 +81,7 @@ runReaderIO a = runReader (liftIO . a)
- -
- The database is created if it doesn't exist yet. -} - The database is created if it doesn't exist yet. -}
runWriter :: (SQL.WriteHandle -> Annex ()) -> Annex () runWriter :: (SQL.WriteHandle -> Annex ()) -> Annex ()
runWriter a = do runWriter a = whenM versionUsesKeysDatabase $ do
h <- getDbHandle h <- getDbHandle
withDbState h go withDbState h go
where where
@ -146,7 +150,8 @@ openDb createdb _ = catchPermissionDenied permerr $ withExclusiveLock gitAnnexKe
- data to it. - data to it.
-} -}
closeDb :: Annex () closeDb :: Annex ()
closeDb = liftIO . closeDbHandle =<< getDbHandle closeDb = whenM versionUsesKeysDatabase $
liftIO . closeDbHandle =<< getDbHandle
addAssociatedFile :: Key -> TopFilePath -> Annex () addAssociatedFile :: Key -> TopFilePath -> Annex ()
addAssociatedFile k f = runWriterIO $ SQL.addAssociatedFile (toIKey k) f addAssociatedFile k f = runWriterIO $ SQL.addAssociatedFile (toIKey k) f

View file

@ -0,0 +1,32 @@
[[!comment format=mdwn
username="joey"
subject="""comment 1"""
date="2016-07-19T15:38:21Z"
content="""
The v6 changes added a sqlite database. Some code will try to query or
write that database even in v5 mode, although it's meant to give up if the
database is not available.
So, the easy fix, at least for the drop problem, is to avoid using the keys
database at all in v5 mode. I've made this change and it will probably fix
the case you reported.
But, that won't help with v6 repos which need to use that sqlite database.
And, incremental fsck uses its own sqlite database too. And,
[[design/caching_database]] plans are to use sqlite databases more broadly
in the future.
I'm sure that it's not a good idea for git-annex to catch "disk IO error"
exceptions from the database layer. So, it seems that most any other fix than
avoiding using the database would need to be made in sqlite or in lustre,
which it seems don't get along. At a guess, sqlite is trying to use some
POSIX filesystem functionality, likely related to locking, that lustre does
not support.
Hmm, what could be done to hack in support for lustre is to
move the sqlite databases to a different filesystem. But, accessing the
same repo from different hosts which have different sqlite databases would
lead to inconsistent and buggy behavior. And repo setup would need to
decide where to put the sqlite databases and manually configure that
location. So this would be very much a caveat empror configuration.
"""]]