mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

remove ContentIndentifiersCidRemoteIndex uniqueness constraint

Browse source 
For reasons explained in the bug report.

Implemented using a persistent migration, which works fine. It may add a
little startup overhead when a remote is enabled that uses this, but
probably un-noticable.

On the next major version, it would be fine to delete this database,
and regenerate it from the git-annex branch information. Then this
change could be reverted.

Did nothing about adding back the data that got dropped from the db
due to the bug. Only the borg special remote was probably affected,
and it's not been released yet. rm -rf .git/annex/cidsdb does work.
This commit is contained in:
Joey Hess 2020-12-23 13:58:01 -04:00
parent b370e6b0ad
commit c6e693b25d
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
2 changed files with 14 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

13
Database/ContentIdentifier.hs
View file

@ -52,7 +52,9 @@ import qualified Utility.RawFilePath as R
import Database.Persist.Sql hiding (Key) import Database.Persist.Sql hiding (Key)
import Database.Persist.TH import Database.Persist.TH
import Database.Persist.Sqlite (runSqlite)
import qualified System.FilePath.ByteString as P import qualified System.FilePath.ByteString as P
import qualified Data.Text as T
data ContentIdentifierHandle = ContentIdentifierHandle H.DbQueue data ContentIdentifierHandle = ContentIdentifierHandle H.DbQueue
@ -62,7 +64,6 @@ ContentIdentifiers
cid ContentIdentifier cid ContentIdentifier
key Key key Key
ContentIndentifiersKeyRemoteCidIndex key remote cid ContentIndentifiersKeyRemoteCidIndex key remote cid
ContentIndentifiersCidRemoteIndex cid remote
-- The last git-annex branch tree sha that was used to update -- The last git-annex branch tree sha that was used to update
-- ContentIdentifiers -- ContentIdentifiers
AnnexBranch AnnexBranch
@ -79,9 +80,15 @@ openDb :: Annex ContentIdentifierHandle
openDb = do openDb = do
dbdir <- fromRepo gitAnnexContentIdentifierDbDir dbdir <- fromRepo gitAnnexContentIdentifierDbDir
let db = dbdir P.</> "db" let db = dbdir P.</> "db"
unlessM (liftIO $ R.doesPathExist db) $ do ifM (liftIO $ not <$> R.doesPathExist db)
initDb db $ void $ ( initDb db $ void $
runMigrationSilent migrateContentIdentifier runMigrationSilent migrateContentIdentifier
-- Migrate from old version of database, which had
-- an incorrect uniqueness constraint on the
-- ContentIdentifiers table.
, liftIO $ runSqlite (T.pack (fromRawFilePath db)) $ void $
runMigrationSilent migrateContentIdentifier
)
h <- liftIO $ H.openDbQueue H.SingleWriter db "content_identifiers" h <- liftIO $ H.openDbQueue H.SingleWriter db "content_identifiers"
return $ ContentIdentifierHandle h return $ ContentIdentifierHandle h

7
doc/bugs/borg_uses_non_unique_contentidentifier_which_gets_lost.mdwn
View file

@ -1,6 +1,6 @@
borg uses a non-unique ContentIdentifier ("") for everything. borg uses a non-unique ContentIdentifier ("") for everything.
I think this is why, it eventually gets lost from the sqlite database, I think this is why, it eventually gets lost from the sqlite database for
preventing retrieval of content from the remote. some keys, preventing retrieval of content from the remote.
Repositories affected by this problem can be fixed up by just: Repositories affected by this problem can be fixed up by just:
`rm -rf .git/annex/cidsdb` `rm -rf .git/annex/cidsdb`
@ -24,4 +24,5 @@ And if a remote uses a hash for generating ContentIdentifiers, two different
Key can have the same content in edge cases. Key can have the same content in edge cases.
So, need to upgrade the database, removing this constraint from it. So, need to upgrade the database, removing this constraint from it.
--[[Joey]]
>> [[done]] --[[Joey]]