mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
git-annex/Upgrade.hs

117 lines
3.2 KiB
Haskell
Raw Normal View History

move stuff out of Core
2010-11-14 18:44:24 +00:00
{- git-annex upgrade support
-
fix problems with upgrade of local remotes Upgrade other repos than the current one by running git-annex upgrade inside them, which avoids problems with upgrade code making assumptions that the cwd will be inside the repo being upgraded. In particular, this fixes a problem where upgrading a v7 repo to v8 caused an ugly git error message. I actually could not find a way to make Upgrade.V7 work properly without changing directory to the remote. Once I got git ls-files to work, the git cat-file failed because :path can only be used in the current git repo.
2020-03-09 20:47:57 +00:00
- Copyright 2010-2020 Joey Hess <id@joeyh.name>
move stuff out of Core
2010-11-14 18:44:24 +00:00
-
update licenses from GPL to AGPL This does not change the overall license of the git-annex program, which was already AGPL due to a number of sources files being AGPL already. Legally speaking, I'm adding a new license under which these files are now available; I already released their current contents under the GPL license. Now they're dual licensed GPL and AGPL. However, I intend for all my future changes to these files to only be released under the AGPL license, and I won't be tracking the dual licensing status, so I'm simply changing the license statement to say it's AGPL. (In some cases, others wrote parts of the code of a file and released it under the GPL; but in all cases I have contributed a significant portion of the code in each file and it's that code that is getting the AGPL license; the GPL license of other contributors allows combining with AGPL code.)
2019-03-13 19:48:14 +00:00
- Licensed under the GNU AGPL version 3 or higher.
move stuff out of Core
2010-11-14 18:44:24 +00:00
-}
git-annex now builds on Windows (doesn't work)
2013-05-11 20:03:00 +00:00
{-# LANGUAGE CPP #-}
move stuff out of Core
2010-11-14 18:44:24 +00:00
module Upgrade where
remove 163 lines of code without changing anything except imports
2016-01-20 20:36:33 +00:00
import Annex.Common
Added annex.autoupgraderepository configuration Can be set to false to prevent any automatic repository upgrades. Also, removed direct mode specific upgrade code in Annex.Init, and made needsUpgrade always include the name/path of the repo, so if there's a problem it's clear what repo has the problem. And, made needsUpgrade catch any exceptions that might occur during the upgrade, so it can display a more useful error message than just the exception.
2019-09-01 17:29:55 +00:00
import qualified Annex
import qualified Git
fix problems with upgrade of local remotes Upgrade other repos than the current one by running git-annex upgrade inside them, which avoids problems with upgrade code making assumptions that the cwd will be inside the repo being upgraded. In particular, this fixes a problem where upgrading a v7 repo to v8 caused an ugly git error message. I actually could not find a way to make Upgrade.V7 work properly without changing directory to the remote. Once I got git ls-files to work, the git cat-file failed because :path can only be used in the current git repo.
2020-03-09 20:47:57 +00:00
import Config
use programPath consistently, not readProgramFile Improve git-annex's ability to find the path to its program, especially when it needs to run itself in another repo to upgrade it. Some parts of the code used readProgramFile, probably because I forgot that programPath exists. I noticed this when a git-annex auto-upgrade failed because it was running git-annex upgrade --autoonly, but the code to run git-annex used readProgramFile, which happened to point to an older build of git-annex.
2020-03-30 20:03:44 +00:00
import Annex.Path
rename
2011-10-04 04:40:47 +00:00
import Annex.Version
add runsGitAnnexChildProcess calls This is all the calls to git-annex that seem capable of possibly locking the same pidlock as their parent. Except possibly for some in the assistant.
2020-06-17 19:31:03 +00:00
import Annex.GitOverlay
v7 Install new git hooks in this version. This does beg the question of what to do if git later gets eg a post-smudge hook, that could run git-annex smudge --update. I think the thing to do in that case would be to make git-annex smudge --update install the new hooks. That way, as the user uses git-annex, the hook would be created pretty quickly and without needing any extra syscalls except for when git-annex smudge --update is called. I considered doing something like that for installation of the post-checkout and post-merge hooks, which would have avoided the need for v7. But the only place it was cheap to do it would be in git-annex smudge which could cheaply notice that smudge.log didn't exist yet and so know the hooks needed to be installed. But since smudge used to populate pointer files, it would be quite surprising if a single git checkout/merge failed to update the work tree, and so that idea didn't work out. The other reason for v7 is psychological -- users don't need to worry about whether they might be running an old version of git-annex that doesn't support their v7 repository very well. And bug reports about "v6" have gotten a bit of a bad association in my head since they often hit one of the known limitations and didn't realize it was experimental. newtyped RepoVersion Int to avoid needing 2 comparisons in versionSupportsUnlockedPointers etc. Also it's just nicer. This commit was sponsored by John Pellman on Patreon.
2018-10-25 21:23:53 +00:00
import Types.RepoVersion
missed some __WINDOWS__ defines
2013-08-04 17:07:55 +00:00
#ifndef mingw32_HOST_OS
upgrade groundwork pulled in old versions of functions for working with keys Wrote a parser from old key filenames to new keys.
2011-03-16 05:23:20 +00:00
import qualified Upgrade.V0
import qualified Upgrade.V1
git-annex now builds on Windows (doesn't work)
2013-05-11 20:03:00 +00:00
#endif
stub in v2 upgrade
2011-06-22 21:51:48 +00:00
import qualified Upgrade.V2
Auto-upgrade v3 indirect repos to v5 with no changes. This also fixes a problem when a direct mode repo was somehow set to v3 rather than v4, and so the automatic direct mode upgrade to v5 was not done.
2013-12-29 17:06:23 +00:00
import qualified Upgrade.V3
v5 for direct mode, with automatic upgrade This includes storing the current state of the HEAD ref, which git annex sync is going to need, but does not make sync use it.
2013-11-05 20:42:59 +00:00
import qualified Upgrade.V4
add v6; keep v5 working for now and manual upgrade Since all places where a repo is used in direct mode need to have git-annex upgraded before the repo can safely be converted to v6, the upgrade needs to be manual for now. I suppose that at some point I'll want to drop all the direct mode support code. At that point, will stop supporting v5, and will need to auto-upgrade any remaining v5 repos. If possible, I'd like to carry the direct mode support for say, a year or so, to give people plenty of time to upgrade and avoid disruption.
2015-12-04 20:14:48 +00:00
import qualified Upgrade.V5
v7 Install new git hooks in this version. This does beg the question of what to do if git later gets eg a post-smudge hook, that could run git-annex smudge --update. I think the thing to do in that case would be to make git-annex smudge --update install the new hooks. That way, as the user uses git-annex, the hook would be created pretty quickly and without needing any extra syscalls except for when git-annex smudge --update is called. I considered doing something like that for installation of the post-checkout and post-merge hooks, which would have avoided the need for v7. But the only place it was cheap to do it would be in git-annex smudge which could cheaply notice that smudge.log didn't exist yet and so know the hooks needed to be installed. But since smudge used to populate pointer files, it would be quite surprising if a single git checkout/merge failed to update the work tree, and so that idea didn't work out. The other reason for v7 is psychological -- users don't need to worry about whether they might be running an old version of git-annex that doesn't support their v7 repository very well. And bug reports about "v6" have gotten a bit of a bad association in my head since they often hit one of the known limitations and didn't realize it was experimental. newtyped RepoVersion Int to avoid needing 2 comparisons in versionSupportsUnlockedPointers etc. Also it's just nicer. This commit was sponsored by John Pellman on Patreon.
2018-10-25 21:23:53 +00:00
import qualified Upgrade.V6
v8 upgrade of keys db Renamed the database to .git/annex/keysdb; the old .git/annex/keys gets deleted during the upgrade. It is possible that an old git-annex process is running during the upgrade. If so, it will be able to continue using the old keys db until the upgrade is complete, and then will presumably fail in some ugly way. Or perhaps the upgrade will be unable to delete the open files on some systems, and so fail with an ugly error message. It's also possible for multiple processes to be running the upgrade concurrently. That should be fine; they will both write the same information into the keys db. Other databases still need to be upgraded.
2019-11-06 19:37:18 +00:00
import qualified Upgrade.V7
move stuff out of Core
2010-11-14 18:44:24 +00:00
fix autoupgrade from v6 to go to v7, not v5 v3 and v4 still autoupgrade to v5 And a few more upgrade doc updates.
2018-10-25 22:33:34 +00:00
import qualified Data.Map as M
v7 Install new git hooks in this version. This does beg the question of what to do if git later gets eg a post-smudge hook, that could run git-annex smudge --update. I think the thing to do in that case would be to make git-annex smudge --update install the new hooks. That way, as the user uses git-annex, the hook would be created pretty quickly and without needing any extra syscalls except for when git-annex smudge --update is called. I considered doing something like that for installation of the post-checkout and post-merge hooks, which would have avoided the need for v7. But the only place it was cheap to do it would be in git-annex smudge which could cheaply notice that smudge.log didn't exist yet and so know the hooks needed to be installed. But since smudge used to populate pointer files, it would be quite surprising if a single git checkout/merge failed to update the work tree, and so that idea didn't work out. The other reason for v7 is psychological -- users don't need to worry about whether they might be running an old version of git-annex that doesn't support their v7 repository very well. And bug reports about "v6" have gotten a bit of a bad association in my head since they often hit one of the known limitations and didn't realize it was experimental. newtyped RepoVersion Int to avoid needing 2 comparisons in versionSupportsUnlockedPointers etc. Also it's just nicer. This commit was sponsored by John Pellman on Patreon.
2018-10-25 21:23:53 +00:00
checkUpgrade :: RepoVersion -> Annex ()
Avoid backtraces on expected failures when built with ghc 8; only use backtraces for unexpected errors. ghc 8 added backtraces on uncaught errors. This is great, but git-annex was using error in many places for a error message targeted at the user, in some known problem case. A backtrace only confuses such a message, so omit it. Notably, commands like git annex drop that failed due to eg, numcopies, used to use error, so had a backtrace. This commit was sponsored by Ethan Aubin.
2016-11-16 01:29:54 +00:00
checkUpgrade = maybe noop giveup <=< needsUpgrade
webapp: Check annex.version.
2013-11-17 18:58:35 +00:00
v7 Install new git hooks in this version. This does beg the question of what to do if git later gets eg a post-smudge hook, that could run git-annex smudge --update. I think the thing to do in that case would be to make git-annex smudge --update install the new hooks. That way, as the user uses git-annex, the hook would be created pretty quickly and without needing any extra syscalls except for when git-annex smudge --update is called. I considered doing something like that for installation of the post-checkout and post-merge hooks, which would have avoided the need for v7. But the only place it was cheap to do it would be in git-annex smudge which could cheaply notice that smudge.log didn't exist yet and so know the hooks needed to be installed. But since smudge used to populate pointer files, it would be quite surprising if a single git checkout/merge failed to update the work tree, and so that idea didn't work out. The other reason for v7 is psychological -- users don't need to worry about whether they might be running an old version of git-annex that doesn't support their v7 repository very well. And bug reports about "v6" have gotten a bit of a bad association in my head since they often hit one of the known limitations and didn't realize it was experimental. newtyped RepoVersion Int to avoid needing 2 comparisons in versionSupportsUnlockedPointers etc. Also it's just nicer. This commit was sponsored by John Pellman on Patreon.
2018-10-25 21:23:53 +00:00
needsUpgrade :: RepoVersion -> Annex (Maybe String)
webapp: Check annex.version.
2013-11-17 18:58:35 +00:00
needsUpgrade v
add v6; keep v5 working for now and manual upgrade Since all places where a repo is used in direct mode need to have git-annex upgraded before the repo can safely be converted to v6, the upgrade needs to be manual for now. I suppose that at some point I'll want to drop all the direct mode support code. At that point, will stop supporting v5, and will need to auto-upgrade any remaining v5 repos. If possible, I'd like to carry the direct mode support for say, a year or so, to give people plenty of time to upgrade and avoid disruption.
2015-12-04 20:14:48 +00:00
| v `elem` supportedVersions = ok
fix autoupgrade from v6 to go to v7, not v5 v3 and v4 still autoupgrade to v5 And a few more upgrade doc updates.
2018-10-25 22:33:34 +00:00
| otherwise = case M.lookup v autoUpgradeableVersions of
Nothing
| v `elem` upgradableVersions ->
err "Upgrade this repository: git-annex upgrade"
| otherwise ->
err "Upgrade git-annex."
Added annex.autoupgraderepository configuration Can be set to false to prevent any automatic repository upgrades. Also, removed direct mode specific upgrade code in Annex.Init, and made needsUpgrade always include the name/path of the repo, so if there's a problem it's clear what repo has the problem. And, made needsUpgrade catch any exceptions that might occur during the upgrade, so it can display a more useful error message than just the exception.
2019-09-01 17:29:55 +00:00
Just newv -> ifM (annexAutoUpgradeRepository <$> Annex.getGitConfig)
( tryNonAsync (upgrade True newv) >>= \case
Right True -> ok
fix build
2019-09-01 20:42:49 +00:00
Right False -> err "Automatic upgrade failed!"
Left ex -> err $ "Automatic upgrade exception! " ++ show ex
Added annex.autoupgraderepository configuration Can be set to false to prevent any automatic repository upgrades. Also, removed direct mode specific upgrade code in Annex.Init, and made needsUpgrade always include the name/path of the repo, so if there's a problem it's clear what repo has the problem. And, made needsUpgrade catch any exceptions that might occur during the upgrade, so it can display a more useful error message than just the exception.
2019-09-01 17:29:55 +00:00
, err "Automatic upgrade is disabled by annex.autoupgraderepository configuration. To upgrade this repository: git-annex upgrade"
fix autoupgrade from v6 to go to v7, not v5 v3 and v4 still autoupgrade to v5 And a few more upgrade doc updates.
2018-10-25 22:33:34 +00:00
)
v5 for direct mode, with automatic upgrade This includes storing the current state of the HEAD ref, which git annex sync is going to need, but does not make sync use it.
2013-11-05 20:42:59 +00:00
where
Added annex.autoupgraderepository configuration Can be set to false to prevent any automatic repository upgrades. Also, removed direct mode specific upgrade code in Annex.Init, and made needsUpgrade always include the name/path of the repo, so if there's a problem it's clear what repo has the problem. And, made needsUpgrade catch any exceptions that might occur during the upgrade, so it can display a more useful error message than just the exception.
2019-09-01 17:29:55 +00:00
err msg = do
g <- Annex.gitRepo
convert TopFilePath to use RawFilePath Adds a dependency on filepath-bytestring, an as yet unreleased fork of filepath that operates on RawFilePath. Git.Repo also changed to use RawFilePath for the path to the repo. This does eliminate some RawFilePath -> FilePath -> RawFilePath conversions. And filepath-bytestring's </> is probably faster. But I don't expect a major performance improvement from this. This is mostly groundwork for making Annex.Location use RawFilePath, which will allow for a conversion-free pipleline.
2019-12-09 17:49:05 +00:00
p <- liftIO $ absPath $ fromRawFilePath $ Git.repoPath g
Added annex.autoupgraderepository configuration Can be set to false to prevent any automatic repository upgrades. Also, removed direct mode specific upgrade code in Annex.Init, and made needsUpgrade always include the name/path of the repo, so if there's a problem it's clear what repo has the problem. And, made needsUpgrade catch any exceptions that might occur during the upgrade, so it can display a more useful error message than just the exception.
2019-09-01 17:29:55 +00:00
return $ Just $ unwords
[ "Repository", p
, "is at unsupported version"
, show (fromRepoVersion v) ++ "."
, msg
]
webapp: Check annex.version.
2013-11-17 18:58:35 +00:00
ok = return Nothing
v5 for direct mode, with automatic upgrade This includes storing the current state of the HEAD ref, which git annex sync is going to need, but does not make sync use it.
2013-11-05 20:42:59 +00:00
v7 Install new git hooks in this version. This does beg the question of what to do if git later gets eg a post-smudge hook, that could run git-annex smudge --update. I think the thing to do in that case would be to make git-annex smudge --update install the new hooks. That way, as the user uses git-annex, the hook would be created pretty quickly and without needing any extra syscalls except for when git-annex smudge --update is called. I considered doing something like that for installation of the post-checkout and post-merge hooks, which would have avoided the need for v7. But the only place it was cheap to do it would be in git-annex smudge which could cheaply notice that smudge.log didn't exist yet and so know the hooks needed to be installed. But since smudge used to populate pointer files, it would be quite surprising if a single git checkout/merge failed to update the work tree, and so that idea didn't work out. The other reason for v7 is psychological -- users don't need to worry about whether they might be running an old version of git-annex that doesn't support their v7 repository very well. And bug reports about "v6" have gotten a bit of a bad association in my head since they often hit one of the known limitations and didn't realize it was experimental. newtyped RepoVersion Int to avoid needing 2 comparisons in versionSupportsUnlockedPointers etc. Also it's just nicer. This commit was sponsored by John Pellman on Patreon.
2018-10-25 21:23:53 +00:00
upgrade :: Bool -> RepoVersion -> Annex Bool
When auto-upgrading a v3 remote, avoid upgrading to version 6, instead keep it at version 5. Fixes a bug introduced with v6 mode that I didn't notice until now. Probably not many v3 repos left out there, and upgrading them to v6 mode is not disastrous, only a little premature. This commit was sponsored by Riku Voipio
2016-10-05 20:23:09 +00:00
upgrade automatic destversion = do
Bug fix: annex.version did not get set on automatic upgrade to v5 direct mode repo, so the upgrade was performed repeatedly, slowing commands down.
2013-11-26 16:54:50 +00:00
upgraded <- go =<< getVersion
fix problems with upgrade of local remotes Upgrade other repos than the current one by running git-annex upgrade inside them, which avoids problems with upgrade code making assumptions that the cwd will be inside the repo being upgraded. In particular, this fixes a problem where upgrading a v7 repo to v8 caused an ugly git error message. I actually could not find a way to make Upgrade.V7 work properly without changing directory to the remote. Once I got git ls-files to work, the git cat-file failed because :path can only be used in the current git repo.
2020-03-09 20:47:57 +00:00
when upgraded
postupgrade
Bug fix: annex.version did not get set on automatic upgrade to v5 direct mode repo, so the upgrade was performed repeatedly, slowing commands down.
2013-11-26 16:54:50 +00:00
return upgraded
where indenting
2012-11-11 04:51:07 +00:00
where
fix upgrade of more than 1 version at a time It was only running the first upgrade and not subsequent ones, but setting annex.version to the final version.
2018-10-26 18:29:52 +00:00
go (Just v)
| v >= destversion = return True
fix problems with upgrade of local remotes Upgrade other repos than the current one by running git-annex upgrade inside them, which avoids problems with upgrade code making assumptions that the cwd will be inside the repo being upgraded. In particular, this fixes a problem where upgrading a v7 repo to v8 caused an ugly git error message. I actually could not find a way to make Upgrade.V7 work properly without changing directory to the remote. Once I got git ls-files to work, the git cat-file failed because :path can only be used in the current git repo.
2020-03-09 20:47:57 +00:00
| otherwise = ifM upgradingRemote
( upgraderemote
, ifM (up v)
( go (Just (RepoVersion (fromRepoVersion v + 1)))
, return False
)
fix upgrade of more than 1 version at a time It was only running the first upgrade and not subsequent ones, but setting annex.version to the final version.
2018-10-26 18:29:52 +00:00
)
go _ = return True
fix problems with upgrade of local remotes Upgrade other repos than the current one by running git-annex upgrade inside them, which avoids problems with upgrade code making assumptions that the cwd will be inside the repo being upgraded. In particular, this fixes a problem where upgrading a v7 repo to v8 caused an ugly git error message. I actually could not find a way to make Upgrade.V7 work properly without changing directory to the remote. Once I got git ls-files to work, the git cat-file failed because :path can only be used in the current git repo.
2020-03-09 20:47:57 +00:00
postupgrade = ifM upgradingRemote
( reloadConfig
, setVersion destversion
)
missed some __WINDOWS__ defines
2013-08-04 17:07:55 +00:00
#ifndef mingw32_HOST_OS
fix upgrade of more than 1 version at a time It was only running the first upgrade and not subsequent ones, but setting annex.version to the final version.
2018-10-26 18:29:52 +00:00
up (RepoVersion 0) = Upgrade.V0.upgrade
up (RepoVersion 1) = Upgrade.V1.upgrade
git-annex now builds on Windows (doesn't work)
2013-05-11 20:03:00 +00:00
#else
fix upgrade of more than 1 version at a time It was only running the first upgrade and not subsequent ones, but setting annex.version to the final version.
2018-10-26 18:29:52 +00:00
up (RepoVersion 0) = giveup "upgrade from v0 on Windows not supported"
up (RepoVersion 1) = giveup "upgrade from v1 on Windows not supported"
git-annex now builds on Windows (doesn't work)
2013-05-11 20:03:00 +00:00
#endif
fix upgrade of more than 1 version at a time It was only running the first upgrade and not subsequent ones, but setting annex.version to the final version.
2018-10-26 18:29:52 +00:00
up (RepoVersion 2) = Upgrade.V2.upgrade
up (RepoVersion 3) = Upgrade.V3.upgrade automatic
up (RepoVersion 4) = Upgrade.V4.upgrade automatic
up (RepoVersion 5) = Upgrade.V5.upgrade automatic
up (RepoVersion 6) = Upgrade.V6.upgrade automatic
v8 upgrade of keys db Renamed the database to .git/annex/keysdb; the old .git/annex/keys gets deleted during the upgrade. It is possible that an old git-annex process is running during the upgrade. If so, it will be able to continue using the old keys db until the upgrade is complete, and then will presumably fail in some ugly way. Or perhaps the upgrade will be unable to delete the open files on some systems, and so fail with an ugly error message. It's also possible for multiple processes to be running the upgrade concurrently. That should be fine; they will both write the same information into the keys db. Other databases still need to be upgraded.
2019-11-06 19:37:18 +00:00
up (RepoVersion 7) = Upgrade.V7.upgrade automatic
fix upgrade of more than 1 version at a time It was only running the first upgrade and not subsequent ones, but setting annex.version to the final version.
2018-10-26 18:29:52 +00:00
up _ = return True
fix problems with upgrade of local remotes Upgrade other repos than the current one by running git-annex upgrade inside them, which avoids problems with upgrade code making assumptions that the cwd will be inside the repo being upgraded. In particular, this fixes a problem where upgrading a v7 repo to v8 caused an ugly git error message. I actually could not find a way to make Upgrade.V7 work properly without changing directory to the remote. Once I got git ls-files to work, the git cat-file failed because :path can only be used in the current git repo.
2020-03-09 20:47:57 +00:00
-- Upgrade local remotes by running git-annex upgrade in them.
-- This avoids complicating the upgrade code by needing to handle
-- upgrading a git repo other than the current repo.
upgraderemote = do
rp <- fromRawFilePath <$> fromRepo Git.repoPath
use programPath consistently, not readProgramFile Improve git-annex's ability to find the path to its program, especially when it needs to run itself in another repo to upgrade it. Some parts of the code used readProgramFile, probably because I forgot that programPath exists. I noticed this when a git-annex auto-upgrade failed because it was running git-annex upgrade --autoonly, but the code to run git-annex used readProgramFile, which happened to point to an older build of git-annex.
2020-03-30 20:03:44 +00:00
cmd <- liftIO programPath
add runsGitAnnexChildProcess calls This is all the calls to git-annex that seem capable of possibly locking the same pidlock as their parent. Except possibly for some in the assistant.
2020-06-17 19:31:03 +00:00
runsGitAnnexChildProcess $ liftIO $ boolSystem' cmd
fix problems with upgrade of local remotes Upgrade other repos than the current one by running git-annex upgrade inside them, which avoids problems with upgrade code making assumptions that the cwd will be inside the repo being upgraded. In particular, this fixes a problem where upgrading a v7 repo to v8 caused an ugly git error message. I actually could not find a way to make Upgrade.V7 work properly without changing directory to the remote. Once I got git ls-files to work, the git cat-file failed because :path can only be used in the current git repo.
2020-03-09 20:47:57 +00:00
[ Param "upgrade"
, Param "--quiet"
, Param "--autoonly"
]
(\p -> p { cwd = Just rp })
upgradingRemote :: Annex Bool
upgradingRemote = isJust <$> fromRepo Git.remoteName
Reference in a new issue Copy permalink