mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
git-annex/Git/FilePath.hs

98 lines
2.8 KiB
Haskell
Raw Normal View History

add support for staging other types of blobs, like symlinks, into the index Also added a utility TopFilePath type, which could stand to be used more widely.
2012-06-06 18:26:15 +00:00
{- git FilePath library
-
- Different git commands use different types of FilePaths to refer to
- files in the repository. Some commands use paths relative to the
- top of the repository even when run in a subdirectory. Adding some
- types helps keep that straight.
-
git style quoting for ActionItemOther Added StringContainingQuotedPath, which is used for ActionItemOther. In the process, checked every ActionItemOther for those containing filenames, and made them use quoting. Sponsored-by: Graham Spencer on Patreon
2023-04-08 19:48:32 +00:00
- Copyright 2012-2023 Joey Hess <id@joeyh.name>
add support for staging other types of blobs, like symlinks, into the index Also added a utility TopFilePath type, which could stand to be used more widely.
2012-06-06 18:26:15 +00:00
-
update licenses from GPL to AGPL This does not change the overall license of the git-annex program, which was already AGPL due to a number of sources files being AGPL already. Legally speaking, I'm adding a new license under which these files are now available; I already released their current contents under the GPL license. Now they're dual licensed GPL and AGPL. However, I intend for all my future changes to these files to only be released under the AGPL license, and I won't be tracking the dual licensing status, so I'm simply changing the license statement to say it's AGPL. (In some cases, others wrote parts of the code of a file and released it under the GPL; but in all cases I have contributed a significant portion of the code in each file and it's that code that is getting the AGPL license; the GPL license of other contributors allows combining with AGPL code.)
2019-03-13 19:48:14 +00:00
- Licensed under the GNU AGPL version 3 or higher.
add support for staging other types of blobs, like symlinks, into the index Also added a utility TopFilePath type, which could stand to be used more widely.
2012-06-06 18:26:15 +00:00
-}
deal with git using / internally, even on DOS
2013-05-12 22:18:48 +00:00
{-# LANGUAGE CPP #-}
benchmark: Add --databases to benchmark sqlite databases Rescued from commit 11d6e2e260d70ba99e35464c19c2b2772ce9efaa which removed db benchmarks in favor of benchmarking arbitrary git-annex commands. Which is nice and general, but microbenchmarks are useful too.
2019-10-29 19:16:15 +00:00
{-# LANGUAGE DeriveGeneric #-}
wip RawFilePath 2x git-annex find speedup Finally builds (oh the agoncy of making it build), but still very unmergable, only Command.Find is included and lots of stuff is badly hacked to make it compile. Benchmarking vs master, this git-annex find is significantly faster! Specifically: num files old new speedup 48500 4.77 3.73 28% 12500 1.36 1.02 66% 20 0.075 0.074 0% (so startup time is unchanged) That's without really finishing the optimization. Things still to do: * Eliminate all the fromRawFilePath, toRawFilePath, encodeBS, decodeBS conversions. * Use versions of IO actions like getFileStatus that take a RawFilePath. * Eliminate some Data.ByteString.Lazy.toStrict, which is a slow copy. * Use ByteString for parsing git config to speed up startup. It's likely several of those will speed up git-annex find further. And other commands will certianly benefit even more.
2019-11-26 19:27:22 +00:00
{-# LANGUAGE OverloadedStrings #-}
deal with git using / internally, even on DOS
2013-05-12 22:18:48 +00:00
add support for staging other types of blobs, like symlinks, into the index Also added a utility TopFilePath type, which could stand to be used more widely.
2012-06-06 18:26:15 +00:00
module Git.FilePath (
clarify absPathFrom The repo path is typically relative, not absolute, so providing it to absPathFrom doesn't yield an absolute path. This is not a bug, just unclear documentation. Indeed, there seem to be no reason to simplifyPath here, which absPathFrom does, so instead just combine the repo path and the TopFilePath. Also, removed an export of the TopFilePath constructor; asTopFilePath is provided to construct one as-is.
2016-01-05 21:33:48 +00:00
TopFilePath,
--branch, stage 2 Show branch:file that is being operated on. I had to make ActionItem a type and not a type class because withKeyOptions' passed two different types of values when using the type class, and I could not get the type checker to accept that.
2016-07-20 19:22:55 +00:00
BranchFilePath(..),
descBranchFilePath,
clarify absPathFrom The repo path is typically relative, not absolute, so providing it to absPathFrom doesn't yield an absolute path. This is not a bug, just unclear documentation. Indeed, there seem to be no reason to simplifyPath here, which absPathFrom does, so instead just combine the repo path and the TopFilePath. Also, removed an export of the TopFilePath constructor; asTopFilePath is provided to construct one as-is.
2016-01-05 21:33:48 +00:00
getTopFilePath,
use TopFilePath for DiffTree and LsTree
2013-10-17 18:51:19 +00:00
fromTopFilePath,
add support for staging other types of blobs, like symlinks, into the index Also added a utility TopFilePath type, which could stand to be used more widely.
2012-06-06 18:26:15 +00:00
toTopFilePath,
asTopFilePath,
deal with git using / internally, even on DOS
2013-05-12 22:18:48 +00:00
InternalGitPath,
toInternalGitPath,
Windows: Fix handling of absolute unix-style git repository paths. Note that on Windows a remote with a path like /home/foo/bar is interpreted by git as being some screwy relative path (relative to what exactly seems ill-defined -- it seemed relative to C:\Program Files\Git\ in my tests!) So no attempt has been made to handle such a path sanely, just not to crash when encountering it. Note that "C:\\foo" </> "/home/foo/bar" yields /home/foo/bar even though that is not absolute! I don't know what to make of all this, except that I will be very happy when this crock of **** vanishes from the face of the earth.
2014-02-08 19:31:03 +00:00
fromInternalGitPath,
absoluteGitPath
add support for staging other types of blobs, like symlinks, into the index Also added a utility TopFilePath type, which could stand to be used more widely.
2012-06-06 18:26:15 +00:00
) where
import Common
import Git
git style filename quoting controlled by core.quotePath This is by no means complete, but escaping filenames in actionItemDesc does cover most commands. Note that for ActionItemBranchFilePath, the value is branch:file, and I choose to only quote the file part (if necessary). I considered quoting the whole thing. But, branch names cannot contain control characters, and while they can contain unicode, git coes not quote unicode when displaying branch names. So, it would be surprising for git-annex to quote unicode in a branch name. The find command is the most obvious command that still needs to be dealt with. There are probably other places that filenames also get displayed, eg embedded in error messages. Some other commands use ActionItemOther with a filename, I think that ActionItemOther should either be pre-sanitized, or should explicitly not be used for filenames, so that needs more work. When --json is used, unicode does not get escaped, but control characters were already escaped in json. (Key escaping may turn out to be needed, but I'm ignoring that for now.) Sponsored-by: unqueued on Patreon
2023-04-08 18:20:02 +00:00
import qualified Git.Filename as Filename
add support for staging other types of blobs, like symlinks, into the index Also added a utility TopFilePath type, which could stand to be used more widely.
2012-06-06 18:26:15 +00:00
convert TopFilePath to use RawFilePath Adds a dependency on filepath-bytestring, an as yet unreleased fork of filepath that operates on RawFilePath. Git.Repo also changed to use RawFilePath for the path to the repo. This does eliminate some RawFilePath -> FilePath -> RawFilePath conversions. And filepath-bytestring's </> is probably faster. But I don't expect a major performance improvement from this. This is mostly groundwork for making Annex.Location use RawFilePath, which will allow for a conversion-free pipleline.
2019-12-09 17:49:05 +00:00
import qualified System.FilePath.ByteString as P
import qualified System.FilePath.Posix.ByteString
benchmark: Add --databases to benchmark sqlite databases Rescued from commit 11d6e2e260d70ba99e35464c19c2b2772ce9efaa which removed db benchmarks in favor of benchmarking arbitrary git-annex commands. Which is nice and general, but microbenchmarks are useful too.
2019-10-29 19:16:15 +00:00
import GHC.Generics
import Control.DeepSeq
wip RawFilePath 2x git-annex find speedup Finally builds (oh the agoncy of making it build), but still very unmergable, only Command.Find is included and lots of stuff is badly hacked to make it compile. Benchmarking vs master, this git-annex find is significantly faster! Specifically: num files old new speedup 48500 4.77 3.73 28% 12500 1.36 1.02 66% 20 0.075 0.074 0% (so startup time is unchanged) That's without really finishing the optimization. Things still to do: * Eliminate all the fromRawFilePath, toRawFilePath, encodeBS, decodeBS conversions. * Use versions of IO actions like getFileStatus that take a RawFilePath. * Eliminate some Data.ByteString.Lazy.toStrict, which is a slow copy. * Use ByteString for parsing git config to speed up startup. It's likely several of those will speed up git-annex find further. And other commands will certianly benefit even more.
2019-11-26 19:27:22 +00:00
import qualified Data.ByteString as S
Windows: Fix handling of absolute unix-style git repository paths. Note that on Windows a remote with a path like /home/foo/bar is interpreted by git as being some screwy relative path (relative to what exactly seems ill-defined -- it seemed relative to C:\Program Files\Git\ in my tests!) So no attempt has been made to handle such a path sanely, just not to crash when encountering it. Note that "C:\\foo" </> "/home/foo/bar" yields /home/foo/bar even though that is not absolute! I don't know what to make of all this, except that I will be very happy when this crock of **** vanishes from the face of the earth.
2014-02-08 19:31:03 +00:00
wip RawFilePath Goal is to make git-annex faster by using ByteString for all the worktree traversal. For now, this is focusing on Command.Find, in order to benchmark how much it helps. (All other commands are temporarily disabled) Currently in a very bad unbuildable in-between state.
2019-11-25 20:18:19 +00:00
{- A RawFilePath, relative to the top of the git repository. -}
convert TopFilePath to use RawFilePath Adds a dependency on filepath-bytestring, an as yet unreleased fork of filepath that operates on RawFilePath. Git.Repo also changed to use RawFilePath for the path to the repo. This does eliminate some RawFilePath -> FilePath -> RawFilePath conversions. And filepath-bytestring's </> is probably faster. But I don't expect a major performance improvement from this. This is mostly groundwork for making Annex.Location use RawFilePath, which will allow for a conversion-free pipleline.
2019-12-09 17:49:05 +00:00
newtype TopFilePath = TopFilePath { getTopFilePath :: RawFilePath }
benchmark: Add --databases to benchmark sqlite databases Rescued from commit 11d6e2e260d70ba99e35464c19c2b2772ce9efaa which removed db benchmarks in favor of benchmarking arbitrary git-annex commands. Which is nice and general, but microbenchmarks are useful too.
2019-10-29 19:16:15 +00:00
deriving (Show, Eq, Ord, Generic)
instance NFData TopFilePath
use TopFilePath for DiffTree and LsTree
2013-10-17 18:51:19 +00:00
--branch, stage 2 Show branch:file that is being operated on. I had to make ActionItem a type and not a type class because withKeyOptions' passed two different types of values when using the type class, and I could not get the type checker to accept that.
2016-07-20 19:22:55 +00:00
{- A file in a branch or other treeish. -}
data BranchFilePath = BranchFilePath Ref TopFilePath
finish CommandStart transition The hoped for optimisation of CommandStart with -J did not materialize. In fact, not runnign CommandStart in parallel is slower than -J3. So, CommandStart are still run in parallel. (The actual bad performance I've been seeing with -J in my big repo has to do with building the remoteList.) But, this is still progress toward making -J faster, because it gets rid of the onlyActionOn roadblock in the way of making CommandCleanup jobs run separate from CommandPerform jobs. Added OnlyActionOn constructor for ActionItem which fixes the onlyActionOn breakage in the last commit. Made CustomOutput include an ActionItem, so even things using it can specify OnlyActionOn. In Command.Move and Command.Sync, there were CommandStarts that used includeCommandAction, so output messages, which is no longer allowed. Fixed by using startingCustomOutput, but that's still not quite right, since it prevents message display for the includeCommandAction run inside it too.
2019-06-12 13:23:26 +00:00
deriving (Show, Eq, Ord)
--branch, stage 2 Show branch:file that is being operated on. I had to make ActionItem a type and not a type class because withKeyOptions' passed two different types of values when using the type class, and I could not get the type checker to accept that.
2016-07-20 19:22:55 +00:00
{- Git uses the branch:file form to refer to a BranchFilePath -}
git style filename quoting controlled by core.quotePath This is by no means complete, but escaping filenames in actionItemDesc does cover most commands. Note that for ActionItemBranchFilePath, the value is branch:file, and I choose to only quote the file part (if necessary). I considered quoting the whole thing. But, branch names cannot contain control characters, and while they can contain unicode, git coes not quote unicode when displaying branch names. So, it would be surprising for git-annex to quote unicode in a branch name. The find command is the most obvious command that still needs to be dealt with. There are probably other places that filenames also get displayed, eg embedded in error messages. Some other commands use ActionItemOther with a filename, I think that ActionItemOther should either be pre-sanitized, or should explicitly not be used for filenames, so that needs more work. When --json is used, unicode does not get escaped, but control characters were already escaped in json. (Key escaping may turn out to be needed, but I'm ignoring that for now.) Sponsored-by: unqueued on Patreon
2023-04-08 18:20:02 +00:00
descBranchFilePath :: Filename.QuotePath -> BranchFilePath -> S.ByteString
descBranchFilePath qp (BranchFilePath b f) =
git style quoting for ActionItemOther Added StringContainingQuotedPath, which is used for ActionItemOther. In the process, checked every ActionItemOther for those containing filenames, and made them use quoting. Sponsored-by: Graham Spencer on Patreon
2023-04-08 19:48:32 +00:00
fromRef' b <> ":" <> Filename.quote qp (getTopFilePath f)
--branch, stage 2 Show branch:file that is being operated on. I had to make ActionItem a type and not a type class because withKeyOptions' passed two different types of values when using the type class, and I could not get the type checker to accept that.
2016-07-20 19:22:55 +00:00
clarify absPathFrom The repo path is typically relative, not absolute, so providing it to absPathFrom doesn't yield an absolute path. This is not a bug, just unclear documentation. Indeed, there seem to be no reason to simplifyPath here, which absPathFrom does, so instead just combine the repo path and the TopFilePath. Also, removed an export of the TopFilePath constructor; asTopFilePath is provided to construct one as-is.
2016-01-05 21:33:48 +00:00
{- Path to a TopFilePath, within the provided git repo. -}
convert TopFilePath to use RawFilePath Adds a dependency on filepath-bytestring, an as yet unreleased fork of filepath that operates on RawFilePath. Git.Repo also changed to use RawFilePath for the path to the repo. This does eliminate some RawFilePath -> FilePath -> RawFilePath conversions. And filepath-bytestring's </> is probably faster. But I don't expect a major performance improvement from this. This is mostly groundwork for making Annex.Location use RawFilePath, which will allow for a conversion-free pipleline.
2019-12-09 17:49:05 +00:00
fromTopFilePath :: TopFilePath -> Git.Repo -> RawFilePath
fromTopFilePath p repo = P.combine (repoPath repo) (getTopFilePath p)
add support for staging other types of blobs, like symlinks, into the index Also added a utility TopFilePath type, which could stand to be used more widely.
2012-06-06 18:26:15 +00:00
{- The input FilePath can be absolute, or relative to the CWD. -}
convert TopFilePath to use RawFilePath Adds a dependency on filepath-bytestring, an as yet unreleased fork of filepath that operates on RawFilePath. Git.Repo also changed to use RawFilePath for the path to the repo. This does eliminate some RawFilePath -> FilePath -> RawFilePath conversions. And filepath-bytestring's </> is probably faster. But I don't expect a major performance improvement from this. This is mostly groundwork for making Annex.Location use RawFilePath, which will allow for a conversion-free pipleline.
2019-12-09 17:49:05 +00:00
toTopFilePath :: RawFilePath -> Git.Repo -> IO TopFilePath
more RawFilePath conversion Most of Git/ builds now. Notable win is toTopFilePath no longer double converts This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
2020-10-28 19:40:50 +00:00
toTopFilePath file repo = TopFilePath <$> relPathDirToFile (repoPath repo) file
add support for staging other types of blobs, like symlinks, into the index Also added a utility TopFilePath type, which could stand to be used more widely.
2012-06-06 18:26:15 +00:00
convert TopFilePath to use RawFilePath Adds a dependency on filepath-bytestring, an as yet unreleased fork of filepath that operates on RawFilePath. Git.Repo also changed to use RawFilePath for the path to the repo. This does eliminate some RawFilePath -> FilePath -> RawFilePath conversions. And filepath-bytestring's </> is probably faster. But I don't expect a major performance improvement from this. This is mostly groundwork for making Annex.Location use RawFilePath, which will allow for a conversion-free pipleline.
2019-12-09 17:49:05 +00:00
{- The input RawFilePath must already be relative to the top of the git
add support for staging other types of blobs, like symlinks, into the index Also added a utility TopFilePath type, which could stand to be used more widely.
2012-06-06 18:26:15 +00:00
- repository -}
convert TopFilePath to use RawFilePath Adds a dependency on filepath-bytestring, an as yet unreleased fork of filepath that operates on RawFilePath. Git.Repo also changed to use RawFilePath for the path to the repo. This does eliminate some RawFilePath -> FilePath -> RawFilePath conversions. And filepath-bytestring's </> is probably faster. But I don't expect a major performance improvement from this. This is mostly groundwork for making Annex.Location use RawFilePath, which will allow for a conversion-free pipleline.
2019-12-09 17:49:05 +00:00
asTopFilePath :: RawFilePath -> TopFilePath
add support for staging other types of blobs, like symlinks, into the index Also added a utility TopFilePath type, which could stand to be used more widely.
2012-06-06 18:26:15 +00:00
asTopFilePath file = TopFilePath file
deal with git using / internally, even on DOS
2013-05-12 22:18:48 +00:00
{- Git may use a different representation of a path when storing
work around msysgit very strange behavior on ./ or .\ at start of path Seems that verify_path() rejects such a path on Windows, but I cannot see why. Git bug?
2013-12-05 03:49:18 +00:00
- it internally.
-
- On Windows, git uses '/' to separate paths stored in the repository,
remove workaround This was needed when absNormPath was not being used on Windows, since path normalization includes removing ./
2014-02-08 18:47:57 +00:00
- despite Windows using '\'.
work around msysgit very strange behavior on ./ or .\ at start of path Seems that verify_path() rejects such a path on Windows, but I cannot see why. Git bug?
2013-12-05 03:49:18 +00:00
-
-}
wip RawFilePath Goal is to make git-annex faster by using ByteString for all the worktree traversal. For now, this is focusing on Command.Find, in order to benchmark how much it helps. (All other commands are temporarily disabled) Currently in a very bad unbuildable in-between state.
2019-11-25 20:18:19 +00:00
type InternalGitPath = RawFilePath
deal with git using / internally, even on DOS
2013-05-12 22:18:48 +00:00
wip RawFilePath Goal is to make git-annex faster by using ByteString for all the worktree traversal. For now, this is focusing on Command.Find, in order to benchmark how much it helps. (All other commands are temporarily disabled) Currently in a very bad unbuildable in-between state.
2019-11-25 20:18:19 +00:00
toInternalGitPath :: RawFilePath -> InternalGitPath
get rid of __WINDOWS__, use mingw32_HOST_OS The latter is harder for me to remember, but avoids build failures in code used by the configure program.
2013-08-02 16:27:32 +00:00
#ifndef mingw32_HOST_OS
deal with git using / internally, even on DOS
2013-05-12 22:18:48 +00:00
toInternalGitPath = id
#else
wip RawFilePath Goal is to make git-annex faster by using ByteString for all the worktree traversal. For now, this is focusing on Command.Find, in order to benchmark how much it helps. (All other commands are temporarily disabled) Currently in a very bad unbuildable in-between state.
2019-11-25 20:18:19 +00:00
toInternalGitPath = encodeBS . replace "\\" "/" . decodeBS
deal with git using / internally, even on DOS
2013-05-12 22:18:48 +00:00
#endif
wip RawFilePath Goal is to make git-annex faster by using ByteString for all the worktree traversal. For now, this is focusing on Command.Find, in order to benchmark how much it helps. (All other commands are temporarily disabled) Currently in a very bad unbuildable in-between state.
2019-11-25 20:18:19 +00:00
fromInternalGitPath :: InternalGitPath -> RawFilePath
get rid of __WINDOWS__, use mingw32_HOST_OS The latter is harder for me to remember, but avoids build failures in code used by the configure program.
2013-08-02 16:27:32 +00:00
#ifndef mingw32_HOST_OS
deal with git using / internally, even on DOS
2013-05-12 22:18:48 +00:00
fromInternalGitPath = id
#else
wip RawFilePath Goal is to make git-annex faster by using ByteString for all the worktree traversal. For now, this is focusing on Command.Find, in order to benchmark how much it helps. (All other commands are temporarily disabled) Currently in a very bad unbuildable in-between state.
2019-11-25 20:18:19 +00:00
fromInternalGitPath = encodeBS . replace "/" "\\" . decodeBS
deal with git using / internally, even on DOS
2013-05-12 22:18:48 +00:00
#endif
Windows: Fix handling of absolute unix-style git repository paths. Note that on Windows a remote with a path like /home/foo/bar is interpreted by git as being some screwy relative path (relative to what exactly seems ill-defined -- it seemed relative to C:\Program Files\Git\ in my tests!) So no attempt has been made to handle such a path sanely, just not to crash when encountering it. Note that "C:\\foo" </> "/home/foo/bar" yields /home/foo/bar even though that is not absolute! I don't know what to make of all this, except that I will be very happy when this crock of **** vanishes from the face of the earth.
2014-02-08 19:31:03 +00:00
{- isAbsolute on Windows does not think "/foo" or "\foo" is absolute,
- so try posix paths.
-}
wip RawFilePath Goal is to make git-annex faster by using ByteString for all the worktree traversal. For now, this is focusing on Command.Find, in order to benchmark how much it helps. (All other commands are temporarily disabled) Currently in a very bad unbuildable in-between state.
2019-11-25 20:18:19 +00:00
absoluteGitPath :: RawFilePath -> Bool
convert TopFilePath to use RawFilePath Adds a dependency on filepath-bytestring, an as yet unreleased fork of filepath that operates on RawFilePath. Git.Repo also changed to use RawFilePath for the path to the repo. This does eliminate some RawFilePath -> FilePath -> RawFilePath conversions. And filepath-bytestring's </> is probably faster. But I don't expect a major performance improvement from this. This is mostly groundwork for making Annex.Location use RawFilePath, which will allow for a conversion-free pipleline.
2019-12-09 17:49:05 +00:00
absoluteGitPath p = P.isAbsolute p ||
System.FilePath.Posix.ByteString.isAbsolute (toInternalGitPath p)
Reference in a new issue Copy permalink