support .git/annex on a different disk than the rest of the repo
The only fully supported thing is to have the main repository on one disk,
and .git/annex on another. Only commands that move data in/out of the annex
will need to copy it across devices.
There is only partial support for putting arbitrary subdirectories of
.git/annex on different devices. For one thing, but this can require more
copies to be done. For example, when .git/annex/tmp is on one device, and
.git/annex/journal on another, every journal write involves a call to
mv(1). Also, there are a few places that make hard links between various
subdirectories of .git/annex with createLink, that are not handled.
In the common case without cross-device, the new moveFile is actually
faster than renameFile, avoiding an unncessary stat to check that a file
(not a directory) is being moved. Of course if a cross-device move is
needed, it is as slow as mv(1) of the data.
2011-11-28 19:26:27 +00:00
|
|
|
{- directory manipulation
|
|
|
|
-
|
2014-02-18 21:38:23 +00:00
|
|
|
- Copyright 2011-2014 Joey Hess <joey@kitenet.net>
|
support .git/annex on a different disk than the rest of the repo
The only fully supported thing is to have the main repository on one disk,
and .git/annex on another. Only commands that move data in/out of the annex
will need to copy it across devices.
There is only partial support for putting arbitrary subdirectories of
.git/annex on different devices. For one thing, but this can require more
copies to be done. For example, when .git/annex/tmp is on one device, and
.git/annex/journal on another, every journal write involves a call to
mv(1). Also, there are a few places that make hard links between various
subdirectories of .git/annex with createLink, that are not handled.
In the common case without cross-device, the new moveFile is actually
faster than renameFile, avoiding an unncessary stat to check that a file
(not a directory) is being moved. Of course if a cross-device move is
needed, it is as slow as mv(1) of the data.
2011-11-28 19:26:27 +00:00
|
|
|
-
|
|
|
|
- Licensed under the GNU GPL version 3 or higher.
|
|
|
|
-}
|
|
|
|
|
2013-05-21 15:45:37 +00:00
|
|
|
{-# LANGUAGE CPP #-}
|
|
|
|
|
support .git/annex on a different disk than the rest of the repo
The only fully supported thing is to have the main repository on one disk,
and .git/annex on another. Only commands that move data in/out of the annex
will need to copy it across devices.
There is only partial support for putting arbitrary subdirectories of
.git/annex on different devices. For one thing, but this can require more
copies to be done. For example, when .git/annex/tmp is on one device, and
.git/annex/journal on another, every journal write involves a call to
mv(1). Also, there are a few places that make hard links between various
subdirectories of .git/annex with createLink, that are not handled.
In the common case without cross-device, the new moveFile is actually
faster than renameFile, avoiding an unncessary stat to check that a file
(not a directory) is being moved. Of course if a cross-device move is
needed, it is as slow as mv(1) of the data.
2011-11-28 19:26:27 +00:00
|
|
|
module Utility.Directory where
|
|
|
|
|
|
|
|
import System.IO.Error
|
|
|
|
import System.Directory
|
2012-10-16 20:47:59 +00:00
|
|
|
import Control.Exception (throw)
|
2011-12-09 05:57:13 +00:00
|
|
|
import Control.Monad
|
2012-01-24 19:28:13 +00:00
|
|
|
import Control.Monad.IfElse
|
2012-03-11 22:12:36 +00:00
|
|
|
import System.FilePath
|
|
|
|
import Control.Applicative
|
2012-05-31 23:25:33 +00:00
|
|
|
import System.IO.Unsafe (unsafeInterleaveIO)
|
support .git/annex on a different disk than the rest of the repo
The only fully supported thing is to have the main repository on one disk,
and .git/annex on another. Only commands that move data in/out of the annex
will need to copy it across devices.
There is only partial support for putting arbitrary subdirectories of
.git/annex on different devices. For one thing, but this can require more
copies to be done. For example, when .git/annex/tmp is on one device, and
.git/annex/journal on another, every journal write involves a call to
mv(1). Also, there are a few places that make hard links between various
subdirectories of .git/annex with createLink, that are not handled.
In the common case without cross-device, the new moveFile is actually
faster than renameFile, avoiding an unncessary stat to check that a file
(not a directory) is being moved. Of course if a cross-device move is
needed, it is as slow as mv(1) of the data.
2011-11-28 19:26:27 +00:00
|
|
|
|
2014-01-29 19:19:03 +00:00
|
|
|
import Utility.PosixFiles
|
support .git/annex on a different disk than the rest of the repo
The only fully supported thing is to have the main repository on one disk,
and .git/annex on another. Only commands that move data in/out of the annex
will need to copy it across devices.
There is only partial support for putting arbitrary subdirectories of
.git/annex on different devices. For one thing, but this can require more
copies to be done. For example, when .git/annex/tmp is on one device, and
.git/annex/journal on another, every journal write involves a call to
mv(1). Also, there are a few places that make hard links between various
subdirectories of .git/annex with createLink, that are not handled.
In the common case without cross-device, the new moveFile is actually
faster than renameFile, avoiding an unncessary stat to check that a file
(not a directory) is being moved. Of course if a cross-device move is
needed, it is as slow as mv(1) of the data.
2011-11-28 19:26:27 +00:00
|
|
|
import Utility.SafeCommand
|
2013-05-12 23:19:28 +00:00
|
|
|
import Utility.Tmp
|
2012-02-03 20:47:24 +00:00
|
|
|
import Utility.Exception
|
2012-04-22 03:32:33 +00:00
|
|
|
import Utility.Monad
|
2014-02-18 21:38:23 +00:00
|
|
|
import Utility.Applicative
|
support .git/annex on a different disk than the rest of the repo
The only fully supported thing is to have the main repository on one disk,
and .git/annex on another. Only commands that move data in/out of the annex
will need to copy it across devices.
There is only partial support for putting arbitrary subdirectories of
.git/annex on different devices. For one thing, but this can require more
copies to be done. For example, when .git/annex/tmp is on one device, and
.git/annex/journal on another, every journal write involves a call to
mv(1). Also, there are a few places that make hard links between various
subdirectories of .git/annex with createLink, that are not handled.
In the common case without cross-device, the new moveFile is actually
faster than renameFile, avoiding an unncessary stat to check that a file
(not a directory) is being moved. Of course if a cross-device move is
needed, it is as slow as mv(1) of the data.
2011-11-28 19:26:27 +00:00
|
|
|
|
2012-05-31 23:25:33 +00:00
|
|
|
dirCruft :: FilePath -> Bool
|
|
|
|
dirCruft "." = True
|
|
|
|
dirCruft ".." = True
|
|
|
|
dirCruft _ = False
|
|
|
|
|
2012-03-11 22:12:36 +00:00
|
|
|
{- Lists the contents of a directory.
|
|
|
|
- Unlike getDirectoryContents, paths are not relative to the directory. -}
|
|
|
|
dirContents :: FilePath -> IO [FilePath]
|
2012-05-31 23:25:33 +00:00
|
|
|
dirContents d = map (d </>) . filter (not . dirCruft) <$> getDirectoryContents d
|
|
|
|
|
2012-06-18 16:53:57 +00:00
|
|
|
{- Gets files in a directory, and then its subdirectories, recursively,
|
2013-12-18 19:05:29 +00:00
|
|
|
- and lazily.
|
|
|
|
-
|
2013-12-24 17:13:17 +00:00
|
|
|
- Does not follow symlinks to other subdirectories.
|
2013-12-18 19:05:29 +00:00
|
|
|
-
|
|
|
|
- When the directory does not exist, no exception is thrown,
|
2012-07-02 14:56:26 +00:00
|
|
|
- instead, [] is returned. -}
|
2012-05-31 23:25:33 +00:00
|
|
|
dirContentsRecursive :: FilePath -> IO [FilePath]
|
2013-12-18 19:05:29 +00:00
|
|
|
dirContentsRecursive topdir = dirContentsRecursiveSkipping (const False) True topdir
|
2012-05-31 23:25:33 +00:00
|
|
|
|
2013-10-07 17:03:05 +00:00
|
|
|
{- Skips directories whose basenames match the skipdir. -}
|
2013-12-18 19:05:29 +00:00
|
|
|
dirContentsRecursiveSkipping :: (FilePath -> Bool) -> Bool -> FilePath -> IO [FilePath]
|
|
|
|
dirContentsRecursiveSkipping skipdir followsubdirsymlinks topdir = go [topdir]
|
2012-12-13 04:24:19 +00:00
|
|
|
where
|
2013-10-05 19:36:09 +00:00
|
|
|
go [] = return []
|
|
|
|
go (dir:dirs)
|
2013-10-07 17:03:05 +00:00
|
|
|
| skipdir (takeFileName dir) = go dirs
|
2013-10-05 19:36:09 +00:00
|
|
|
| otherwise = unsafeInterleaveIO $ do
|
|
|
|
(files, dirs') <- collect [] []
|
|
|
|
=<< catchDefaultIO [] (dirContents dir)
|
|
|
|
files' <- go (dirs' ++ dirs)
|
|
|
|
return (files ++ files')
|
2012-12-13 04:24:19 +00:00
|
|
|
collect files dirs' [] = return (reverse files, reverse dirs')
|
|
|
|
collect files dirs' (entry:entries)
|
|
|
|
| dirCruft entry = collect files dirs' entries
|
|
|
|
| otherwise = do
|
2013-12-18 19:20:26 +00:00
|
|
|
let skip = collect (entry:files) dirs' entries
|
|
|
|
let recurse = collect files (entry:dirs') entries
|
|
|
|
ms <- catchMaybeIO $ getSymbolicLinkStatus entry
|
2013-12-18 19:05:29 +00:00
|
|
|
case ms of
|
2013-12-18 19:20:26 +00:00
|
|
|
(Just s)
|
|
|
|
| isDirectory s -> recurse
|
|
|
|
| isSymbolicLink s && followsubdirsymlinks ->
|
|
|
|
ifM (doesDirectoryExist entry)
|
|
|
|
( recurse
|
|
|
|
, skip
|
|
|
|
)
|
|
|
|
_ -> skip
|
2012-03-11 22:12:36 +00:00
|
|
|
|
2014-02-18 21:38:23 +00:00
|
|
|
{- Gets the directory tree from a point, recursively and lazily,
|
|
|
|
- with leaf directories **first**, skipping any whose basenames
|
|
|
|
- match the skipdir. Does not follow symlinks. -}
|
|
|
|
dirTreeRecursiveSkipping :: (FilePath -> Bool) -> FilePath -> IO [FilePath]
|
|
|
|
dirTreeRecursiveSkipping skipdir topdir = go [] [topdir]
|
|
|
|
where
|
|
|
|
go c [] = return c
|
|
|
|
go c (dir:dirs)
|
|
|
|
| skipdir (takeFileName dir) = go c dirs
|
|
|
|
| otherwise = unsafeInterleaveIO $ do
|
|
|
|
subdirs <- go c
|
|
|
|
=<< filterM (isDirectory <$$> getSymbolicLinkStatus)
|
|
|
|
=<< catchDefaultIO [] (dirContents dir)
|
|
|
|
go (subdirs++[dir]) dirs
|
|
|
|
|
support .git/annex on a different disk than the rest of the repo
The only fully supported thing is to have the main repository on one disk,
and .git/annex on another. Only commands that move data in/out of the annex
will need to copy it across devices.
There is only partial support for putting arbitrary subdirectories of
.git/annex on different devices. For one thing, but this can require more
copies to be done. For example, when .git/annex/tmp is on one device, and
.git/annex/journal on another, every journal write involves a call to
mv(1). Also, there are a few places that make hard links between various
subdirectories of .git/annex with createLink, that are not handled.
In the common case without cross-device, the new moveFile is actually
faster than renameFile, avoiding an unncessary stat to check that a file
(not a directory) is being moved. Of course if a cross-device move is
needed, it is as slow as mv(1) of the data.
2011-11-28 19:26:27 +00:00
|
|
|
{- Moves one filename to another.
|
|
|
|
- First tries a rename, but falls back to moving across devices if needed. -}
|
|
|
|
moveFile :: FilePath -> FilePath -> IO ()
|
2012-02-03 20:47:24 +00:00
|
|
|
moveFile src dest = tryIO (rename src dest) >>= onrename
|
2012-12-13 04:24:19 +00:00
|
|
|
where
|
|
|
|
onrename (Right _) = noop
|
|
|
|
onrename (Left e)
|
|
|
|
| isPermissionError e = rethrow
|
|
|
|
| isDoesNotExistError e = rethrow
|
|
|
|
| otherwise = do
|
|
|
|
-- copyFile is likely not as optimised as
|
|
|
|
-- the mv command, so we'll use the latter.
|
|
|
|
-- But, mv will move into a directory if
|
|
|
|
-- dest is one, which is not desired.
|
|
|
|
whenM (isdir dest) rethrow
|
|
|
|
viaTmp mv dest undefined
|
|
|
|
where
|
|
|
|
rethrow = throw e
|
|
|
|
mv tmp _ = do
|
|
|
|
ok <- boolSystem "mv" [Param "-f", Param src, Param tmp]
|
|
|
|
unless ok $ do
|
|
|
|
-- delete any partial
|
|
|
|
_ <- tryIO $ removeFile tmp
|
|
|
|
rethrow
|
|
|
|
|
|
|
|
isdir f = do
|
|
|
|
r <- tryIO $ getFileStatus f
|
|
|
|
case r of
|
|
|
|
(Left _) -> return False
|
|
|
|
(Right s) -> return $ isDirectory s
|
Clean up handling of git directory and git worktree.
Baked into the code was an assumption that a repository's git directory
could be determined by adding ".git" to its work tree (or nothing for bare
repos). That fails when core.worktree, or GIT_DIR and GIT_WORK_TREE are
used to separate the two.
This was attacked at the type level, by storing the gitdir and worktree
separately, so Nothing for the worktree means a bare repo.
A complication arose because we don't learn where a repository is bare
until its configuration is read. So another Location type handles
repositories that have not had their config read yet. I am not entirely
happy with this being a Location type, rather than representing them
entirely separate from the Git type. The new code is not worse than the
old, but better types could enforce more safety.
Added support for core.worktree. Overriding it with -c isn't supported
because it's not really clear what to do if a git repo's config is read, is
not bare, and is then overridden to bare. What is the right git directory
in this case? I will worry about this if/when someone has a use case for
overriding core.worktree with -c. (See Git.Config.updateLocation)
Also removed and renamed some functions like gitDir and workTree that
misused git's terminology.
One minor regression is known: git annex add in a bare repository does not
print a nice error message, but runs git ls-files in a way that fails
earlier with a less nice error message. This is because before --work-tree
was always passed to git commands, even in a bare repo, while now it's not.
2012-05-18 20:38:26 +00:00
|
|
|
|
2013-05-21 15:45:37 +00:00
|
|
|
{- Removes a file, which may or may not exist, and does not have to
|
|
|
|
- be a regular file.
|
2012-06-06 17:13:13 +00:00
|
|
|
-
|
|
|
|
- Note that an exception is thrown if the file exists but
|
|
|
|
- cannot be removed. -}
|
|
|
|
nukeFile :: FilePath -> IO ()
|
2013-05-21 17:03:46 +00:00
|
|
|
nukeFile file = void $ tryWhenExists go
|
|
|
|
where
|
2013-05-21 15:45:37 +00:00
|
|
|
#ifndef mingw32_HOST_OS
|
2013-05-21 17:03:46 +00:00
|
|
|
go = removeLink file
|
2013-05-21 15:45:37 +00:00
|
|
|
#else
|
2013-05-21 17:03:46 +00:00
|
|
|
go = removeFile file
|
2013-05-21 15:45:37 +00:00
|
|
|
#endif
|