2014-07-10 03:36:53 +00:00
|
|
|
{- directory traversal and manipulation
|
support .git/annex on a different disk than the rest of the repo
The only fully supported thing is to have the main repository on one disk,
and .git/annex on another. Only commands that move data in/out of the annex
will need to copy it across devices.
There is only partial support for putting arbitrary subdirectories of
.git/annex on different devices. For one thing, but this can require more
copies to be done. For example, when .git/annex/tmp is on one device, and
.git/annex/journal on another, every journal write involves a call to
mv(1). Also, there are a few places that make hard links between various
subdirectories of .git/annex with createLink, that are not handled.
In the common case without cross-device, the new moveFile is actually
faster than renameFile, avoiding an unncessary stat to check that a file
(not a directory) is being moved. Of course if a cross-device move is
needed, it is as slow as mv(1) of the data.
2011-11-28 19:26:27 +00:00
|
|
|
-
|
2023-08-15 16:57:41 +00:00
|
|
|
- Copyright 2011-2023 Joey Hess <id@joeyh.name>
|
support .git/annex on a different disk than the rest of the repo
The only fully supported thing is to have the main repository on one disk,
and .git/annex on another. Only commands that move data in/out of the annex
will need to copy it across devices.
There is only partial support for putting arbitrary subdirectories of
.git/annex on different devices. For one thing, but this can require more
copies to be done. For example, when .git/annex/tmp is on one device, and
.git/annex/journal on another, every journal write involves a call to
mv(1). Also, there are a few places that make hard links between various
subdirectories of .git/annex with createLink, that are not handled.
In the common case without cross-device, the new moveFile is actually
faster than renameFile, avoiding an unncessary stat to check that a file
(not a directory) is being moved. Of course if a cross-device move is
needed, it is as slow as mv(1) of the data.
2011-11-28 19:26:27 +00:00
|
|
|
-
|
2014-05-10 14:01:27 +00:00
|
|
|
- License: BSD-2-clause
|
support .git/annex on a different disk than the rest of the repo
The only fully supported thing is to have the main repository on one disk,
and .git/annex on another. Only commands that move data in/out of the annex
will need to copy it across devices.
There is only partial support for putting arbitrary subdirectories of
.git/annex on different devices. For one thing, but this can require more
copies to be done. For example, when .git/annex/tmp is on one device, and
.git/annex/journal on another, every journal write involves a call to
mv(1). Also, there are a few places that make hard links between various
subdirectories of .git/annex with createLink, that are not handled.
In the common case without cross-device, the new moveFile is actually
faster than renameFile, avoiding an unncessary stat to check that a file
(not a directory) is being moved. Of course if a cross-device move is
needed, it is as slow as mv(1) of the data.
2011-11-28 19:26:27 +00:00
|
|
|
-}
|
|
|
|
|
2013-05-21 15:45:37 +00:00
|
|
|
{-# LANGUAGE CPP #-}
|
2020-03-05 17:56:39 +00:00
|
|
|
{-# LANGUAGE LambdaCase #-}
|
2016-05-22 19:51:31 +00:00
|
|
|
{-# OPTIONS_GHC -fno-warn-tabs #-}
|
2013-05-21 15:45:37 +00:00
|
|
|
|
2016-04-28 19:18:11 +00:00
|
|
|
module Utility.Directory (
|
|
|
|
module Utility.Directory,
|
2016-05-22 19:51:31 +00:00
|
|
|
module Utility.SystemDirectory
|
2016-04-28 19:18:11 +00:00
|
|
|
) where
|
support .git/annex on a different disk than the rest of the repo
The only fully supported thing is to have the main repository on one disk,
and .git/annex on another. Only commands that move data in/out of the annex
will need to copy it across devices.
There is only partial support for putting arbitrary subdirectories of
.git/annex on different devices. For one thing, but this can require more
copies to be done. For example, when .git/annex/tmp is on one device, and
.git/annex/journal on another, every journal write involves a call to
mv(1). Also, there are a few places that make hard links between various
subdirectories of .git/annex with createLink, that are not handled.
In the common case without cross-device, the new moveFile is actually
faster than renameFile, avoiding an unncessary stat to check that a file
(not a directory) is being moved. Of course if a cross-device move is
needed, it is as slow as mv(1) of the data.
2011-11-28 19:26:27 +00:00
|
|
|
|
2011-12-09 05:57:13 +00:00
|
|
|
import Control.Monad
|
2012-03-11 22:12:36 +00:00
|
|
|
import System.FilePath
|
2023-03-01 19:55:58 +00:00
|
|
|
import System.PosixCompat.Files (isDirectory, isSymbolicLink)
|
2012-03-11 22:12:36 +00:00
|
|
|
import Control.Applicative
|
2012-05-31 23:25:33 +00:00
|
|
|
import System.IO.Unsafe (unsafeInterleaveIO)
|
2014-07-10 03:36:53 +00:00
|
|
|
import Data.Maybe
|
2015-05-10 20:19:56 +00:00
|
|
|
import Prelude
|
support .git/annex on a different disk than the rest of the repo
The only fully supported thing is to have the main repository on one disk,
and .git/annex on another. Only commands that move data in/out of the annex
will need to copy it across devices.
There is only partial support for putting arbitrary subdirectories of
.git/annex on different devices. For one thing, but this can require more
copies to be done. For example, when .git/annex/tmp is on one device, and
.git/annex/journal on another, every journal write involves a call to
mv(1). Also, there are a few places that make hard links between various
subdirectories of .git/annex with createLink, that are not handled.
In the common case without cross-device, the new moveFile is actually
faster than renameFile, avoiding an unncessary stat to check that a file
(not a directory) is being moved. Of course if a cross-device move is
needed, it is as slow as mv(1) of the data.
2011-11-28 19:26:27 +00:00
|
|
|
|
2016-05-22 19:51:31 +00:00
|
|
|
import Utility.SystemDirectory
|
2012-02-03 20:47:24 +00:00
|
|
|
import Utility.Exception
|
2012-04-22 03:32:33 +00:00
|
|
|
import Utility.Monad
|
2023-03-01 19:55:58 +00:00
|
|
|
import Utility.FileSystemEncoding
|
|
|
|
import qualified Utility.RawFilePath as R
|
support .git/annex on a different disk than the rest of the repo
The only fully supported thing is to have the main repository on one disk,
and .git/annex on another. Only commands that move data in/out of the annex
will need to copy it across devices.
There is only partial support for putting arbitrary subdirectories of
.git/annex on different devices. For one thing, but this can require more
copies to be done. For example, when .git/annex/tmp is on one device, and
.git/annex/journal on another, every journal write involves a call to
mv(1). Also, there are a few places that make hard links between various
subdirectories of .git/annex with createLink, that are not handled.
In the common case without cross-device, the new moveFile is actually
faster than renameFile, avoiding an unncessary stat to check that a file
(not a directory) is being moved. Of course if a cross-device move is
needed, it is as slow as mv(1) of the data.
2011-11-28 19:26:27 +00:00
|
|
|
|
2012-05-31 23:25:33 +00:00
|
|
|
dirCruft :: FilePath -> Bool
|
|
|
|
dirCruft "." = True
|
|
|
|
dirCruft ".." = True
|
|
|
|
dirCruft _ = False
|
|
|
|
|
2012-03-11 22:12:36 +00:00
|
|
|
{- Lists the contents of a directory.
|
|
|
|
- Unlike getDirectoryContents, paths are not relative to the directory. -}
|
|
|
|
dirContents :: FilePath -> IO [FilePath]
|
2012-05-31 23:25:33 +00:00
|
|
|
dirContents d = map (d </>) . filter (not . dirCruft) <$> getDirectoryContents d
|
|
|
|
|
2012-06-18 16:53:57 +00:00
|
|
|
{- Gets files in a directory, and then its subdirectories, recursively,
|
2013-12-18 19:05:29 +00:00
|
|
|
- and lazily.
|
|
|
|
-
|
2013-12-24 17:13:17 +00:00
|
|
|
- Does not follow symlinks to other subdirectories.
|
2013-12-18 19:05:29 +00:00
|
|
|
-
|
2023-08-15 16:57:41 +00:00
|
|
|
- Throws exception if the directory does not exist or otherwise cannot be
|
|
|
|
- accessed. However, does not throw exceptions when subdirectories cannot
|
|
|
|
- be accessed (the use of unsafeInterleaveIO would make it difficult to
|
|
|
|
- trap such exceptions).
|
|
|
|
-}
|
2012-05-31 23:25:33 +00:00
|
|
|
dirContentsRecursive :: FilePath -> IO [FilePath]
|
2014-04-26 23:25:05 +00:00
|
|
|
dirContentsRecursive = dirContentsRecursiveSkipping (const False) True
|
2012-05-31 23:25:33 +00:00
|
|
|
|
2013-10-07 17:03:05 +00:00
|
|
|
{- Skips directories whose basenames match the skipdir. -}
|
2013-12-18 19:05:29 +00:00
|
|
|
dirContentsRecursiveSkipping :: (FilePath -> Bool) -> Bool -> FilePath -> IO [FilePath]
|
2023-08-15 16:57:41 +00:00
|
|
|
dirContentsRecursiveSkipping skipdir followsubdirsymlinks topdir
|
|
|
|
| skipdir (takeFileName topdir) = return []
|
|
|
|
| otherwise = do
|
|
|
|
-- Get the contents of the top directory outside of
|
|
|
|
-- unsafeInterleaveIO, which allows throwing exceptions if
|
|
|
|
-- it cannot be accessed.
|
|
|
|
(files, dirs) <- collect [] []
|
|
|
|
=<< dirContents topdir
|
|
|
|
files' <- go dirs
|
|
|
|
return (files ++ files')
|
2012-12-13 04:24:19 +00:00
|
|
|
where
|
2014-10-09 18:53:13 +00:00
|
|
|
go [] = return []
|
2013-10-05 19:36:09 +00:00
|
|
|
go (dir:dirs)
|
2013-10-07 17:03:05 +00:00
|
|
|
| skipdir (takeFileName dir) = go dirs
|
2013-10-05 19:36:09 +00:00
|
|
|
| otherwise = unsafeInterleaveIO $ do
|
|
|
|
(files, dirs') <- collect [] []
|
|
|
|
=<< catchDefaultIO [] (dirContents dir)
|
|
|
|
files' <- go (dirs' ++ dirs)
|
|
|
|
return (files ++ files')
|
2012-12-13 04:24:19 +00:00
|
|
|
collect files dirs' [] = return (reverse files, reverse dirs')
|
|
|
|
collect files dirs' (entry:entries)
|
|
|
|
| dirCruft entry = collect files dirs' entries
|
|
|
|
| otherwise = do
|
2013-12-18 19:20:26 +00:00
|
|
|
let skip = collect (entry:files) dirs' entries
|
|
|
|
let recurse = collect files (entry:dirs') entries
|
2023-03-01 19:55:58 +00:00
|
|
|
ms <- catchMaybeIO $ R.getSymbolicLinkStatus (toRawFilePath entry)
|
2013-12-18 19:05:29 +00:00
|
|
|
case ms of
|
2013-12-18 19:20:26 +00:00
|
|
|
(Just s)
|
|
|
|
| isDirectory s -> recurse
|
|
|
|
| isSymbolicLink s && followsubdirsymlinks ->
|
|
|
|
ifM (doesDirectoryExist entry)
|
|
|
|
( recurse
|
|
|
|
, skip
|
|
|
|
)
|
|
|
|
_ -> skip
|
2012-03-11 22:12:36 +00:00
|
|
|
|
2014-02-18 21:38:23 +00:00
|
|
|
{- Gets the directory tree from a point, recursively and lazily,
|
|
|
|
- with leaf directories **first**, skipping any whose basenames
|
2023-08-15 16:57:41 +00:00
|
|
|
- match the skipdir. Does not follow symlinks.
|
|
|
|
-
|
|
|
|
- Throws exception if the directory does not exist or otherwise cannot be
|
|
|
|
- accessed. However, does not throw exceptions when subdirectories cannot
|
|
|
|
- be accessed (the use of unsafeInterleaveIO would make it difficult to
|
|
|
|
- trap such exceptions).
|
|
|
|
-}
|
2014-02-18 21:38:23 +00:00
|
|
|
dirTreeRecursiveSkipping :: (FilePath -> Bool) -> FilePath -> IO [FilePath]
|
2023-08-15 16:57:41 +00:00
|
|
|
dirTreeRecursiveSkipping skipdir topdir
|
|
|
|
| skipdir (takeFileName topdir) = return []
|
|
|
|
| otherwise = do
|
|
|
|
subdirs <- filterM isdir =<< dirContents topdir
|
|
|
|
go [] subdirs
|
2014-02-18 21:38:23 +00:00
|
|
|
where
|
2014-10-09 18:53:13 +00:00
|
|
|
go c [] = return c
|
2014-02-18 21:38:23 +00:00
|
|
|
go c (dir:dirs)
|
|
|
|
| skipdir (takeFileName dir) = go c dirs
|
|
|
|
| otherwise = unsafeInterleaveIO $ do
|
2017-05-16 15:33:53 +00:00
|
|
|
subdirs <- go []
|
2023-03-01 19:55:58 +00:00
|
|
|
=<< filterM isdir
|
2014-02-18 21:38:23 +00:00
|
|
|
=<< catchDefaultIO [] (dirContents dir)
|
2017-05-16 15:33:53 +00:00
|
|
|
go (subdirs++dir:c) dirs
|
2023-03-01 19:55:58 +00:00
|
|
|
isdir p = isDirectory <$> R.getSymbolicLinkStatus (toRawFilePath p)
|
2014-02-18 21:38:23 +00:00
|
|
|
|
2023-08-15 16:57:41 +00:00
|
|
|
{- When the action fails due to the directory not existing, returns []. -}
|
|
|
|
emptyWhenDoesNotExist :: IO [a] -> IO [a]
|
|
|
|
emptyWhenDoesNotExist a = tryWhenExists a >>= return . \case
|
|
|
|
Just v -> v
|
|
|
|
Nothing -> []
|
|
|
|
|
2020-10-29 14:33:12 +00:00
|
|
|
{- Use with an action that removes something, which may or may not exist.
|
2012-06-06 17:13:13 +00:00
|
|
|
-
|
2020-10-29 14:33:12 +00:00
|
|
|
- If an exception is thrown due to it not existing, it is ignored.
|
|
|
|
-}
|
|
|
|
removeWhenExistsWith :: (a -> IO ()) -> a -> IO ()
|
|
|
|
removeWhenExistsWith f a = void $ tryWhenExists $ f a
|