git-annex/Utility/Tmp.hs

117 lines
3.7 KiB
Haskell
Raw Permalink Normal View History

{- Temporary files.
2011-10-16 04:31:25 +00:00
-
- Copyright 2010-2020 Joey Hess <id@joeyh.name>
2011-10-16 04:31:25 +00:00
-
- License: BSD-2-clause
2011-10-16 04:31:25 +00:00
-}
{-# LANGUAGE CPP #-}
{-# OPTIONS_GHC -fno-warn-tabs #-}
module Utility.Tmp (
Template,
viaTmp,
withTmpFile,
withTmpFileIn,
relatedTemplate,
openTmpFileIn,
) where
2011-10-16 04:31:25 +00:00
import System.IO
import System.FilePath
import System.Directory
import Control.Monad.IO.Class
import System.IO.Error
2011-10-16 04:31:25 +00:00
import Utility.Exception
Fix a few bugs involving filenames that are at or near the filesystem's maximum filename length limit. Started with a problem when running addurl on a really long url, because the whole url is munged into the filename. Ended up doing a fairly extensive review for places where filenames could get too large, although it's hard to say I'm not missed any.. Backend.Url had a 128 character limit, which is fine when the limit is 255, but not if it's a lot shorter on some systems. So check the pathconf() limit. Note that this could result in fromUrl creating different keys for the same url, if run on systems with different limits. I don't see this is likely to cause any problems. That can already happen when using addurl --fast, or if the content of an url changes. Both Command.AddUrl and Backend.Url assumed that urls don't contain a lot of multi-byte unicode, and would fail to truncate an url that did properly. A few places use a filename as the template to make a temp file. While that's nice in that the temp file name can be easily related back to the original filename, it could lead to `git annex add` failing to add a filename that was at or close to the maximum length. Note that in Command.Add.lockdown, the template is still derived from the filename, just with enough space left to turn it into a temp file. This is an important optimisation, because the assistant may lock down a bunch of files all at once, and using the same template for all of them would cause openTempFile to iterate through the same set of names, looking for an unused temp file. I'm not very happy with the relatedTemplate hack, but it avoids that slowdown. Backend.WORM does not limit the filename stored in the key. I have not tried to change that; so git annex add will fail on really long filenames when using the WORM backend. It seems better to preserve the invariant that a WORM key always contains the complete filename, since the filename is the only unique material in the key, other than mtime and size. Since nobody has complained about add failing (I think I saw it once?) on WORM, probably it's ok, or nobody but me uses it. There may be compatability problems if using git annex addurl --fast or the WORM backend on a system with the 255 limit and then trying to use that repo in a system with a smaller limit. I have not tried to deal with those. This commit was sponsored by Alexander Brem. Thanks!
2013-07-30 21:49:11 +00:00
import Utility.FileSystemEncoding
import Utility.FileMode
import qualified Utility.RawFilePath as R
2011-10-16 04:31:25 +00:00
2013-05-12 20:38:00 +00:00
type Template = String
{- This is the same as openTempFile, except when there is an
- error, it displays the template as well as the directory,
- to help identify what call was responsible.
-}
openTmpFileIn :: FilePath -> String -> IO (FilePath, Handle)
2023-12-05 19:56:58 +00:00
openTmpFileIn dir template = openTempFile dir template
`catchIO` decoraterrror
where
decoraterrror e = throwM $
let loc = ioeGetLocation e ++ " template " ++ template
in annotateIOError e loc Nothing Nothing
2011-10-16 04:31:25 +00:00
{- Runs an action like writeFile, writing to a temp file first and
- then moving it into place. The temp file is stored in the same
- directory as the final file to avoid cross-device renames.
-
- While this uses a temp file, the file will end up with the same
- mode as it would when using writeFile, unless the writer action changes
- it.
-}
viaTmp :: (MonadMask m, MonadIO m) => (FilePath -> v -> m ()) -> FilePath -> v -> m ()
2014-11-12 18:59:24 +00:00
viaTmp a file content = bracketIO setup cleanup use
where
(dir, base) = splitFileName file
2019-03-04 18:12:00 +00:00
template = relatedTemplate (base ++ ".tmp")
setup = do
createDirectoryIfMissing True dir
openTmpFileIn dir template
cleanup (tmpfile, h) = do
_ <- tryIO $ hClose h
tryIO $ removeFile tmpfile
use (tmpfile, h) = do
let tmpfile' = toRawFilePath tmpfile
-- Make mode the same as if the file were created usually,
-- not as a temp file. (This may fail on some filesystems
-- that don't support file modes well, so ignore
-- exceptions.)
_ <- liftIO $ tryIO $ R.setFileMode tmpfile' =<< defaultFileMode
2014-11-12 18:59:24 +00:00
liftIO $ hClose h
a tmpfile content
liftIO $ R.rename tmpfile' (toRawFilePath file)
2012-01-21 06:24:12 +00:00
2013-05-12 20:38:00 +00:00
{- Runs an action with a tmp file located in the system's tmp directory
- (or in "." if there is none) then removes the file. -}
withTmpFile :: (MonadIO m, MonadMask m) => Template -> (FilePath -> Handle -> m a) -> m a
2013-05-12 23:19:28 +00:00
withTmpFile template a = do
tmpdir <- liftIO $ catchDefaultIO "." getTemporaryDirectory
2013-05-12 23:19:28 +00:00
withTmpFileIn tmpdir template a
2013-05-12 20:38:00 +00:00
{- Runs an action with a tmp file located in the specified directory,
- then removes the file.
-
- Note that the tmp file will have a file mode that only allows the
- current user to access it.
-}
withTmpFileIn :: (MonadIO m, MonadMask m) => FilePath -> Template -> (FilePath -> Handle -> m a) -> m a
2013-05-12 23:19:28 +00:00
withTmpFileIn tmpdir template a = bracket create remove use
2012-11-12 21:43:10 +00:00
where
create = liftIO $ openTmpFileIn tmpdir template
remove (name, h) = liftIO $ do
hClose h
2012-11-12 21:43:10 +00:00
catchBoolIO (removeFile name >> return True)
use (name, h) = a name h
2012-09-10 18:09:13 +00:00
Fix a few bugs involving filenames that are at or near the filesystem's maximum filename length limit. Started with a problem when running addurl on a really long url, because the whole url is munged into the filename. Ended up doing a fairly extensive review for places where filenames could get too large, although it's hard to say I'm not missed any.. Backend.Url had a 128 character limit, which is fine when the limit is 255, but not if it's a lot shorter on some systems. So check the pathconf() limit. Note that this could result in fromUrl creating different keys for the same url, if run on systems with different limits. I don't see this is likely to cause any problems. That can already happen when using addurl --fast, or if the content of an url changes. Both Command.AddUrl and Backend.Url assumed that urls don't contain a lot of multi-byte unicode, and would fail to truncate an url that did properly. A few places use a filename as the template to make a temp file. While that's nice in that the temp file name can be easily related back to the original filename, it could lead to `git annex add` failing to add a filename that was at or close to the maximum length. Note that in Command.Add.lockdown, the template is still derived from the filename, just with enough space left to turn it into a temp file. This is an important optimisation, because the assistant may lock down a bunch of files all at once, and using the same template for all of them would cause openTempFile to iterate through the same set of names, looking for an unused temp file. I'm not very happy with the relatedTemplate hack, but it avoids that slowdown. Backend.WORM does not limit the filename stored in the key. I have not tried to change that; so git annex add will fail on really long filenames when using the WORM backend. It seems better to preserve the invariant that a WORM key always contains the complete filename, since the filename is the only unique material in the key, other than mtime and size. Since nobody has complained about add failing (I think I saw it once?) on WORM, probably it's ok, or nobody but me uses it. There may be compatability problems if using git annex addurl --fast or the WORM backend on a system with the 255 limit and then trying to use that repo in a system with a smaller limit. I have not tried to deal with those. This commit was sponsored by Alexander Brem. Thanks!
2013-07-30 21:49:11 +00:00
{- It's not safe to use a FilePath of an existing file as the template
- for openTempFile, because if the FilePath is really long, the tmpfile
- will be longer, and may exceed the maximum filename length.
-
- This generates a template that is never too long.
- (Well, it allocates 20 characters for use in making a unique temp file,
- anyway, which is enough for the current implementation and any
- likely implementation.)
-}
relatedTemplate :: FilePath -> FilePath
relatedTemplate f
| len > 20 =
{- Some filesystems like FAT have issues with filenames
- ending in ".", so avoid truncating a filename to end
- that way. -}
reverse $ dropWhile (== '.') $ reverse $
truncateFilePath (len - 20) f
Fix a few bugs involving filenames that are at or near the filesystem's maximum filename length limit. Started with a problem when running addurl on a really long url, because the whole url is munged into the filename. Ended up doing a fairly extensive review for places where filenames could get too large, although it's hard to say I'm not missed any.. Backend.Url had a 128 character limit, which is fine when the limit is 255, but not if it's a lot shorter on some systems. So check the pathconf() limit. Note that this could result in fromUrl creating different keys for the same url, if run on systems with different limits. I don't see this is likely to cause any problems. That can already happen when using addurl --fast, or if the content of an url changes. Both Command.AddUrl and Backend.Url assumed that urls don't contain a lot of multi-byte unicode, and would fail to truncate an url that did properly. A few places use a filename as the template to make a temp file. While that's nice in that the temp file name can be easily related back to the original filename, it could lead to `git annex add` failing to add a filename that was at or close to the maximum length. Note that in Command.Add.lockdown, the template is still derived from the filename, just with enough space left to turn it into a temp file. This is an important optimisation, because the assistant may lock down a bunch of files all at once, and using the same template for all of them would cause openTempFile to iterate through the same set of names, looking for an unused temp file. I'm not very happy with the relatedTemplate hack, but it avoids that slowdown. Backend.WORM does not limit the filename stored in the key. I have not tried to change that; so git annex add will fail on really long filenames when using the WORM backend. It seems better to preserve the invariant that a WORM key always contains the complete filename, since the filename is the only unique material in the key, other than mtime and size. Since nobody has complained about add failing (I think I saw it once?) on WORM, probably it's ok, or nobody but me uses it. There may be compatability problems if using git annex addurl --fast or the WORM backend on a system with the 255 limit and then trying to use that repo in a system with a smaller limit. I have not tried to deal with those. This commit was sponsored by Alexander Brem. Thanks!
2013-07-30 21:49:11 +00:00
| otherwise = f
where
len = length f