mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

directory CoW on retrieve

Browse source 
directory: When cp supports reflinks, use it when getting content from a
directory special remote.

Not yet for imports from directory though, and not for store.

Note that, when it's chunked, using cp --reflink would not speed it up, and
when reflink was not supported, would unnecessarily write the chunk to a
file before reading it back in. So, only using a fileRetriever in the
NoChunks case is necessary to keep chunking fast.

fileCopier is told not to verify, because the special remote interface
does not yet support verification in passing. AFAICS, fileCopies can
never return False when not verifying so the added giveup should never
actually happen.
This commit is contained in:
Joey Hess 2021-04-14 14:43:08 -04:00
parent a36be49b01
commit 7bb93896af
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
2 changed files with 12 additions and 4 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
CHANGELOG
View file

@ -9,6 +9,8 @@ git-annex (8.20210331) UNRELEASED; urgency=medium
exporttree remotes in some unusual circumstances.
* fsck: When downloading content from a remote, if the content is able
to be verified during the transfer, skip checksumming it a second time.
* directory: When cp supports reflinks, use it when getting content from
a directory special remote.
-- Joey Hess <id@joeyh.name> Thu, 01 Apr 2021 12:17:26 -0400

14
Remote/Directory.hs
View file

@ -31,6 +31,7 @@ import Remote.Helper.Special
import Remote.Helper.ExportImport
import Types.Import
import qualified Remote.Directory.LegacyChunked as Legacy
import Annex.CopyFile
import Annex.Content
import Annex.Perms
import Annex.UUID
@ -67,9 +68,10 @@ gen r u rc gc rs = do
c <- parsedRemoteConfig remote rc
cst <- remoteCost gc cheapRemoteCost
let chunkconfig = getChunkConfig c
cow <- liftIO newCopyCoWTried
return $ Just $ specialRemote c
(storeKeyM dir chunkconfig)
(retrieveKeyFileM dir chunkconfig)
(retrieveKeyFileM dir chunkconfig cow)
(removeKeyM dir)
(checkPresentM dir chunkconfig)
Remote
@ -220,9 +222,13 @@ finalizeStoreGeneric d tmp dest = do
where
dest' = fromRawFilePath dest
retrieveKeyFileM :: RawFilePath -> ChunkConfig -> Retriever
retrieveKeyFileM d (LegacyChunks _) = Legacy.retrieve locations d
retrieveKeyFileM d _ = byteRetriever $ \k sink ->
retrieveKeyFileM :: RawFilePath -> ChunkConfig -> CopyCoWTried -> Retriever
retrieveKeyFileM d (LegacyChunks _) _ = Legacy.retrieve locations d
retrieveKeyFileM d NoChunks cow = fileRetriever $ \dest k p -> do
src <- liftIO $ fromRawFilePath <$> getLocation d k
(ok, _verification) <- fileCopier cow src dest k p (return True) NoVerify
unless ok $ giveup "failed to copy file from remote"
retrieveKeyFileM d _ _ = byteRetriever $ \k sink ->
sink =<< liftIO (L.readFile . fromRawFilePath =<< getLocation d k)
retrieveKeyFileCheapM :: RawFilePath -> ChunkConfig -> Maybe (Key -> AssociatedFile -> FilePath -> Annex ())