if external hash command fails for any reason, fall back to internal hashing

This way, if a system's sha1sum etc is broken, it will be tried if
git-annex was built to use it, but at least it will fall back to using
internal hashing when it fails.

A side benefit of this is that hashFile consistently throws an IOError if
the file is unable to be read. In particular, if the disk is failing with
IO errors, and external hash command is used, it used to throw a user error
with the error message from externalSHA. Now, the external hash command
will fail, that message will be printed as a warning, and it'll fall back
to the internal hash command. If the disk IO error is not intermittent, it
will re-occur, and so an IOError will be thrown.

Of course, this can mean it reads a file twice, but only in edge cases.
This commit is contained in:
Joey Hess 2015-05-27 15:58:32 -04:00
parent 7cb16b9efb
commit b256f861ca

View file

@ -146,17 +146,25 @@ trivialMigrate oldkey newbackend afile
| otherwise = Nothing | otherwise = Nothing
hashFile :: Hash -> FilePath -> Integer -> Annex String hashFile :: Hash -> FilePath -> Integer -> Annex String
hashFile hash file filesize = liftIO $ go hash hashFile hash file filesize = go hash
where where
go (SHAHash hashsize) = case shaHasher hashsize filesize of go (SHAHash hashsize) = case shaHasher hashsize filesize of
Left sha -> sha <$> L.readFile file Left sha -> use sha
Right command -> Right (external, internal) -> do
either error return v <- liftIO $ externalSHA external hashsize file
=<< externalSHA command hashsize file case v of
go (SkeinHash hashsize) = skeinHasher hashsize <$> L.readFile file Right r -> return r
go MD5Hash = md5Hasher <$> L.readFile file Left e -> do
warning e
-- fall back to internal since
-- external command failed
use internal
go (SkeinHash hashsize) = use (skeinHasher hashsize)
go MD5Hash = use md5Hasher
use hasher = liftIO $ hasher <$> L.readFile file
shaHasher :: HashSize -> Integer -> Either (L.ByteString -> String) String shaHasher :: HashSize -> Integer -> Either (L.ByteString -> String) (String, L.ByteString -> String)
shaHasher hashsize filesize shaHasher hashsize filesize
| hashsize == 1 = use SysConfig.sha1 sha1 | hashsize == 1 = use SysConfig.sha1 sha1
| hashsize == 256 = use SysConfig.sha256 sha256 | hashsize == 256 = use SysConfig.sha256 sha256
@ -173,7 +181,7 @@ shaHasher hashsize filesize
- and system. So there is no point forking an external - and system. So there is no point forking an external
- process unless the file is large. -} - process unless the file is large. -}
| filesize < 1048576 = use Nothing hasher | filesize < 1048576 = use Nothing hasher
| otherwise = Right c | otherwise = Right (c, show . hasher)
skeinHasher :: HashSize -> (L.ByteString -> String) skeinHasher :: HashSize -> (L.ByteString -> String)
skeinHasher hashsize skeinHasher hashsize