if external hash command fails for any reason, fall back to internal hashing

This way, if a system's sha1sum etc is broken, it will be tried if
git-annex was built to use it, but at least it will fall back to using
internal hashing when it fails.

A side benefit of this is that hashFile consistently throws an IOError if
the file is unable to be read. In particular, if the disk is failing with
IO errors, and external hash command is used, it used to throw a user error
with the error message from externalSHA. Now, the external hash command
will fail, that message will be printed as a warning, and it'll fall back
to the internal hash command. If the disk IO error is not intermittent, it
will re-occur, and so an IOError will be thrown.

Of course, this can mean it reads a file twice, but only in edge cases.
This commit is contained in:
Joey Hess 2015-05-27 15:58:32 -04:00
parent 7cb16b9efb
commit b256f861ca

View file

@ -146,17 +146,25 @@ trivialMigrate oldkey newbackend afile
| otherwise = Nothing
hashFile :: Hash -> FilePath -> Integer -> Annex String
hashFile hash file filesize = liftIO $ go hash
hashFile hash file filesize = go hash
where
go (SHAHash hashsize) = case shaHasher hashsize filesize of
Left sha -> sha <$> L.readFile file
Right command ->
either error return
=<< externalSHA command hashsize file
go (SkeinHash hashsize) = skeinHasher hashsize <$> L.readFile file
go MD5Hash = md5Hasher <$> L.readFile file
Left sha -> use sha
Right (external, internal) -> do
v <- liftIO $ externalSHA external hashsize file
case v of
Right r -> return r
Left e -> do
warning e
-- fall back to internal since
-- external command failed
use internal
go (SkeinHash hashsize) = use (skeinHasher hashsize)
go MD5Hash = use md5Hasher
shaHasher :: HashSize -> Integer -> Either (L.ByteString -> String) String
use hasher = liftIO $ hasher <$> L.readFile file
shaHasher :: HashSize -> Integer -> Either (L.ByteString -> String) (String, L.ByteString -> String)
shaHasher hashsize filesize
| hashsize == 1 = use SysConfig.sha1 sha1
| hashsize == 256 = use SysConfig.sha256 sha256
@ -173,7 +181,7 @@ shaHasher hashsize filesize
- and system. So there is no point forking an external
- process unless the file is large. -}
| filesize < 1048576 = use Nothing hasher
| otherwise = Right c
| otherwise = Right (c, show . hasher)
skeinHasher :: HashSize -> (L.ByteString -> String)
skeinHasher hashsize