mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
git-annex/Utility/Hash.hs

276 lines
8.7 KiB
Haskell
Raw Normal View History

stop using MissingH for MD5 Cryptonite is faster and allocates less, and I want to get rid of MissingH use. Note that the new dependency on memory is free; it's a dependency of cryptonite. This commit was supported by the NSF-funded DataLad project.
2017-05-15 22:10:13 +00:00
{- Convenience wrapper around cryptonite's hashing. -}
Added support for SHA3 hashed keys (in 8 varieties), when git-annex is built using the cryptonite library. While cryptohash has SHA3 support, it has not been updated for the final version of the spec. Note that cryptonite has not been ported to all arches that cryptohash builds on yet.
2015-08-06 19:02:25 +00:00
allow building w/o cryptohash Mostly for the debian stable autobuilds, which have a too old version to use the Crypto.Hash module.
2013-10-03 16:33:38 +00:00
module Utility.Hash (
sha1,
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
sha1_context,
Added support for SHA3 hashed keys (in 8 varieties), when git-annex is built using the cryptonite library. While cryptohash has SHA3 support, it has not been updated for the final version of the spec. Note that cryptonite has not been ported to all arches that cryptohash builds on yet.
2015-08-06 19:02:25 +00:00
sha2_224,
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
sha2_224_context,
Added support for SHA3 hashed keys (in 8 varieties), when git-annex is built using the cryptonite library. While cryptohash has SHA3 support, it has not been updated for the final version of the spec. Note that cryptonite has not been ported to all arches that cryptohash builds on yet.
2015-08-06 19:02:25 +00:00
sha2_256,
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
sha2_256_context,
Added support for SHA3 hashed keys (in 8 varieties), when git-annex is built using the cryptonite library. While cryptohash has SHA3 support, it has not been updated for the final version of the spec. Note that cryptonite has not been ported to all arches that cryptohash builds on yet.
2015-08-06 19:02:25 +00:00
sha2_384,
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
sha2_384_context,
Added support for SHA3 hashed keys (in 8 varieties), when git-annex is built using the cryptonite library. While cryptohash has SHA3 support, it has not been updated for the final version of the spec. Note that cryptonite has not been ported to all arches that cryptohash builds on yet.
2015-08-06 19:02:25 +00:00
sha2_512,
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
sha2_512_context,
Added support for SHA3 hashed keys (in 8 varieties), when git-annex is built using the cryptonite library. While cryptohash has SHA3 support, it has not been updated for the final version of the spec. Note that cryptonite has not been ported to all arches that cryptohash builds on yet.
2015-08-06 19:02:25 +00:00
sha3_224,
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
sha3_224_context,
Added support for SHA3 hashed keys (in 8 varieties), when git-annex is built using the cryptonite library. While cryptohash has SHA3 support, it has not been updated for the final version of the spec. Note that cryptonite has not been ported to all arches that cryptohash builds on yet.
2015-08-06 19:02:25 +00:00
sha3_256,
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
sha3_256_context,
Added support for SHA3 hashed keys (in 8 varieties), when git-annex is built using the cryptonite library. While cryptohash has SHA3 support, it has not been updated for the final version of the spec. Note that cryptonite has not been ported to all arches that cryptohash builds on yet.
2015-08-06 19:02:25 +00:00
sha3_384,
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
sha3_384_context,
Added support for SHA3 hashed keys (in 8 varieties), when git-annex is built using the cryptonite library. While cryptohash has SHA3 support, it has not been updated for the final version of the spec. Note that cryptonite has not been ported to all arches that cryptohash builds on yet.
2015-08-06 19:02:25 +00:00
sha3_512,
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
sha3_512_context,
allow building w/o cryptohash Mostly for the debian stable autobuilds, which have a too old version to use the Crypto.Hash module.
2013-10-03 16:33:38 +00:00
skein256,
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
skein256_context,
allow building w/o cryptohash Mostly for the debian stable autobuilds, which have a too old version to use the Crypto.Hash module.
2013-10-03 16:33:38 +00:00
skein512,
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
skein512_context,
Added backends for the BLAKE2 family of hashes. There are a lot of different variants and sizes, I suppose we might as well export all the common ones. Bump dep to cryptonite to 0.16, earlier versions lacked BLAKE2 support. Even android has 0.16 or newer. On Debian, Blake2bp_512 is buggy, so I have omitted it for now. http://bugs.debian.org/892855 This commit was sponsored by andrea rota.
2018-03-13 20:15:35 +00:00
blake2s_160,
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
blake2s_160_context,
Added backends for the BLAKE2 family of hashes. There are a lot of different variants and sizes, I suppose we might as well export all the common ones. Bump dep to cryptonite to 0.16, earlier versions lacked BLAKE2 support. Even android has 0.16 or newer. On Debian, Blake2bp_512 is buggy, so I have omitted it for now. http://bugs.debian.org/892855 This commit was sponsored by andrea rota.
2018-03-13 20:15:35 +00:00
blake2s_224,
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
blake2s_224_context,
Added backends for the BLAKE2 family of hashes. There are a lot of different variants and sizes, I suppose we might as well export all the common ones. Bump dep to cryptonite to 0.16, earlier versions lacked BLAKE2 support. Even android has 0.16 or newer. On Debian, Blake2bp_512 is buggy, so I have omitted it for now. http://bugs.debian.org/892855 This commit was sponsored by andrea rota.
2018-03-13 20:15:35 +00:00
blake2s_256,
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
blake2s_256_context,
Added backends for the BLAKE2 family of hashes. There are a lot of different variants and sizes, I suppose we might as well export all the common ones. Bump dep to cryptonite to 0.16, earlier versions lacked BLAKE2 support. Even android has 0.16 or newer. On Debian, Blake2bp_512 is buggy, so I have omitted it for now. http://bugs.debian.org/892855 This commit was sponsored by andrea rota.
2018-03-13 20:15:35 +00:00
blake2sp_224,
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
blake2sp_224_context,
Added backends for the BLAKE2 family of hashes. There are a lot of different variants and sizes, I suppose we might as well export all the common ones. Bump dep to cryptonite to 0.16, earlier versions lacked BLAKE2 support. Even android has 0.16 or newer. On Debian, Blake2bp_512 is buggy, so I have omitted it for now. http://bugs.debian.org/892855 This commit was sponsored by andrea rota.
2018-03-13 20:15:35 +00:00
blake2sp_256,
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
blake2sp_256_context,
Added backends for the BLAKE2 family of hashes. There are a lot of different variants and sizes, I suppose we might as well export all the common ones. Bump dep to cryptonite to 0.16, earlier versions lacked BLAKE2 support. Even android has 0.16 or newer. On Debian, Blake2bp_512 is buggy, so I have omitted it for now. http://bugs.debian.org/892855 This commit was sponsored by andrea rota.
2018-03-13 20:15:35 +00:00
blake2b_160,
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
blake2b_160_context,
Added backends for the BLAKE2 family of hashes. There are a lot of different variants and sizes, I suppose we might as well export all the common ones. Bump dep to cryptonite to 0.16, earlier versions lacked BLAKE2 support. Even android has 0.16 or newer. On Debian, Blake2bp_512 is buggy, so I have omitted it for now. http://bugs.debian.org/892855 This commit was sponsored by andrea rota.
2018-03-13 20:15:35 +00:00
blake2b_224,
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
blake2b_224_context,
Added backends for the BLAKE2 family of hashes. There are a lot of different variants and sizes, I suppose we might as well export all the common ones. Bump dep to cryptonite to 0.16, earlier versions lacked BLAKE2 support. Even android has 0.16 or newer. On Debian, Blake2bp_512 is buggy, so I have omitted it for now. http://bugs.debian.org/892855 This commit was sponsored by andrea rota.
2018-03-13 20:15:35 +00:00
blake2b_256,
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
blake2b_256_context,
Added backends for the BLAKE2 family of hashes. There are a lot of different variants and sizes, I suppose we might as well export all the common ones. Bump dep to cryptonite to 0.16, earlier versions lacked BLAKE2 support. Even android has 0.16 or newer. On Debian, Blake2bp_512 is buggy, so I have omitted it for now. http://bugs.debian.org/892855 This commit was sponsored by andrea rota.
2018-03-13 20:15:35 +00:00
blake2b_384,
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
blake2b_384_context,
Added backends for the BLAKE2 family of hashes. There are a lot of different variants and sizes, I suppose we might as well export all the common ones. Bump dep to cryptonite to 0.16, earlier versions lacked BLAKE2 support. Even android has 0.16 or newer. On Debian, Blake2bp_512 is buggy, so I have omitted it for now. http://bugs.debian.org/892855 This commit was sponsored by andrea rota.
2018-03-13 20:15:35 +00:00
blake2b_512,
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
blake2b_512_context,
Add BLAKE2BP512 and BLAKE2BP512E backends using a blake2 variant optimised for 4-way CPUs This had been deferred because the Debian package of cryptonite, and possibly other builds, was broken for blake2bp, but I've confirmed #892855 is fixed. This commit was sponsored by Brett Eisenberg on Patreon.
2019-07-05 19:29:00 +00:00
blake2bp_512,
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
blake2bp_512_context,
Added MD5 and MD5E backends.
2015-02-04 17:47:54 +00:00
md5,
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
md5_context,
cache the serialization of a Key This will speed up the common case where a Key is deserialized from disk, but is then serialized to build eg, the path to the annex object. Previously attempted in 4536c93bb2ecf114ab711beac33fa358facd6985 and reverted in 96aba8eff7597898f53bfbab5865ff30a927d355. The problems mentioned in the latter commit are addressed now: Read/Show of KeyData is backwards-compatible with Read/Show of Key from before this change, so Types.Distribution will keep working. The Eq instance is fixed. Also, Key has smart constructors, avoiding needing to remember to update the cached serialization. Used git-annex benchmark: find is 7% faster whereis is 3% faster get when all files are already present is 5% faster Generally, the benchmarks are running 0.1 seconds faster per 2000 files, on a ram disk in my laptop.
2019-11-22 20:24:04 +00:00
md5s,
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
hashUpdate,
hashFinalize,
Digest,
HashAlgorithm,
Context,
split up quickcheck tests for hashes and macs So when one fais, it's clear which one is the problem.
2020-03-02 18:25:45 +00:00
props_hashes_stable,
refactor
2015-04-19 14:57:14 +00:00
Mac(..),
calcMac,
split up quickcheck tests for hashes and macs So when one fais, it's clear which one is the problem.
2020-03-02 18:25:45 +00:00
props_macs_stable,
allow building w/o cryptohash Mostly for the debian stable autobuilds, which have a too old version to use the Crypto.Hash module.
2013-10-03 16:33:38 +00:00
) where
Use cryptohash rather than SHA for hashing. This is a massive win on OSX, which doesn't have a sha256sum normally. Only use external hash commands when the file is > 1 mb, since cryptohash is quite close to them in speed. SHA is still used to calculate HMACs. I don't quite understand cryptohash's API for those. Used the following benchmark to arrive at the 1 mb number. 1 mb file: benchmarking sha256/internal mean: 13.86696 ms, lb 13.83010 ms, ub 13.93453 ms, ci 0.950 std dev: 249.3235 us, lb 162.0448 us, ub 458.1744 us, ci 0.950 found 5 outliers among 100 samples (5.0%) 4 (4.0%) high mild 1 (1.0%) high severe variance introduced by outliers: 10.415% variance is moderately inflated by outliers benchmarking sha256/external mean: 14.20670 ms, lb 14.17237 ms, ub 14.27004 ms, ci 0.950 std dev: 230.5448 us, lb 150.7310 us, ub 427.6068 us, ci 0.950 found 3 outliers among 100 samples (3.0%) 2 (2.0%) high mild 1 (1.0%) high severe 2 mb file: benchmarking sha256/internal mean: 26.44270 ms, lb 26.23701 ms, ub 26.63414 ms, ci 0.950 std dev: 1.012303 ms, lb 925.8921 us, ub 1.122267 ms, ci 0.950 variance introduced by outliers: 35.540% variance is moderately inflated by outliers benchmarking sha256/external mean: 26.84521 ms, lb 26.77644 ms, ub 26.91433 ms, ci 0.950 std dev: 347.7867 us, lb 210.6283 us, ub 571.3351 us, ci 0.950 found 6 outliers among 100 samples (6.0%) import Crypto.Hash import Data.ByteString.Lazy as L import Criterion.Main import Common testfile :: FilePath testfile = "/run/shm/data" -- on ram disk main = defaultMain [ bgroup "sha256" [ bench "internal" $ whnfIO internal , bench "external" $ whnfIO external ] ] sha256 :: L.ByteString -> Digest SHA256 sha256 = hashlazy internal :: IO String internal = show . sha256 <$> L.readFile testfile external :: IO String external = do s <- readProcess "sha256sum" [testfile] return $ fst $ separate (== ' ') s
2013-09-22 23:45:08 +00:00
convert serializeKey' to strict ByteString The builder produces a lazy ByteString, and L.toStrict has to copy it, but needing to use the builder is no longer to common case; the serialization will normally be cached already as a strict ByteString, and this avoids keyFile' needing to use L.toStrict . serializeKey'
2019-01-14 20:59:27 +00:00
import qualified Data.ByteString as S
Use cryptohash rather than SHA for hashing. This is a massive win on OSX, which doesn't have a sha256sum normally. Only use external hash commands when the file is > 1 mb, since cryptohash is quite close to them in speed. SHA is still used to calculate HMACs. I don't quite understand cryptohash's API for those. Used the following benchmark to arrive at the 1 mb number. 1 mb file: benchmarking sha256/internal mean: 13.86696 ms, lb 13.83010 ms, ub 13.93453 ms, ci 0.950 std dev: 249.3235 us, lb 162.0448 us, ub 458.1744 us, ci 0.950 found 5 outliers among 100 samples (5.0%) 4 (4.0%) high mild 1 (1.0%) high severe variance introduced by outliers: 10.415% variance is moderately inflated by outliers benchmarking sha256/external mean: 14.20670 ms, lb 14.17237 ms, ub 14.27004 ms, ci 0.950 std dev: 230.5448 us, lb 150.7310 us, ub 427.6068 us, ci 0.950 found 3 outliers among 100 samples (3.0%) 2 (2.0%) high mild 1 (1.0%) high severe 2 mb file: benchmarking sha256/internal mean: 26.44270 ms, lb 26.23701 ms, ub 26.63414 ms, ci 0.950 std dev: 1.012303 ms, lb 925.8921 us, ub 1.122267 ms, ci 0.950 variance introduced by outliers: 35.540% variance is moderately inflated by outliers benchmarking sha256/external mean: 26.84521 ms, lb 26.77644 ms, ub 26.91433 ms, ci 0.950 std dev: 347.7867 us, lb 210.6283 us, ub 571.3351 us, ci 0.950 found 6 outliers among 100 samples (6.0%) import Crypto.Hash import Data.ByteString.Lazy as L import Criterion.Main import Common testfile :: FilePath testfile = "/run/shm/data" -- on ram disk main = defaultMain [ bgroup "sha256" [ bench "internal" $ whnfIO internal , bench "external" $ whnfIO external ] ] sha256 :: L.ByteString -> Digest SHA256 sha256 = hashlazy internal :: IO String internal = show . sha256 <$> L.readFile testfile external :: IO String external = do s <- readProcess "sha256sum" [testfile] return $ fst $ separate (== ' ') s
2013-09-22 23:45:08 +00:00
import qualified Data.ByteString.Lazy as L
get rid of (completely safe) uses of Char8 Char8 often indicates an encoding bug. It didn't here, but I can avoid it and not worry about it.
2014-05-28 00:23:53 +00:00
import qualified Data.Text as T
import qualified Data.Text.Encoding as T
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
import "cryptonite" Crypto.MAC.HMAC hiding (Context)
package qualify imports needed for "make fast" to work
2015-08-14 21:23:25 +00:00
import "cryptonite" Crypto.Hash
allow building w/o cryptohash Mostly for the debian stable autobuilds, which have a too old version to use the Crypto.Hash module.
2013-10-03 16:33:38 +00:00
Use cryptohash rather than SHA for hashing. This is a massive win on OSX, which doesn't have a sha256sum normally. Only use external hash commands when the file is > 1 mb, since cryptohash is quite close to them in speed. SHA is still used to calculate HMACs. I don't quite understand cryptohash's API for those. Used the following benchmark to arrive at the 1 mb number. 1 mb file: benchmarking sha256/internal mean: 13.86696 ms, lb 13.83010 ms, ub 13.93453 ms, ci 0.950 std dev: 249.3235 us, lb 162.0448 us, ub 458.1744 us, ci 0.950 found 5 outliers among 100 samples (5.0%) 4 (4.0%) high mild 1 (1.0%) high severe variance introduced by outliers: 10.415% variance is moderately inflated by outliers benchmarking sha256/external mean: 14.20670 ms, lb 14.17237 ms, ub 14.27004 ms, ci 0.950 std dev: 230.5448 us, lb 150.7310 us, ub 427.6068 us, ci 0.950 found 3 outliers among 100 samples (3.0%) 2 (2.0%) high mild 1 (1.0%) high severe 2 mb file: benchmarking sha256/internal mean: 26.44270 ms, lb 26.23701 ms, ub 26.63414 ms, ci 0.950 std dev: 1.012303 ms, lb 925.8921 us, ub 1.122267 ms, ci 0.950 variance introduced by outliers: 35.540% variance is moderately inflated by outliers benchmarking sha256/external mean: 26.84521 ms, lb 26.77644 ms, ub 26.91433 ms, ci 0.950 std dev: 347.7867 us, lb 210.6283 us, ub 571.3351 us, ci 0.950 found 6 outliers among 100 samples (6.0%) import Crypto.Hash import Data.ByteString.Lazy as L import Criterion.Main import Common testfile :: FilePath testfile = "/run/shm/data" -- on ram disk main = defaultMain [ bgroup "sha256" [ bench "internal" $ whnfIO internal , bench "external" $ whnfIO external ] ] sha256 :: L.ByteString -> Digest SHA256 sha256 = hashlazy internal :: IO String internal = show . sha256 <$> L.readFile testfile external :: IO String external = do s <- readProcess "sha256sum" [testfile] return $ fst $ separate (== ' ') s
2013-09-22 23:45:08 +00:00
sha1 :: L.ByteString -> Digest SHA1
sha1 = hashlazy
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
sha1_context :: Context SHA1
sha1_context = hashInit
Added support for SHA3 hashed keys (in 8 varieties), when git-annex is built using the cryptonite library. While cryptohash has SHA3 support, it has not been updated for the final version of the spec. Note that cryptonite has not been ported to all arches that cryptohash builds on yet.
2015-08-06 19:02:25 +00:00
sha2_224 :: L.ByteString -> Digest SHA224
sha2_224 = hashlazy
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
sha2_224_context :: Context SHA224
sha2_224_context = hashInit
Added support for SHA3 hashed keys (in 8 varieties), when git-annex is built using the cryptonite library. While cryptohash has SHA3 support, it has not been updated for the final version of the spec. Note that cryptonite has not been ported to all arches that cryptohash builds on yet.
2015-08-06 19:02:25 +00:00
sha2_256 :: L.ByteString -> Digest SHA256
sha2_256 = hashlazy
Use cryptohash rather than SHA for hashing. This is a massive win on OSX, which doesn't have a sha256sum normally. Only use external hash commands when the file is > 1 mb, since cryptohash is quite close to them in speed. SHA is still used to calculate HMACs. I don't quite understand cryptohash's API for those. Used the following benchmark to arrive at the 1 mb number. 1 mb file: benchmarking sha256/internal mean: 13.86696 ms, lb 13.83010 ms, ub 13.93453 ms, ci 0.950 std dev: 249.3235 us, lb 162.0448 us, ub 458.1744 us, ci 0.950 found 5 outliers among 100 samples (5.0%) 4 (4.0%) high mild 1 (1.0%) high severe variance introduced by outliers: 10.415% variance is moderately inflated by outliers benchmarking sha256/external mean: 14.20670 ms, lb 14.17237 ms, ub 14.27004 ms, ci 0.950 std dev: 230.5448 us, lb 150.7310 us, ub 427.6068 us, ci 0.950 found 3 outliers among 100 samples (3.0%) 2 (2.0%) high mild 1 (1.0%) high severe 2 mb file: benchmarking sha256/internal mean: 26.44270 ms, lb 26.23701 ms, ub 26.63414 ms, ci 0.950 std dev: 1.012303 ms, lb 925.8921 us, ub 1.122267 ms, ci 0.950 variance introduced by outliers: 35.540% variance is moderately inflated by outliers benchmarking sha256/external mean: 26.84521 ms, lb 26.77644 ms, ub 26.91433 ms, ci 0.950 std dev: 347.7867 us, lb 210.6283 us, ub 571.3351 us, ci 0.950 found 6 outliers among 100 samples (6.0%) import Crypto.Hash import Data.ByteString.Lazy as L import Criterion.Main import Common testfile :: FilePath testfile = "/run/shm/data" -- on ram disk main = defaultMain [ bgroup "sha256" [ bench "internal" $ whnfIO internal , bench "external" $ whnfIO external ] ] sha256 :: L.ByteString -> Digest SHA256 sha256 = hashlazy internal :: IO String internal = show . sha256 <$> L.readFile testfile external :: IO String external = do s <- readProcess "sha256sum" [testfile] return $ fst $ separate (== ' ') s
2013-09-22 23:45:08 +00:00
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
sha2_256_context :: Context SHA256
sha2_256_context = hashInit
Added support for SHA3 hashed keys (in 8 varieties), when git-annex is built using the cryptonite library. While cryptohash has SHA3 support, it has not been updated for the final version of the spec. Note that cryptonite has not been ported to all arches that cryptohash builds on yet.
2015-08-06 19:02:25 +00:00
sha2_384 :: L.ByteString -> Digest SHA384
sha2_384 = hashlazy
Use cryptohash rather than SHA for hashing. This is a massive win on OSX, which doesn't have a sha256sum normally. Only use external hash commands when the file is > 1 mb, since cryptohash is quite close to them in speed. SHA is still used to calculate HMACs. I don't quite understand cryptohash's API for those. Used the following benchmark to arrive at the 1 mb number. 1 mb file: benchmarking sha256/internal mean: 13.86696 ms, lb 13.83010 ms, ub 13.93453 ms, ci 0.950 std dev: 249.3235 us, lb 162.0448 us, ub 458.1744 us, ci 0.950 found 5 outliers among 100 samples (5.0%) 4 (4.0%) high mild 1 (1.0%) high severe variance introduced by outliers: 10.415% variance is moderately inflated by outliers benchmarking sha256/external mean: 14.20670 ms, lb 14.17237 ms, ub 14.27004 ms, ci 0.950 std dev: 230.5448 us, lb 150.7310 us, ub 427.6068 us, ci 0.950 found 3 outliers among 100 samples (3.0%) 2 (2.0%) high mild 1 (1.0%) high severe 2 mb file: benchmarking sha256/internal mean: 26.44270 ms, lb 26.23701 ms, ub 26.63414 ms, ci 0.950 std dev: 1.012303 ms, lb 925.8921 us, ub 1.122267 ms, ci 0.950 variance introduced by outliers: 35.540% variance is moderately inflated by outliers benchmarking sha256/external mean: 26.84521 ms, lb 26.77644 ms, ub 26.91433 ms, ci 0.950 std dev: 347.7867 us, lb 210.6283 us, ub 571.3351 us, ci 0.950 found 6 outliers among 100 samples (6.0%) import Crypto.Hash import Data.ByteString.Lazy as L import Criterion.Main import Common testfile :: FilePath testfile = "/run/shm/data" -- on ram disk main = defaultMain [ bgroup "sha256" [ bench "internal" $ whnfIO internal , bench "external" $ whnfIO external ] ] sha256 :: L.ByteString -> Digest SHA256 sha256 = hashlazy internal :: IO String internal = show . sha256 <$> L.readFile testfile external :: IO String external = do s <- readProcess "sha256sum" [testfile] return $ fst $ separate (== ' ') s
2013-09-22 23:45:08 +00:00
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
sha2_384_context :: Context SHA384
sha2_384_context = hashInit
Added support for SHA3 hashed keys (in 8 varieties), when git-annex is built using the cryptonite library. While cryptohash has SHA3 support, it has not been updated for the final version of the spec. Note that cryptonite has not been ported to all arches that cryptohash builds on yet.
2015-08-06 19:02:25 +00:00
sha2_512 :: L.ByteString -> Digest SHA512
sha2_512 = hashlazy
Use cryptohash rather than SHA for hashing. This is a massive win on OSX, which doesn't have a sha256sum normally. Only use external hash commands when the file is > 1 mb, since cryptohash is quite close to them in speed. SHA is still used to calculate HMACs. I don't quite understand cryptohash's API for those. Used the following benchmark to arrive at the 1 mb number. 1 mb file: benchmarking sha256/internal mean: 13.86696 ms, lb 13.83010 ms, ub 13.93453 ms, ci 0.950 std dev: 249.3235 us, lb 162.0448 us, ub 458.1744 us, ci 0.950 found 5 outliers among 100 samples (5.0%) 4 (4.0%) high mild 1 (1.0%) high severe variance introduced by outliers: 10.415% variance is moderately inflated by outliers benchmarking sha256/external mean: 14.20670 ms, lb 14.17237 ms, ub 14.27004 ms, ci 0.950 std dev: 230.5448 us, lb 150.7310 us, ub 427.6068 us, ci 0.950 found 3 outliers among 100 samples (3.0%) 2 (2.0%) high mild 1 (1.0%) high severe 2 mb file: benchmarking sha256/internal mean: 26.44270 ms, lb 26.23701 ms, ub 26.63414 ms, ci 0.950 std dev: 1.012303 ms, lb 925.8921 us, ub 1.122267 ms, ci 0.950 variance introduced by outliers: 35.540% variance is moderately inflated by outliers benchmarking sha256/external mean: 26.84521 ms, lb 26.77644 ms, ub 26.91433 ms, ci 0.950 std dev: 347.7867 us, lb 210.6283 us, ub 571.3351 us, ci 0.950 found 6 outliers among 100 samples (6.0%) import Crypto.Hash import Data.ByteString.Lazy as L import Criterion.Main import Common testfile :: FilePath testfile = "/run/shm/data" -- on ram disk main = defaultMain [ bgroup "sha256" [ bench "internal" $ whnfIO internal , bench "external" $ whnfIO external ] ] sha256 :: L.ByteString -> Digest SHA256 sha256 = hashlazy internal :: IO String internal = show . sha256 <$> L.readFile testfile external :: IO String external = do s <- readProcess "sha256sum" [testfile] return $ fst $ separate (== ' ') s
2013-09-22 23:45:08 +00:00
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
sha2_512_context :: Context SHA512
sha2_512_context = hashInit
Added support for SHA3 hashed keys (in 8 varieties), when git-annex is built using the cryptonite library. While cryptohash has SHA3 support, it has not been updated for the final version of the spec. Note that cryptonite has not been ported to all arches that cryptohash builds on yet.
2015-08-06 19:02:25 +00:00
sha3_224 :: L.ByteString -> Digest SHA3_224
sha3_224 = hashlazy
Use cryptohash rather than SHA for hashing. This is a massive win on OSX, which doesn't have a sha256sum normally. Only use external hash commands when the file is > 1 mb, since cryptohash is quite close to them in speed. SHA is still used to calculate HMACs. I don't quite understand cryptohash's API for those. Used the following benchmark to arrive at the 1 mb number. 1 mb file: benchmarking sha256/internal mean: 13.86696 ms, lb 13.83010 ms, ub 13.93453 ms, ci 0.950 std dev: 249.3235 us, lb 162.0448 us, ub 458.1744 us, ci 0.950 found 5 outliers among 100 samples (5.0%) 4 (4.0%) high mild 1 (1.0%) high severe variance introduced by outliers: 10.415% variance is moderately inflated by outliers benchmarking sha256/external mean: 14.20670 ms, lb 14.17237 ms, ub 14.27004 ms, ci 0.950 std dev: 230.5448 us, lb 150.7310 us, ub 427.6068 us, ci 0.950 found 3 outliers among 100 samples (3.0%) 2 (2.0%) high mild 1 (1.0%) high severe 2 mb file: benchmarking sha256/internal mean: 26.44270 ms, lb 26.23701 ms, ub 26.63414 ms, ci 0.950 std dev: 1.012303 ms, lb 925.8921 us, ub 1.122267 ms, ci 0.950 variance introduced by outliers: 35.540% variance is moderately inflated by outliers benchmarking sha256/external mean: 26.84521 ms, lb 26.77644 ms, ub 26.91433 ms, ci 0.950 std dev: 347.7867 us, lb 210.6283 us, ub 571.3351 us, ci 0.950 found 6 outliers among 100 samples (6.0%) import Crypto.Hash import Data.ByteString.Lazy as L import Criterion.Main import Common testfile :: FilePath testfile = "/run/shm/data" -- on ram disk main = defaultMain [ bgroup "sha256" [ bench "internal" $ whnfIO internal , bench "external" $ whnfIO external ] ] sha256 :: L.ByteString -> Digest SHA256 sha256 = hashlazy internal :: IO String internal = show . sha256 <$> L.readFile testfile external :: IO String external = do s <- readProcess "sha256sum" [testfile] return $ fst $ separate (== ' ') s
2013-09-22 23:45:08 +00:00
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
sha3_224_context :: Context SHA3_224
sha3_224_context = hashInit
Added support for SHA3 hashed keys (in 8 varieties), when git-annex is built using the cryptonite library. While cryptohash has SHA3 support, it has not been updated for the final version of the spec. Note that cryptonite has not been ported to all arches that cryptohash builds on yet.
2015-08-06 19:02:25 +00:00
sha3_256 :: L.ByteString -> Digest SHA3_256
sha3_256 = hashlazy
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
sha3_256_context :: Context SHA3_256
sha3_256_context = hashInit
Added support for SHA3 hashed keys (in 8 varieties), when git-annex is built using the cryptonite library. While cryptohash has SHA3 support, it has not been updated for the final version of the spec. Note that cryptonite has not been ported to all arches that cryptohash builds on yet.
2015-08-06 19:02:25 +00:00
sha3_384 :: L.ByteString -> Digest SHA3_384
sha3_384 = hashlazy
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
sha3_384_context :: Context SHA3_384
sha3_384_context = hashInit
Added support for SHA3 hashed keys (in 8 varieties), when git-annex is built using the cryptonite library. While cryptohash has SHA3 support, it has not been updated for the final version of the spec. Note that cryptonite has not been ported to all arches that cryptohash builds on yet.
2015-08-06 19:02:25 +00:00
sha3_512 :: L.ByteString -> Digest SHA3_512
sha3_512 = hashlazy
Use cryptohash rather than SHA for hashing. This is a massive win on OSX, which doesn't have a sha256sum normally. Only use external hash commands when the file is > 1 mb, since cryptohash is quite close to them in speed. SHA is still used to calculate HMACs. I don't quite understand cryptohash's API for those. Used the following benchmark to arrive at the 1 mb number. 1 mb file: benchmarking sha256/internal mean: 13.86696 ms, lb 13.83010 ms, ub 13.93453 ms, ci 0.950 std dev: 249.3235 us, lb 162.0448 us, ub 458.1744 us, ci 0.950 found 5 outliers among 100 samples (5.0%) 4 (4.0%) high mild 1 (1.0%) high severe variance introduced by outliers: 10.415% variance is moderately inflated by outliers benchmarking sha256/external mean: 14.20670 ms, lb 14.17237 ms, ub 14.27004 ms, ci 0.950 std dev: 230.5448 us, lb 150.7310 us, ub 427.6068 us, ci 0.950 found 3 outliers among 100 samples (3.0%) 2 (2.0%) high mild 1 (1.0%) high severe 2 mb file: benchmarking sha256/internal mean: 26.44270 ms, lb 26.23701 ms, ub 26.63414 ms, ci 0.950 std dev: 1.012303 ms, lb 925.8921 us, ub 1.122267 ms, ci 0.950 variance introduced by outliers: 35.540% variance is moderately inflated by outliers benchmarking sha256/external mean: 26.84521 ms, lb 26.77644 ms, ub 26.91433 ms, ci 0.950 std dev: 347.7867 us, lb 210.6283 us, ub 571.3351 us, ci 0.950 found 6 outliers among 100 samples (6.0%) import Crypto.Hash import Data.ByteString.Lazy as L import Criterion.Main import Common testfile :: FilePath testfile = "/run/shm/data" -- on ram disk main = defaultMain [ bgroup "sha256" [ bench "internal" $ whnfIO internal , bench "external" $ whnfIO external ] ] sha256 :: L.ByteString -> Digest SHA256 sha256 = hashlazy internal :: IO String internal = show . sha256 <$> L.readFile testfile external :: IO String external = do s <- readProcess "sha256sum" [testfile] return $ fst $ separate (== ' ') s
2013-09-22 23:45:08 +00:00
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
sha3_512_context :: Context SHA3_512
sha3_512_context = hashInit
Added SKEIN256 and SKEIN512 backends SHA3 is still waiting for final standardization. Although this is looking less likely given https://www.cdt.org/blogs/joseph-lorenzo-hall/2409-nist-sha-3 In the meantime, cryptohash implements skein, and it's used by some of the haskell ecosystem (for yesod sessions, IIRC), so this implementation is likely to continue working. Also, I've talked with the cryprohash author and he's a reasonable guy. It makes sense to have an alternate high security hash, in case some horrible attack is found against SHA2 tomorrow, or in case SHA3 comes out and worst fears are realized. I'd also like to support using skein for HMAC. But no hurry there and a new version of cryptohash has much nicer HMAC code, so I will probably wait until I can use that version.
2013-10-02 00:34:06 +00:00
skein256 :: L.ByteString -> Digest Skein256_256
skein256 = hashlazy
Use cryptohash rather than SHA for hashing. This is a massive win on OSX, which doesn't have a sha256sum normally. Only use external hash commands when the file is > 1 mb, since cryptohash is quite close to them in speed. SHA is still used to calculate HMACs. I don't quite understand cryptohash's API for those. Used the following benchmark to arrive at the 1 mb number. 1 mb file: benchmarking sha256/internal mean: 13.86696 ms, lb 13.83010 ms, ub 13.93453 ms, ci 0.950 std dev: 249.3235 us, lb 162.0448 us, ub 458.1744 us, ci 0.950 found 5 outliers among 100 samples (5.0%) 4 (4.0%) high mild 1 (1.0%) high severe variance introduced by outliers: 10.415% variance is moderately inflated by outliers benchmarking sha256/external mean: 14.20670 ms, lb 14.17237 ms, ub 14.27004 ms, ci 0.950 std dev: 230.5448 us, lb 150.7310 us, ub 427.6068 us, ci 0.950 found 3 outliers among 100 samples (3.0%) 2 (2.0%) high mild 1 (1.0%) high severe 2 mb file: benchmarking sha256/internal mean: 26.44270 ms, lb 26.23701 ms, ub 26.63414 ms, ci 0.950 std dev: 1.012303 ms, lb 925.8921 us, ub 1.122267 ms, ci 0.950 variance introduced by outliers: 35.540% variance is moderately inflated by outliers benchmarking sha256/external mean: 26.84521 ms, lb 26.77644 ms, ub 26.91433 ms, ci 0.950 std dev: 347.7867 us, lb 210.6283 us, ub 571.3351 us, ci 0.950 found 6 outliers among 100 samples (6.0%) import Crypto.Hash import Data.ByteString.Lazy as L import Criterion.Main import Common testfile :: FilePath testfile = "/run/shm/data" -- on ram disk main = defaultMain [ bgroup "sha256" [ bench "internal" $ whnfIO internal , bench "external" $ whnfIO external ] ] sha256 :: L.ByteString -> Digest SHA256 sha256 = hashlazy internal :: IO String internal = show . sha256 <$> L.readFile testfile external :: IO String external = do s <- readProcess "sha256sum" [testfile] return $ fst $ separate (== ' ') s
2013-09-22 23:45:08 +00:00
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
skein256_context :: Context Skein256_256
skein256_context = hashInit
Added SKEIN256 and SKEIN512 backends SHA3 is still waiting for final standardization. Although this is looking less likely given https://www.cdt.org/blogs/joseph-lorenzo-hall/2409-nist-sha-3 In the meantime, cryptohash implements skein, and it's used by some of the haskell ecosystem (for yesod sessions, IIRC), so this implementation is likely to continue working. Also, I've talked with the cryprohash author and he's a reasonable guy. It makes sense to have an alternate high security hash, in case some horrible attack is found against SHA2 tomorrow, or in case SHA3 comes out and worst fears are realized. I'd also like to support using skein for HMAC. But no hurry there and a new version of cryptohash has much nicer HMAC code, so I will probably wait until I can use that version.
2013-10-02 00:34:06 +00:00
skein512 :: L.ByteString -> Digest Skein512_512
skein512 = hashlazy
ensure that hash representations don't change in future
2013-10-02 01:10:56 +00:00
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
skein512_context :: Context Skein512_512
skein512_context = hashInit
Added backends for the BLAKE2 family of hashes. There are a lot of different variants and sizes, I suppose we might as well export all the common ones. Bump dep to cryptonite to 0.16, earlier versions lacked BLAKE2 support. Even android has 0.16 or newer. On Debian, Blake2bp_512 is buggy, so I have omitted it for now. http://bugs.debian.org/892855 This commit was sponsored by andrea rota.
2018-03-13 20:15:35 +00:00
blake2s_160 :: L.ByteString -> Digest Blake2s_160
blake2s_160 = hashlazy
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
blake2s_160_context :: Context Blake2s_160
blake2s_160_context = hashInit
Added backends for the BLAKE2 family of hashes. There are a lot of different variants and sizes, I suppose we might as well export all the common ones. Bump dep to cryptonite to 0.16, earlier versions lacked BLAKE2 support. Even android has 0.16 or newer. On Debian, Blake2bp_512 is buggy, so I have omitted it for now. http://bugs.debian.org/892855 This commit was sponsored by andrea rota.
2018-03-13 20:15:35 +00:00
blake2s_224 :: L.ByteString -> Digest Blake2s_224
blake2s_224 = hashlazy
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
blake2s_224_context :: Context Blake2s_224
blake2s_224_context = hashInit
Added backends for the BLAKE2 family of hashes. There are a lot of different variants and sizes, I suppose we might as well export all the common ones. Bump dep to cryptonite to 0.16, earlier versions lacked BLAKE2 support. Even android has 0.16 or newer. On Debian, Blake2bp_512 is buggy, so I have omitted it for now. http://bugs.debian.org/892855 This commit was sponsored by andrea rota.
2018-03-13 20:15:35 +00:00
blake2s_256 :: L.ByteString -> Digest Blake2s_256
blake2s_256 = hashlazy
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
blake2s_256_context :: Context Blake2s_256
blake2s_256_context = hashInit
Added backends for the BLAKE2 family of hashes. There are a lot of different variants and sizes, I suppose we might as well export all the common ones. Bump dep to cryptonite to 0.16, earlier versions lacked BLAKE2 support. Even android has 0.16 or newer. On Debian, Blake2bp_512 is buggy, so I have omitted it for now. http://bugs.debian.org/892855 This commit was sponsored by andrea rota.
2018-03-13 20:15:35 +00:00
blake2sp_224 :: L.ByteString -> Digest Blake2sp_224
blake2sp_224 = hashlazy
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
blake2sp_224_context :: Context Blake2sp_224
blake2sp_224_context = hashInit
Added backends for the BLAKE2 family of hashes. There are a lot of different variants and sizes, I suppose we might as well export all the common ones. Bump dep to cryptonite to 0.16, earlier versions lacked BLAKE2 support. Even android has 0.16 or newer. On Debian, Blake2bp_512 is buggy, so I have omitted it for now. http://bugs.debian.org/892855 This commit was sponsored by andrea rota.
2018-03-13 20:15:35 +00:00
blake2sp_256 :: L.ByteString -> Digest Blake2sp_256
blake2sp_256 = hashlazy
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
blake2sp_256_context :: Context Blake2sp_256
blake2sp_256_context = hashInit
Added backends for the BLAKE2 family of hashes. There are a lot of different variants and sizes, I suppose we might as well export all the common ones. Bump dep to cryptonite to 0.16, earlier versions lacked BLAKE2 support. Even android has 0.16 or newer. On Debian, Blake2bp_512 is buggy, so I have omitted it for now. http://bugs.debian.org/892855 This commit was sponsored by andrea rota.
2018-03-13 20:15:35 +00:00
blake2b_160 :: L.ByteString -> Digest Blake2b_160
blake2b_160 = hashlazy
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
blake2b_160_context :: Context Blake2b_160
blake2b_160_context = hashInit
Added backends for the BLAKE2 family of hashes. There are a lot of different variants and sizes, I suppose we might as well export all the common ones. Bump dep to cryptonite to 0.16, earlier versions lacked BLAKE2 support. Even android has 0.16 or newer. On Debian, Blake2bp_512 is buggy, so I have omitted it for now. http://bugs.debian.org/892855 This commit was sponsored by andrea rota.
2018-03-13 20:15:35 +00:00
blake2b_224 :: L.ByteString -> Digest Blake2b_224
blake2b_224 = hashlazy
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
blake2b_224_context :: Context Blake2b_224
blake2b_224_context = hashInit
Added backends for the BLAKE2 family of hashes. There are a lot of different variants and sizes, I suppose we might as well export all the common ones. Bump dep to cryptonite to 0.16, earlier versions lacked BLAKE2 support. Even android has 0.16 or newer. On Debian, Blake2bp_512 is buggy, so I have omitted it for now. http://bugs.debian.org/892855 This commit was sponsored by andrea rota.
2018-03-13 20:15:35 +00:00
blake2b_256 :: L.ByteString -> Digest Blake2b_256
blake2b_256 = hashlazy
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
blake2b_256_context :: Context Blake2b_256
blake2b_256_context = hashInit
Added backends for the BLAKE2 family of hashes. There are a lot of different variants and sizes, I suppose we might as well export all the common ones. Bump dep to cryptonite to 0.16, earlier versions lacked BLAKE2 support. Even android has 0.16 or newer. On Debian, Blake2bp_512 is buggy, so I have omitted it for now. http://bugs.debian.org/892855 This commit was sponsored by andrea rota.
2018-03-13 20:15:35 +00:00
blake2b_384 :: L.ByteString -> Digest Blake2b_384
blake2b_384 = hashlazy
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
blake2b_384_context :: Context Blake2b_384
blake2b_384_context = hashInit
Added backends for the BLAKE2 family of hashes. There are a lot of different variants and sizes, I suppose we might as well export all the common ones. Bump dep to cryptonite to 0.16, earlier versions lacked BLAKE2 support. Even android has 0.16 or newer. On Debian, Blake2bp_512 is buggy, so I have omitted it for now. http://bugs.debian.org/892855 This commit was sponsored by andrea rota.
2018-03-13 20:15:35 +00:00
blake2b_512 :: L.ByteString -> Digest Blake2b_512
blake2b_512 = hashlazy
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
blake2b_512_context :: Context Blake2b_512
blake2b_512_context = hashInit
Add BLAKE2BP512 and BLAKE2BP512E backends using a blake2 variant optimised for 4-way CPUs This had been deferred because the Debian package of cryptonite, and possibly other builds, was broken for blake2bp, but I've confirmed #892855 is fixed. This commit was sponsored by Brett Eisenberg on Patreon.
2019-07-05 19:29:00 +00:00
blake2bp_512 :: L.ByteString -> Digest Blake2bp_512
blake2bp_512 = hashlazy
Added backends for the BLAKE2 family of hashes. There are a lot of different variants and sizes, I suppose we might as well export all the common ones. Bump dep to cryptonite to 0.16, earlier versions lacked BLAKE2 support. Even android has 0.16 or newer. On Debian, Blake2bp_512 is buggy, so I have omitted it for now. http://bugs.debian.org/892855 This commit was sponsored by andrea rota.
2018-03-13 20:15:35 +00:00
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
blake2bp_512_context :: Context Blake2bp_512
blake2bp_512_context = hashInit
Added MD5 and MD5E backends.
2015-02-04 17:47:54 +00:00
md5 :: L.ByteString -> Digest MD5
md5 = hashlazy
add incremental hashing interface to Backend As yet unused. Backend.External could perhaps implement it too, although that would involve sending chunks of data to it via a pipe or something, so likely to be slow.
2021-02-09 19:00:51 +00:00
md5_context :: Context MD5
md5_context = hashInit
cache the serialization of a Key This will speed up the common case where a Key is deserialized from disk, but is then serialized to build eg, the path to the annex object. Previously attempted in 4536c93bb2ecf114ab711beac33fa358facd6985 and reverted in 96aba8eff7597898f53bfbab5865ff30a927d355. The problems mentioned in the latter commit are addressed now: Read/Show of KeyData is backwards-compatible with Read/Show of Key from before this change, so Types.Distribution will keep working. The Eq instance is fixed. Also, Key has smart constructors, avoiding needing to remember to update the cached serialization. Used git-annex benchmark: find is 7% faster whereis is 3% faster get when all files are already present is 5% faster Generally, the benchmarks are running 0.1 seconds faster per 2000 files, on a ram disk in my laptop.
2019-11-22 20:24:04 +00:00
md5s :: S.ByteString -> Digest MD5
md5s = hash
ensure that hash representations don't change in future
2013-10-02 01:10:56 +00:00
{- Check that all the hashes continue to hash the same. -}
split up quickcheck tests for hashes and macs So when one fais, it's clear which one is the problem.
2020-03-02 18:25:45 +00:00
props_hashes_stable :: [(String, Bool)]
props_hashes_stable = map (\(desc, hasher, result) -> (desc ++ " stable", hasher foo == result))
[ ("sha1", show . sha1, "0beec7b5ea3f0fdbc95d0dd47f3c5bc275da8a33")
, ("sha2_224", show . sha2_224, "0808f64e60d58979fcb676c96ec938270dea42445aeefcd3a4e6f8db")
, ("sha2_256", show . sha2_256, "2c26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae")
, ("sha2_384", show . sha2_384, "98c11ffdfdd540676b1a137cb1a22b2a70350c9a44171d6b1180c6be5cbb2ee3f79d532c8a1dd9ef2e8e08e752a3babb")
, ("sha2_512", show . sha2_512, "f7fbba6e0636f890e56fbbf3283e524c6fa3204ae298382d624741d0dc6638326e282c41be5e4254d8820772c5518a2c5a8c0c7f7eda19594a7eb539453e1ed7")
, ("skein256", show . skein256, "a04efd9a0aeed6ede40fe5ce0d9361ae7b7d88b524aa19917b9315f1ecf00d33")
, ("skein512", show . skein512, "fd8956898113510180aa4658e6c0ac85bd74fb47f4a4ba264a6b705d7a8e8526756e75aecda12cff4f1aca1a4c2830fbf57f458012a66b2b15a3dd7d251690a7")
, ("sha3_224", show . sha3_224, "f4f6779e153c391bbd29c95e72b0708e39d9166c7cea51d1f10ef58a")
, ("sha3_256", show . sha3_256, "76d3bc41c9f588f7fcd0d5bf4718f8f84b1c41b20882703100b9eb9413807c01")
, ("sha3_384", show . sha3_384, "665551928d13b7d84ee02734502b018d896a0fb87eed5adb4c87ba91bbd6489410e11b0fbcc06ed7d0ebad559e5d3bb5")
, ("sha3_512", show . sha3_512, "4bca2b137edc580fe50a88983ef860ebaca36c857b1f492839d6d7392452a63c82cbebc68e3b70a2a1480b4bb5d437a7cba6ecf9d89f9ff3ccd14cd6146ea7e7")
, ("blake2s_160", show . blake2s_160, "52fb63154f958a5c56864597273ea759e52c6f00")
, ("blake2s_224", show . blake2s_224, "9466668503ac415d87b8e1dfd7f348ab273ac1d5e4f774fced5fdb55")
, ("blake2s_256", show . blake2s_256, "08d6cad88075de8f192db097573d0e829411cd91eb6ec65e8fc16c017edfdb74")
, ("blake2sp_224", show . blake2sp_224, "8492d356fbac99f046f55e114301f7596649cb590e5b083d1a19dcdb")
, ("blake2sp_256", show . blake2sp_256, "050dc5786037ea72cb9ed9d0324afcab03c97ec02e8c47368fc5dfb4cf49d8c9")
, ("blake2b_160", show . blake2b_160, "983ceba2afea8694cc933336b27b907f90c53a88")
, ("blake2b_224", show . blake2b_224, "853986b3fe231d795261b4fb530e1a9188db41e460ec4ca59aafef78")
, ("blake2b_256", show . blake2b_256, "b8fe9f7f6255a6fa08f668ab632a8d081ad87983c77cd274e48ce450f0b349fd")
, ("blake2b_384", show . blake2b_384, "e629ee880953d32c8877e479e3b4cb0a4c9d5805e2b34c675b5a5863c4ad7d64bb2a9b8257fac9d82d289b3d39eb9cc2")
, ("blake2b_512", show . blake2b_512, "ca002330e69d3e6b84a46a56a6533fd79d51d97a3bb7cad6c2ff43b354185d6dc1e723fb3db4ae0737e120378424c714bb982d9dc5bbd7a0ab318240ddd18f8d")
, ("blake2bp_512", show . blake2bp_512, "8ca9ccee7946afcb686fe7556628b5ba1bf9a691da37ca58cd049354d99f37042c007427e5f219b9ab5063707ec6823872dee413ee014b4d02f2ebb6abb5f643")
, ("md5", show . md5, "acbd18db4cc2f85cedef654fccc4a4d8")
ensure that hash representations don't change in future
2013-10-02 01:10:56 +00:00
]
where
lazy bytestring fromStrict is not available in stable; use fromChunks instead
2014-05-30 00:25:01 +00:00
foo = L.fromChunks [T.encodeUtf8 $ T.pack "foo"]
refactor
2015-04-19 14:57:14 +00:00
{- File names are (client-side) MAC'ed on special remotes.
- The chosen MAC algorithm needs to be same for all files stored on the
- remote.
-}
data Mac = HmacSha1 | HmacSha224 | HmacSha256 | HmacSha384 | HmacSha512
deriving (Eq)
calcMac
:: Mac -- ^ MAC
-> S.ByteString -- ^ secret key
-> S.ByteString -- ^ message
-> String -- ^ MAC'ed message, in hexadecimal
calcMac mac = case mac of
HmacSha1 -> use SHA1
HmacSha224 -> use SHA224
HmacSha256 -> use SHA256
HmacSha384 -> use SHA384
HmacSha512 -> use SHA512
where
Added support for SHA3 hashed keys (in 8 varieties), when git-annex is built using the cryptonite library. While cryptohash has SHA3 support, it has not been updated for the final version of the spec. Note that cryptonite has not been ported to all arches that cryptohash builds on yet.
2015-08-06 19:02:25 +00:00
use alg k m = show (hmacGetDigest (hmacWitnessAlg alg k m))
hmacWitnessAlg :: HashAlgorithm a => a -> S.ByteString -> S.ByteString -> HMAC a
hmacWitnessAlg _ = hmac
refactor
2015-04-19 14:57:14 +00:00
-- Check that all the MACs continue to produce the same.
split up quickcheck tests for hashes and macs So when one fais, it's clear which one is the problem.
2020-03-02 18:25:45 +00:00
props_macs_stable :: [(String, Bool)]
props_macs_stable = map (\(desc, mac, result) -> (desc ++ " stable", calcMac mac key msg == result))
[ ("HmacSha1", HmacSha1, "46b4ec586117154dacd49d664e5d63fdc88efb51")
, ("HmacSha224", HmacSha224, "4c1f774863acb63b7f6e9daa9b5c543fa0d5eccf61e3ffc3698eacdd")
, ("HmacSha256", HmacSha256, "f9320baf0249169e73850cd6156ded0106e2bb6ad8cab01b7bbbebe6d1065317")
, ("HmacSha384", HmacSha384, "3d10d391bee2364df2c55cf605759373e1b5a4ca9355d8f3fe42970471eca2e422a79271a0e857a69923839015877fc6")
, ("HmacSha512", HmacSha512, "114682914c5d017dfe59fdc804118b56a3a652a0b8870759cf9e792ed7426b08197076bf7d01640b1b0684df79e4b67e37485669e8ce98dbab60445f0db94fce")
refactor
2015-04-19 14:57:14 +00:00
]
where
key = T.encodeUtf8 $ T.pack "foo"
msg = T.encodeUtf8 $ T.pack "bar"
Reference in a new issue Copy permalink