CI to automate build of cargo lockfiles on different Alpine releases for git-annex aports
https://gitlab.alpinelinux.org/alpine/aports/-/tree/master/community/git-annex
7390f08ef9
This is a massive win on OSX, which doesn't have a sha256sum normally.
Only use external hash commands when the file is > 1 mb,
since cryptohash is quite close to them in speed.
SHA is still used to calculate HMACs. I don't quite understand
cryptohash's API for those.
Used the following benchmark to arrive at the 1 mb number.
1 mb file:
benchmarking sha256/internal
mean: 13.86696 ms, lb 13.83010 ms, ub 13.93453 ms, ci 0.950
std dev: 249.3235 us, lb 162.0448 us, ub 458.1744 us, ci 0.950
found 5 outliers among 100 samples (5.0%)
4 (4.0%) high mild
1 (1.0%) high severe
variance introduced by outliers: 10.415%
variance is moderately inflated by outliers
benchmarking sha256/external
mean: 14.20670 ms, lb 14.17237 ms, ub 14.27004 ms, ci 0.950
std dev: 230.5448 us, lb 150.7310 us, ub 427.6068 us, ci 0.950
found 3 outliers among 100 samples (3.0%)
2 (2.0%) high mild
1 (1.0%) high severe
2 mb file:
benchmarking sha256/internal
mean: 26.44270 ms, lb 26.23701 ms, ub 26.63414 ms, ci 0.950
std dev: 1.012303 ms, lb 925.8921 us, ub 1.122267 ms, ci 0.950
variance introduced by outliers: 35.540%
variance is moderately inflated by outliers
benchmarking sha256/external
mean: 26.84521 ms, lb 26.77644 ms, ub 26.91433 ms, ci 0.950
std dev: 347.7867 us, lb 210.6283 us, ub 571.3351 us, ci 0.950
found 6 outliers among 100 samples (6.0%)
import Crypto.Hash
import Data.ByteString.Lazy as L
import Criterion.Main
import Common
testfile :: FilePath
testfile = "/run/shm/data" -- on ram disk
main = defaultMain
[ bgroup "sha256"
[ bench "internal" $ whnfIO internal
, bench "external" $ whnfIO external
]
]
sha256 :: L.ByteString -> Digest SHA256
sha256 = hashlazy
internal :: IO String
internal = show . sha256 <$> L.readFile testfile
external :: IO String
external = do
s <- readProcess "sha256sum" [testfile]
return $ fst $ separate (== ' ') s
|
||
|---|---|---|
| Annex | ||
| Assistant | ||
| Backend | ||
| Build | ||
| Command | ||
| Common | ||
| Config | ||
| debian | ||
| doc | ||
| Git | ||
| GitAnnex | ||
| Logs | ||
| Messages | ||
| Remote | ||
| standalone | ||
| static | ||
| templates | ||
| Types | ||
| Upgrade | ||
| Utility | ||
| .ghci | ||
| .gitattributes | ||
| .gitignore | ||
| Annex.hs | ||
| Assistant.hs | ||
| Backend.hs | ||
| BuildFlags.hs | ||
| CHANGELOG | ||
| Checks.hs | ||
| CmdLine.hs | ||
| Command.hs | ||
| Common.hs | ||
| Config.hs | ||
| configure.hs | ||
| COPYRIGHT | ||
| Creds.hs | ||
| Crypto.hs | ||
| Fields.hs | ||
| ghci | ||
| git-annex.cabal | ||
| git-annex.hs | ||
| git-union-merge.hs | ||
| Git.hs | ||
| GitAnnex.hs | ||
| GitAnnexShell.hs | ||
| Init.hs | ||
| INSTALL | ||
| Limit.hs | ||
| Locations.hs | ||
| Logs.hs | ||
| Makefile | ||
| Messages.hs | ||
| NEWS | ||
| Option.hs | ||
| README | ||
| Remote.hs | ||
| Seek.hs | ||
| Setup.hs | ||
| Test.hs | ||
| Types.hs | ||
| Upgrade.hs | ||
| Usage.hs | ||
git-annex allows managing files with git, without checking the file contents into git. While that may seem paradoxical, it is useful when dealing with files larger than git can currently easily handle, whether due to limitations in memory, checksumming time, or disk space. For documentation, see doc/ or <http://git-annex.branchable.com/>