0c6b7e288d
using a blake2 variant optimised for 4-way CPUs This had been deferred because the Debian package of cryptonite, and possibly other builds, was broken for blake2bp, but I've confirmed #892855 is fixed. This commit was sponsored by Brett Eisenberg on Patreon.
87 lines
4.2 KiB
Markdown
87 lines
4.2 KiB
Markdown
When a file is annexed, a key is generated from its content and/or filesystem
|
|
metadata. The file checked into git symlinks to the key. This key can later
|
|
be used to retrieve the file's content (its value).
|
|
|
|
Multiple pluggable key-value backends are supported, and a single repository
|
|
can use different ones for different files.
|
|
|
|
These are the recommended backends to use.
|
|
|
|
* `SHA256E` -- The default backend for new files, combines a 256 bit SHA-2
|
|
hash of the file's content with the file's extension. This allows
|
|
verifying that the file content is right, and can avoid duplicates of
|
|
files with the same content. Its need to generate checksums
|
|
can make it slower for large files.
|
|
* `SHA256` -- SHA-2 hash that does not include the file extension in the
|
|
key, which can lead to better deduplication but can confuse some programs.
|
|
* `SHA512`, `SHA512E` -- Best SHA-2 hash, for the very paranoid.
|
|
* `SHA384`, `SHA384E`, `SHA224`, `SHA224E` -- SHA-2 hashes for
|
|
people who like unusual sizes.
|
|
* `SHA3_512`, `SHA3_512E`, `SHA3_384`, `SHA3_384E`, `SHA3_256`, `SHA3_256E`, `SHA3_224`, `SHA3_224E`
|
|
-- SHA-3 hashes, for bleeding edge fun.
|
|
* `SKEIN512`, `SKEIN512E`, `SKEIN256`, `SKEIN256E`
|
|
-- [Skein hash](http://en.wikipedia.org/wiki/Skein_hash),
|
|
a well-regarded SHA3 hash competition finalist.
|
|
* `BLAKE2B160`, `BLAKE2B224`, `BLAKE2B256`, `BLAKE2B384`, `BLAKE2B512`
|
|
`BLAKE2B160E`, `BLAKE2B224E`, `BLAKE2B256E`, `BLAKE2B384E`, `BLAKE2B512E`
|
|
-- Fast [Blake2 hash](https://blake2.net/) variants optimised for 64 bit
|
|
platforms.
|
|
* `BLAKE2S160`, `BLAKE2S224`, `BLAKE2S256`
|
|
`BLAKE2S160E`, `BLAKE2S224E`, `BLAKE2S256E`
|
|
-- Fast [Blake2 hash](https://blake2.net/) variants optimised for 32 bit
|
|
platforms.
|
|
* `BLAKE2BP512`, `BLAKE2BP512E`
|
|
-- Fast [Blake2 hash](https://blake2.net/) variants optimised for
|
|
4-way CPUs.
|
|
* `BLAKE2SP224`, `BLAKE2SP256`
|
|
`BLAKE2SP224E`, `BLAKE2SP256E`
|
|
-- Fast [Blake2 hash](https://blake2.net/) variants optimised for
|
|
8-way CPUs.
|
|
|
|
The backends below do not guarantee cryptographically that the
|
|
content of an annexed file remains unchanged.
|
|
|
|
* `SHA1`, `SHA1E`, `MD5`, `MD5E` -- Smaller hashes than `SHA256`
|
|
for those who want a checksum but are not concerned about security.
|
|
* `WORM` ("Write Once, Read Many") -- This assumes that any file with
|
|
the same filename, size, and modification time has the same content.
|
|
This is the least expensive backend, recommended for really large
|
|
files or slow systems.
|
|
* `URL` -- This is a key that is generated from the url to a file.
|
|
It's generated when using eg, `git annex addurl --fast`, when the file
|
|
content is not available for hashing. The key may not contain the full
|
|
URL; for long URLs, part of the URL may be represented by a checksum.
|
|
The URL key may contain `&` characters; be sure to quote the key if
|
|
passing it to a shell script. The URL-backend key is distinct from URLs/URIs
|
|
that may be attached to a key (from any backend) indicating the key's location
|
|
on the web or in one of [[special_remotes]].
|
|
|
|
If you want to be able to prove that you're working with the same file
|
|
contents that were checked into a repository earlier, you should avoid
|
|
using the non-cryptographically-secure backends, and will need to use
|
|
signed git commits. See [[tips/using_signed_git_commits]] for details.
|
|
|
|
Retrieval of WORM and URL from many [[special_remotes]] is prohibited
|
|
for [[security_reasons|security/CVE-2018-10857_and_CVE-2018-10859]].
|
|
|
|
Note that the various 512 and 384 length hashes result in long paths,
|
|
which are known to not work on Windows. If interoperability on Windows is a
|
|
concern, avoid those.
|
|
|
|
The `annex.backend` git-config setting can be used to configure the
|
|
default backend to use when adding new files.
|
|
|
|
For finer control of what backend is used when adding different types of
|
|
files, the `.gitattributes` file can be used. The `annex.backend`
|
|
attribute can be set to the name of the backend to use for matching files.
|
|
|
|
For example, to use the SHA256E backend for sound files, which tend to be
|
|
smallish and might be modified or copied over time,
|
|
while using the WORM backend for everything else, you could set
|
|
in `.gitattributes`:
|
|
|
|
* annex.backend=WORM
|
|
*.mp3 annex.backend=SHA256E
|
|
*.ogg annex.backend=SHA256E
|
|
|
|
See also: [[git-annex-examinekey]]
|