mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
git-annex/doc/backends.mdwn

43 lines
2 KiB
Text
Raw Normal View History

update
2010-10-19 19:59:40 +00:00
When a file is annexed, a key is generated from its content and/or metadata.
The file checked into git symlinks to the key. This key can later be used
to retrieve the file's content (its value).
update documentation for new, neutered key-value backends Backends are now only used to generate keys (and check them); they are not arbitrary key-value stores for data, because it turned out such a store is better modeled as a special remote. Updated docs to not imply backends do more than they do now. Sometimes I'm tempted to rename "backend" to "keytype" or something, which would really be more clear. But it would be an annoying transition for users, with annex.backends etc.
2011-08-28 20:28:38 +00:00
Multiple pluggable key-value backends are supported, and a single repository
can use different ones for different files.
document S3 remotes
2011-03-28 02:52:13 +00:00
make edits that a commenter apparently didn't realize they could make. this is a wiki..
2013-07-31 16:31:30 +00:00
* `SHA256E` -- The default backend for new files, combines a SHA256 hash of
the file's content with the file's extension. This allows
update to not overstate the danger or WORM
2011-08-29 16:08:54 +00:00
verifying that the file content is right, and can avoid duplicates of
files with the same content. Its need to generate checksums
use SHA256 by default To get old behavior, add a .gitattributes containing: * annex.backend=WORM I feel that SHA256 is a better default for most people, as long as their systems are fast enough that checksumming their files isn't a problem. git-annex should default to preserving the integrity of data as well as git does. Checksum backends also work better with editing files via unlock/lock. I considered just using SHA1, but since that hash is believed to be somewhat near to being broken, and git-annex deals with large files which would be a perfect exploit medium, I decided to go to a SHA-2 hash. SHA512 is annoyingly long when displayed, and git-annex displays it in a few places (and notably it is shown in ls -l), so I picked the shorter hash. Considered SHA224 as it's even shorter, but feel it's a bit weird. I expect git-annex will use SHA-3 at some point in the future, but probably not soon! Note that systems without a sha256sum (or sha256) program will fall back to defaulting to SHA1.
2011-11-04 19:21:45 +00:00
can make it slower for large files.
SHA256E is new default backend The default backend used when adding files to the annex is changed from SHA256 to SHA256E, to simplify interoperability with OSX, media players, and various programs that needlessly look at symlink targets. To get old behavior, add a .gitattributes containing: * annex.backend=SHA256
2012-09-12 17:22:16 +00:00
* `SHA256` -- Does not include the file extension in the key, which can
make edits that a commenter apparently didn't realize they could make. this is a wiki..
2013-07-31 16:31:30 +00:00
lead to better deduplication but can confuse some programs.
use SHA256 by default To get old behavior, add a .gitattributes containing: * annex.backend=WORM I feel that SHA256 is a better default for most people, as long as their systems are fast enough that checksumming their files isn't a problem. git-annex should default to preserving the integrity of data as well as git does. Checksum backends also work better with editing files via unlock/lock. I considered just using SHA1, but since that hash is believed to be somewhat near to being broken, and git-annex deals with large files which would be a perfect exploit medium, I decided to go to a SHA-2 hash. SHA512 is annoyingly long when displayed, and git-annex displays it in a few places (and notably it is shown in ls -l), so I picked the shorter hash. Considered SHA224 as it's even shorter, but feel it's a bit weird. I expect git-annex will use SHA-3 at some point in the future, but probably not soon! Note that systems without a sha256sum (or sha256) program will fall back to defaulting to SHA1.
2011-11-04 19:21:45 +00:00
* `WORM` ("Write Once, Read Many") This assumes that any file with
the same basename, size, and modification time has the same content.
Typo: double 'the'
2013-08-18 11:47:39 +00:00
This is the least expensive backend, recommended for really large
use SHA256 by default To get old behavior, add a .gitattributes containing: * annex.backend=WORM I feel that SHA256 is a better default for most people, as long as their systems are fast enough that checksumming their files isn't a problem. git-annex should default to preserving the integrity of data as well as git does. Checksum backends also work better with editing files via unlock/lock. I considered just using SHA1, but since that hash is believed to be somewhat near to being broken, and git-annex deals with large files which would be a perfect exploit medium, I decided to go to a SHA-2 hash. SHA512 is annoyingly long when displayed, and git-annex displays it in a few places (and notably it is shown in ls -l), so I picked the shorter hash. Considered SHA224 as it's even shorter, but feel it's a bit weird. I expect git-annex will use SHA-3 at some point in the future, but probably not soon! Note that systems without a sha256sum (or sha256) program will fall back to defaulting to SHA1.
2011-11-04 19:21:45 +00:00
files or slow systems.
SHA256E is new default backend The default backend used when adding files to the annex is changed from SHA256 to SHA256E, to simplify interoperability with OSX, media players, and various programs that needlessly look at symlink targets. To get old behavior, add a .gitattributes containing: * annex.backend=SHA256
2012-09-12 17:22:16 +00:00
* `SHA512`, `SHA512E` -- Best currently available hash, for the very paranoid.
* `SHA1`, `SHA1E` -- Smaller hash than `SHA256` for those who want a checksum
use SHA256 by default To get old behavior, add a .gitattributes containing: * annex.backend=WORM I feel that SHA256 is a better default for most people, as long as their systems are fast enough that checksumming their files isn't a problem. git-annex should default to preserving the integrity of data as well as git does. Checksum backends also work better with editing files via unlock/lock. I considered just using SHA1, but since that hash is believed to be somewhat near to being broken, and git-annex deals with large files which would be a perfect exploit medium, I decided to go to a SHA-2 hash. SHA512 is annoyingly long when displayed, and git-annex displays it in a few places (and notably it is shown in ls -l), so I picked the shorter hash. Considered SHA224 as it's even shorter, but feel it's a bit weird. I expect git-annex will use SHA-3 at some point in the future, but probably not soon! Note that systems without a sha256sum (or sha256) program will fall back to defaulting to SHA1.
2011-11-04 19:21:45 +00:00
but are not concerned about security.
SHA256E is new default backend The default backend used when adding files to the annex is changed from SHA256 to SHA256E, to simplify interoperability with OSX, media players, and various programs that needlessly look at symlink targets. To get old behavior, add a .gitattributes containing: * annex.backend=SHA256
2012-09-12 17:22:16 +00:00
* `SHA384`, `SHA384E`, `SHA224`, `SHA224E` -- Hashes for people who like
unusual sizes.
Added SKEIN256 and SKEIN512 backends SHA3 is still waiting for final standardization. Although this is looking less likely given https://www.cdt.org/blogs/joseph-lorenzo-hall/2409-nist-sha-3 In the meantime, cryptohash implements skein, and it's used by some of the haskell ecosystem (for yesod sessions, IIRC), so this implementation is likely to continue working. Also, I've talked with the cryprohash author and he's a reasonable guy. It makes sense to have an alternate high security hash, in case some horrible attack is found against SHA2 tomorrow, or in case SHA3 comes out and worst fears are realized. I'd also like to support using skein for HMAC. But no hurry there and a new version of cryptohash has much nicer HMAC code, so I will probably wait until I can use that version.
2013-10-02 00:34:06 +00:00
* `SKEIN512`, `SKEIN256` -- [Skein hash](http://en.wikipedia.org/wiki/Skein_hash),
a well-regarded SHA3 hash competition finalist.
document S3 remotes
2011-03-28 02:52:13 +00:00
expand docs
2010-11-01 23:11:06 +00:00
The `annex.backends` git-config setting can be used to list the backends
git-annex should use. The first one listed will be used by default when
new files are added.
For finer control of what backend is used when adding different types of
The git-annex-backend attribute has been renamed to annex.backend.
2010-11-28 22:58:03 +00:00
files, the `.gitattributes` file can be used. The `annex.backend`
expand docs
2010-11-01 23:11:06 +00:00
attribute can be set to the name of the backend to use for matching files.
SHA256E is new default backend The default backend used when adding files to the annex is changed from SHA256 to SHA256E, to simplify interoperability with OSX, media players, and various programs that needlessly look at symlink targets. To get old behavior, add a .gitattributes containing: * annex.backend=SHA256
2012-09-12 17:22:16 +00:00
For example, to use the SHA256E backend for sound files, which tend to be
use SHA256 by default To get old behavior, add a .gitattributes containing: * annex.backend=WORM I feel that SHA256 is a better default for most people, as long as their systems are fast enough that checksumming their files isn't a problem. git-annex should default to preserving the integrity of data as well as git does. Checksum backends also work better with editing files via unlock/lock. I considered just using SHA1, but since that hash is believed to be somewhat near to being broken, and git-annex deals with large files which would be a perfect exploit medium, I decided to go to a SHA-2 hash. SHA512 is annoyingly long when displayed, and git-annex displays it in a few places (and notably it is shown in ls -l), so I picked the shorter hash. Considered SHA224 as it's even shorter, but feel it's a bit weird. I expect git-annex will use SHA-3 at some point in the future, but probably not soon! Note that systems without a sha256sum (or sha256) program will fall back to defaulting to SHA1.
2011-11-04 19:21:45 +00:00
smallish and might be modified or copied over time,
while using the WORM backend for everything else, you could set
in `.gitattributes`:
expand docs
2010-11-01 23:11:06 +00:00
use SHA256 by default To get old behavior, add a .gitattributes containing: * annex.backend=WORM I feel that SHA256 is a better default for most people, as long as their systems are fast enough that checksumming their files isn't a problem. git-annex should default to preserving the integrity of data as well as git does. Checksum backends also work better with editing files via unlock/lock. I considered just using SHA1, but since that hash is believed to be somewhat near to being broken, and git-annex deals with large files which would be a perfect exploit medium, I decided to go to a SHA-2 hash. SHA512 is annoyingly long when displayed, and git-annex displays it in a few places (and notably it is shown in ls -l), so I picked the shorter hash. Considered SHA224 as it's even shorter, but feel it's a bit weird. I expect git-annex will use SHA-3 at some point in the future, but probably not soon! Note that systems without a sha256sum (or sha256) program will fall back to defaulting to SHA1.
2011-11-04 19:21:45 +00:00
* annex.backend=WORM
SHA256E is new default backend The default backend used when adding files to the annex is changed from SHA256 to SHA256E, to simplify interoperability with OSX, media players, and various programs that needlessly look at symlink targets. To get old behavior, add a .gitattributes containing: * annex.backend=SHA256
2012-09-12 17:22:16 +00:00
*.mp3 annex.backend=SHA256E
*.ogg annex.backend=SHA256E
Reference in a new issue Copy permalink