diff --git a/doc/special_remotes.mdwn b/doc/special_remotes.mdwn index 0ae4346ed0..0203f5778f 100644 --- a/doc/special_remotes.mdwn +++ b/doc/special_remotes.mdwn @@ -15,6 +15,7 @@ the git history is not stored in them. * [[ddar]] * [[directory]] * [[gcrypt]] (encrypted git repositories!) +* [[git-lfs]] * [[hook]] * [[rclone]] * [[rsync]] diff --git a/doc/special_remotes/git-lfs.mdwn b/doc/special_remotes/git-lfs.mdwn new file mode 100644 index 0000000000..21e07fa223 --- /dev/null +++ b/doc/special_remotes/git-lfs.mdwn @@ -0,0 +1,64 @@ +git-annex has a special remote that lets it store content in git-lfs +repositories. + +See [[tips/storing_data_in_git-lfs]] for some examples of how to use this. + +## configuration + +These parameters can be passed to `git annex initremote` to configure +the git-lfs special remote: + +* `url` - Required. The url to the git-lfs repository to use. + Can be either a ssh url (scp-style is also accepted) or a http url. + But currently, a http url accesses the git-lfs repository without + authentication. To authenticate, you will need to use a ssh url. + + This parameter needs to be specified in the initial `git annex + initremote` but also each time you `git annex enableremote` + an existing git-lfs special remote. It's fine to use different urls + at different times as long as they point to the same git-lfs repository. + +* `encryption` - One of "none", "hybrid", "shared", or "pubkey". + Required. See [[encryption]]. + +* `keyid` - Specifies the gpg key to use for encryption of both the files + git-annex stores in the repository, as well as to encrypt the git + repository itself. May be repeated when multiple participants + should have access to the repository. + +## efficiency note + +Since git-lfs uses SHA256 checksums, git-annex needs to keep track of the +SHA256 of content stored in it, in order to be able to retrieve that +content. When a git-annex key uses a [[backend|backends]] +of SHA256 or SHA256E, that's easy. But, if a git-annex key uses some +other backend, git-annex has to additionally store the SHA256 checksum +into the git-annex branch when storing content in git-lfs. That adds a +small bit of size overhead to using this remote. + +## encryption notes + +The encryption= parameter only makes git-annex encrypt data stored on the +remote, `git push` can also be used with the remote (it is a git repository +after all), and data pushed to it with git will *not* be encrypted. + +When encrypting data sent to the git-lfs remote, git-annex always has to +store its SHA256 checksum in the git-annex branch. + +## limitations + +The git-lfs protocol does not support resuming uploads, and so an +interrupted upload will have to restart from the beginning. Interrupted +downloads will resume. + +git-lfs has a concept of git ref based access control, so a user may only +be able to send content associated with a particular git ref. git-annex +does not currently provide any git ref, so won't work with a git-lfs server +that uses that. + +git-annex only supports the "basic" git-lfs transfer adapter, but that's +the one used by most git-lfs servers. + +The git-lfs protocol is designed around batching of transfers, but +git-annex doesn't do batching. This may cause it to fall afoul of +rate limiting of git-lfs servers when transferring a lot of files. diff --git a/doc/tips/storing_data_in_git-lfs.mdwn b/doc/tips/storing_data_in_git-lfs.mdwn new file mode 100644 index 0000000000..d6ab8827ef --- /dev/null +++ b/doc/tips/storing_data_in_git-lfs.mdwn @@ -0,0 +1,23 @@ +git-annex can store data in [git-lfs](https://git-lfs.github.com/) +repositories, using the [[git-lfs special remote|special_remotes/git-lfs]]. + +Here's how to initialize a git-lfs special remote on Github. + + git annex initremote lfs type=git-lfs encryption=none url=git@github.com:yourname/yourrepo.git + +If you want git-annex to encrypt the objects it stores in the remote, +change the encryption= parameter. But be sure to read the +[[git-lfs special remote|special_remotes/git-lfs]] page's +**encryption notes** first! + +To enable the same remote in another clone of the repository, +you'll need to provide an url to it again. It's ok to provide a different +url as long as it points to the same git-lfs repository. + + git annex enableremote lfs url=https://github.com/yourname/yourrepo.git + +Note that http urls currently only allow read access to the git-lfs +repository. + +A git-lfs special remote also functions as a regular git remote. You can +use things like `git push` and `git pull` with it.