Added support for git-remote-gcrypt's rsync URIs

Which access a remote using rsync over ssh, and which git pushes to much
more efficiently than ssh urls.

There was some old partial support for rsync URIs from 2013, but it seemed
incomplete, and did not use rsync over ssh. Weird.

I'm not sure if there's any remaining benefit to using the non-rsync url
forms with gcrypt, now that this is implemented? Updated docs to encourage
using the rsync urls.

This commit was sponsored by Svenne Krap on Patreon.
This commit is contained in:
Joey Hess 2021-03-09 15:58:09 -04:00
parent 15891441f1
commit 1d7fa63149
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
6 changed files with 127 additions and 68 deletions

View file

@ -28,3 +28,5 @@ is running. The upload of the actual changeset starts after this, the processes
git-annex is great and revolutionized my file organization and backup structure (if they were even existing before)
[[!meta title="gcrypt special remotes should support rsync:// and perhaps also sftp://"]]
> [[done]] --[[Joey]]

View file

@ -0,0 +1,24 @@
[[!comment format=mdwn
username="joey"
subject="""comment 10"""
date="2021-03-09T18:10:38Z"
content="""
Actually, it supports rsync urls already. But git-annex treats the url as
literally the rsync protocol, so it will run rsync with an url like
rsync://localhost/foo. Which will mostly fail unless there's actually a
rsync server running.
git-remote-gcrypt treats the same url as rsync over ssh.
See [[!commit 3e079cdcd1ac4f52ceb1c3ad483917d7c6133d0b]] for when that was
added in 2013. Although I'm doubtful if it actually worked; it seems like
initremote supported it, but it was never implemented the rest of
the way.
If anyone has been using that, it seems git-annex and git-remote-gcrypt
would be accessing the server in different ways, so the same server must
support rsync over ssh too, or git pushes to the remote would fail. So, it
seems all that's needed is for git-annex to treat that url as rsync over
ssh, and finish the incomplete implementation of rsync urls.
Ok, done!
"""]]

View file

@ -26,10 +26,21 @@ gcrypt:
repository itself. May be repeated when multiple participants
should have access to the repository.
* `gitrepo` - Required. The path or url to the git repository
* `gitrepo` - Required. The location of the git repository
for gcrypt to use. This repository should be either an unpopulated
bare git repo, or an existing gcrypt repository.
To use a local git repository, use: `gitrepo=/path/to/repo`
For a git repository accessed using rsync over ssh, use:
`gitrepo=rsync://user@host/path/to/repo`
For a git repository accessed over ssh, and using git-annex-shell
to transfer data, use:
`gitrepo=ssh://user@host/path/to/repo` or `host:path`
Note that each `git push` has to re-send the whole content of the git
repository when using this option.
* `chunk` - Enables [[chunking]] when storing large files.
* `shellescape` - See [[rsync]] for the details of this option.
@ -37,9 +48,7 @@ gcrypt:
## notes
For git-annex to store files in a repository on a remote server, you need
shell access, and `rsync` must be installed. Those are the minimum
requirements, but it's also recommended to install git-annex on the remote
server, so that [[git-annex-shell]] can be used.
shell access, and it needs to be able to run `rsync` or `git-annex-shell`.
If you can't run `rsync` or `git-annex-shell` on the remote server,
you can't use this special remote. Other options are the [[git-lfs]]

View file

@ -66,9 +66,9 @@ the gpg key used to encrypt it, and then:
## encrypted git-annex repository on a ssh server
If you have a ssh server that has git-annex or rsync installed on it, you
can set up an encrypted repository there. Works just like the encrypted
drive except without the cable.
If you have a server that has ssh and rsync installed on it, you can set up an
encrypted repository there. Works just like the encrypted drive except
without the cable.
First, on the server, run:
@ -76,14 +76,14 @@ First, on the server, run:
Now, in your existing git-annex repository, set up the encrypted remote:
git annex initremote encryptedrepo type=gcrypt gitrepo=ssh://my.server/home/me/encryptedrepo keyid=$mykey
git annex initremote encryptedrepo type=gcrypt gitrepo=rsync://my.server/home/me/encryptedrepo keyid=$mykey
git annex sync encryptedrepo
(Remember to replace "$mykey" with the keyid of your gpg key.)
This uses the [[gcrypt special remote|special_remotes/gcrypt]] to encrypt
pushes to the git remote, and git-annex will also encrypt the files it
stores there.
stores there. Data is transferred using rsync over ssh.
If you're going to be sharing this repository with others, be sure to also
include their keyids, by specifying keyid= repeatedly.
@ -98,9 +98,9 @@ the git repository changes to it as well.
Anyone who has access to the repo it and has one of the keys
used to encrypt it can check it out:
git clone gcrypt::ssh://my.server/home/me/encryptedrepo myrepo
git clone gcrypt::rsync://my.server/home/me/encryptedrepo myrepo
cd myrepo
git annex enableremote encryptedrepo gitrepo=ssh://my.server/home/me/encryptedrepo
git annex enableremote encryptedrepo gitrepo=rsync://my.server/home/me/encryptedrepo
git annex get --from encryptedrepo
## private encrypted git remote on a git-lfs hosting site