2024-05-06 16:44:33 +00:00
|
|
|
# NAME
|
|
|
|
|
|
2024-05-06 17:06:22 +00:00
|
|
|
git-remote-annex - remote helper program to store a git repository in a git-annex special remote
|
2024-05-06 16:44:33 +00:00
|
|
|
|
|
|
|
|
# SYNOPSIS
|
|
|
|
|
|
2024-05-07 18:37:29 +00:00
|
|
|
git fetch annex::uuid?param=value¶m=value...
|
2024-05-06 16:44:33 +00:00
|
|
|
|
|
|
|
|
# DESCRIPTION
|
|
|
|
|
|
|
|
|
|
This is a git remote helper program that allows git to clone,
|
|
|
|
|
pull and push from a git repository that is stored in a git-annex
|
git-remote-annex: Support urls like annex::https://example.com/foo-repo
Using the usual url download machinery even allows these urls to need
http basic auth, which is prompted for with git-credential. Which opens
the possibility for urls that contain a secret to be used, eg the cipher
for encryption=shared. Although the user is currently on their own
constructing such an url, I do think it would work.
Limited to httpalso for now, for security reasons. Since both httpalso
(and retrieving this very url) is limited by the usual
annex.security.allowed-ip-addresses configs, it's not possible for an
attacker to make one of these urls that sets up a httpalso url that
opens the garage door. Which is one class of attacks to keep in mind
with this thing.
It seems that there could be either a git-config that allows other types
of special remotes to be set up this way, or special remotes could
indicate when they are safe. I do worry that the git-config would
encourage users to set it without thinking through the security
implications. One remote config might be safe to access this way, but
another config, for one with the same type, might not be. This will need
further thought, and real-world examples to decide what to do.
2024-05-30 16:19:46 +00:00
|
|
|
special remote with an URL that starts with "annex::"
|
2024-05-06 16:44:33 +00:00
|
|
|
|
git-remote-annex: Support urls like annex::https://example.com/foo-repo
Using the usual url download machinery even allows these urls to need
http basic auth, which is prompted for with git-credential. Which opens
the possibility for urls that contain a secret to be used, eg the cipher
for encryption=shared. Although the user is currently on their own
constructing such an url, I do think it would work.
Limited to httpalso for now, for security reasons. Since both httpalso
(and retrieving this very url) is limited by the usual
annex.security.allowed-ip-addresses configs, it's not possible for an
attacker to make one of these urls that sets up a httpalso url that
opens the garage door. Which is one class of attacks to keep in mind
with this thing.
It seems that there could be either a git-config that allows other types
of special remotes to be set up this way, or special remotes could
indicate when they are safe. I do worry that the git-config would
encourage users to set it without thinking through the security
implications. One remote config might be safe to access this way, but
another config, for one with the same type, might not be. This will need
further thought, and real-world examples to decide what to do.
2024-05-30 16:19:46 +00:00
|
|
|
The special remote needs to have a `remote.<name>.url`
|
|
|
|
|
configured to use this. That is set up automatically when git
|
|
|
|
|
cloning from a special remote.
|
2024-05-06 16:44:33 +00:00
|
|
|
|
git-remote-annex: Support urls like annex::https://example.com/foo-repo
Using the usual url download machinery even allows these urls to need
http basic auth, which is prompted for with git-credential. Which opens
the possibility for urls that contain a secret to be used, eg the cipher
for encryption=shared. Although the user is currently on their own
constructing such an url, I do think it would work.
Limited to httpalso for now, for security reasons. Since both httpalso
(and retrieving this very url) is limited by the usual
annex.security.allowed-ip-addresses configs, it's not possible for an
attacker to make one of these urls that sets up a httpalso url that
opens the garage door. Which is one class of attacks to keep in mind
with this thing.
It seems that there could be either a git-config that allows other types
of special remotes to be set up this way, or special remotes could
indicate when they are safe. I do worry that the git-config would
encourage users to set it without thinking through the security
implications. One remote config might be safe to access this way, but
another config, for one with the same type, might not be. This will need
further thought, and real-world examples to decide what to do.
2024-05-30 16:19:46 +00:00
|
|
|
To make [[git-annex-initremote]](1) and [[git-annex-enableremote]](1)
|
|
|
|
|
configure the url, pass them the `--with-url` option.
|
2024-05-06 16:44:33 +00:00
|
|
|
|
git-remote-annex: Support urls like annex::https://example.com/foo-repo
Using the usual url download machinery even allows these urls to need
http basic auth, which is prompted for with git-credential. Which opens
the possibility for urls that contain a secret to be used, eg the cipher
for encryption=shared. Although the user is currently on their own
constructing such an url, I do think it would work.
Limited to httpalso for now, for security reasons. Since both httpalso
(and retrieving this very url) is limited by the usual
annex.security.allowed-ip-addresses configs, it's not possible for an
attacker to make one of these urls that sets up a httpalso url that
opens the garage door. Which is one class of attacks to keep in mind
with this thing.
It seems that there could be either a git-config that allows other types
of special remotes to be set up this way, or special remotes could
indicate when they are safe. I do worry that the git-config would
encourage users to set it without thinking through the security
implications. One remote config might be safe to access this way, but
another config, for one with the same type, might not be. This will need
further thought, and real-world examples to decide what to do.
2024-05-30 16:19:46 +00:00
|
|
|
Or, to configure an existing special remote with a shorthand URL, run:
|
2024-05-06 16:44:33 +00:00
|
|
|
|
git-remote-annex: Support urls like annex::https://example.com/foo-repo
Using the usual url download machinery even allows these urls to need
http basic auth, which is prompted for with git-credential. Which opens
the possibility for urls that contain a secret to be used, eg the cipher
for encryption=shared. Although the user is currently on their own
constructing such an url, I do think it would work.
Limited to httpalso for now, for security reasons. Since both httpalso
(and retrieving this very url) is limited by the usual
annex.security.allowed-ip-addresses configs, it's not possible for an
attacker to make one of these urls that sets up a httpalso url that
opens the garage door. Which is one class of attacks to keep in mind
with this thing.
It seems that there could be either a git-config that allows other types
of special remotes to be set up this way, or special remotes could
indicate when they are safe. I do worry that the git-config would
encourage users to set it without thinking through the security
implications. One remote config might be safe to access this way, but
another config, for one with the same type, might not be. This will need
further thought, and real-world examples to decide what to do.
2024-05-30 16:19:46 +00:00
|
|
|
git config remote.name.url annex::
|
2024-05-06 16:44:33 +00:00
|
|
|
|
git-remote-annex: Support urls like annex::https://example.com/foo-repo
Using the usual url download machinery even allows these urls to need
http basic auth, which is prompted for with git-credential. Which opens
the possibility for urls that contain a secret to be used, eg the cipher
for encryption=shared. Although the user is currently on their own
constructing such an url, I do think it would work.
Limited to httpalso for now, for security reasons. Since both httpalso
(and retrieving this very url) is limited by the usual
annex.security.allowed-ip-addresses configs, it's not possible for an
attacker to make one of these urls that sets up a httpalso url that
opens the garage door. Which is one class of attacks to keep in mind
with this thing.
It seems that there could be either a git-config that allows other types
of special remotes to be set up this way, or special remotes could
indicate when they are safe. I do worry that the git-config would
encourage users to set it without thinking through the security
implications. One remote config might be safe to access this way, but
another config, for one with the same type, might not be. This will need
further thought, and real-world examples to decide what to do.
2024-05-30 16:19:46 +00:00
|
|
|
Once the URL is configured, you can use `git pull`, `git push`, etc
|
|
|
|
|
with the special remote much like with any other git remote.
|
|
|
|
|
But see CONFLICTING PUSHES below for some situations where it behaves
|
|
|
|
|
slightly differently.
|
2024-05-06 16:44:33 +00:00
|
|
|
|
git-remote-annex: Support urls like annex::https://example.com/foo-repo
Using the usual url download machinery even allows these urls to need
http basic auth, which is prompted for with git-credential. Which opens
the possibility for urls that contain a secret to be used, eg the cipher
for encryption=shared. Although the user is currently on their own
constructing such an url, I do think it would work.
Limited to httpalso for now, for security reasons. Since both httpalso
(and retrieving this very url) is limited by the usual
annex.security.allowed-ip-addresses configs, it's not possible for an
attacker to make one of these urls that sets up a httpalso url that
opens the garage door. Which is one class of attacks to keep in mind
with this thing.
It seems that there could be either a git-config that allows other types
of special remotes to be set up this way, or special remotes could
indicate when they are safe. I do worry that the git-config would
encourage users to set it without thinking through the security
implications. One remote config might be safe to access this way, but
another config, for one with the same type, might not be. This will need
further thought, and real-world examples to decide what to do.
2024-05-30 16:19:46 +00:00
|
|
|
# URL FORMAT
|
2024-05-06 16:44:33 +00:00
|
|
|
|
git-remote-annex: Support urls like annex::https://example.com/foo-repo
Using the usual url download machinery even allows these urls to need
http basic auth, which is prompted for with git-credential. Which opens
the possibility for urls that contain a secret to be used, eg the cipher
for encryption=shared. Although the user is currently on their own
constructing such an url, I do think it would work.
Limited to httpalso for now, for security reasons. Since both httpalso
(and retrieving this very url) is limited by the usual
annex.security.allowed-ip-addresses configs, it's not possible for an
attacker to make one of these urls that sets up a httpalso url that
opens the garage door. Which is one class of attacks to keep in mind
with this thing.
It seems that there could be either a git-config that allows other types
of special remotes to be set up this way, or special remotes could
indicate when they are safe. I do worry that the git-config would
encourage users to set it without thinking through the security
implications. One remote config might be safe to access this way, but
another config, for one with the same type, might not be. This will need
further thought, and real-world examples to decide what to do.
2024-05-30 16:19:46 +00:00
|
|
|
This uses an URL that starts with "annex::". There are three forms of such
|
|
|
|
|
URLs:
|
|
|
|
|
|
|
|
|
|
* Complete URL
|
|
|
|
|
|
|
|
|
|
This contains the UUID and all configuration parameters
|
|
|
|
|
of the special remote that were passed when using
|
|
|
|
|
`git-annex initremote`.
|
|
|
|
|
|
|
|
|
|
For example, to clone from a directory special remote:
|
|
|
|
|
|
|
|
|
|
git clone annex::358ff77e-0bc3-11ef-bc49-872e6695c0e3?type=directory&encryption=none&directory=/mnt/foo/
|
|
|
|
|
|
|
|
|
|
* Shorthand URL
|
|
|
|
|
|
|
|
|
|
This makes it easy to configure an existing special remote with an URL
|
|
|
|
|
without having to come up with the complete URL.
|
|
|
|
|
|
|
|
|
|
annex::
|
|
|
|
|
|
|
|
|
|
When using this shorthand URL, the full URL will be displayed each time you
|
|
|
|
|
git pull or push, when it's possible for git-annex to determine it.
|
|
|
|
|
(Although in some cases, like the directory special remote, some
|
|
|
|
|
parameters may be left off of the displayed URL.)
|
|
|
|
|
|
|
|
|
|
* Web URL
|
|
|
|
|
|
|
|
|
|
This URL points at a file on the web, which contains the complete annex::
|
|
|
|
|
URL.
|
|
|
|
|
|
|
|
|
|
annex::https://example.com/foo-repo
|
|
|
|
|
|
|
|
|
|
Not all special remotes can be accessed by such an URL,
|
|
|
|
|
for security reasons. Currently, this is limited to httpalso special
|
|
|
|
|
remotes.
|
2024-05-24 21:15:31 +00:00
|
|
|
|
2024-05-28 19:29:38 +00:00
|
|
|
# CONFLICTING PUSHES
|
2024-05-28 17:26:21 +00:00
|
|
|
|
|
|
|
|
Like any git repository, a git repository stored on a special remote can
|
|
|
|
|
have conflicting things pushed to it from different places. This mostly
|
|
|
|
|
works the same as any other git repository, eg a push that overwrites other
|
2024-05-28 19:29:38 +00:00
|
|
|
work will be prevented unless forced.
|
|
|
|
|
|
|
|
|
|
However, it is possible, when conflicting pushes are being done at the same
|
|
|
|
|
time, for one of the pushes to be overwritten by the other one. In this
|
|
|
|
|
situation, the overwritten push will appear to have succeeded, but pulling
|
|
|
|
|
later will show the true situation.
|
|
|
|
|
|
|
|
|
|
# REPOSITORY FORMAT
|
2024-05-14 18:23:40 +00:00
|
|
|
|
2024-05-06 16:44:33 +00:00
|
|
|
The git repository is stored in the special remote using special annex objects
|
2024-05-28 17:26:21 +00:00
|
|
|
with names starting with "GITMANIFEST" and "GITBUNDLE". For details, see:
|
2024-05-06 16:44:33 +00:00
|
|
|
<https://git-annex.branchable.com/internals/git-remote-annex/>
|
|
|
|
|
|
|
|
|
|
Pushes to a special remote are usually done incrementally. However,
|
2024-05-28 19:29:38 +00:00
|
|
|
sometimes the whole git repository is re-uploaded. That is done when force
|
|
|
|
|
pushing a ref, or deleting a ref from the remote. It's also done when too
|
|
|
|
|
many git bundles accumulate in the special remote, as configured by the
|
2024-05-28 17:26:21 +00:00
|
|
|
`remote.<name>.annex-max-git-bundles` git config.
|
2024-05-21 14:41:48 +00:00
|
|
|
|
2024-05-28 17:26:21 +00:00
|
|
|
Note that a re-upload of the repository does not delete old GITBUNDLE
|
|
|
|
|
objects from it. This means that refs pushed to the special
|
2024-05-21 14:41:48 +00:00
|
|
|
remote can still be accessed even after deleting or overwriting them.
|
2024-05-28 19:29:38 +00:00
|
|
|
|
2024-05-28 17:26:21 +00:00
|
|
|
A push that deletes every ref from the special remote will delete all
|
2024-05-21 14:41:48 +00:00
|
|
|
the accumulated GITBUNDLE objects. But of course, making such a push
|
2024-05-28 17:26:21 +00:00
|
|
|
means that someone who clones from the special remote at that point in time
|
2024-05-21 14:41:48 +00:00
|
|
|
will see an empty remote.
|
2024-05-06 16:44:33 +00:00
|
|
|
|
|
|
|
|
# SEE ALSO
|
|
|
|
|
|
|
|
|
|
gitremote-helpers(1)
|
|
|
|
|
|
|
|
|
|
[[git-annex]](1)
|
|
|
|
|
|
|
|
|
|
[[git-annex-initremote]](1)
|
|
|
|
|
|
|
|
|
|
# AUTHOR
|
|
|
|
|
|
|
|
|
|
Joey Hess <id@joeyh.name>
|
|
|
|
|
|
|
|
|
|
Warning: Automatically converted into a man page by mdwn2man. Edit with care.
|