8a305e5fa3
When a web special remote does not have urlinclude/urlexclude configured, make it respect the configuration of other web special remotes and avoid using urls that match the config of another. Note that the other web special remote does not have to be enabled. That seems ok, it would have been extra work to check for only ones that are enabled. The implementation does mean that the web special remote re-parses its own config once at startup, as well as re-parsing the configs of any other web special remotes. This should be a very small slowdown unless there are lots of web special remotes. Sponsored-by: Dartmouth College's DANDI project
42 lines
1.9 KiB
Markdown
42 lines
1.9 KiB
Markdown
git-annex can use the web as a special remote, associating an url with an
|
|
annexed file, and downloading the file content from the web.
|
|
See [[tips/using_the_web_as_a_special_remote]] for usage examples.
|
|
|
|
The web special remote is always enabled, without any manual setup being
|
|
needed. Its name is "web".
|
|
|
|
This special remote can only be used for downloading content,
|
|
not uploading content, or removing content from the web.
|
|
|
|
This special remote uses urls on the web as the source for content.
|
|
There are several other ways http can be used to download annexed objects,
|
|
including a git remote accessible by http, S3 with a `publicurl` configured,
|
|
and the [[httpalso]] special remote.
|
|
|
|
## configuration
|
|
|
|
These parameters can be passed to `git annex initremote` or
|
|
`git-annex enableremote` to configure a web remote:
|
|
|
|
* `urlinclude` - Only use urls that match the specified glob.
|
|
For example, `urlinclude="https://s3.amazonaws.com/*"`
|
|
* `urlexclude` - Don't use urls that match the specified glob.
|
|
For example, to prohibit http urls, but allow https,
|
|
use `urlexclude="http:*"`
|
|
|
|
Globs are matched case-insensitively.
|
|
|
|
When there are multiple special remotes of type web, and some are not
|
|
configured with `urlinclude` and/or `urlexclude`, those will avoid using
|
|
urls that are matched by the configuration of other web remotes.
|
|
|
|
For example, this creates a second web special remote named "slowweb" that
|
|
is only used for urls on one host, and that has a higher cost than the
|
|
"web" special remote. With this configuration, `git-annex get` will first
|
|
try to get the file from the "web" special remote, which will avoid
|
|
using any urls that match slowweb's urlinclude. Only if the content
|
|
can't be downloaded from "web" (or some other remote) will it fall back
|
|
to downloading from slowweb.
|
|
|
|
git-annex initremote --sameas=web slowweb type=web urlinclude='*//slowhost.com/*'
|
|
git config remote.slowweb.cost 300
|