git-annex/doc/git-annex-addurl.mdwn

# NAME

git-annex addurl - add urls to annex

# SYNOPSIS

git annex addurl `[url ...]`

# DESCRIPTION

Downloads each url to its own file, which is added to the annex.

When `youtube-dl` is installed, it can be used to check for a video
embedded in  a web page at the url, and that is added to the annex instead.
(However, this is disabled by default as it can be a security risk. 
See the documentation of annex.security.allowed-http-addresses
in [[git-annex]](1) for details.)

Urls to torrent files (including magnet links) will cause the content of
the torrent to be downloaded, using `aria2c`.

Normally the filename is based on the full url, so will look like
"www.example.com_dir_subdir_bigfile". In some cases, addurl is able to
come up with a better filename based on other information. Options can also
be used to get better filenames.

# OPTIONS

* `--fast`

  Avoid immediately downloading the url. The url is still checked
  (via HEAD) to verify that it exists, and to get its size if possible.

* `--relaxed`

  Don't immediately download the url, and avoid storing the size of the
  url's content. This makes git-annex accept whatever content is there
  at a future point.

  This is the fastest option, but it still has to access the network
  to check if the url contains embedded media. When adding large numbers
  of urls, using `--relaxed --raw` is much faster.
  
* `--raw`

  Prevent special handling of urls by youtube-dl, bittorrent, and other
  special remotes. This will for example, make addurl
  download the .torrent file and not the contents it points to.

* `--file=name`

  Use with a filename that does not yet exist to add a new file
  with the specified name and the content downloaded from the url.

  If the file already exists, addurl will record that it can be downloaded
  from the specified url(s).

* `--pathdepth=N`

  Rather than basing the filename on the whole url, this causes a path to
  be constructed, starting at the specified depth within the path of the
  url.

  For example, adding the url http://www.example.com/dir/subdir/bigfile
  with `--pathdepth=1` will use "dir/subdir/bigfile",
  while `--pathdepth=3` will use "bigfile". 

  It can also be negative; `--pathdepth=-2` will use the last
  two parts of the url.

* `--prefix=foo` `--suffix=bar`

  Use to adjust the filenames that are created by addurl. For example,
  `--suffix=.mp3` can be used to add an extension to the file.

* `--jobs=N` `-JN`

  Enables parallel downloads when multiple urls are being added.
  For example: `-J4`  

  Setting this to "cpus" will run one job per CPU core.

* `--batch`

  Enables batch mode, in which lines containing urls to add are read from
  stdin.

* `-z`

  Makes the `--batch` input be delimited by nulls instead of the usual
  newlines. 

* `--with-files`

  When batch mode is enabled, makes it parse lines of the form: "$url $file"

  That adds the specified url to the specified file, downloading its
  content if the file does not yet exist; the same as
  `git annex addurl $url --file $file`

* `--json`

  Enable JSON output. This is intended to be parsed by programs that use
  git-annex. Each line of output is a JSON object.

* `--json-progress`

  Include progress objects in JSON output.

* `--json-error-messages`

  Messages that would normally be output to standard error are included in
  the json instead.

# CAVEATS

If annex.largefiles is configured, and does not match a file, `git annex
addurl` will add the non-large file directly to the git repository,
instead of to the annex. However, this is not done when --fast or --relaxed
is used.

# SEE ALSO

[[git-annex]](1)

[[git-annex-rmurl]](1)

[[git-annex-registerurl]](1)

[[git-annex-importfeed]](1)

# AUTHOR

Joey Hess <id@joeyh.name>

Warning: Automatically converted into a man page by mdwn2man. Edit with care.