git-annex/doc/git-annex-export.mdwn
Joey Hess 0bcecb67f5
export: Let concurrent transfers be done with -J or annex.jobs
Tested working, although I did find this bug in my testing, which also
afflicts sync -J to an export remote.
2020-05-26 11:44:07 -04:00

169 lines
5.7 KiB
Markdown

# NAME
git-annex export - export content to a remote
# SYNOPSIS
git annex export `treeish --to remote`
# DESCRIPTION
Use this command to export a tree of files from a git-annex repository.
Normally files are stored on a git-annex special remote named by their
keys. That is great for reliable data storage, but your filenames are
obscured. Exporting replicates the tree to the special remote as-is.
Mixing key/value storage and exports in the same remote would be a mess and
so is not allowed. You have to configure a special remote with
`exporttree=yes` when initially setting it up with
[[git-annex-initremote]](1).
The treeish to export can be the name of a git branch, or a tag, or any
other treeish accepted by git, including eg master:subdir to only export a
subdirectory from a branch.
When the remote has a preferred content setting, the treeish is filtered
through it, excluding files it does not want from being exported to it.
Repeated exports are done efficiently, by diffing the old and new tree,
and transferring only the changed files, and renaming files as necessary.
Exports can be interrupted and resumed. However, partially uploaded files
will be re-started from the beginning in most cases.
Once content has been exported to a remote, commands like `git annex get`
can download content from there the same as from other remotes. However,
since an export is not a key/value store, git-annex has to do more
verification of content downloaded from an export. Some types of keys,
that are not based on checksums, cannot be downloaded from an export.
And, git-annex will never trust an export to retain the content of a key.
However, some special remotes, notably S3, support keeping track of old
versions of files stored in them. If a special remote is set up to do
that, it can be used as a key/value store and the limitations in the above
paragraph do not apply. Note that dropping content from such a remote is
not supported. See individual special remotes' documentation for
details of how to enable such versioning.
The `git annex sync --content` command (and the git-annex assistant)
can also be used to export a branch to a special remote,
updating the special remote whenever the branch is changed.
To do this, you need to configure "remote.<name>.annex-tracking-branch"
to tell it what branch to track.
For example:
git config remote.myremote.annex-tracking-branch master
git annex sync --content
You can combine using `git annex export` to send changes to a special
remote with `git annex import` to fetch changes from a special remote.
When a file on a special remote has been modified by software other than
git-annex, exporting to it will not overwrite the modified file, and the
export will not succeed. You can resolve this conflict by using
`git annex import`.
(Some types of special remotes such as S3 with versioning may instead
let an export overwrite the modified file; then `git annex import`
will create a sequence of commits that includes the modified file,
so the overwritten modification is not lost.)
# OPTIONS
* `--to=remote`
Specify the special remote to export to.
* `--tracking`
This is a deprecated way to set "remote.<name>.annex-tracking-branch".
Instead of using this option, you should just set the git configuration
yourself.
* `--fast`
This sets up an export of a tree, but avoids any expensive file uploads to
the remote. You can later run `git annex sync --content` to upload
the files to the export.
* `--jobs=N` `-JN`
Exports multiple files in parallel. This may be faster.
For example: `-J4`
Setting this to "cpus" will run one job per CPU core.
* `--json`
Enable JSON output. This is intended to be parsed by programs that use
git-annex. Each line of output is a JSON object.
* `--json-progress`
Include progress objects in JSON output.
* `--json-error-messages`
Messages that would normally be output to standard error are included in
the json instead.
# EXAMPLE
git annex initremote myremote type=directory directory=/mnt/myremote \
exporttree=yes encryption=none
git annex export master --to myremote
After that, /mnt/myremote will contain the same tree of files as the master
branch does.
git mv myfile subdir/myfile
git commit -m renamed
git annex export master --to myremote
That updates /mnt/myremote to reflect the renamed file.
git annex export master:subdir --to myremote
That updates /mnt/myremote, to contain only the files in the "subdir"
directory of the master branch.
# EXPORT CONFLICTS
If two different git-annex repositories are both exporting different trees
to the same special remote, it's possible for an export conflict to occur.
This leaves the special remote with some files from one tree, and some
files from the other. Files in the special remote may have entirely the
wrong content as well.
It's not possible for git-annex to detect when making an export will result
in an export conflict. The best way to avoid export conflicts is to either
only ever export to a special remote from a single repository, or to have a
rule about the tree that you export to the special remote. For example, if
you always export origin/master after pushing to origin, then an export
conflict can't happen.
An export conflict can only be detected after the two git repositories
that produced it get back in sync. Then the next time you run `git annex
export`, it will detect the export conflict, and resolve it.
# SEE ALSO
[[git-annex]](1)
[[git-annex-initremote]](1)
[[git-annex-import]](1)
[[git-annex-sync]](1)
[[git-annex-preferred-content]](1)
# HISTORY
The `export` command was introduced in git-annex version 6.20170925.
# AUTHOR
Joey Hess <id@joeyh.name>
Warning: Automatically converted into a man page by mdwn2man. Edit with care.