277 lines
10 KiB
Markdown
277 lines
10 KiB
Markdown
git-annex tries to ensure that the configured number of [[copies]] of your
|
|
data always exist, and leaves it up to you to use commands like `git annex
|
|
get` and `git annex drop` to move the content to the repositories you want
|
|
to contain it. But often, it can be good to have more fine-grained
|
|
control over which content is wanted by which repositories. Configuring
|
|
this allows the git-annex assistant as well as
|
|
`git annex get --auto`, `git annex drop --auto`, `git annex sync --content`,
|
|
etc to do smarter things.
|
|
|
|
Preferred content settings can be edited using `git
|
|
annex vicfg`, or viewed and set at the command line with `git annex wanted`.
|
|
Each repository can have its own settings, and other repositories will
|
|
try to honor those settings when interacting with it.
|
|
(So there's no local `.git/config` for preferred content settings.)
|
|
|
|
The idea is that you write an expression that files are matched against.
|
|
If a file matches, the repository wants to store its content.
|
|
If it doesn't, the repository wants to drop its content
|
|
(if there are enough copies elsewhere to allow removing it).
|
|
|
|
## finding preferred content
|
|
|
|
To check at the command line which files are matched by preferred content
|
|
settings, you can use the --want-get and --want-drop options.
|
|
|
|
For example, `git annex find --want-get --not --in .` will find all the
|
|
files that `git annex get --auto` will want to get, and `git annex find
|
|
--want-drop --in .` will find all the files that `git annex drop --auto`
|
|
will want to drop.
|
|
|
|
## writing expressions
|
|
|
|
[[!template id=note text="""
|
|
### [[quickstart|standard_groups]]
|
|
|
|
Rather than writing your own preferred content expression, you can use
|
|
several standard ones included in git-annex that are tuned to cover different
|
|
common use cases.
|
|
|
|
You do this by putting a repository in a group,
|
|
and simply setting its preferred content to "standard" to match whatever
|
|
is standard for that group. See [[standard_groups]] for a list.
|
|
"""]]
|
|
|
|
|
|
The expressions are very similar to the matching options documented
|
|
on the [[git-annex-matching-options]] man page.
|
|
At the command line, you can use those options in commands like this:
|
|
|
|
git annex get --include='*.mp3' --and -'(' --not --largerthan=100mb -')'
|
|
|
|
The equivalent preferred content expression looks like this:
|
|
|
|
include=*.mp3 and (not largerthan=100mb)
|
|
|
|
So, just remove the dashes, basically. But, there are some differences
|
|
between the command line options and expressions, so see the documentation
|
|
below to get the full story.
|
|
|
|
* `include=glob` and `exclude=glob`
|
|
|
|
Match files to include, or exclude.
|
|
|
|
While --include=glob and --exclude=glob match files relative to the current
|
|
directory, preferred content expressions always match files relative to the
|
|
top of the git repository.
|
|
|
|
For example, suppose you put files into `archive` directories
|
|
when you're done with them. Then you could configure your laptop to prefer
|
|
to not retain those files, like this: `exclude=*/archive/*`
|
|
|
|
* `copies=number`
|
|
|
|
Matches only files that git-annex believes to have the specified number
|
|
of copies, or more. Note that it does not check remotes to verify that
|
|
the copies still exist.
|
|
|
|
To decide if content should be dropped, git-annex evaluates the preferred
|
|
content expression under the assumption that the content has *already* been
|
|
dropped. If the content would not be wanted then, the drop can be done.
|
|
So, for example, `copies=2` in a preferred content expression lets
|
|
content be dropped only when there are currently 3 copies of it, including
|
|
the repo it's being dropped from. This is different than running `git annex
|
|
drop --copies=2`, which will drop files that currently have 2 copies.
|
|
|
|
* `copies=trustlevel:number`
|
|
|
|
Matches only files that git-annex believes have the specified number
|
|
copies, on remotes with the specified trust level. For example,
|
|
`copies=trusted:2`
|
|
|
|
To match any trust level at or higher than a given level,
|
|
use `trustlevel+`. For example, `--copies=semitrusted+:2`
|
|
|
|
* `copies=groupname:number`
|
|
|
|
Matches only files that git-annex believes have the specified number of
|
|
copies, on remotes in the specified group. For example,
|
|
`copies=archive:2`
|
|
|
|
Preferred content expressions have no equivilant to the `--in`
|
|
option, but groups can accomplish similar things. You can add
|
|
repositories to groups, and match against the groups in a
|
|
preferred content expression. So rather than `--in=usbdrive`,
|
|
put all the USB drives into a "transfer" group, and use
|
|
`copies=transfer:1`
|
|
|
|
* `lackingcopies=number`
|
|
|
|
Matches only files that git-annex believes need the specified number or
|
|
more additional copies to be made in order to satisfy their numcopies
|
|
settings.
|
|
|
|
* `approxlackingcopies=number`
|
|
|
|
Like lackingcopies, but does not look at .gitattributes annex.numcopies
|
|
settings. This makes it significantly faster.
|
|
|
|
* `inbackend=name`
|
|
|
|
Matches only files whose content is stored using the specified key-value
|
|
backend.
|
|
|
|
* `inallgroup=groupname`
|
|
|
|
Matches only files that git-annex believes are present in all repositories
|
|
in the specified group.
|
|
|
|
* `smallerthan=size` and `largerthan=size`
|
|
|
|
Matches only files whose content is smaller than, or larger than the
|
|
specified size.
|
|
|
|
The size can be specified with any commonly used units, for example,
|
|
"0.5 gb" or "100 KiloBytes"
|
|
|
|
* `metadata=field=glob`
|
|
|
|
Matches only files that have a metadata field attached with a value that
|
|
matches the glob. The values of metadata fields are matched case
|
|
insensitively.
|
|
|
|
To match a tag "done", use `metadata=tag=done`
|
|
|
|
To match author metadata, use `metadata=author=* Smith`
|
|
|
|
* `present`
|
|
|
|
Makes content be wanted if it's present, but not otherwise.
|
|
|
|
This leaves it up to you to use git-annex manually
|
|
to move content around. You can use this to avoid preferred content
|
|
settings from affecting a subdirectory. For example:
|
|
`auto/* or (include=ad-hoc/* and present)`
|
|
|
|
Note that `not present` is a very bad thing to put in a preferred content
|
|
expression. It'll make it want to get content that's not present, and
|
|
drop content that is present! Don't go there..
|
|
|
|
* `inpreferreddir`
|
|
|
|
Makes content be preferred if it's in a directory (located anywhere
|
|
in the tree) with a particular name.
|
|
|
|
The name of the directory can be configured using
|
|
`git annex enableremote $remote preferreddir=$dirname`
|
|
|
|
(If no directory name is configured, it uses "public" by default.)
|
|
|
|
* `standard`
|
|
|
|
git-annex comes with some built-in preferred content expressions, that
|
|
can be used with repositories that are in some [[standard_groups]].
|
|
|
|
When a repository is in exactly one such group, you can use the "standard"
|
|
keyword in its preferred content expression, to match whatever content
|
|
the group's expression matches.
|
|
(If a repository is put into multiple standard
|
|
groups, "standard" will match anything.. so don't do that!)
|
|
|
|
Most often, the whole preferred content expression is simply "standard".
|
|
But, you can do more complicated things, for example:
|
|
`standard or include=otherdir/*`
|
|
|
|
* `groupwanted`
|
|
|
|
The "groupwanted" keyword can be used to refer to a preferred content
|
|
expression that is associated with a group. This is like the "standard"
|
|
keyword, but you can configure the preferred content expressions
|
|
using `git annex groupwanted`.
|
|
|
|
Note that when writing a groupwanted preferred content expression,
|
|
you can use all of the keywords listed above, including "standard".
|
|
(But not "groupwanted".)
|
|
|
|
For example, to make a variant of the standard client preferred content
|
|
expression that does not want files in the "out" directory, you
|
|
could run: `git annex groupwanted client "standard and exclude=out/*"`
|
|
|
|
Then repositories that are in the client group and have their preferred
|
|
content expression set to "groupwanted" will use that, while
|
|
other client repositories that have their preferred content expression
|
|
set to "standard" will use the standard expression.
|
|
|
|
Or, you could make a new group, with your own custom preferred content
|
|
expression tuned for your needs, and every repository you put in this
|
|
group and make its preferred content be "groupwanted" will use it.
|
|
|
|
For example, the archive group only wants to archive 1 copy of each file,
|
|
spread among every repository in the group.
|
|
Here's how to configure a group named redundantarchive, that instead
|
|
wants to contain 3 copies of each file:
|
|
|
|
git annex groupwanted redundantarchive "not (copies=redundantarchive:3)"
|
|
for repo in foo bar baz; do
|
|
git annex group $repo redundantarchive
|
|
git annex wanted $repo groupwanted
|
|
done
|
|
|
|
* `unused`
|
|
|
|
Matches only keys that `git annex unused` has determined to be unused.
|
|
|
|
This is related the the --unused option.
|
|
However, putting `unused` in a preferred content expression
|
|
doesn't make git-annex consider those unused keys. So when git-annex is
|
|
only checking preferred content expressions against files in the
|
|
repository (which are obviously used), `unused` in a preferred
|
|
content expression won't match anything.
|
|
|
|
So when is `unused` useful in a preferred content expression?
|
|
|
|
1. Using `git annex sync --content --all` will operate on all files,
|
|
including unused ones, and take `unused` in preferred content expressions
|
|
into account.
|
|
2. The git-annex assistant periodically scans for unused files, and
|
|
moves them to some repository whose preferred content expression
|
|
says it wants them. (Or, if annex.expireunused is set, it may just delete
|
|
them.)
|
|
|
|
* `anything`
|
|
|
|
Matches any version of any file.
|
|
|
|
* `not expression`
|
|
|
|
Inverts what the expression matches. For example, `not include=archive/*`
|
|
is the same as `exclude=archive/*`
|
|
|
|
* `and` / `or` / `( expression )`
|
|
|
|
These can be used to build up more complicated expressions.
|
|
|
|
## upgrades
|
|
|
|
It's important that all clones of a repository can understand one-another's
|
|
preferred content expressions, especially when using the git-annex
|
|
assistant. So using newly added keywords can cause a problem if
|
|
an older version of git-annex is in use elsewhere.
|
|
|
|
Before git-annex version 5.20140320, when git-annex saw a keyword it
|
|
did not understand, it defaulted to assuming *all* files were
|
|
preferred content. From version 5.20140320, git-annex has a nicer fallback
|
|
behavior: When it is unable to parse a preferred content expression,
|
|
it assumes all files that are currently present are preferred content.
|
|
|
|
Here are recent changes to preferred content expressions, and the version
|
|
they were added in.
|
|
|
|
* "anything" 5.20150616
|
|
* "standard" 5.20140314
|
|
(only when used in a more complicated expression; "standard" by
|
|
itself has been supported for a long time)
|
|
* "groupwanted=" 5.20140314
|
|
* "metadata=" 5.20140221
|
|
* "lackingcopies=", "approxlackingcopies=", "unused=" 5.20140127
|
|
* "inpreferreddir=" 4.20130501
|