313 lines
9.3 KiB
Markdown
313 lines
9.3 KiB
Markdown
# NAME
|
|
|
|
git-annex sim - simulate a network of repositories
|
|
|
|
# SYNOPSIS
|
|
|
|
git annex sim start [my.sim]
|
|
|
|
git annex sim command
|
|
|
|
git annex sim visit repo
|
|
|
|
git annex sim end
|
|
|
|
# DESCRIPTION
|
|
|
|
This command simulates the behavior of git-annex in a network of
|
|
repositories, determining which files would reach which repositories
|
|
according to the configuration of preferred content, numcopies,
|
|
trust level, etc.
|
|
|
|
The input to the simulation is a sim file, and/or sim commands that are
|
|
run after starting it. These are in the form "git annex sim command"
|
|
with the command in the same format used in the sim file (see sim commands
|
|
list below). For example, "git annex sim step 1" runs the simulation one step.
|
|
|
|
The simulation writes to an output sim file as it runs, which contains the
|
|
entire simulation input, as well as the actions performed in the
|
|
simulation, and the results of the simulation.
|
|
This allows re-running the same simulation later, as well as analyzing
|
|
the results of the simulation.
|
|
|
|
While a simulation is running, the command "git annex visit repo", where
|
|
"repo" is the name of one of the repositories in the simulation, will spawn
|
|
a subshell in a git repository whose git-annex branch contains the state of
|
|
that simulated repository. This allows running any git-annex command, such
|
|
as `git-annex whereis` to examine the state of the simulation.
|
|
Exit the subshell to end the visit.
|
|
|
|
Use "git annex sim end" to finish the simulation, and clean up.
|
|
|
|
# THE SIM FILE
|
|
|
|
This text file is used to configure the simulation and also to report on
|
|
the results of the simulation. Each line takes the form of a command
|
|
followed by parameters to the command. Lines starting with "#" are comments.
|
|
|
|
Here is an example sim file:
|
|
|
|
# add repositories to the simulation and connect them as remotes
|
|
init foo
|
|
init bar
|
|
connect foo <-> bar
|
|
|
|
# add a special remote
|
|
initremote baz
|
|
connect foo -> baz <- bar
|
|
|
|
# configure repositories
|
|
numcopies 2
|
|
group foo client
|
|
wanted foo standard
|
|
group bar archive
|
|
wanted bar standard
|
|
wanted baz include=*.mp3
|
|
|
|
# add annexed files in the working tree to the simulation, as if they
|
|
# were just added to repository foo
|
|
addtree foo include=*.mp3
|
|
addtree foo include=*.jpg
|
|
addtree foo include=bigfiles/
|
|
|
|
# add simulated annexed files
|
|
add bigfile 100gb bar
|
|
add hugefile 10tb foo
|
|
|
|
# run the simulation forward by ten steps
|
|
step 10
|
|
|
|
# remove foo's remote bar and see if a new file added to foo reaches bar
|
|
disconnect foo -> bar
|
|
add foo.mp3 2mb foo
|
|
step 5
|
|
|
|
# SIM COMMANDS
|
|
|
|
This is the full set of commands that can be used in the sim file as well
|
|
as passed to "git annex sim" while a simulation is running.
|
|
|
|
* `init name`
|
|
|
|
Initialize a simulated repository, giving it a name that will be used
|
|
in the simulation.
|
|
|
|
* `initremote name`
|
|
|
|
Initialize a simulated special remote.
|
|
|
|
* `use name here|remote|description|uuid`
|
|
|
|
Use an existing repository in the simulation, with its existing
|
|
configuration (trust level, groups, preferred and required content,
|
|
maxsize, and the groupwanted configuration of its groups).
|
|
|
|
The repository is given a name for the purposes of the simulation.
|
|
The repository to use can be specified by remote name, uuid, etc.
|
|
Example: "use myrepo here"
|
|
|
|
* `connect repo [<-|->|<->] repo [...]`
|
|
|
|
Add a connection between two or more repositories. The arrow indicates
|
|
which direction the connection runs, and it can be bidirectional. For
|
|
example, "connect foo -> bar" makes bar be a remote of foo, while
|
|
"connect foo <-> bar" makes each be the remote of the other. A chain
|
|
of connections can extend to many repositories, eg
|
|
"connect foo -> bar -> baz -> foo"
|
|
|
|
* `disconnect repo [<-|->|<->] repo [...]`
|
|
|
|
Removes connections between repositories.
|
|
|
|
For example, "disconnect foo -> bar" makes foo no longer have bar as a
|
|
remote.
|
|
|
|
* `addtree repo expression`
|
|
|
|
Adds annexed files from the git repository to the simulation making them
|
|
be present in the specified repository.
|
|
|
|
The expression is a preferred content expression
|
|
(see [[git-annex-preferred-content]](1)) specifying which annexed files
|
|
to add. While it is possible to include all or a large number of files
|
|
this way, note that often it's more efficient to simulate a small
|
|
quantity of files that have the particular properties you are interested
|
|
in.
|
|
|
|
This can be used with the same files more than once, to make multiple
|
|
repositories in the simulation contain the same files.
|
|
|
|
* `add filename size repo [repo ...]`
|
|
|
|
Create a simulated annexed file with the specified filename and size,
|
|
that is present in the specified repository, or repositories.
|
|
|
|
The size can be specified using any usual units, eg "10mb" or
|
|
"3.3terabytes"
|
|
|
|
The filename cannot contain a space.
|
|
|
|
This stages a file in the index, so that regular git-annex commands can
|
|
be used to query the state of the simulated annexed file. If there is
|
|
already an annexed file by that name, it will be overwritten with the new
|
|
file.
|
|
|
|
Note that the simulation does not cover adding conflicting files to
|
|
different repositories. The files in the simulation are the same across
|
|
all simulated repositories.
|
|
|
|
* `step N`
|
|
|
|
Run the simulation forward by this many steps.
|
|
|
|
On each step of the simulation, a simulated repository is selected
|
|
at random, and a random action is performed in it.
|
|
|
|
* `action repo getwanted remote`
|
|
|
|
Simulate the repository getting files it wants from the remote.
|
|
|
|
* `action repo dropunwanted`
|
|
|
|
Simulate the repository dropping files it does not want,
|
|
when it is able to verify enough copies exist on remotes.
|
|
|
|
* `action repo dropunwanted remote`
|
|
|
|
Simulate the repository dropping files from the remote that the remote
|
|
does not want, when it is able to verify enouh copies exist.
|
|
|
|
* `action repo sendwanted remote`
|
|
|
|
Simulate the repository sending files that the remote wants to it.
|
|
|
|
* `action repo gitpush remote`
|
|
|
|
Simulate the repository pushing the git-annex branch to the remote.
|
|
|
|
* `action repo gitpull remote`
|
|
|
|
Simulate the repository pulling the git-annex branch from the remote.
|
|
|
|
* `action repo pull remote`
|
|
|
|
Simulate the equivilant of [[git-annex-pull]](1), by combining
|
|
the actions gitpull, getwanted, and dropunwanted.
|
|
|
|
* `action repo push remote`
|
|
|
|
Simulate the equivilant of [[git-annex-push]](1) by combining
|
|
the actions sendwanted, dropunwanted, and gitpush.
|
|
|
|
* `action repo sync remote`
|
|
|
|
Simulate the equivilant of [[git-annex-sync]](1) by combining
|
|
the actions gitpull, getwanted, sendwanted, dropunwanted, and gitpush.
|
|
|
|
* `seed N`
|
|
|
|
Sets the random seed to a given number. Using this should make the
|
|
results of the simulation deterministic. The output sim file
|
|
always has the random seed included in it, so usually you don't need to
|
|
specify this.
|
|
|
|
* `present repo file`
|
|
|
|
This indicates the expected state of the simulation at this point. The
|
|
repository should contain the content of the file. If it does not, the
|
|
discrepancy will be indicated on standard error, and the `git-annex sim`
|
|
command will eventually exit nonzero.
|
|
|
|
This is added to the output sim file as the simulation runs.
|
|
|
|
* `notpresent repo file`
|
|
|
|
This indicates the expected state of the simulation at this point. The
|
|
repository should not contain the content of the file. If it does, the
|
|
discrepancy will be indicated on standard error, and the `git-annex sim`
|
|
command will eventually exit nonzero.
|
|
|
|
This is added to the output sim file as the simulation runs.
|
|
|
|
* `numcopies N`
|
|
|
|
Sets the desired number of copies. This is equivilant to
|
|
[[git-annex-numcopies]](1).
|
|
|
|
Note that other configuration that sets numcopies, such as .gitattributes
|
|
files, is not used by the simulation.
|
|
|
|
* `mincopies N`
|
|
|
|
Sets the minimum number of copies. This is equivilant to
|
|
[[git-annex-mincopies]](1).
|
|
|
|
* `trustlevel repo trusted|untrusted|semitrusted|dead`
|
|
|
|
Sets the trust level of the repository. This is equivilant to
|
|
[[git-annex-trust]](1), [[git-annex-untrust]](1), etc.
|
|
|
|
* `group repo group`
|
|
|
|
Add a repository to a group. This is equivilant to
|
|
[[git-annex-group]](1).
|
|
|
|
* `ungroup repo group`
|
|
|
|
Remove a repository from a group. This is equivilant to
|
|
[[git-annex-ungroup]](1).
|
|
|
|
* `wanted repo expression`
|
|
|
|
Configure the preferred content of a repository. This is equivilant
|
|
to [[git-annex-wanted]](1).
|
|
|
|
* `required repo expression`
|
|
|
|
Configure the required content of a repository. This is equivilant
|
|
to [[git-annex-required]](1).
|
|
|
|
* `groupwanted group expression`
|
|
|
|
Configure the groupwanted expression. This is equivilant to
|
|
[[git-annex-groupwanted]](1).
|
|
|
|
* `maxsize repo size`
|
|
|
|
Configure the maximum size of a repository. This is equivilant to
|
|
[[git-annex-maxsize]](1).
|
|
|
|
* `rebalance [on|off]`
|
|
|
|
Setting "rebalance on" is the equivilant of passing the --rebalance
|
|
option to git-annex. Setting "rebalance off" undoes that.
|
|
|
|
For example:
|
|
|
|
maxsize foo 1tb
|
|
rebalance on
|
|
step 100
|
|
rebalance off
|
|
|
|
# OPTIONS
|
|
|
|
* The [[git-annex-common-options]](1) can be used.
|
|
|
|
# HASKELL INTERFACE
|
|
|
|
There is also a Haskell interface to the simulation,
|
|
in the git-annex source tree in the Annex.Sim module. This allows
|
|
implementing simulations in pure Haskell code, without the overhead of
|
|
using a git repository.
|
|
|
|
# SEE ALSO
|
|
|
|
[[git-annex]](1)
|
|
|
|
[[git-annex-test]](1)
|
|
|
|
# AUTHOR
|
|
|
|
Joey Hess <id@joeyh.name>
|
|
|
|
Warning: Automatically converted into a man page by mdwn2man. Edit with care.
|