From 428d228ee5d3a6c5c9a8802285f2d3269049a5dd Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 5 Jan 2021 14:22:27 -0400 Subject: [PATCH] docs for requirednumcopies Not implemented yet. --- CHANGELOG | 6 +++ doc/copies.mdwn | 66 +++++++++++++++++----------- doc/git-annex-numcopies.mdwn | 17 ++++--- doc/git-annex-requirednumcopies.mdwn | 43 ++++++++++++++++++ 4 files changed, 101 insertions(+), 31 deletions(-) create mode 100644 doc/git-annex-requirednumcopies.mdwn diff --git a/CHANGELOG b/CHANGELOG index 80859fac97..ca841d47d0 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -1,5 +1,11 @@ git-annex (8.20201130) UNRELEASED; urgency=medium + * Added requirednumcopies configuration. This is like numcopies, but is + enforced even more strictly. While numcopies can be violated in + concurrent drop situations involving special remotes that do not + support locking, requirednumcopies cannot be. The default value is 1, + which is not a behavior change, but now it can be set to higher + values if desired. * add: Significantly speed up adding lots of non-large files to git, by disabling the annex smudge filter when running git add. * add --force-small: Run git add rather than updating the index itself, diff --git a/doc/copies.mdwn b/doc/copies.mdwn index 3e0ebc6c69..73482da7be 100644 --- a/doc/copies.mdwn +++ b/doc/copies.mdwn @@ -1,35 +1,49 @@ Annexed data is stored inside your git repository's `.git/annex` directory. Some [[special_remotes]] can store annexed data elsewhere. -It's important that data not get lost by an ill-considered `git annex drop` -command. So, git-annex can be configured to try -to keep N copies of a file's content available across all repositories. -(Although [[untrusted_repositories|trust]] don't count toward this total.) +It's important that data not get lost by an ill-considered `git-annex drop` +command. So, git-annex can be configured to try to keep a number of copies +of a file's content available across all repositories. -By default, N is 1; it is configured by running `git annex numcopies N`. -This default can be overridden on a per-file-type basis by the annex.numcopies -setting in `.gitattributes` files. The --numcopies switch allows -temporarily using a different value. +By default, it keeps 1 copy; this is configured by running `git-annex +numcopies N`, or can be overridden on a per-file-type basis by the +annex.numcopies setting in `.gitattributes` files. The --numcopies switch +allows temporarily using a different value. -`git annex drop` attempts to check with other git remotes, to check that N -copies of the file exist. If enough repositories cannot be verified to have -it, it will retain the file content to avoid data loss. Note that -[[trusted_repositories|trust]] are not explicitly checked. +When dropping content, git-annex checks with remotes to make sure +If enough repositories cannot be verified to have it, it will retain +the file content to avoid data loss. -For example, consider three repositories: Server, Laptop, and USB. Both Server -and USB have a copy of a file, and N=1. If on Laptop, you `git annex get -$file`, this will transfer it from either Server or USB (depending on which -is available), and there are now 3 copies of the file. +When it can, git-annex locks enough copies on other repositories, to allow +it to safely drop a copy without any possibility that numcopies will be +violated. There are some exceptions, including special remotes not +supporting locking, and [[trusted repositories|trust]] that are not +accessible, where locking is not done. -Suppose you want to free up space on Laptop again, and you `git annex drop` the file -there. If USB is connected, or Server can be contacted, git-annex can check -that it still has a copy of the file, and the content is removed from -Laptop. But if USB is currently disconnected, and Server also cannot be -contacted, it can't verify that it is safe to drop the file, and will +If such a repository is being relied on to contain a copy and drops it at +the wrong time, the configured numcopies setting can be violated. To avoid +losing the last copy in such an unusual situation, git-annex requires that +at least 1 copy is locked in place when dropping content. If 1 does not +seem like enough, you can override this default by running `git-annex +requirednumcopies or setting annex.requirednumcopies in `.gitattributes` +files. + +To express more detailed requirements about which repositories contain which +content, see [[required_content]]. + +## example + +For example, consider three repositories: Server, Laptop, and USB. Both +Server and USB have a copy of a file, and numcopies is 1. If on Laptop, you +`git-annex get $file`, this will transfer it from either Server or USB +(depending on which is available), and there are now 3 copies of the file. + +Suppose you want to free up space on Laptop again, and you `git-annex drop` +the file there. If USB is connected, or Server can be contacted, git-annex +can check that it still has a copy of the file, and the content is removed +from Laptop. But if USB is currently disconnected, and Server also cannot +be contacted, it can't verify that it is safe to drop the file, and will refuse to do so. -With N=2, in order to drop the file content from Laptop, it would need access -to both USB and Server. - -For more complicated requirements about which repositories contain which -content, see [[required_content]]. +With numcopies of 2, in order to drop the file content from Laptop, it +would need access to both USB and Server. diff --git a/doc/git-annex-numcopies.mdwn b/doc/git-annex-numcopies.mdwn index 96701e1ef5..15ddb06fba 100644 --- a/doc/git-annex-numcopies.mdwn +++ b/doc/git-annex-numcopies.mdwn @@ -14,18 +14,25 @@ repositories. The default is 1. Run without a number to get the current value. This configuration is stored in the git-annex branch, so it will be seen -by all clones of the repository. +by all clones of the repository. It can be overridden on a per-file basis +by the annex.numcopies setting in .gitattributes files, or can be +overridden temporarily with the --numcopies option. When git-annex is asked to drop a file, it first verifies that the -required number of copies can be satisfied among all the other +number of copies can be satisfied among all the other repositories that have a copy of the file. - -This can be overridden on a per-file basis by the annex.numcopies setting -in .gitattributes files. + +In situations involving trusted repositories or special remotes that +cannot lock content in place, the numcopies setting may be violated +when the same file is being dropped at the same time from multiple +repositories. In these unusual situations, git-annex ensures that +the requirednumcopies setting (default 1) is not violated. See +[[git-annex-requirednumcopies]](1) for more about this setting. # SEE ALSO [[git-annex]](1) +[[git-annex-requirednumcopies]](1) # AUTHOR diff --git a/doc/git-annex-requirednumcopies.mdwn b/doc/git-annex-requirednumcopies.mdwn new file mode 100644 index 0000000000..7e25b7ec73 --- /dev/null +++ b/doc/git-annex-requirednumcopies.mdwn @@ -0,0 +1,43 @@ +# NAME + +git-annex requirednumcopies - configure required number of copies + +# SYNOPSIS + +git annex requirednumcopies `N` + +# DESCRIPTION + +Tells git-annex how many copies it is required to preserve of files, over all +repositories. The default is 1. + +Run without a number to get the current value. + +This configuration is stored in the git-annex branch, so it will be seen +by all clones of the repository. It can be overridden on a per-file basis +by the annex.requirednumcopies setting in .gitattributes files, or can be +overridden temporarily with the --requirednumcopies option. + +When git-annex is asked to drop a file, it makes sure that +that the required number of copies will still exist in other +repositories, by locking the content in them, preventing it from +being dropped. + +This supplements the [[git-annex-numcopies]](1) setting. git-annex +checks that numcopies is met before dropping. But in situations +involving trusted repositories or special remotes that +cannot lock content in place, the numcopies setting may be violated +when the same file is being dropped at the same time from multiple +repositories. In these unusual situations, git-annex ensures that +the requirednumcopies setting is not violated. + +# SEE ALSO + +[[git-annex]](1) +[[git-annex-numcopies]](1) + +# AUTHOR + +Joey Hess + +Warning: Automatically converted into a man page by mdwn2man. Edit with care.