config: Added the --show-origin and --for-file options

* config: Added the --show-origin and --for-file options.
* config: Support annex.numcopies and annex.mincopies.

There is a little bit of redundancy here with other code elsewhere that
combines the various configs and selects which to use. But really only
for the special case of annex.numcopies, which is a git config that does
not override the annex branch setting and for annex.mincopies, which does
not have a git config but does have gitattributes settings as well as the
annex branch setting.

That seems small enough, and unlikely enough to grow into a mess that it was
worth supporting annex.numcopies and annex.mincopies in git-annex config
--show-origin. Because these settings are a prime thing that someone might
get confused about and want to know where they were configured.

And, it followed that git-annex config might as well support those two
for --set and --get as well. While this is redundant with the speclialized
commands, it's only a little code and it makes it more consistent.

Note that --set does not have as nice output as numcopies/mincopies
commands in some special cases like setting to 0 or a negative number.
It does avoid setting to a bad value thanks to the smart
constructors (eg configuredNumCopies).

As for other git-annex branch configurations that are not set by git-annex
config, things like trust and wanted that are specific to a repository
don't map to a git config name, so don't really fit into git-annex config.
And they are only configured in the git-annex branch with no local override
(at least so far), so --show-origin would not be useful for them.

Sponsored-by: Dartmouth College's DANDI project

Annex/CheckAttr.hs

@@ -6,6 +6,7 @@
  -}
 
 module Annex.CheckAttr (
+	annexAttrs,
 	checkAttr,
 	checkAttrs,
 	checkAttrStop,

CHANGELOG

@@ -88,6 +88,8 @@ git-annex (10.20230408) UNRELEASED; urgency=medium
     or --unlock-present.
   * assistant: Add dotfiles to git by default, unless annex.dotfiles
     is configured, the same as git-annex add does.
+  * config: Added the --show-origin and --for-file options.
+  * config: Support annex.numcopies and annex.mincopies.
 
  -- Joey Hess <id@joeyh.name>  Sat, 08 Apr 2023 13:57:18 -0400
 

Command/Config.hs

@@ -1,6 +1,6 @@
 {- git-annex command
  -
- - Copyright 2017-2020 Joey Hess <id@joeyh.name>
+ - Copyright 2017-2023 Joey Hess <id@joeyh.name>
  -
  - Licensed under the GNU AGPL version 3 or higher.
  -}
@@ -13,8 +13,12 @@ import Command
 import Logs.Config
 import Config
 import Types.GitConfig (globalConfigs)
-import Git.Types (fromConfigValue)
+import Git.Types (fromConfigValue, fromConfigKey)
+import qualified Git.Command
 import Utility.SafeOutput
+import Annex.CheckAttr
+import Types.NumCopies
+import Logs.NumCopies
 
 import qualified Data.ByteString.Char8 as S8
 
@@ -27,12 +31,13 @@ data Action
 	= SetConfig ConfigKey ConfigValue
 	| GetConfig ConfigKey
 	| UnsetConfig ConfigKey
+	| ShowOrigin ConfigKey (Maybe FilePath)
 
 type Name = String
 type Value = String
 
 optParser :: CmdParamsDesc -> Parser Action
-optParser _ = setconfig <|> getconfig <|> unsetconfig
+optParser _ = setconfig <|> getconfig <|> unsetconfig <|> showorigin
   where
 	setconfig = SetConfig
 		<$> strOption
@@ -53,40 +58,158 @@ optParser _ = setconfig <|> getconfig <|> unsetconfig
 		<> help "unset configuration"
 		<> metavar paramName
 		)
+	showorigin = ShowOrigin
+		<$> strOption
+			( long "show-origin"
+			<> help "explain where a value is configured"
+			<> metavar paramName
+			)
+		<*> optional (strOption
+			( long "for-file"
+			<> help "filename to check for in gitattributes"
+			<> metavar paramFile
+			))
 
 seek :: Action -> CommandSeek
-seek (SetConfig ck@(ConfigKey name) val) = checkIsGlobalConfig ck $ commandAction $
+seek (SetConfig ck@(ConfigKey name) val) = checkIsGlobalConfig ck $ \setter _unsetter _getter ->
-	startingUsualMessages (decodeBS name) ai si $ do
+	commandAction $ startingUsualMessages (decodeBS name) ai si $ do
-		setGlobalConfig ck val
+		setter val
 		when (needLocalUpdate ck) $
 			setConfig ck (fromConfigValue val)
 		next $ return True
   where
 	ai = ActionItemOther (Just (UnquotedString (fromConfigValue val)))
 	si = SeekInput [decodeBS name]
-seek (UnsetConfig ck@(ConfigKey name)) = checkIsGlobalConfig ck $ commandAction $
+seek (UnsetConfig ck@(ConfigKey name)) = checkIsGlobalConfig ck $ \_setter unsetter _getter ->
-	startingUsualMessages (decodeBS name) ai si $ do
+	commandAction $ startingUsualMessages (decodeBS name) ai si $ do
-		unsetGlobalConfig ck
+		unsetter
 		when (needLocalUpdate ck) $
 			unsetConfig ck
 		next $ return True
   where
 	ai = ActionItemOther (Just "unset")
 	si = SeekInput [decodeBS name]
-seek (GetConfig ck) = checkIsGlobalConfig ck $ commandAction $
+seek (GetConfig ck) = checkIsGlobalConfig ck $ \_setter _unsetter getter ->
-	startingCustomOutput ai $ do
+	commandAction $	startingCustomOutput ai $ do
-		getGlobalConfig ck >>= \case
+		getter >>= \case
 			Just (ConfigValue v) -> liftIO $ S8.putStrLn $ safeOutput v
 			Just NoConfigValue -> return ()
 			Nothing -> return ()
 		next $ return True
+  where
+	ai = ActionItemOther Nothing
+seek (ShowOrigin ck@(ConfigKey name) forfile) = commandAction $
+	startingCustomOutput ai $ next $ checknotconfigured $
+		case checkIsGlobalConfig' ck of
+			Just (_setter, _unsetter, getter) ->
+				ifM gitconfigorigin
+					( return True
+					, checkattrs (checkconfigbranch getter)
+					)
+			Nothing -> ifM gitconfigorigin
+				( return True
+				, checkattrs checkgitconfigunderride
+				)
   where
 	ai = ActionItemOther Nothing
 
-checkIsGlobalConfig :: ConfigKey -> Annex a -> Annex a
+	gitconfigorigin
-checkIsGlobalConfig ck@(ConfigKey name) a
+		| name `elem` gitconfigdoesnotoverride = return False
-	| elem ck globalConfigs = a
+		| otherwise = gitconfigorigin'
-	| otherwise = giveup $ decodeBS name ++ " is not a configuration setting that can be stored in the git-annex branch"
+	gitconfigorigin' = inRepo $ Git.Command.runBool
+			[ Param "config"
+			, Param "--show-origin"
+			, Param (decodeBS name)
+			]
+	
+	-- git configs for these do not override values from git attributes
+	-- or the branch
+	gitconfigdoesnotoverride =
+		[ "annex.numcopies"
+		, "annex.mincopies"
+		]
+
+	-- the git config for annex.numcopies is a special case; it's only
+	-- used if not configured anywhere else
+	checkgitconfigunderride
+		| name == "annex.numcopies" = gitconfigorigin'
+		| otherwise = return False
+
+	-- Display similar to git config --show-origin
+	showval loc v = liftIO $ do
+		putStrLn $ loc ++ "\t" ++ v
+		return True
+	
+	configbranch v
+		| needLocalUpdate ck = checkgitconfigunderride
+		| otherwise = showval "branch:git-annex" (decodeBS v)
+	
+	checkconfigbranch getter = getter >>= \case
+		Just (ConfigValue v) -> configbranch v
+		_ -> checkgitconfigunderride
+	
+	checkattrs cont
+		| decodeBS name `elem` annexAttrs =
+			case forfile of
+				Just file -> do
+					v <- checkAttr (decodeBS name) (toRawFilePath file)
+					if null v
+						then cont
+						else showval "gitattributes" v		
+				Nothing -> do
+					warnforfile
+					cont
+		| otherwise = cont
+	
+	warnforfile = warning $ UnquotedString $ configKeyMessage ck $ unwords
+		[ "may be configured in gitattributes."
+		, "Pass --for-file= with a filename to check"
+		]
+	
+	checknotconfigured a = do
+		ok <- a
+		unless ok $
+			warning $ UnquotedString $ configKeyMessage ck
+				"is not configured"
+		return ok
+
+type Setter = ConfigValue -> Annex ()
+type Unsetter = Annex ()
+type Getter = Annex (Maybe ConfigValue)
+
+checkIsGlobalConfig :: ConfigKey -> (Setter -> Unsetter -> Getter -> Annex a) -> Annex a
+checkIsGlobalConfig ck a = case checkIsGlobalConfig' ck of
+	Just (setter, unsetter, getter) -> a setter unsetter getter
+	Nothing -> giveup $ configKeyMessage ck "is not a configuration setting that can be stored in the git-annex branch"
+
+checkIsGlobalConfig' :: ConfigKey -> Maybe (Setter, Unsetter, Getter)
+checkIsGlobalConfig' ck
+	| elem ck globalConfigs = Just
+		( setGlobalConfig ck
+		, unsetGlobalConfig ck
+		, getGlobalConfig ck
+		)
+	-- These came before this command, but are also global configs,
+	-- so support them here as well.
+	| ck == ConfigKey "annex.numcopies" = Just
+		( mksetter (setGlobalNumCopies . configuredNumCopies)
+		, error "unsetting annex.numcopies is not supported"
+		, mkgetter fromNumCopies getGlobalNumCopies
+		)
+	| ck == ConfigKey "annex.mincopies" = Just
+		( mksetter (setGlobalMinCopies . configuredMinCopies)
+		, error "unsetting annex.mincopies is not supported"
+		, mkgetter fromMinCopies getGlobalMinCopies
+		)
+	| otherwise = Nothing
+  where
+	mksetter f = 
+		maybe (error ("invalid value for " ++ fromConfigKey ck)) f 
+			. readish . decodeBS . fromConfigValue
+	mkgetter f g = fmap (ConfigValue . encodeBS . show . f) <$> g
+
+configKeyMessage :: ConfigKey -> String -> String
+configKeyMessage (ConfigKey name) msg = decodeBS name ++ " " ++ msg
 
 needLocalUpdate :: ConfigKey -> Bool
 needLocalUpdate (ConfigKey "annex.securehashesonly") = True

doc/git-annex-config.mdwn

@@ -10,6 +10,8 @@ git annex config --get name
 
 git annex config --unset name
 
+git annex config --show-origin name
+
 # DESCRIPTION
 
 Set or get configuration settings stored in the git-annex branch.
@@ -29,6 +31,47 @@ looks for these.
 
 # SUPPORTED SETTINGS
 
+* `annex.numcopies`
+
+  Tells git-annex how many copies it should preserve of files, over all
+  repositories. The default is 1.
+
+  When git-annex is asked to drop a file, it first verifies that the
+  number of copies can be satisfied among all the other
+  repositories that have a copy of the file.
+  
+  In unusual situations, involving special remotes that do not support
+  locking, and concurrent drops of the same content from multiple
+  repositories, git-annex may violate the numcopies setting. It still
+  guarantees at least 1 copy is preserved. This can be configured by
+  setting annex.mincopies.
+
+  This is the same setting that the [[git-annex-numcopies]](1) command
+  configures. It can be overridden on a per-file basis
+  by the annex.numcopies setting in `.gitattributes` files.
+
+* `annex.mincopies`
+
+  Tells git-annex how many copies it is required to preserve of files, 
+  over all repositories. The default is 1.
+
+  This supplements the annex.numcopies setting. 
+  In unusual situations, involving special remotes that do not support
+  locking, and concurrent drops of the same content from multiple
+  repositories, git-annex may violate the numcopies setting.
+  In these unusual situations, git-annex ensures that the number of copies
+  never goes below mincopies.
+  
+  It is a good idea to not only rely on only setting mincopies. Set
+  numcopies as well, to a larger number, and keep mincopies at the
+  bare minimum you're comfortable with. Setting mincopies to a large
+  number, rather than setting numcopies will in some cases prevent
+  droping content in entirely safe situations.
+
+  This is the same setting that the [[git-annex-mincopies]](1) command
+  configures. It can be overridden on a per-file basis
+  by the annex.mincopies setting in `.gitattributes` files.
+
 * `annex.largefiles`
 
   Used to configure which files are large enough to be added to the annex.
@@ -143,6 +186,24 @@ looks for these.
 
   Unset a value.
 
+* `--show-origin name`
+
+  Explain where the value is configured, whether in the git-annex branch,
+  or in a `git config` file, or `.gitattributes` file. When a value is
+  configured in multiple places, displays the place and the value that
+  will be used.
+
+  Note that the parameter can be the name of one of the settings listed
+  above, but also any other configuration setting supported by git-annex.
+  For example, "annex.backend" cannot be set in the git-annex branch, but
+  it can be set in `.gitattributes` or `git config` and this option can
+  explain which setting will be used for it.
+
+* `--for-file file`
+
+  Can be used in combination with `--show-origin` to specify what
+  filename to check for in `.gitattributes`.
+
 * Also the [[git-annex-common-options]](1) can be used.
 
 # EXAMPLE

doc/git-annex-mincopies.mdwn

@@ -39,6 +39,7 @@ droping content in entirely safe situations.
 
 [[git-annex]](1)
 [[git-annex-numcopies]](1)
+[[git-annex-config]](1)
 
 # AUTHOR
 

doc/git-annex-numcopies.mdwn

@@ -36,6 +36,7 @@ using [[git-annex-mincopies]](1)
 
 [[git-annex]](1)
 [[git-annex-mincopies]](1)
+[[git-annex-config]](1)
 
 # AUTHOR
 

doc/todo/a_way_to_figure_out_the_origin_of_largefiles.mdwn

@@ -6,3 +6,5 @@ I guess the same functionality might be useful for some other options which coul
 
 [[!meta author=yoh]]
 [[!tag projects/dandi]]
+
+> [[done]] --[[Joey]]

doc/todo/a_way_to_figure_out_the_origin_of_largefiles/comment_2_6bfe8e1b6b2c3334803d8a6676862e8a._comment

@@ -0,0 +1,20 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 2"""
+ date="2023-06-12T20:08:40Z"
+ content="""
+I think it makes sense to have a command that displays the value of a
+config that will be used, and where that value is configured. I've implemented 
+`git-annex config --show-origin` to do that.
+
+For example:
+
+	% git-annex config --show-origin annex.largefiles
+	  annex.largefiles may be configured in gitattributes. Pass --for-file= with a filename to check
+	branch:git-annex	exclude=*.c
+	% git-annex config --show-origin annex.largefiles --for-file=foo
+	gitattributes	largerthan=100kb
+	% git config annex.largefiles oops
+	% git-annex config --show-origin annex.largefiles --for-file=foo
+	file:.git/config	oops
+"""]]