add --size-limit option

When this option is not used, there should be effectively no added
overhead, thanks to the optimisation in
b3cd0cc6ba.

When an action fails on a file, the size of the file still counts toward
the size limit. This was necessary to support concurrency, but also
generally seems like the right choice.

Most commands that operate on annexed files support the option.
export and import do not, and I don't know if it would make sense for
export to.. Why would you want an incomplete export? sync doesn't, and
while it would be easy to make it support it for transferring files,
it's not clear if dropping files should also take the size limit into
account. Commands like add that don't operate on annexed files don't
support the option either.

Exiting 101 not yet implemented.

Sponsored-by: Denis Dzyubenko on Patreon

CmdLine/GitAnnex/Options.hs

@@ -12,6 +12,7 @@ module CmdLine.GitAnnex.Options where
 import Control.Monad.Fail as Fail (MonadFail(..))
 import Options.Applicative
 import Data.Time.Clock.POSIX
+import Control.Concurrent.STM
 import qualified Data.Map as M
 
 import Annex.Common
@@ -37,6 +38,7 @@ import CmdLine.GlobalSetter
 import qualified Backend
 import qualified Types.Backend as Backend
 import Utility.HumanTime
+import Utility.DataUnits
 import Annex.Concurrent
 
 -- Global options that are accepted by all git-annex sub-commands,
@@ -233,11 +235,12 @@ annexedMatchingOptions = concat
 	, fileMatchingOptions' Limit.LimitAnnexFiles
 	, combiningOptions
 	, timeLimitOption
+	, sizeLimitOption
 	]
 
 -- Matching options that can operate on keys as well as files.
 keyMatchingOptions :: [GlobalOption]
-keyMatchingOptions = keyMatchingOptions' ++ combiningOptions ++ timeLimitOption
+keyMatchingOptions = keyMatchingOptions' ++ combiningOptions ++ timeLimitOption ++ sizeLimitOption
 
 keyMatchingOptions' :: [GlobalOption]
 keyMatchingOptions' = 
@@ -435,6 +438,19 @@ timeLimitOption =
 		let cutoff = start + durationToPOSIXTime duration
 		Annex.changeState $ \s -> s { Annex.timelimit = Just (duration, cutoff) }
 
+sizeLimitOption :: [GlobalOption]
+sizeLimitOption =
+	[ globalOption setsizelimit $ option (maybeReader (readSize dataUnits))
+		( long "size-limit" <> metavar paramSize
+		<> help "total size of annexed files to process"
+		<> hidden
+		)
+	]
+  where
+	setsizelimit n = setAnnexState $ do
+		v <- liftIO $ newTVarIO n
+		Annex.changeState $ \s -> s { Annex.sizelimit = Just v }
+
 data DaemonOptions = DaemonOptions
 	{ foregroundDaemonOption :: Bool
 	, stopDaemonOption :: Bool