2015-05-06 17:44:53 +00:00
|
|
|
{- git-annex batch commands
|
|
|
|
-
|
|
|
|
- Copyright 2015 Joey Hess <id@joeyh.name>
|
|
|
|
-
|
2019-03-13 19:48:14 +00:00
|
|
|
- Licensed under the GNU AGPL version 3 or higher.
|
2015-05-06 17:44:53 +00:00
|
|
|
-}
|
|
|
|
|
|
|
|
module CmdLine.Batch where
|
|
|
|
|
2016-01-20 20:36:33 +00:00
|
|
|
import Annex.Common
|
|
|
|
import Types.Command
|
|
|
|
import CmdLine.Action
|
|
|
|
import CmdLine.GitAnnex.Options
|
|
|
|
import Options.Applicative
|
make --batch honor matching options
When --batch is used with matching options like --in, --metadata, etc, only
operate on the provided files when they match those options. Otherwise, a
blank line is output in the batch protocol.
Affected commands: find, add, whereis, drop, copy, move, get
In the case of find, the documentation for --batch already said it honored
the matching options. The docs for the rest didn't, but it makes sense to
have them honor them. While this is a behavior change, why specify the
matching options with --batch if you didn't want them to apply?
Note that the batch output for all of the affected commands could
already output a blank line in other cases, so batch users should
already be prepared to deal with it.
git-annex metadata didn't seem worth making support the matching options,
since all it does is output metadata or set metadata, the use cases for
using it in combination with the martching options seem small. Made it
refuse to run when they're combined, leaving open the possibility for later
support if a use case develops.
This commit was sponsored by Brett Eisenberg on Patreon.
2018-08-08 16:03:30 +00:00
|
|
|
import Limit
|
|
|
|
import Types.FileMatcher
|
2015-05-06 17:44:53 +00:00
|
|
|
|
added -z
Added -z option to git-annex commands that use --batch, useful for
supporting filenames containing newlines.
It only controls input to --batch, the output will still be line delimited
unless --json or etc is used to get some other output. While git often
makes -z affect both input and output, I don't like trying them together,
and making it affect output would have been a significant complication,
and also git-annex output is generally not intended to be machine parsed,
unless using --json or a format option.
Commands that take pairs like "file key" still separate them with a space
in --batch mode. All such commands take care to support filenames with
spaces when parsing that, so there was no need to change it, and it would
have needed significant changes to the batch machinery to separate tose
with a null.
To make fromkey and registerurl support -z, I had to give them a --batch
option. The implicit batch mode they enter when not provided with input
parameters does not support -z as that would have complicated option
parsing. Seemed better to move these toward using the same --batch as
everything else, though the implicit batch mode can still be used.
This commit was sponsored by Ole-Morten Duesund on Patreon.
2018-09-20 20:09:21 +00:00
|
|
|
data BatchMode = Batch BatchFormat | NoBatch
|
|
|
|
|
|
|
|
data BatchFormat = BatchLine | BatchNull
|
2015-07-12 00:43:45 +00:00
|
|
|
|
2015-12-21 16:57:13 +00:00
|
|
|
parseBatchOption :: Parser BatchMode
|
added -z
Added -z option to git-annex commands that use --batch, useful for
supporting filenames containing newlines.
It only controls input to --batch, the output will still be line delimited
unless --json or etc is used to get some other output. While git often
makes -z affect both input and output, I don't like trying them together,
and making it affect output would have been a significant complication,
and also git-annex output is generally not intended to be machine parsed,
unless using --json or a format option.
Commands that take pairs like "file key" still separate them with a space
in --batch mode. All such commands take care to support filenames with
spaces when parsing that, so there was no need to change it, and it would
have needed significant changes to the batch machinery to separate tose
with a null.
To make fromkey and registerurl support -z, I had to give them a --batch
option. The implicit batch mode they enter when not provided with input
parameters does not support -z as that would have complicated option
parsing. Seemed better to move these toward using the same --batch as
everything else, though the implicit batch mode can still be used.
This commit was sponsored by Ole-Morten Duesund on Patreon.
2018-09-20 20:09:21 +00:00
|
|
|
parseBatchOption = go
|
|
|
|
<$> switch
|
|
|
|
( long "batch"
|
|
|
|
<> help "enable batch mode"
|
|
|
|
)
|
|
|
|
<*> switch
|
|
|
|
( short 'z'
|
|
|
|
<> help "null delimited batch input"
|
|
|
|
)
|
|
|
|
where
|
|
|
|
go True False = Batch BatchLine
|
|
|
|
go True True = Batch BatchNull
|
|
|
|
go False _ = NoBatch
|
2015-07-12 00:43:45 +00:00
|
|
|
|
2015-12-21 16:57:13 +00:00
|
|
|
-- A batchable command can run in batch mode, or not.
|
2015-05-06 17:44:53 +00:00
|
|
|
-- In batch mode, one line at a time is read, parsed, and a reply output to
|
|
|
|
-- stdout. In non batch mode, the command's parameters are parsed and
|
|
|
|
-- a reply output for each.
|
2015-07-12 00:43:45 +00:00
|
|
|
batchable :: (opts -> String -> Annex Bool) -> Parser opts -> CmdParamsDesc -> CommandParser
|
|
|
|
batchable handler parser paramdesc = batchseeker <$> batchparser
|
2015-05-06 17:44:53 +00:00
|
|
|
where
|
2015-07-12 00:43:45 +00:00
|
|
|
batchparser = (,,)
|
|
|
|
<$> parser
|
2015-12-21 16:57:13 +00:00
|
|
|
<*> parseBatchOption
|
2015-07-12 00:43:45 +00:00
|
|
|
<*> cmdParams paramdesc
|
|
|
|
|
added -z
Added -z option to git-annex commands that use --batch, useful for
supporting filenames containing newlines.
It only controls input to --batch, the output will still be line delimited
unless --json or etc is used to get some other output. While git often
makes -z affect both input and output, I don't like trying them together,
and making it affect output would have been a significant complication,
and also git-annex output is generally not intended to be machine parsed,
unless using --json or a format option.
Commands that take pairs like "file key" still separate them with a space
in --batch mode. All such commands take care to support filenames with
spaces when parsing that, so there was no need to change it, and it would
have needed significant changes to the batch machinery to separate tose
with a null.
To make fromkey and registerurl support -z, I had to give them a --batch
option. The implicit batch mode they enter when not provided with input
parameters does not support -z as that would have complicated option
parsing. Seemed better to move these toward using the same --batch as
everything else, though the implicit batch mode can still be used.
This commit was sponsored by Ole-Morten Duesund on Patreon.
2018-09-20 20:09:21 +00:00
|
|
|
batchseeker (opts, NoBatch, params) =
|
|
|
|
mapM_ (go NoBatch opts) params
|
|
|
|
batchseeker (opts, batchmode@(Batch fmt), _) =
|
|
|
|
batchInput fmt Right (go batchmode opts)
|
2015-07-12 00:43:45 +00:00
|
|
|
|
|
|
|
go batchmode opts p =
|
|
|
|
unlessM (handler opts p) $
|
|
|
|
batchBadInput batchmode
|
2015-05-06 17:44:53 +00:00
|
|
|
|
|
|
|
-- bad input is indicated by an empty line in batch mode. In non batch
|
|
|
|
-- mode, exit on bad input.
|
|
|
|
batchBadInput :: BatchMode -> Annex ()
|
|
|
|
batchBadInput NoBatch = liftIO exitFailure
|
added -z
Added -z option to git-annex commands that use --batch, useful for
supporting filenames containing newlines.
It only controls input to --batch, the output will still be line delimited
unless --json or etc is used to get some other output. While git often
makes -z affect both input and output, I don't like trying them together,
and making it affect output would have been a significant complication,
and also git-annex output is generally not intended to be machine parsed,
unless using --json or a format option.
Commands that take pairs like "file key" still separate them with a space
in --batch mode. All such commands take care to support filenames with
spaces when parsing that, so there was no need to change it, and it would
have needed significant changes to the batch machinery to separate tose
with a null.
To make fromkey and registerurl support -z, I had to give them a --batch
option. The implicit batch mode they enter when not provided with input
parameters does not support -z as that would have complicated option
parsing. Seemed better to move these toward using the same --batch as
everything else, though the implicit batch mode can still be used.
This commit was sponsored by Ole-Morten Duesund on Patreon.
2018-09-20 20:09:21 +00:00
|
|
|
batchBadInput (Batch _) = liftIO $ putStrLn ""
|
2015-12-21 16:57:13 +00:00
|
|
|
|
|
|
|
-- Reads lines of batch mode input and passes to the action to handle.
|
added -z
Added -z option to git-annex commands that use --batch, useful for
supporting filenames containing newlines.
It only controls input to --batch, the output will still be line delimited
unless --json or etc is used to get some other output. While git often
makes -z affect both input and output, I don't like trying them together,
and making it affect output would have been a significant complication,
and also git-annex output is generally not intended to be machine parsed,
unless using --json or a format option.
Commands that take pairs like "file key" still separate them with a space
in --batch mode. All such commands take care to support filenames with
spaces when parsing that, so there was no need to change it, and it would
have needed significant changes to the batch machinery to separate tose
with a null.
To make fromkey and registerurl support -z, I had to give them a --batch
option. The implicit batch mode they enter when not provided with input
parameters does not support -z as that would have complicated option
parsing. Seemed better to move these toward using the same --batch as
everything else, though the implicit batch mode can still be used.
This commit was sponsored by Ole-Morten Duesund on Patreon.
2018-09-20 20:09:21 +00:00
|
|
|
batchInput :: BatchFormat -> (String -> Either String a) -> (a -> Annex ()) -> Annex ()
|
|
|
|
batchInput fmt parser a = go =<< batchLines fmt
|
2015-12-22 16:20:39 +00:00
|
|
|
where
|
2016-12-13 19:35:04 +00:00
|
|
|
go [] = return ()
|
|
|
|
go (l:rest) = do
|
|
|
|
either parseerr a (parser l)
|
|
|
|
go rest
|
2016-11-16 01:29:54 +00:00
|
|
|
parseerr s = giveup $ "Batch input parse failure: " ++ s
|
2016-01-19 21:46:46 +00:00
|
|
|
|
added -z
Added -z option to git-annex commands that use --batch, useful for
supporting filenames containing newlines.
It only controls input to --batch, the output will still be line delimited
unless --json or etc is used to get some other output. While git often
makes -z affect both input and output, I don't like trying them together,
and making it affect output would have been a significant complication,
and also git-annex output is generally not intended to be machine parsed,
unless using --json or a format option.
Commands that take pairs like "file key" still separate them with a space
in --batch mode. All such commands take care to support filenames with
spaces when parsing that, so there was no need to change it, and it would
have needed significant changes to the batch machinery to separate tose
with a null.
To make fromkey and registerurl support -z, I had to give them a --batch
option. The implicit batch mode they enter when not provided with input
parameters does not support -z as that would have complicated option
parsing. Seemed better to move these toward using the same --batch as
everything else, though the implicit batch mode can still be used.
This commit was sponsored by Ole-Morten Duesund on Patreon.
2018-09-20 20:09:21 +00:00
|
|
|
batchLines :: BatchFormat -> Annex [String]
|
|
|
|
batchLines fmt = liftIO $ splitter <$> getContents
|
|
|
|
where
|
|
|
|
splitter = case fmt of
|
|
|
|
BatchLine -> lines
|
|
|
|
BatchNull -> splitc '\0'
|
2016-12-13 19:35:04 +00:00
|
|
|
|
2016-01-19 21:46:46 +00:00
|
|
|
-- Runs a CommandStart in batch mode.
|
|
|
|
--
|
|
|
|
-- The batch mode user expects to read a line of output, and it's up to the
|
|
|
|
-- CommandStart to generate that output as it succeeds or fails to do its
|
|
|
|
-- job. However, if it stops without doing anything, it won't generate
|
|
|
|
-- any output, so in that case, batchBadInput is used to provide the caller
|
|
|
|
-- with an empty line.
|
|
|
|
batchCommandAction :: CommandStart -> Annex ()
|
added -z
Added -z option to git-annex commands that use --batch, useful for
supporting filenames containing newlines.
It only controls input to --batch, the output will still be line delimited
unless --json or etc is used to get some other output. While git often
makes -z affect both input and output, I don't like trying them together,
and making it affect output would have been a significant complication,
and also git-annex output is generally not intended to be machine parsed,
unless using --json or a format option.
Commands that take pairs like "file key" still separate them with a space
in --batch mode. All such commands take care to support filenames with
spaces when parsing that, so there was no need to change it, and it would
have needed significant changes to the batch machinery to separate tose
with a null.
To make fromkey and registerurl support -z, I had to give them a --batch
option. The implicit batch mode they enter when not provided with input
parameters does not support -z as that would have complicated option
parsing. Seemed better to move these toward using the same --batch as
everything else, though the implicit batch mode can still be used.
This commit was sponsored by Ole-Morten Duesund on Patreon.
2018-09-20 20:09:21 +00:00
|
|
|
batchCommandAction a = maybe (batchBadInput (Batch BatchLine)) (const noop)
|
2016-01-19 21:46:46 +00:00
|
|
|
=<< callCommandAction' a
|
2016-01-20 16:46:00 +00:00
|
|
|
|
|
|
|
-- Reads lines of batch input and passes the filepaths to a CommandStart
|
|
|
|
-- to handle them.
|
make --batch honor matching options
When --batch is used with matching options like --in, --metadata, etc, only
operate on the provided files when they match those options. Otherwise, a
blank line is output in the batch protocol.
Affected commands: find, add, whereis, drop, copy, move, get
In the case of find, the documentation for --batch already said it honored
the matching options. The docs for the rest didn't, but it makes sense to
have them honor them. While this is a behavior change, why specify the
matching options with --batch if you didn't want them to apply?
Note that the batch output for all of the affected commands could
already output a blank line in other cases, so batch users should
already be prepared to deal with it.
git-annex metadata didn't seem worth making support the matching options,
since all it does is output metadata or set metadata, the use cases for
using it in combination with the martching options seem small. Made it
refuse to run when they're combined, leaving open the possibility for later
support if a use case develops.
This commit was sponsored by Brett Eisenberg on Patreon.
2018-08-08 16:03:30 +00:00
|
|
|
--
|
|
|
|
-- File matching options are not checked.
|
2019-02-05 18:03:29 +00:00
|
|
|
batchStart :: BatchFormat -> (String -> CommandStart) -> Annex ()
|
|
|
|
batchStart fmt a = batchInput fmt Right $ batchCommandAction . a
|
make --batch honor matching options
When --batch is used with matching options like --in, --metadata, etc, only
operate on the provided files when they match those options. Otherwise, a
blank line is output in the batch protocol.
Affected commands: find, add, whereis, drop, copy, move, get
In the case of find, the documentation for --batch already said it honored
the matching options. The docs for the rest didn't, but it makes sense to
have them honor them. While this is a behavior change, why specify the
matching options with --batch if you didn't want them to apply?
Note that the batch output for all of the affected commands could
already output a blank line in other cases, so batch users should
already be prepared to deal with it.
git-annex metadata didn't seem worth making support the matching options,
since all it does is output metadata or set metadata, the use cases for
using it in combination with the martching options seem small. Made it
refuse to run when they're combined, leaving open the possibility for later
support if a use case develops.
This commit was sponsored by Brett Eisenberg on Patreon.
2018-08-08 16:03:30 +00:00
|
|
|
|
2019-02-05 18:03:29 +00:00
|
|
|
-- Like batchStart, but checks the file matching options
|
make --batch honor matching options
When --batch is used with matching options like --in, --metadata, etc, only
operate on the provided files when they match those options. Otherwise, a
blank line is output in the batch protocol.
Affected commands: find, add, whereis, drop, copy, move, get
In the case of find, the documentation for --batch already said it honored
the matching options. The docs for the rest didn't, but it makes sense to
have them honor them. While this is a behavior change, why specify the
matching options with --batch if you didn't want them to apply?
Note that the batch output for all of the affected commands could
already output a blank line in other cases, so batch users should
already be prepared to deal with it.
git-annex metadata didn't seem worth making support the matching options,
since all it does is output metadata or set metadata, the use cases for
using it in combination with the martching options seem small. Made it
refuse to run when they're combined, leaving open the possibility for later
support if a use case develops.
This commit was sponsored by Brett Eisenberg on Patreon.
2018-08-08 16:03:30 +00:00
|
|
|
-- and skips non-matching files.
|
added -z
Added -z option to git-annex commands that use --batch, useful for
supporting filenames containing newlines.
It only controls input to --batch, the output will still be line delimited
unless --json or etc is used to get some other output. While git often
makes -z affect both input and output, I don't like trying them together,
and making it affect output would have been a significant complication,
and also git-annex output is generally not intended to be machine parsed,
unless using --json or a format option.
Commands that take pairs like "file key" still separate them with a space
in --batch mode. All such commands take care to support filenames with
spaces when parsing that, so there was no need to change it, and it would
have needed significant changes to the batch machinery to separate tose
with a null.
To make fromkey and registerurl support -z, I had to give them a --batch
option. The implicit batch mode they enter when not provided with input
parameters does not support -z as that would have complicated option
parsing. Seemed better to move these toward using the same --batch as
everything else, though the implicit batch mode can still be used.
This commit was sponsored by Ole-Morten Duesund on Patreon.
2018-09-20 20:09:21 +00:00
|
|
|
batchFilesMatching :: BatchFormat -> (FilePath -> CommandStart) -> Annex ()
|
|
|
|
batchFilesMatching fmt a = do
|
make --batch honor matching options
When --batch is used with matching options like --in, --metadata, etc, only
operate on the provided files when they match those options. Otherwise, a
blank line is output in the batch protocol.
Affected commands: find, add, whereis, drop, copy, move, get
In the case of find, the documentation for --batch already said it honored
the matching options. The docs for the rest didn't, but it makes sense to
have them honor them. While this is a behavior change, why specify the
matching options with --batch if you didn't want them to apply?
Note that the batch output for all of the affected commands could
already output a blank line in other cases, so batch users should
already be prepared to deal with it.
git-annex metadata didn't seem worth making support the matching options,
since all it does is output metadata or set metadata, the use cases for
using it in combination with the martching options seem small. Made it
refuse to run when they're combined, leaving open the possibility for later
support if a use case develops.
This commit was sponsored by Brett Eisenberg on Patreon.
2018-08-08 16:03:30 +00:00
|
|
|
matcher <- getMatcher
|
2019-02-05 18:03:29 +00:00
|
|
|
batchStart fmt $ \f ->
|
make --batch honor matching options
When --batch is used with matching options like --in, --metadata, etc, only
operate on the provided files when they match those options. Otherwise, a
blank line is output in the batch protocol.
Affected commands: find, add, whereis, drop, copy, move, get
In the case of find, the documentation for --batch already said it honored
the matching options. The docs for the rest didn't, but it makes sense to
have them honor them. While this is a behavior change, why specify the
matching options with --batch if you didn't want them to apply?
Note that the batch output for all of the affected commands could
already output a blank line in other cases, so batch users should
already be prepared to deal with it.
git-annex metadata didn't seem worth making support the matching options,
since all it does is output metadata or set metadata, the use cases for
using it in combination with the martching options seem small. Made it
refuse to run when they're combined, leaving open the possibility for later
support if a use case develops.
This commit was sponsored by Brett Eisenberg on Patreon.
2018-08-08 16:03:30 +00:00
|
|
|
ifM (matcher $ MatchingFile $ FileInfo f f)
|
|
|
|
( a f
|
|
|
|
, return Nothing
|
|
|
|
)
|