export: Let concurrent transfers be done with -J or annex.jobs
Tested working, although I did find this bug in my testing, which also afflicts sync -J to an export remote.
This commit is contained in:
parent
f7fe71602c
commit
0bcecb67f5
4 changed files with 23 additions and 2 deletions
|
@ -4,6 +4,7 @@ git-annex (8.20200523) UNRELEASED; urgency=medium
|
|||
* import: Added --json-progress.
|
||||
* addurl: Make --preserve-filename also apply when eg a torrent contains
|
||||
multiple files.
|
||||
* export: Let concurrent transfers be done with -J or annex.jobs.
|
||||
|
||||
-- Joey Hess <id@joeyh.name> Tue, 26 May 2020 10:20:52 -0400
|
||||
|
||||
|
|
|
@ -44,7 +44,7 @@ import qualified Data.Map as M
|
|||
import Control.Concurrent
|
||||
|
||||
cmd :: Command
|
||||
cmd = withGlobalOptions [jsonOptions, jsonProgressOption] $
|
||||
cmd = withGlobalOptions [jobsOption, jsonOptions, jsonProgressOption] $
|
||||
command "export" SectionCommon
|
||||
"export content to a remote"
|
||||
paramTreeish (seek <$$> optParser)
|
||||
|
@ -77,7 +77,7 @@ exportTempName ek = mkExportLocation $ toRawFilePath $
|
|||
".git-annex-tmp-content-" ++ serializeKey (asKey (ek))
|
||||
|
||||
seek :: ExportOptions -> CommandSeek
|
||||
seek o = do
|
||||
seek o = startConcurrency commandStages $ do
|
||||
r <- getParsed (exportRemote o)
|
||||
unlessM (isExportSupported r) $
|
||||
giveup "That remote does not support exports."
|
||||
|
|
13
doc/bugs/sync_to_export_with-J_STM_error.mdwn
Normal file
13
doc/bugs/sync_to_export_with-J_STM_error.mdwn
Normal file
|
@ -0,0 +1,13 @@
|
|||
git annex sync exportremote -J2 --content
|
||||
|
||||
git-annex: thread blocked indefinitely in an MVar operation
|
||||
failed
|
||||
git-annex: thread blocked indefinitely in an STM transaction
|
||||
|
||||
Also, git-annex export -J2 crashes the same way. I discovered this bug
|
||||
when adding -J to export, but then found sync had the same bug.
|
||||
|
||||
To reproduce this, there may need there to be a tree of several annexed
|
||||
files whose content is not locally available. In my case,
|
||||
there were 338 of them. It seems to export all, or all but 1
|
||||
before crashing. --[[Joey]]
|
|
@ -86,6 +86,13 @@ so the overwritten modification is not lost.)
|
|||
the remote. You can later run `git annex sync --content` to upload
|
||||
the files to the export.
|
||||
|
||||
* `--jobs=N` `-JN`
|
||||
|
||||
Exports multiple files in parallel. This may be faster.
|
||||
For example: `-J4`
|
||||
|
||||
Setting this to "cpus" will run one job per CPU core.
|
||||
|
||||
* `--json`
|
||||
|
||||
Enable JSON output. This is intended to be parsed by programs that use
|
||||
|
|
Loading…
Reference in a new issue