mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

test: Added --jobs option

Browse source 
Default to the number of CPU cores, which seems about optimal
on my laptop. Using one more saves me 2 seconds actually.

Better packing of workers improves speed significantly.

In 2 tests runs, I saw segfaulting workers despite my attempt
to work around that issue. So detect when a worker does, and re-run it.

Removed installSignalHandlers again, because I was seeing an
error "lost signal due to full pipe", which I guess was somehow caused
by using it.

Sponsored-by: Dartmouth College's Datalad project
This commit is contained in:
Joey Hess 2022-03-16 14:42:07 -04:00
parent be31a8a3d2
commit 025c18128b
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
6 changed files with 77 additions and 24 deletions
Download patch file Download diff file Expand all files Collapse all files

5
doc/git-annex-test.mdwn
View file

@ -20,6 +20,11 @@ or to verify your local installation of git-annex.
There are several options, provided by Haskell's tasty test
framework. Pass --help for details about those.
* `--jobs=N` `-JN`
How many tests to run in parallel. The default is "cpus", which will
runs one job per CPU core.
* `--keep-failures`
When there are test failures, leave the `.t` directory populated with

17
doc/todo/speed_up___34__standalone_build__34___and__47__or_tests/comment_9_2253c6916a1c6e4f1393b8656617aa10._comment Normal file
View file

@ -0,0 +1,17 @@
[[!comment format=mdwn
username="joey"
subject="""comment 9"""
date="2022-03-16T17:55:52Z"
content="""
I've finished up parallelizing git-annex test.
Splitting up the test groups further and improved scheduling sped it up more.
On my laptop, it's dropped from 444 to 334 to now 289 seconds.
Also, the `-J` option is now supported by git-annex test, so you can experiment
to find the number of jobs where it runs fastest in your particular situation.
The default is one job per CPU core.
My guess is that on NFS, it's not CPU bound but is network latency bound,
and so a rather high -J value like -J10 may behave better.
"""]]