Commit graph

41421 commits

Author SHA1 Message Date
Joey Hess
d3b7c6705c
clean up concurrent output of tests
Using concurrent-output this is easy. Just have to check if tasty has
color enabled, and propagate it into the worker processes, some of which
will be run without a controlling console.

Also added a call to installSignalHandlers; I noticed that interrupting
the test suite could leave the console in a bad state and this fixes
that.

The ansi-terminal dependency is free, since tasty also depends on it.

Sponsored-by: Dartmouth College's Datalad project
2022-03-16 12:41:28 -04:00
Joey Hess
ca711f1741
link to issue for tasty segfault 2022-03-15 19:30:57 -04:00
Joey Hess
57c01b2a63
allow tests to be split for more granularity
Unit tests are the main bulk of runtime, so splitting them into 2 or 3
parts should help.

For now, the number of parts is still 1, because on my 4 core laptop,
2 was a little bit slower, and 3 slower yet. However, this probably does
vary based on the number of cores, so needs to be revisited, and perhaps
made dynamic.

Since each test mode gets split into the specified number of parts,
plus property and remote tests, 2 gives 8 parts, and 3 gives 11 parts.
Load went to maybe 18, so there was probably contention slowing things
down.

So probably it needs to start N workers with some parts, and when a
worker finishes, run it with the next part, until all parts are
processed.

Sponsored-by: Dartmouth College's Datalad project
2022-03-14 17:23:52 -04:00
Joey Hess
8d14ce8f38
parallelize git-annex test for 25% speedup
Note the very weird workaround for what appears to be some kind of tasty
bug, which causes a segfault. This is not new to this modification,
I was seeing a segfault before at least intermittently when limiting
git-annex test -p to only run a single test group.

Also, the path from one test repo to a remote test repo used to be
"../../foo", which somehow broke when moving the test repos from .t to
.t/N. I don't actually quite understand how it used to work, but
"../foo" seems correct and works in the new situation.

Test output from the concurrent processes is not yet serialized.
Should be easy to do using concurrent-output.

More test groups will probably make the speedup larger. It would
probably be best to have a larger number of test groups and divvy them
amoung subprocesses numbered based on the number of CPU cores, perhaps
times 2 or 3.

Sponsored-by: Dartmouth College's Datalad project
2022-03-14 15:24:37 -04:00
Joey Hess
62e6c6afb9
fix accidental change to Test.hs while making comment 2022-03-14 13:10:37 -04:00
Joey Hess
8f05bcc98f
indentation 2022-03-14 12:30:27 -04:00
git-annex.visiteur@e9d364191d2ffc1b163c8d9e4c57dbadf58aad8e
35e91ecb55 Added a comment 2022-03-11 18:46:40 +00:00
ErrGe
7580d787cc Added a comment 2022-03-11 02:26:56 +00:00
ErrGe
4d593d4461 Added a comment 2022-03-11 02:23:20 +00:00
Joey Hess
afeb9b728e
comment 2022-03-10 16:10:46 -04:00
Joey Hess
feaf16141e
comment 2022-03-10 13:22:32 -04:00
Joey Hess
6f468a8bfc
comment 2022-03-10 13:10:39 -04:00
Atemu
0e39304905 Added a comment 2022-03-09 12:07:45 +00:00
Atemu
d476fc5a75 Added a comment 2022-03-09 11:40:28 +00:00
git-annex.visiteur@e9d364191d2ffc1b163c8d9e4c57dbadf58aad8e
7365aed80a Added a comment 2022-03-09 06:33:52 +00:00
ErrGe
f706a68c43 2022-03-09 01:08:23 +00:00
Joey Hess
c7f7be0236
comment 2022-03-08 15:50:31 -04:00
Joey Hess
82f1d82286
comment 2022-03-08 15:46:29 -04:00
Joey Hess
4cab0c1b05
comment 2022-03-08 14:51:01 -04:00
Joey Hess
14add55c2b
reopen 2022-03-08 14:08:27 -04:00
Joey Hess
64e9fa3267
comment 2022-03-08 14:07:24 -04:00
Joey Hess
7424ce54f4
comment 2022-03-08 13:54:07 -04:00
Joey Hess
2c5bf952cf
comment 2022-03-08 13:47:06 -04:00
Joey Hess
88d5a61fb7
comment 2022-03-08 13:08:16 -04:00
Atemu
6253bd391c Added a comment 2022-03-08 13:24:21 +00:00
Atemu
c7e3414d8d Added a comment 2022-03-08 13:21:03 +00:00
Atemu
327d06eb9d Added a comment 2022-03-08 11:56:42 +00:00
anamyk
a5086345bc Added a comment 2022-03-08 08:11:17 +00:00
anamyk
d4aa2dfb94 Added a comment 2022-03-08 07:49:38 +00:00
yarikoptic
875a04e1e2 Added a comment: still slow 2022-03-08 00:28:00 +00:00
git-annex.visiteur@e9d364191d2ffc1b163c8d9e4c57dbadf58aad8e
041cea21ba Added a comment 2022-03-07 19:59:18 +00:00
Joey Hess
cd473f071c
comment 2022-03-07 15:30:16 -04:00
Joey Hess
434812bcfb
Merge branch 'master' of ssh://git-annex.branchable.com 2022-03-07 15:25:57 -04:00
Joey Hess
1cbbd23109
comment 2022-03-07 15:25:32 -04:00
git-annex.visiteur@e9d364191d2ffc1b163c8d9e4c57dbadf58aad8e
85eb26e746 Added a comment 2022-03-07 19:22:54 +00:00
Joey Hess
5fb98d5f29
Merge branch 'master' of ssh://git-annex.branchable.com 2022-03-07 15:20:00 -04:00
Joey Hess
06c2f625a9
comment 2022-03-07 15:18:30 -04:00
tomdhunt
0127e5f4e4 Added a comment 2022-03-07 18:27:38 +00:00
Joey Hess
da698437b6
close 2022-03-07 14:12:11 -04:00
Joey Hess
9770fde5f4
hard link git-annex to fix ./git-annex test
2fc46e1871 broke running ./git-annex test
when git-annex-shell is not in PATH from another install, and also
caused the wrong git-annex-shell to be tested.

Using a symlink made programPath return the .cabal/ directory where the
git-annex binary is, but git-annex-shell is not in that directory.
Changing to a hard link avoids that problem.

Sponsored-by: Dartmouth College's Datalad project
2022-03-07 14:01:45 -04:00
Joey Hess
5b6518d4a3
fix build warning with old aeson 2022-03-07 13:29:24 -04:00
Joey Hess
dab9078ab7
close 2022-03-07 13:23:19 -04:00
Joey Hess
675a491632
document nix-on-droid as an alternative to termux
Sponsored-by: Graham Spencer on Patreon
2022-03-07 12:57:16 -04:00
Joey Hess
5154d15183
comment 2022-03-07 12:11:18 -04:00
Joey Hess
5804ea67a6
comment; close 2022-03-07 12:03:11 -04:00
Joey Hess
568c469e7f
Merge branch 'master' of ssh://git-annex.branchable.com 2022-03-07 12:00:14 -04:00
nowkixide
6a45d7e5d0 Added a comment: Works fine 2022-03-07 12:07:54 +00:00
git-annex.visiteur@e9d364191d2ffc1b163c8d9e4c57dbadf58aad8e
ce523f7561 Added a comment 2022-03-06 14:37:42 +00:00
git-annex.visiteur@e9d364191d2ffc1b163c8d9e4c57dbadf58aad8e
6050b839f4 2022-03-06 06:46:01 +00:00
Atemu
54a43f68ed Added a comment 2022-03-05 21:46:23 +00:00