mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

allow tests to be split for more granularity

Browse source 
Unit tests are the main bulk of runtime, so splitting them into 2 or 3
parts should help.

For now, the number of parts is still 1, because on my 4 core laptop,
2 was a little bit slower, and 3 slower yet. However, this probably does
vary based on the number of cores, so needs to be revisited, and perhaps
made dynamic.

Since each test mode gets split into the specified number of parts,
plus property and remote tests, 2 gives 8 parts, and 3 gives 11 parts.
Load went to maybe 18, so there was probably contention slowing things
down.

So probably it needs to start N workers with some parts, and when a
worker finishes, run it with the next part, until all parts are
processed.

Sponsored-by: Dartmouth College's Datalad project
This commit is contained in:
Joey Hess 2022-03-14 17:23:52 -04:00
parent 8d14ce8f38
commit 57c01b2a63
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
2 changed files with 18 additions and 9 deletions
Download patch file Download diff file Expand all files Collapse all files

20
Test.hs
View file

@ -90,7 +90,7 @@ import qualified Utility.Gpg
optParser :: Parser TestOptions
optParser = TestOptions
<$> snd (tastyParser (tests False True mempty))
<$> snd (tastyParser (tests 1 False True mempty))
<*> switch
( long "keep-failures"
<> help "preserve repositories on test failure"
@ -104,11 +104,11 @@ optParser = TestOptions
runner :: TestOptions -> IO ()
runner opts = parallelTestRunner opts tests
tests :: Bool -> Bool -> TestOptions -> [TestTree]
tests crippledfilesystem adjustedbranchok opts =
tests :: Int -> Bool -> Bool -> TestOptions -> [TestTree]
tests n crippledfilesystem adjustedbranchok opts =
properties
: withTestMode remotetestmode Nothing testRemotes
: map (\(d, te) -> withTestMode te (Just initTests) (unitTests d)) testmodes
: concatMap mkunittests testmodes
where
testmodes = catMaybes
[ canadjust ("v8 adjusted unlocked branch", (testMode opts (RepoVersion 8)) { adjustedUnlockedBranch = True })
@ -122,6 +122,9 @@ tests crippledfilesystem adjustedbranchok opts =
canadjust v
| adjustedbranchok = Just v
| otherwise = Nothing
mkunittests (d, te) = map
(\uts -> withTestMode te (Just initTests) uts)
(unitTests d n)
properties :: TestTree
properties = localOption (QuickCheckTests 1000) $ testGroup "QuickCheck" $
@ -248,8 +251,8 @@ initTests = testGroup "Init Tests"
, testCase "add" test_add
]
unitTests :: String -> TestTree
unitTests note = testGroup ("Unit Tests " ++ note)
unitTests :: String -> Int -> [TestTree]
unitTests note numparts = map (testGroup ("Unit Tests " ++ note)) $ sep
[ testCase "add dup" test_add_dup
, testCase "add extras" test_add_extras
, testCase "readonly remote" test_readonly_remote
@ -328,6 +331,11 @@ unitTests note = testGroup ("Unit Tests " ++ note)
, testCase "add subdirs" test_add_subdirs
, testCase "addurl" test_addurl
]
where
sep = sep' (replicate numparts [])
sep' (p:ps) (l:ls) = sep' (ps++[l:p]) ls
sep' ps [] = ps
sep' [] _ = []
-- this test case creates the main repo
test_init :: Assertion