mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge branch 'master' of ssh://git-annex.branchable.com

Browse source
This commit is contained in:
Joey Hess 2021-06-09 15:38:58 -04:00
commit 4b09b93a18
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
3 changed files with 28 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

12
doc/bugs/significant_performance_regression_impacting_datal/comment_17_8df0e346ed85e52ec94763d11c174ab3._comment Normal file
View file

@ -0,0 +1,12 @@
[[!comment format=mdwn
username="yarikoptic"
avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
subject="OSX mystery resolved. add --batch is effective mitigation"
date="2021-06-08T21:56:53Z"
content="""
> Perhaps on OSX something is making the write-tree significantly slower. Or something is making it run the command more with fewer files per run.
The latter would be my guess... We seems to get 2 vs 5 splits on Linux vs OSX... our `datalad.utils.CMD_MAX_ARG` (logic is [here](https://github.com/datalad/datalad/blob/HEAD/datalad/utils.py#L103)) gets set to 1048576 on linux and 524288 on OSX. So \"matches\" and \"OSX mystery resolved\"! ;)
Meanwhile confirming that using `add --batch` mitigates it. With [ad-hoc patch to add add --batched to datalad](https://github.com/datalad/datalad/pull/5722) I get 187.8053s run for our test using `annex add --batch` and bleeding edge annex 57b567ac8 and 183.1654s (so about the same) using annex 9a5981a15 from 20210525 (which takes over hour with splitting); and then with our old splitting and that old git-annex 9a5981a15 I get 188.3987s run.
"""]]

8
doc/bugs/significant_performance_regression_impacting_datal/comment_18_3e57d7d4640de95a059b2ab3d1e9d54e._comment Normal file
View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="yarikoptic"
avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
subject="comment 18"
date="2021-06-08T21:58:20Z"
content="""
clarification: the \"(which takes over hour with splitting)\" was intended for the \"bleeding edge\" one, not the older one. That one is ok even with splitting.
"""]]

8
doc/bugs/significant_performance_regression_impacting_datal/comment_19_80372cd15a6a8812072a446c41fed4e7._comment Normal file
View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="yarikoptic"
avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
subject="comment 19"
date="2021-06-08T22:02:33Z"
content="""
and we seems to not run that test on windows (marked as `@known_failure_windows` ;)) so we do not see that \"regression\" on windows runs (was yet another part of mystery to me why OSX only ;))
"""]]