mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions
git-annex/doc/bugs/significant_performance_regression_impacting_datal
History
Joey Hess 3af4c9a29a
fix exponential blowup when adding lots of identical files 
This was an old problem when the files were being added unlocked,
so the changelog mentions that being fixed. However, recently it's also
affected locked files.

The fix for locked files is kind of stupidly simple. moveAnnex already
handles populating unlocked files, and only does it when the object file
was not already present. So remove the redundant populateUnlockedFiles
call. (That call was added all the way back in
cfaac52b88, and has always been
unncessary.)

Sponsored-by: Dartmouth College's Datalad project
2021-06-15 09:45:55 -04:00
..
comment_1_3eb1b092f41dc5c04d74f7a03249aa0f._comment
comment_2_521f784d686fcaeb5e081599fbdf9903._comment
comment_3_d30a515103d04fd966aadee7f141aeee._comment comment 2021-06-07 11:53:25 -04:00
comment_4_85d1031d2b51c0fc1271c283d8ee7888._comment avoid double work in git-annex init 2021-06-07 16:50:14 -04:00
comment_4_4011786784a140442dd7ecd1cefe559b._comment Added a comment: deferring the scan 2021-06-07 17:41:45 +00:00
comment_5_b0f1e94e96cf8fe992425cf81f8d1105._comment avoid double work in git-annex init 2021-06-07 16:50:14 -04:00
comment_7_35217e61ccf6936e105df8de7e1775d0._comment Added a comment: clarification 2021-06-07 21:20:35 +00:00
comment_8_29bde900ef625702b49cf020118f940f._comment Added a comment 2021-06-07 21:39:05 +00:00
comment_9_431582f1c218f27e50849df1444dc845._comment comment 2021-06-08 10:19:04 -04:00
comment_10_37d5186dfa2da31526c4eafbbbfdbc33._comment Added a comment: slow down is OSX specific 2021-06-08 14:28:18 +00:00
comment_11_51a3a86a290b0e3994507f3f64e7c72a._comment clarification 2021-06-08 12:00:01 -04:00
comment_12_c841c9db2000c65c56ce8e26e58a3f62._comment Added a comment: all recent builds/logs are fetched to smaug 2021-06-08 16:50:12 +00:00
comment_13_f6bec279f43603719694e44d99309fc7._comment Added a comment 2021-06-08 19:08:01 +00:00
comment_14_4d2998ea843dd8adee8b7b066d97d942._comment Added a comment: getting closer... 2021-06-08 19:21:59 +00:00
comment_15_c9488d6180e741dfec0793f546c9eb29._comment Added a comment 2021-06-08 20:23:09 +00:00
comment_16_65aaf8e15cd15187cd63863634f25091._comment comments 2021-06-08 17:38:56 -04:00
comment_17_8df0e346ed85e52ec94763d11c174ab3._comment Added a comment: OSX mystery resolved. add --batch is effective mitigation 2021-06-08 21:56:53 +00:00
comment_17_045fd8ebe1f9441e42c9c6fbf5c06563._comment comment 2021-06-09 15:38:55 -04:00
comment_18_3e57d7d4640de95a059b2ab3d1e9d54e._comment Added a comment 2021-06-08 21:58:20 +00:00
comment_19_80372cd15a6a8812072a446c41fed4e7._comment Added a comment 2021-06-08 22:02:34 +00:00
comment_21_0119751108d8d9fe6848269594bec273._comment Added a comment: more "mystery resolved" -- identical (empty) keys 2021-06-09 21:00:34 +00:00
comment_22_192f2055345f1156a06d93b4c25e6722._comment Added a comment 2021-06-09 22:28:06 +00:00
comment_23_8693e7e9c800f25cbd274b6781d834d6._comment reproduced 2021-06-14 11:37:21 -04:00
comment_24_6d11f6aa4b1a435bdf6d165eb8e6db8a._comment reproduced 2021-06-14 11:37:21 -04:00
comment_25_5525b58ec94084a926c71f749d1f4233._comment check symlink before reading file 2021-06-14 11:53:51 -04:00
comment_26_cb64b1798af6ad507ce4a4196e9c5b1d._comment Added a comment: git-annex-add slowdown 2021-06-14 16:00:44 +00:00
comment_27_db91e10d0bea246686ba2241b348e67d._comment Added a comment 2021-06-14 16:23:41 +00:00
comment_28_0d6a37f823cd9cb3ed1e6e90066ebd2c._comment avoid sometimes expensive operations when annex.supportunlocked = false 2021-06-14 12:40:41 -04:00
comment_29_0f3c8949ae362b43ac7db02eb4e11890._comment response 2021-06-14 12:36:42 -04:00
comment_30_386c0cfe688effb1543ffd01a54717e0._comment Added a comment 2021-06-14 17:36:16 +00:00
comment_31_97fa9b7729805704a6a22cf18082e203._comment going round and round, boredly 2021-06-14 14:37:06 -04:00
comment_32_0ab5e774dad1e6e74a5c72b95e40d659._comment bloom doesn't work, but this should I hope 2021-06-14 17:53:01 -04:00
comment_33_6e5121e066998a303cf68ebc53e9fc15._comment fix exponential blowup when adding lots of identical files 2021-06-15 09:45:55 -04:00