From 9f4b99a0e748a494a10b3418cc574fe57b6e0720 Mon Sep 17 00:00:00 2001 From: Ilya_Shlyakhter Date: Wed, 27 Nov 2019 17:23:15 +0000 Subject: [PATCH] Added a comment: parallelization --- .../comment_1_bad182de605b7b47d66dcfe583acd4f1._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/todo/parallel_possibilities/comment_1_bad182de605b7b47d66dcfe583acd4f1._comment diff --git a/doc/todo/parallel_possibilities/comment_1_bad182de605b7b47d66dcfe583acd4f1._comment b/doc/todo/parallel_possibilities/comment_1_bad182de605b7b47d66dcfe583acd4f1._comment new file mode 100644 index 0000000000..36a15aef89 --- /dev/null +++ b/doc/todo/parallel_possibilities/comment_1_bad182de605b7b47d66dcfe583acd4f1._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="Ilya_Shlyakhter" + avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0" + subject="parallelization" + date="2019-11-27T17:23:14Z" + content=""" +When operating on many files, maybe run N parallel commands where i'th command ignores paths for which `(hash(filename) module N) != i`. Or, if git index has size I, i'th command ignores paths that are not legixographically between `index[(I/N)*i]` and `index[(I/N)*(i+1)]` (for index state at command start). Extending [[git-annex-matching-options]] with `--block=i` would let this be done using `xargs`. +"""]]