From 4088ce49ce990f4411c2c9162bb74b9991e92cd3 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Wed, 5 Jun 2019 20:18:59 -0400 Subject: [PATCH] devblog --- .../day_591__superscalar_pipelining.mdwn | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) create mode 100644 doc/devblog/day_591__superscalar_pipelining.mdwn diff --git a/doc/devblog/day_591__superscalar_pipelining.mdwn b/doc/devblog/day_591__superscalar_pipelining.mdwn new file mode 100644 index 0000000000..f6bcc7f9db --- /dev/null +++ b/doc/devblog/day_591__superscalar_pipelining.mdwn @@ -0,0 +1,17 @@ +A long day spent making CommandCleanup actions run in a separate job pool +than CommandPerform actions. I don't think this will speed anything up much +yet, but it's useful groundwork. Now expensive things that are not +the main action of a command can be moved into CommandCleanup and won't +delay git-annex moving on to the next file. The main thing I want to move +is checksum verification after a transfer. But there are probably other +things I have not thought of. + +CommandCleanup was always not well distinguised from CommandPerform, +and so there was little incentive to put things in it. Now that's changed. + +I also noticed that with -J, git-annex takes significantly longer than +without to get started, when the first file it needs to process is quite a +way down the ls-tree. This must be concurrency overhead. But, when +CommandStart is skipping over a file that it doesn't need to process, +there is no need to do that bookkeeping. Planning to take some time +tomorrow to see if I can refactor CommandStart to avoid that overhead.