git-annex/doc/todo/optimize_by_converting_String_to_ByteString.mdwn

git-annex uses FilePath (String) extensively. That's a slow data type.
Converting to ByteString, and RawFilePath, should speed it up
significantly, according to [[/profiling]].

I've made a test branch, `bs`, to see what kind of performance improvement
to expect. 

Benchmarking `git-annex find`, speedups range from 28-66%. The files fly by
much more snappily. Other commands likely also speed up, but do more work
than find so the improvement is not as large.

The `bs` branch is in a mergeable state now. [[done]]

Stuff not entirely finished:

* Profile various commands and look for hot spots involving conversion
  between RawFilePath and FilePath.
todo for bs branch 2019-11-26 20:11:55 +00:00			`git-annex uses FilePath (String) extensively. That's a slow data type.`
			`Converting to ByteString, and RawFilePath, should speed it up`
			`significantly, according to [[/profiling]].`

			I've made a test branch, `bs`, to see what kind of performance improvement
update re state of bs branch 2019-12-06 19:13:13 +00:00			`to expect.`
todo for bs branch 2019-11-26 20:11:55 +00:00
update re state of bs branch 2019-12-06 19:13:13 +00:00			Benchmarking `git-annex find`, speedups range from 28-66%. The files fly by
			`much more snappily. Other commands likely also speed up, but do more work`
			`than find so the improvement is not as large.`

wrap up 2019-12-18 19:19:19 +00:00			The `bs` branch is in a mergeable state now. [[done]]
use filepath-bytestring for annex object manipulations git-annex find is now RawFilePath end to end, no string conversions. So is git-annex get when it does not need to get anything. So this is a major milestone on optimisation. Benchmarks indicate around 30% speedup in both commands. Probably many other performance improvements. All or nearly all places where a file is statted use RawFilePath now. 2019-12-11 18:12:22 +00:00
update 2019-12-18 16:12:51 +00:00			`Stuff not entirely finished:`
fix fileJournal My ByteString rewrite oversimplified it, resulting in any _ in a journal file turning into a / in the git-annex branch, which was often the wrong filename, or sometimes (//) an invalid filename that git refused to add. 2019-12-18 15:29:34 +00:00
update 2019-12-18 16:12:51 +00:00			`* Profile various commands and look for hot spots involving conversion`
			`between RawFilePath and FilePath.`