mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

2x speedup for annex file seeking on the horizon

Browse source
This commit is contained in:
Joey Hess 2020-07-10 14:02:48 -04:00
parent 1df9e72a78
commit 2468eefc6d
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
1 changed files with 13 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

13
doc/todo/precache_logs_for_speed_with_cat-file_--buffer.mdwn
View file

@ -19,3 +19,16 @@ A complication is that, non-symlinks could be large files that are not
annexed but in git; don't want to cat those when looking for annex links. annexed but in git; don't want to cat those when looking for annex links.
That would probably need pre-filtering through a cat-file --buffer that That would probably need pre-filtering through a cat-file --buffer that
only gets the size of the blob, not its content. only gets the size of the blob, not its content.
> WIP implementation in the `streamkeys` branch is a win! Nearly 2x faster
> `git-annex get` seeking. It needs more work, to avoid catting large
> non-annexex files.
>
> And, only `git-annex get` uses it, other commands need to be converted.
>
> Also, limits that look up keys still do, before they're looked up
> efficiently. Avoiding that would speed up --in etc, probably another
> 1.5x-2x speedup when such limits are used. What that optimisation needs
> is a way to tell if the current limit needs the key or not.
> If it does, then match on it after getting the key, otherwise before
> getting the key.