This commit is contained in:
Joey Hess 2023-06-09 16:13:52 -04:00
parent a0ab425c95
commit c33c226abd
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
3 changed files with 26 additions and 1 deletions

View file

@ -79,7 +79,7 @@ git-annex (10.20230408) UNRELEASED; urgency=medium
* Large speed up to importing trees from special remotes that contain a lot
of files, by only processing changed files.
* Optimise database to further speed up importing large trees from special
remotes.
remotes, and getting files from importtree=yes remotes.
* Some other speedups to importing trees from special remotes.
* Cache negative lookups of global numcopies and mincopies.
Speeds up eg git-annex sync --content by up to 50%.

View file

@ -46,3 +46,5 @@ I have tested this on a source directory that's 2.2G and another that's 1.1T, bo
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
Yes, and I hope to use it for a project to archive family photos and videos to BD-R (that's what this is about here)
> [[fixed|done]] --[[Joey]]

View file

@ -0,0 +1,23 @@
[[!comment format=mdwn
username="joey"
subject="""comment 21"""
date="2023-06-09T17:54:29Z"
content="""
Here's the final timing from the second test case with
150000 files after all of the above optimisation:
1. 0m0.131s
2. 33m40.490s
3. 0m41.027s
4. 2m59.070s
5. 14m5.679s
6. 0m3.673s
7. 17m39.271s
8. 11m15.317s
(git-annex get from an importtree remote was also sped up a lot;
it makes the same cid queries as sync #7)
I'm satisfied with this, considering this bug done, unless it's
somehow still being slow for you.
"""]]