mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

fixed

Browse source
This commit is contained in:
Joey Hess 2023-06-09 16:13:52 -04:00
parent a0ab425c95
commit c33c226abd
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
3 changed files with 26 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
CHANGELOG
View file

@ -79,7 +79,7 @@ git-annex (10.20230408) UNRELEASED; urgency=medium
* Large speed up to importing trees from special remotes that contain a lot * Large speed up to importing trees from special remotes that contain a lot
of files, by only processing changed files. of files, by only processing changed files.
* Optimise database to further speed up importing large trees from special * Optimise database to further speed up importing large trees from special
remotes. remotes, and getting files from importtree=yes remotes.
* Some other speedups to importing trees from special remotes. * Some other speedups to importing trees from special remotes.
* Cache negative lookups of global numcopies and mincopies. * Cache negative lookups of global numcopies and mincopies.
Speeds up eg git-annex sync --content by up to 50%. Speeds up eg git-annex sync --content by up to 50%.

2
doc/bugs/importtree_spends_hours_reading_cidsdb.mdwn
View file

@ -46,3 +46,5 @@ I have tested this on a source directory that's 2.2G and another that's 1.1T, bo
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) ### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
Yes, and I hope to use it for a project to archive family photos and videos to BD-R (that's what this is about here) Yes, and I hope to use it for a project to archive family photos and videos to BD-R (that's what this is about here)
> [[fixed|done]] --[[Joey]]

23
doc/bugs/importtree_spends_hours_reading_cidsdb/comment_21_9320fcfb8bd47d4ee547e81fb9e01f38._comment Normal file
View file

@ -0,0 +1,23 @@
[[!comment format=mdwn
username="joey"
subject="""comment 21"""
date="2023-06-09T17:54:29Z"
content="""
Here's the final timing from the second test case with
150000 files after all of the above optimisation:
1. 0m0.131s
2. 33m40.490s
3. 0m41.027s
4. 2m59.070s
5. 14m5.679s
6. 0m3.673s
7. 17m39.271s
8. 11m15.317s
(git-annex get from an importtree remote was also sped up a lot;
it makes the same cid queries as sync #7)
I'm satisfied with this, considering this bug done, unless it's
somehow still being slow for you.
"""]]