Commit graph

3167 commits

Author SHA1 Message Date
Joey Hess
9d9f1f85d6
comment 2020-07-27 11:37:12 -04:00
Joey Hess
36d1621c35
Merge branch 'master' of ssh://git-annex.branchable.com 2020-07-27 11:34:03 -04:00
Joey Hess
3953c7a0ce
add DEBUG 2020-07-27 11:31:00 -04:00
ghen1
ad391da6b8 2020-07-27 15:24:21 +00:00
Joey Hess
32e1d7bc31
add 2020-07-24 14:11:08 -04:00
Joey Hess
c5ea2e9d12
better benchmark for move/copy speedup 2020-07-24 13:34:12 -04:00
Joey Hess
18f1fb5841
drop performance improvements
Sped up seeking files to drop by 2x, and also some performance
improvements to checking numcopies.

Interestingly, the seek speedup is not due to precaching, but I think is
due to calling getParsed earlier.

Annex.Drop had to be changed to check inAnnex there, since it was removed
from Command.Drop. All other users of Command.Drop already checked inAnnex
themselves.

This commit was sponsored by Ryan Newton on Patreon.
2020-07-24 13:27:46 -04:00
Joey Hess
d732ef1a89
move, copy: Sped up seeking for annexed files to operate on by a factor of nearly 2x. 2020-07-24 12:56:02 -04:00
Joey Hess
4685612f43
small git-annex get speedup
Remove an redundant inAnnex check. The checkContentPresent handles that,
and after the last commit also does in batch mode.
2020-07-22 14:29:30 -04:00
Joey Hess
1be92381ec
unify batch mode with non-batch by using AnnexedFileSeeker 2020-07-22 14:23:28 -04:00
Ilya_Shlyakhter
59917f8a6d Added a comment: external backend protocol 2020-07-21 17:43:27 +00:00
Joey Hess
abd56fb019
Fix a bug in find --batch in the previous version. 2020-07-20 19:50:53 -04:00
Joey Hess
f71310fed0
comment 2020-07-20 14:19:13 -04:00
Joey Hess
d1300eca2e
draft external backend protocol 2020-07-20 14:05:49 -04:00
Joey Hess
1489fbbdde
bug 2020-07-19 18:26:57 -04:00
yarikoptic
6a05388877 Added a comment 2020-07-18 05:09:54 +00:00
yarikoptic
7ee0bcbee7 Added a comment 2020-07-18 05:09:32 +00:00
yarikoptic
4ab711e153 Added a comment 2020-07-18 04:50:12 +00:00
yarikoptic
a6b0147b7f Added a comment 2020-07-18 04:49:49 +00:00
yarikoptic
d7b4df85e4 Added a comment 2020-07-18 04:34:26 +00:00
yarikoptic
5215fe92b9 Added a comment 2020-07-18 04:34:05 +00:00
yarikoptic
360de9446e Added a comment 2020-07-18 03:57:20 +00:00
yarikoptic
c46b9ac4ae initial 2nd wave of whining about startup time and to consider prelink or alike 2020-07-18 03:54:34 +00:00
yarikoptic
4f152089eb Added a comment: Windows build of file (which includes libmagic) 2020-07-16 22:01:17 +00:00
Joey Hess
a3a8779501
Merge branch 'master' of ssh://git-annex.branchable.com 2020-07-16 15:08:51 -04:00
Ilya_Shlyakhter
77299ae6e5 Added a comment: external backends 2020-07-16 17:30:55 +00:00
Joey Hess
5ab3849da3
thought 2020-07-15 20:42:53 -04:00
Joey Hess
034f958b09
comment 2020-07-15 14:02:31 -04:00
Joey Hess
360dc386e7
comment 2020-07-15 10:08:37 -04:00
Joey Hess
1bc015bff4
tag datalad at yoh's req 2020-07-15 09:51:57 -04:00
Joey Hess
e66ba410fc
todo 2020-07-14 21:44:31 -04:00
Joey Hess
f9b4a9f650
update 2020-07-14 14:47:22 -04:00
Joey Hess
7b2d236556
importfeed: stream metadata for 5% speedup
On top of the 10% speedup from streaming url logs.
2020-07-14 14:35:26 -04:00
Joey Hess
535cdc8d48
importfeed: Made checking known urls step around 10% faster.
This was a bit disappointing, I was hoping for a 2x speedup. But, I think
the metadata lookup is wasting a lot of time and also needs to be made to
stream.

The changes to catObjectStreamLsTree were benchmarked to not also speed
up --all around 3% more. Seems I managed to make it polymorphic after all.
2020-07-14 12:47:51 -04:00
Joey Hess
75aab72d23
mostly done with location log precaching
Some nice wins.
2020-07-13 17:04:02 -04:00
Joey Hess
df58609804
convert sync to use seekFilteredKeys
This only speeds up sync --content from 34.75 to 33.17 seconds;
location log precaching will probably be a bigger win.
2020-07-13 15:02:52 -04:00
Joey Hess
c70ae68d7e
update 2020-07-13 11:49:24 -04:00
Joey Hess
415d394222
thought 2020-07-13 11:04:57 -04:00
Joey Hess
a32b6f9812
update 2020-07-10 15:49:03 -04:00
Joey Hess
412b09e17e
update 2020-07-10 15:23:12 -04:00
Joey Hess
2468eefc6d
2x speedup for annex file seeking on the horizon 2020-07-10 14:02:48 -04:00
Joey Hess
1df9e72a78
update 2020-07-10 13:31:47 -04:00
Joey Hess
6b9d1c1317
Merge branch 'master' of ssh://git-annex.branchable.com 2020-07-10 13:16:11 -04:00
Joey Hess
6e9fcf468d
streamkeys branch 2020-07-09 14:48:03 -04:00
branchable@bafd175a4b99afd6ed72501042e364ebd3e0c45e
bbc3800369 Added a comment: Update on my auto-commit / auto-sync scripts 2020-07-09 14:23:15 +00:00
Ilya_Shlyakhter
96aad5458b Added a comment: re: git-annex-cat 2020-07-09 01:06:37 +00:00
Ilya_Shlyakhter
75b96059af Added a comment: git-annex-cat 2020-07-09 00:21:02 +00:00
Joey Hess
9f6bd6cc05
add inRepoDetails
planned to use for an optimisation

most things using stagedDetails were not expecting to get dup files in a
conflicted merge and deal with them, so converted them to use
inRepoDetails.
2020-07-08 15:36:35 -04:00
Joey Hess
c1eaf5b930
note 2020-07-08 14:21:37 -04:00
Joey Hess
d08c178f97
avoid catObjectStream skipping over unavailable shas
Not needed as it's used for --all, but will be needed later.
2020-07-08 13:57:17 -04:00