Commit graph

28251 commits

Author SHA1 Message Date
Joey Hess
6d0ed43786
Merge branch 'master' of ssh://git-annex.branchable.com 2020-07-14 21:44:51 -04:00
Joey Hess
e66ba410fc
todo 2020-07-14 21:44:31 -04:00
kyle
e55263046f tag issue 2020-07-15 00:58:03 +00:00
kyle
8c65cab4c6 recent test failures 2020-07-15 00:52:39 +00:00
Joey Hess
9a3b0500e8
Merge branch 'master' of ssh://git-annex.branchable.com 2020-07-14 20:04:47 -04:00
flpgdt@f64318f00d9e1c9535e11f5d27c80c1d799cce00
2d5cb06ae2 removed 2020-07-14 22:47:22 +00:00
flpgdt@f64318f00d9e1c9535e11f5d27c80c1d799cce00
1ed3b6567d Added a comment 2020-07-14 22:40:56 +00:00
kyle
13bff1a8e1 Added a comment 2020-07-14 20:13:22 +00:00
karel.de.macil@58e46458e7db066384a55b59a366d2025c4b4523
425717f571 2020-07-14 19:43:49 +00:00
kyle
ed99f652fa Added a comment 2020-07-14 19:18:10 +00:00
Joey Hess
f9b4a9f650
update 2020-07-14 14:47:22 -04:00
Joey Hess
7b2d236556
importfeed: stream metadata for 5% speedup
On top of the 10% speedup from streaming url logs.
2020-07-14 14:35:26 -04:00
Joey Hess
535cdc8d48
importfeed: Made checking known urls step around 10% faster.
This was a bit disappointing, I was hoping for a 2x speedup. But, I think
the metadata lookup is wasting a lot of time and also needs to be made to
stream.

The changes to catObjectStreamLsTree were benchmarked to not also speed
up --all around 3% more. Seems I managed to make it polymorphic after all.
2020-07-14 12:47:51 -04:00
Joey Hess
e223ddb774
comments 2020-07-13 17:52:03 -04:00
Joey Hess
c893743819
Merge branch 'master' of ssh://git-annex.branchable.com 2020-07-13 17:13:02 -04:00
Joey Hess
75aab72d23
mostly done with location log precaching
Some nice wins.
2020-07-13 17:04:02 -04:00
kyle
592548a843 Added a comment 2020-07-13 19:27:18 +00:00
kyle
b94481a3b9 Added a comment 2020-07-13 19:04:45 +00:00
Joey Hess
df58609804
convert sync to use seekFilteredKeys
This only speeds up sync --content from 34.75 to 33.17 seconds;
location log precaching will probably be a bigger win.
2020-07-13 15:02:52 -04:00
Joey Hess
f34edf0011
close 2020-07-13 13:27:43 -04:00
Joey Hess
4dd5cfa463
Merge branch 'master' of ssh://git-annex.branchable.com 2020-07-13 13:26:28 -04:00
Joey Hess
8219dbcb95
comment 2020-07-13 13:25:10 -04:00
Joey Hess
ba345e90b5
test metadata 2020-07-13 12:53:09 -04:00
kyle
800b36ccff Added a comment 2020-07-13 16:50:37 +00:00
kyle
ed7afd4f6d Added a comment 2020-07-13 16:46:54 +00:00
Joey Hess
918b1faa3d
avoid hanging on exception 2020-07-13 12:36:15 -04:00
Joey Hess
bff5694f55
Merge branch 'master' of ssh://git-annex.branchable.com 2020-07-13 12:18:05 -04:00
Joey Hess
1b2f29b207
comment 2020-07-13 12:16:48 -04:00
Joey Hess
c70ae68d7e
update 2020-07-13 11:49:24 -04:00
Joey Hess
415d394222
thought 2020-07-13 11:04:57 -04:00
kyle
575317a444 metadata --remove hang on 88a7fb5cb 2020-07-11 01:12:31 +00:00
kyle
780f8675f6 git-annex-init hang on older systems 2020-07-10 20:57:47 +00:00
Joey Hess
a7fcda861f
comment 2020-07-10 16:09:13 -04:00
Joey Hess
2b62e7dad4
Merge branch 'master' of ssh://git-annex.branchable.com 2020-07-10 16:04:54 -04:00
Joey Hess
5b3bed33c5
devblog 2020-07-10 16:00:39 -04:00
barak
7397bd75ea 2020-07-10 19:52:11 +00:00
Joey Hess
a32b6f9812
update 2020-07-10 15:49:03 -04:00
Joey Hess
412b09e17e
update 2020-07-10 15:23:12 -04:00
Joey Hess
2468eefc6d
2x speedup for annex file seeking on the horizon 2020-07-10 14:02:48 -04:00
Joey Hess
1df9e72a78
update 2020-07-10 13:31:47 -04:00
Joey Hess
6b9d1c1317
Merge branch 'master' of ssh://git-annex.branchable.com 2020-07-10 13:16:11 -04:00
barak
59c90643cd typo 2020-07-10 14:26:22 +00:00
Joey Hess
6e9fcf468d
streamkeys branch 2020-07-09 14:48:03 -04:00
branchable@bafd175a4b99afd6ed72501042e364ebd3e0c45e
b2581d4dd1 Added a comment: I've moved my auto-sync-daemon script 2020-07-09 14:24:48 +00:00
branchable@bafd175a4b99afd6ed72501042e364ebd3e0c45e
bbc3800369 Added a comment: Update on my auto-commit / auto-sync scripts 2020-07-09 14:23:15 +00:00
Ilya_Shlyakhter
96aad5458b Added a comment: re: git-annex-cat 2020-07-09 01:06:37 +00:00
Ilya_Shlyakhter
75b96059af Added a comment: git-annex-cat 2020-07-09 00:21:02 +00:00
Joey Hess
9f6bd6cc05
add inRepoDetails
planned to use for an optimisation

most things using stagedDetails were not expecting to get dup files in a
conflicted merge and deal with them, so converted them to use
inRepoDetails.
2020-07-08 15:36:35 -04:00
Joey Hess
c1eaf5b930
note 2020-07-08 14:21:37 -04:00
Joey Hess
d08c178f97
avoid catObjectStream skipping over unavailable shas
Not needed as it's used for --all, but will be needed later.
2020-07-08 13:57:17 -04:00
Joey Hess
de3d7d044d
make catObjectStream support newline and carriage return in filenames
Turns out the %(rest) trick was not needed. Instead, just maintain a
list of files we've asked for, and each cat-file response is for the
next file in the list.

This actually benchmarks 25% faster than before! Very surprising, but it
must be due to needing to shove less data through the pipe, and parse
less.
2020-07-08 13:49:03 -04:00
Joey Hess
2cf6717aec
thoughts 2020-07-08 10:51:24 -04:00
Joey Hess
5849bd6340
Merge branch 'master' of ssh://git-annex.branchable.com 2020-07-07 16:50:26 -04:00
Joey Hess
afd9b2f667
idea 2020-07-07 16:49:44 -04:00
yarikoptic
c9d0bf0e6a reassign to datalad - generic enhancement 2020-07-07 19:05:59 +00:00
Joey Hess
ba0adefe4c
Merge branch 'master' of ssh://git-annex.branchable.com 2020-07-07 14:19:46 -04:00
Joey Hess
d010ab04be
sped up the --all option by 2x to 16x by using git cat-file --buffer
This assumes that no location log files will have a newline or carriage
return in their name. catObjectStream skips any such files due to
cat-file not supporting them.

Keys have been prevented from containing newlines since 2011,
commit 480495beb4. If some old repo
had a key with a newline in it, --all will just skip processing that key.
Other things, like .git/annex/unused files certianly assume no newlines in
keys too, and AFAICR, such keys never actually worked.

Carriage return is escaped by preSanitizeKeyName since 2013. WORM keys
generated before that point could perhaps contain a CR. (URL probably not,
http probably doesn't support an URL with a raw CR in it.) So, added
a warning in fsck about such keys. Although, fsck --all will naturally
skip them, so won't be able to warn about them. Not entirely
satisfactory, but I'll bet there are not really any such keys in
existence.

Thanks to Lukey for finding this optimisation.
2020-07-07 13:54:04 -04:00
flpgdt@f64318f00d9e1c9535e11f5d27c80c1d799cce00
c4994c608e Added a comment 2020-07-07 16:54:46 +00:00
kyle
a5142499ce rsync hang 2020-07-07 15:26:38 +00:00
timothy.sanders@a7ce3a8bae11a60e0c4cda9cb4aef24ec459bbab
3b6754e2a5 2020-07-07 10:26:00 +00:00
timothy.sanders@a7ce3a8bae11a60e0c4cda9cb4aef24ec459bbab
8a9323f5b5 2020-07-07 10:24:29 +00:00
Lukey
56f5d99ceb Added a comment 2020-07-06 21:20:58 +00:00
Joey Hess
9468675ba9
note 2020-07-06 15:12:26 -04:00
Joey Hess
d66fc1a464
Revert "async exception safety for coprocesses"
This reverts commit 7013798df5.
2020-07-06 15:11:28 -04:00
Joey Hess
6b8c961e1f
some analysis but stuck 2020-07-06 14:46:05 -04:00
Joey Hess
dfa1c21b8a
comment
and update changelog with benchmark results
2020-07-06 13:39:42 -04:00
Joey Hess
0518b62d2b
update 2020-07-06 12:58:29 -04:00
Joey Hess
9a2fbc2ea8
comment 2020-07-06 11:58:14 -04:00
Joey Hess
27bbeea00e
close 2020-07-06 10:49:06 -04:00
Joey Hess
e2a4c49004
Merge branch 'master' of ssh://git-annex.branchable.com 2020-07-06 10:46:52 -04:00
andrew
48a6978d55 Added a comment: transfer repos 2020-07-05 17:25:49 +00:00
Joey Hess
4ac504cd2e
update 2020-07-05 11:12:10 -04:00
flpgdt@f64318f00d9e1c9535e11f5d27c80c1d799cce00
bbd5d1503d 2020-07-04 23:47:54 +00:00
jenkin.schibel@286264d9ceb79998aecff0d5d1a4ffe34f8b8421
151efb9c3c Added a comment: problem fixed itself 2020-07-04 03:28:26 +00:00
jenkin.schibel@286264d9ceb79998aecff0d5d1a4ffe34f8b8421
e40ba93924 Added a comment: problem fixed itself 2020-07-04 03:27:59 +00:00
Ilya_Shlyakhter
f6af30a7af Added a comment 2020-07-03 19:55:36 +00:00
Joey Hess
52e72f878e
expand 2020-07-03 14:42:04 -04:00
Joey Hess
c016527cb5
link 2020-07-03 14:40:13 -04:00
Joey Hess
d89b52086e
close 2020-07-03 14:31:12 -04:00
Joey Hess
b1fb4f9f58
clarify 2020-07-03 14:31:08 -04:00
Joey Hess
28fee95d4e
extend proposed interface with IMPORTKEY 2020-07-03 14:23:04 -04:00
Joey Hess
85506a7015
import: Added --no-content option, which avoids downloading files from a special remote
Only supported by some special remotes: directory
I need to check the rest and they're currently missing methods until I do.

git-annex sync --no-content does not yet use this to do imports
2020-07-03 13:41:57 -04:00
Joey Hess
a8099b9896
thought 2020-07-03 12:02:07 -04:00
Joey Hess
8ca0cdb0ac
reflow to fix man page conversion 2020-07-03 11:54:21 -04:00
Joey Hess
6953cc830a
Merge branch 'master' of ssh://git-annex.branchable.com 2020-07-03 10:34:18 -04:00
petersjt014@c20995a2e26a13566cd49aef30a99a7dff732a4e
668f3278b9 2020-07-03 07:36:41 +00:00
petersjt014@c20995a2e26a13566cd49aef30a99a7dff732a4e
ba209702c1 Added a comment: tor remote 2020-07-03 07:33:54 +00:00
Joey Hess
89108d6f5a
thought 2020-07-02 21:56:00 -04:00
Joey Hess
e463ef1b91
comment 2020-07-02 20:13:19 -04:00
Joey Hess
a3e125c1fe
move to re-notify 2020-07-02 20:05:54 -04:00
Joey Hess
8fc9788363
fix commment 2020-07-02 20:05:36 -04:00
jenkin.schibel@286264d9ceb79998aecff0d5d1a4ffe34f8b8421
51aa291b20 2020-07-02 21:23:18 +00:00
yarikoptic
edef3c25b3 Added a comment: map2url? 2020-07-02 20:41:15 +00:00
Ilya_Shlyakhter
df65c4796d Added a comment 2020-07-02 20:22:26 +00:00
yarikoptic
b7a78cbb26 Added a comment 2020-07-02 20:14:20 +00:00
kyle
a379ea833e Added a comment 2020-07-02 20:07:10 +00:00
Joey Hess
3353ff236a
comment 2020-07-02 15:30:16 -04:00
Joey Hess
629026bdbc
followup 2020-07-02 15:06:00 -04:00
Joey Hess
d8ad2204c7
comment 2020-07-02 14:43:52 -04:00
Joey Hess
f8ed8a916c
design 2020-07-02 14:35:59 -04:00