Commit graph

28331 commits

Author SHA1 Message Date
Joey Hess
bdda55ab6f
devblog 2020-07-24 14:15:29 -04:00
Joey Hess
32e1d7bc31
add 2020-07-24 14:11:08 -04:00
nix.zahlen@1211ac6c964ba2d68b70655f747bef1383032e77
df77d63e76 Added a comment: It is really a pitty that the Windows port / version is unmaintained 2020-07-24 17:48:10 +00:00
Joey Hess
c5ea2e9d12
better benchmark for move/copy speedup 2020-07-24 13:34:12 -04:00
Joey Hess
18f1fb5841
drop performance improvements
Sped up seeking files to drop by 2x, and also some performance
improvements to checking numcopies.

Interestingly, the seek speedup is not due to precaching, but I think is
due to calling getParsed earlier.

Annex.Drop had to be changed to check inAnnex there, since it was removed
from Command.Drop. All other users of Command.Drop already checked inAnnex
themselves.

This commit was sponsored by Ryan Newton on Patreon.
2020-07-24 13:27:46 -04:00
Joey Hess
d732ef1a89
move, copy: Sped up seeking for annexed files to operate on by a factor of nearly 2x. 2020-07-24 12:56:02 -04:00
erewhon
694c2dc16c Added a comment 2020-07-24 16:05:08 +00:00
yarikoptic
d7e04e5e3a Added a comment 2020-07-24 15:22:40 +00:00
yarikoptic
4ff59a85d5 report on intolerance of IFS 2020-07-24 01:42:59 +00:00
flpgdt@f64318f00d9e1c9535e11f5d27c80c1d799cce00
d604cd621b 2020-07-23 13:12:38 +00:00
karel-de-macil
39337c604b 2020-07-23 10:30:35 +00:00
karel-de-macil
8325db46a6 2020-07-23 10:28:43 +00:00
karel-de-macil
205d6ea3e7 Added a comment 2020-07-23 10:17:09 +00:00
karel-de-macil
c6454b9b22 2020-07-23 08:32:56 +00:00
Joey Hess
4685612f43
small git-annex get speedup
Remove an redundant inAnnex check. The checkContentPresent handles that,
and after the last commit also does in batch mode.
2020-07-22 14:29:30 -04:00
Joey Hess
1be92381ec
unify batch mode with non-batch by using AnnexedFileSeeker 2020-07-22 14:23:28 -04:00
kyle
155b56f015 Added a comment 2020-07-22 16:12:25 +00:00
mreiden
6dbe68bac5 Added a comment: Autobuild permissions 2020-07-22 16:07:22 +00:00
kyle
0538dc425c Added a comment 2020-07-22 16:01:05 +00:00
Joey Hess
d9ae6ab0b4
comment 2020-07-22 11:12:38 -04:00
Joey Hess
8da851d585
Merge branch 'master' of ssh://git-annex.branchable.com 2020-07-22 11:06:13 -04:00
Joey Hess
aa492bc659
Fix a hang when using git-annex with an old openssh 7.2p2
This does mean a 2 second delay after transfers when using that ssh, but
it's an old and apparently quite weirdly broken version of ssh.
2020-07-22 11:04:33 -04:00
Barafu
6dbef88eee 2020-07-21 23:25:06 +00:00
Joey Hess
afaae84f49
comment 2020-07-21 16:50:26 -04:00
Joey Hess
cb74cefde7
Fix a hang when using git-annex with an old openssh 7.2p2
Which had some weird inheriting of ssh FDs by sshd.

Bug was introduced in git-annex version 7.20200202.7.
2020-07-21 16:14:25 -04:00
Joey Hess
b75aa68bfa
Merge branch 'master' of ssh://git-annex.branchable.com 2020-07-21 15:31:04 -04:00
Joey Hess
ac56a5c2a0
Fix a lock file descriptor leak that could occur when running commands like git-annex add with -J
Bug was introduced as part of a different FD leak fix in version 6.20160318.
2020-07-21 15:30:47 -04:00
Joey Hess
fd8339005a
update 2020-07-21 14:44:07 -04:00
Barafu
314aef3471 2020-07-21 18:43:40 +00:00
Barafu
75d59a47d7 2020-07-21 18:42:32 +00:00
Ilya_Shlyakhter
59917f8a6d Added a comment: external backend protocol 2020-07-21 17:43:27 +00:00
Joey Hess
473e9767f4
Merge branch 'master' of ssh://git-annex.branchable.com 2020-07-21 13:01:11 -04:00
Ilya_Shlyakhter
b1e84e61e1 posted build questions 2020-07-21 15:06:23 +00:00
Joey Hess
77f7157ddb
analysis 2020-07-21 10:30:46 -04:00
Joey Hess
4de4c36d2b
dup 2020-07-21 10:07:23 -04:00
flpgdt@f64318f00d9e1c9535e11f5d27c80c1d799cce00
c129221686 2020-07-21 12:49:12 +00:00
Joey Hess
2ce34f1870
add news item for git-annex 8.20200720.1 2020-07-20 19:58:35 -04:00
Joey Hess
1ccb6699a1
guidance on size and mtime fields 2020-07-20 19:56:47 -04:00
Joey Hess
abd56fb019
Fix a bug in find --batch in the previous version. 2020-07-20 19:50:53 -04:00
Joey Hess
8499c6d65a
cleanup 2020-07-20 14:42:02 -04:00
Joey Hess
5c5e2ed8cf
add news item for git-annex 8.20200720 2020-07-20 14:41:43 -04:00
Joey Hess
2630bb766a
fix link 2020-07-20 14:25:24 -04:00
Joey Hess
8b0fbc1391
formatting 2020-07-20 14:23:01 -04:00
Joey Hess
adf4f25a1c
Merge branch 'master' of ssh://git-annex.branchable.com 2020-07-20 14:19:24 -04:00
Joey Hess
f71310fed0
comment 2020-07-20 14:19:13 -04:00
Joey Hess
d1300eca2e
draft external backend protocol 2020-07-20 14:05:49 -04:00
gueux
225fd08b6d Added a comment 2020-07-20 06:45:25 +00:00
Joey Hess
4be5567372
Merge branch 'master' of ssh://git-annex.branchable.com 2020-07-19 21:31:37 -04:00
Joey Hess
889603336a
fix reversion in skipping deleted files
And add a test case for that.

This certianly loses some of the 2x performance improvement in file
seeking that seekFilteredKeys led to, because now it has to stat the
worktree files again. Without benchmarking, I expect there will still be
a sizable improvement, and also the git-annex branch precaching that
seekFilteredKeys can do will still be a win of its approach.

Also worth noting that lookupKey, when the file DNE, check if it's in an
adjusted branch with hidden files, and if so, finds the key for the
file anyway. That was intended to make git-annex sync --content be able
to process those files, but a side effect was that, when a file was
deleted but the deletion not yet staged, git-annex commands used to
still list it. That was actually a bug. This commit fixes that bug too.
(git-annex sync --content on such a branch does not use seekFilteredKeys
so was not affected by the reversion or by this behavior change)

This commit was sponsored by Jake Vosloo on Patreon.
2020-07-19 21:25:01 -04:00
Joey Hess
b5590e916e
analysis 2020-07-19 20:14:49 -04:00
Joey Hess
1489fbbdde
bug 2020-07-19 18:26:57 -04:00
mzimann
7d09ca4363 Added a comment: Use curl instead of wget 2020-07-18 22:44:20 +00:00
yarikoptic
6a05388877 Added a comment 2020-07-18 05:09:54 +00:00
yarikoptic
7ee0bcbee7 Added a comment 2020-07-18 05:09:32 +00:00
yarikoptic
4ab711e153 Added a comment 2020-07-18 04:50:12 +00:00
yarikoptic
a6b0147b7f Added a comment 2020-07-18 04:49:49 +00:00
yarikoptic
d7b4df85e4 Added a comment 2020-07-18 04:34:26 +00:00
yarikoptic
5215fe92b9 Added a comment 2020-07-18 04:34:05 +00:00
Ilya_Shlyakhter
526a549766 Added a comment 2020-07-18 03:58:18 +00:00
yarikoptic
360de9446e Added a comment 2020-07-18 03:57:20 +00:00
yarikoptic
c46b9ac4ae initial 2nd wave of whining about startup time and to consider prelink or alike 2020-07-18 03:54:34 +00:00
erewhon
aa554e1adb 2020-07-17 16:38:22 +00:00
yarikoptic
4f152089eb Added a comment: Windows build of file (which includes libmagic) 2020-07-16 22:01:17 +00:00
kyle
f818da2f85 bug: find --batch regression 2020-07-16 21:53:14 +00:00
Joey Hess
a3a8779501
Merge branch 'master' of ssh://git-annex.branchable.com 2020-07-16 15:08:51 -04:00
Ilya_Shlyakhter
77299ae6e5 Added a comment: external backends 2020-07-16 17:30:55 +00:00
branchable@bafd175a4b99afd6ed72501042e364ebd3e0c45e
059d56659b Added a comment: GIT_TRACE=1 debug shows smudge --clean is run twice per file 2020-07-16 17:06:13 +00:00
branchable@bafd175a4b99afd6ed72501042e364ebd3e0c45e
1836f7a318 2020-07-16 16:32:41 +00:00
Joey Hess
5ab3849da3
thought 2020-07-15 20:42:53 -04:00
Joey Hess
ebdbb87716
comment 2020-07-15 16:11:34 -04:00
Joey Hess
fdf4bdd875
Merge branch 'master' of ssh://git-annex.branchable.com 2020-07-15 16:08:49 -04:00
Joey Hess
034f958b09
comment 2020-07-15 14:02:31 -04:00
kyle
e8197c9e74 fixed elsewhere 2020-07-15 16:35:46 +00:00
Joey Hess
aa44126bc0
Merge branch 'master' of ssh://git-annex.branchable.com 2020-07-15 11:27:38 -04:00
Joey Hess
a7156b875c
fix fsck reversion
75aab72d23 made fsck skip files whose
content is not present, but it should complain if there are not enough
copies.
2020-07-15 11:21:43 -04:00
Joey Hess
32c847d1a1
analysis 2020-07-15 11:13:02 -04:00
Joey Hess
5d59bb9554
add 2020-07-15 10:58:45 -04:00
Joey Hess
360dc386e7
comment 2020-07-15 10:08:37 -04:00
Joey Hess
1bc015bff4
tag datalad at yoh's req 2020-07-15 09:51:57 -04:00
nolan.nichols@fe8d6ffa163484d0632235838db69df04538caf1
af6088bcba 2020-07-15 03:32:24 +00:00
Joey Hess
6d0ed43786
Merge branch 'master' of ssh://git-annex.branchable.com 2020-07-14 21:44:51 -04:00
Joey Hess
e66ba410fc
todo 2020-07-14 21:44:31 -04:00
kyle
e55263046f tag issue 2020-07-15 00:58:03 +00:00
kyle
8c65cab4c6 recent test failures 2020-07-15 00:52:39 +00:00
Joey Hess
9a3b0500e8
Merge branch 'master' of ssh://git-annex.branchable.com 2020-07-14 20:04:47 -04:00
flpgdt@f64318f00d9e1c9535e11f5d27c80c1d799cce00
2d5cb06ae2 removed 2020-07-14 22:47:22 +00:00
flpgdt@f64318f00d9e1c9535e11f5d27c80c1d799cce00
1ed3b6567d Added a comment 2020-07-14 22:40:56 +00:00
kyle
13bff1a8e1 Added a comment 2020-07-14 20:13:22 +00:00
karel.de.macil@58e46458e7db066384a55b59a366d2025c4b4523
425717f571 2020-07-14 19:43:49 +00:00
kyle
ed99f652fa Added a comment 2020-07-14 19:18:10 +00:00
Joey Hess
f9b4a9f650
update 2020-07-14 14:47:22 -04:00
Joey Hess
7b2d236556
importfeed: stream metadata for 5% speedup
On top of the 10% speedup from streaming url logs.
2020-07-14 14:35:26 -04:00
Joey Hess
535cdc8d48
importfeed: Made checking known urls step around 10% faster.
This was a bit disappointing, I was hoping for a 2x speedup. But, I think
the metadata lookup is wasting a lot of time and also needs to be made to
stream.

The changes to catObjectStreamLsTree were benchmarked to not also speed
up --all around 3% more. Seems I managed to make it polymorphic after all.
2020-07-14 12:47:51 -04:00
Joey Hess
e223ddb774
comments 2020-07-13 17:52:03 -04:00
Joey Hess
c893743819
Merge branch 'master' of ssh://git-annex.branchable.com 2020-07-13 17:13:02 -04:00
Joey Hess
75aab72d23
mostly done with location log precaching
Some nice wins.
2020-07-13 17:04:02 -04:00
kyle
592548a843 Added a comment 2020-07-13 19:27:18 +00:00
kyle
b94481a3b9 Added a comment 2020-07-13 19:04:45 +00:00
Joey Hess
df58609804
convert sync to use seekFilteredKeys
This only speeds up sync --content from 34.75 to 33.17 seconds;
location log precaching will probably be a bigger win.
2020-07-13 15:02:52 -04:00
Joey Hess
f34edf0011
close 2020-07-13 13:27:43 -04:00