Commit graph

3157 commits

Author SHA1 Message Date
Joey Hess
9468675ba9
note 2020-07-06 15:12:26 -04:00
Joey Hess
d66fc1a464
Revert "async exception safety for coprocesses"
This reverts commit 7013798df5.
2020-07-06 15:11:28 -04:00
Joey Hess
dfa1c21b8a
comment
and update changelog with benchmark results
2020-07-06 13:39:42 -04:00
Joey Hess
9a2fbc2ea8
comment 2020-07-06 11:58:14 -04:00
Ilya_Shlyakhter
f6af30a7af Added a comment 2020-07-03 19:55:36 +00:00
Joey Hess
d89b52086e
close 2020-07-03 14:31:12 -04:00
Joey Hess
85506a7015
import: Added --no-content option, which avoids downloading files from a special remote
Only supported by some special remotes: directory
I need to check the rest and they're currently missing methods until I do.

git-annex sync --no-content does not yet use this to do imports
2020-07-03 13:41:57 -04:00
Joey Hess
a8099b9896
thought 2020-07-03 12:02:07 -04:00
Joey Hess
89108d6f5a
thought 2020-07-02 21:56:00 -04:00
Joey Hess
e463ef1b91
comment 2020-07-02 20:13:19 -04:00
Joey Hess
8fc9788363
fix commment 2020-07-02 20:05:36 -04:00
yarikoptic
edef3c25b3 Added a comment: map2url? 2020-07-02 20:41:15 +00:00
Ilya_Shlyakhter
df65c4796d Added a comment 2020-07-02 20:22:26 +00:00
yarikoptic
b7a78cbb26 Added a comment 2020-07-02 20:14:20 +00:00
Joey Hess
3353ff236a
comment 2020-07-02 15:30:16 -04:00
Joey Hess
f8ed8a916c
design 2020-07-02 14:35:59 -04:00
Joey Hess
a88b671bd9
Merge branch 'master' of ssh://git-annex.branchable.com 2020-07-02 14:17:31 -04:00
Joey Hess
caaeba0be9
thoughts 2020-07-02 14:15:47 -04:00
yarikoptic
1d51db3b02 Added a comment: more ideas for async implementation 2020-07-02 17:44:44 +00:00
Joey Hess
fe1f4632a4
Merge branch 'master' of ssh://git-annex.branchable.com 2020-07-02 10:00:11 -04:00
Joey Hess
00c9eb4c78
comment 2020-07-01 20:12:10 -04:00
Ilya_Shlyakhter
d03902f7ff Added a comment: annex.thin for importing from directory special remote 2020-07-01 22:23:58 +00:00
Lukey
5a64acf790 Added a comment 2020-07-01 20:37:13 +00:00
Joey Hess
640dbaaaf8
Merge branch 'master' of ssh://git-annex.branchable.com 2020-07-01 15:15:47 -04:00
Joey Hess
11c2886578
overlapping todos 2020-07-01 15:06:36 -04:00
Ilya_Shlyakhter
d1232e385b Added a comment 2020-07-01 17:33:49 +00:00
Ilya_Shlyakhter
6eb318cd53 Added a comment: git pack files 2020-07-01 17:32:45 +00:00
Joey Hess
424b1912d6
followup and add link 2020-07-01 12:28:44 -04:00
Joey Hess
a496ab602d
todo 2020-07-01 12:07:11 -04:00
Joey Hess
07dff32bd4
Merge branch 'master' of ssh://git-annex.branchable.com 2020-07-01 11:23:39 -04:00
Lukey
ffb03cc959 Added a comment 2020-07-01 14:32:12 +00:00
Joey Hess
98a8a6da81
todo 2020-06-30 18:41:47 -04:00
Joey Hess
8f508d4406
comments 2020-06-30 16:41:31 -04:00
Joey Hess
1d335520df
Merge branch 'master' of ssh://git-annex.branchable.com 2020-06-30 12:27:19 -04:00
Joey Hess
137450c9fe
thoughts 2020-06-30 12:24:08 -04:00
Lukey
e6ca4cd0df 2020-06-30 15:46:57 +00:00
yarikoptic
692cea01e4 an idea on a (more) efficient transfer via async external remote protocol 2020-06-30 04:37:22 +00:00
Joey Hess
7fd20146e1
all easy cases done
bup can't do it after all, because removeKey deletes the git branch. And
the rest seem too hard to tackle today.
2020-06-26 14:24:48 -04:00
Joey Hess
76721b62dd
does not make sense to lockContent on web
Looked into this, and dropKey from web actually removes the url,
so git-annex won't try to get content from it.

So, if lockContent were implemented for web, and the web was left as the
only thing containing an object, another repo could at the same time
drop from web and remove its url, leaving no way to get the object.

Add to that, of course, the web is typically set untrusted, and so
implementing lockContent would not then be useful.

Similar reasoning applies to the bittorrent special remote, as well
as the fact that it does not even implement checkKey.
2020-06-26 13:58:28 -04:00
Joey Hess
b316a85ede
update 2020-06-26 13:54:23 -04:00
Joey Hess
4229713e63
importfeed: Added some additional --template variables for date and time
This commit was sponsored by Ethan Aubin.
2020-06-24 14:24:50 -04:00
Joey Hess
7757c0e900
Honor annex.largefiles when importing a tree from a special remote.
This commit was sponsored by Martin D on Patreon.
2020-06-23 16:07:18 -04:00
Joey Hess
d045e39058
thoughts 2020-06-23 14:14:13 -04:00
Joey Hess
3da4caa785
thoughts 2020-06-23 13:51:10 -04:00
Joey Hess
400b03115e
close 2020-06-22 14:46:02 -04:00
Joey Hess
b97d8030f3
respond and close 2020-06-22 14:44:14 -04:00
Joey Hess
d4d0951c07
followup and break out todo item to prioritize 2020-06-16 14:25:51 -04:00
Ilya_Shlyakhter
c003237871 Added a comment: separate processes for parallel transfers 2020-06-12 15:47:28 +00:00
Joey Hess
0017d9a347
Merge branch 'master' of ssh://git-annex.branchable.com 2020-06-11 15:44:28 -04:00
Joey Hess
e15ab727eb
comment and todo 2020-06-11 14:05:01 -04:00
branchable@bafd175a4b99afd6ed72501042e364ebd3e0c45e
512f77fe2a Added a comment: I've hacked up a Python script for policy-based automatic commits 2020-06-11 10:10:52 +00:00
Joey Hess
266ec93237
ugh 2020-06-09 17:39:03 -04:00
Joey Hess
e0d8ab75dd
Merge branch 'master' of ssh://git-annex.branchable.com 2020-06-09 16:20:45 -04:00
Joey Hess
24766125d9
update 2020-06-09 16:20:08 -04:00
Joey Hess
8a824147e4
horrible realization 2020-06-09 16:10:27 -04:00
yarikoptic
02ee2d0e47 2020-06-07 19:38:34 +00:00
yarikoptic
8e3cdfa90e Added a comment 2020-06-05 22:15:09 +00:00
yarikoptic
04aa8360a9 Added a comment: reply to Joey's comments 2020-06-05 22:11:48 +00:00
Joey Hess
0210e81d83
async exception safety for openFd
Audited for openFile and openFd, and this fixes all the ones I found
where an async exception could prevent the file getting closed.

Except for the lock pool, which is a whole other can of worms.
2020-06-05 15:48:00 -04:00
Joey Hess
dca19099a9
async exception safety
Masking ensures that EndStderrHandler gets written, so the helper
threads shut down.

However, nothing currently guarantees that calls to closeP2PSshConnection
are async exception safe, so made a note about it.

At this point, I've audited all calls to async, and made them all async
exception safe, except for ones in the assistant, and a few in leaf
commands (remotedaemon, enable-tor, multicast, p2p) which don't need to
be.
2020-06-05 14:56:41 -04:00
Joey Hess
42756b972e
few more things 2020-06-05 11:15:29 -04:00
Joey Hess
db270bd6ba
Merge branch 'master' of ssh://git-annex.branchable.com 2020-06-05 11:13:15 -04:00
Joey Hess
2bff3b7c49
init: When annex.pidlock is set, skip lock probing. 2020-06-05 11:12:16 -04:00
Joey Hess
1d41ae5d2a
init warning on stalled lock probe
init: If lock probing stalls for a long time (eg a broken NFS server),
display a message to let the user know what's taking so long.
2020-06-05 11:06:19 -04:00
Ilya_Shlyakhter
1d189cd08e Added a comment 2020-06-05 14:51:08 +00:00
Joey Hess
e998e4a29a
Merge branch 'master' of ssh://git-annex.branchable.com 2020-06-05 10:41:30 -04:00
yarikoptic
8d3ffe50a1 initial whining about long pidlock need detection time 2020-06-05 02:18:31 +00:00
Joey Hess
a3d1f967a8
almost forgot 2020-06-04 16:22:45 -04:00
Joey Hess
2670890b17
convert to withCreateProcess for async exception safety
This handles all createProcessSuccess callers, and aside from process
pools, the complete conversion of all process running to async exception
safety should be complete now.

Also, was able to remove from Utility.Process the old API that I now
know was not a good idea. And proof it was bad: The code size went *down*,
despite there being a fair bit of boilerplate for some future API to
reduce.
2020-06-04 15:45:52 -04:00
Joey Hess
12e7d52c8b
another wrinkle 2020-06-04 14:46:28 -04:00
Joey Hess
e1fc4f7594
make safeCommand stop the process if the thread gets killed
And a comment on a todo item that this commit is perhaps the start of
solving.
2020-06-03 12:52:11 -04:00
Joey Hess
81a811d935
comment 2020-06-03 12:00:44 -04:00
superhair123
4b0dd1fcb2 2020-06-01 13:47:30 +00:00
superhair123
401f0f7d5a 2020-06-01 13:44:15 +00:00
Joey Hess
89b2542d3c
annex.skipunknown with transition plan
Added annex.skipunknown git config, that can be set to false to change the
behavior of commands like `git annex get foo*`, to not skip over files/dirs
that are not checked into git and are explicitly listed in the command
line.

Significant complexity was needed to handle git-annex add, which uses some
git ls-files calls, but needs to not use --error-unmatch because of course
the files are not known to git.

annex.skipunknown is planned to change to default to false in a
git-annex release in early 2022. There's a todo for that.
2020-05-28 15:55:17 -04:00
Joey Hess
a6271b1323
Merge branch 'master' of ssh://git-annex.branchable.com 2020-05-27 12:46:52 -04:00
mike@2d6d71f56ce2a992244350475251df87c26fe351
311e605c28 Added a comment: .noannex 2020-05-27 15:34:54 +00:00
kyle
3e717128f8 Added a comment 2020-05-27 15:19:13 +00:00
mike@2d6d71f56ce2a992244350475251df87c26fe351
7776691ead 2020-05-27 15:10:57 +00:00
Joey Hess
298fa1c081
done 2020-05-27 11:00:57 -04:00
Joey Hess
fc9833f68d
export: Added options for json output
Just worked, no need to do anything except add the options.
2020-05-26 10:31:10 -04:00
Joey Hess
bbdba72624
Merge branch 'master' of ssh://git-annex.branchable.com 2020-05-26 10:21:21 -04:00
yarikoptic
d30dd6daa4 TODO for export --json* options 2020-05-26 04:14:57 +00:00
Joey Hess
6eea4969d3
close 2020-05-25 12:04:56 -04:00
Joey Hess
234764518a
close 2020-05-25 12:02:23 -04:00
Joey Hess
e46717bf18
close 2020-05-25 11:59:21 -04:00
Joey Hess
133de41e49
close 2020-05-25 11:56:27 -04:00
Joey Hess
5f5a2a6b56
close 2020-05-25 11:52:45 -04:00
Joey Hess
fcb288be13
close dup 2020-05-25 11:48:39 -04:00
Joey Hess
4d87d255ee
comment 2020-05-25 11:47:10 -04:00
Joey Hess
7cc5e955b6
close 2020-05-25 11:31:38 -04:00
https://christian.amsuess.com/chrysn
112d2f8e7b Added a comment: Re: Usefulness of batch key processing 2020-05-15 09:33:22 +00:00
https://christian.amsuess.com/chrysn
340287ef94 Added a comment: Usefulness of batch key processing 2020-05-15 09:21:15 +00:00
Joey Hess
5b5a9aaf92
comment 2020-05-05 12:33:28 -04:00
Joey Hess
d2e78dfc0d
prove this optimisation would not be safe, so close 2020-05-04 14:49:42 -04:00
Joey Hess
9fa940569c
added remote variants
Todo item is done at last.

Might later want to think about testing some other types of remotes that
can be tested locally. The git remote itself is probably already well
enough tested by the test suite that testremote is not needed. Could
test things like bup, or rsync to a local directory. Or even external,
although that would require embedding an external special remote program
into the test suite..
2020-04-30 13:52:03 -04:00
Joey Hess
b41c0edebd
todo 2020-04-28 17:24:53 -04:00
ct.git-annex@230092d9bd3cf09ced2b9605cdb14ad0a3db265d
d3270a8b9a Added a comment 2020-04-26 20:18:48 +00:00
Joey Hess
985dd4f2ad
comment 2020-04-23 16:21:18 -04:00
Joey Hess
957a87b437
fix absolute filenames fed into --batch and git-annex info 2020-04-15 16:04:05 -04:00