Commit graph

28407 commits

Author SHA1 Message Date
Joey Hess
95d9a3cf8a
Merge branch 'asyncexternal' 2020-08-14 16:00:49 -04:00
Joey Hess
05b2b46a82
async extension done 2020-08-14 15:24:34 -04:00
Joey Hess
198b709561
switch to TMVars for thread safety when using the async extension
TVars were not updated atomically, which was ok when each thread got its
own External that was the only thing using these TVars. But, with the
async extension, several External instances can share the same var, so
it needs to be a TMVar to avoid read/write conflicts.

In particular, this makes PREPARE only be sent once.
2020-08-14 14:50:09 -04:00
Joey Hess
7da2d4dd2d
one jobid per thread
And, relay ERROR on to all listening threads.
2020-08-14 14:24:46 -04:00
Joey Hess
72561563d9
rethought the async protocol some more
Moving jobid generation to the git-annex side lets it be simplified a
lot.

Note that it will also be possible to generate one jobid per connection,
rather than a new job per request. That will make overflow not an issue,
and will avoid some work, and will simplify some of the code.
2020-08-13 20:18:06 -04:00
Joey Hess
7546e686a2
async proto basically working
Simplified the protocol by removing END-ASYNC.

There's a STM crash when a non-async protocol message is sent, which
needs to be fixed.
2020-08-13 15:52:12 -04:00
Joey Hess
0900bb2ca9
Merge branch 'master' of ssh://git-annex.branchable.com 2020-08-12 16:27:40 -04:00
Joey Hess
1ecbac4025
branch 2020-08-12 16:27:02 -04:00
Joey Hess
15706e6991
relayer receive loop is done
Receive loop looks right. Still need the send loop.

And, a complication is that some messages git-annex
sends need to be wrapped in REPLY_ASYNC, while others
do not. So will probably need to split externalSend
into two.
2020-08-12 15:56:58 -04:00
Joey Hess
7a21492f49
rethought this protocol again
Now that I've started implementation, I see it's really necessary that
every message the special remote sends use the protocol, otherwise
nasty edge cases abound.
2020-08-12 15:12:09 -04:00
git-annex@32dadd8082caa7c4ec9b28cf7dcf78e0d9fe58a8
c68bf6fc90 Bug: addurl --file not working as expected with youtube-dl 2020-08-12 13:09:02 +00:00
raf@4b53f20230978a98fc814b6e5565d9a30567a4f8
7be4eb71f2 removed 2020-08-12 12:54:46 +00:00
raf@4b53f20230978a98fc814b6e5565d9a30567a4f8
928787af63 2020-08-12 12:47:19 +00:00
cardoso-neto
2e5fe87ef0 Added a comment 2020-08-11 22:10:27 +00:00
Joey Hess
482422b9c5
Merge branch 'master' of ssh://git-annex.branchable.com 2020-08-11 16:42:37 -04:00
Joey Hess
ddf69bf5b8
draft async extension 2020-08-11 16:42:09 -04:00
cardoso-neto
d79bb64ea4 2020-08-11 20:23:25 +00:00
Ilya_Shlyakhter
14bbf17bb5 Added a comment: strange conda install behavior 2020-08-11 20:19:49 +00:00
cardoso-neto
46ca795432 Added a comment 2020-08-11 19:57:06 +00:00
Joey Hess
db1c6da84b
close 2020-08-11 14:01:22 -04:00
Ilya_Shlyakhter
b41f774451 added note on installing the standalone git-annex distribution with conda 2020-08-11 16:03:40 +00:00
Ilya_Shlyakhter
2df83e0511 Added a comment: checking the conda-forge recipe 2020-08-11 15:51:28 +00:00
cardoso-neto
1518e15123 Added a comment: Great tip about conda. 2020-08-11 13:23:00 +00:00
Ilya_Shlyakhter
042714fed7 Added a comment: please keep the hackage tarballs 2020-08-11 00:43:51 +00:00
Joey Hess
766a501fa1
add news item for git-annex 8.20200810 2020-08-10 19:22:55 -04:00
Joey Hess
3400b01887
comment 2020-08-10 18:46:44 -04:00
Ilya_Shlyakhter
693f2093a0 Added a comment 2020-08-10 21:52:32 +00:00
kyle
a186c4a4c4 Added a comment 2020-08-10 21:34:46 +00:00
Ilya_Shlyakhter
3b573c3735 Added a comment: full source tarballs for specific releases? 2020-08-10 21:14:05 +00:00
Joey Hess
c59a51a065
discard any exception thrown while trying to kill worker threads
Since there's a race here, and since Kyle saw an exception leak out,
which I have not been able to reproduce that. See my comment for what
I think might be going on.

Note that, I used tryNonAsync, because it seems a later tryNonAsync
caught the exception. I don't actually understand how it did, as I
understand exception classification, it's the data type, not the way it
was thrown. One possibility is that the async exception may have been wrapped
in some other, non-async exception, and Show displayed it the same way.
2020-08-10 16:24:51 -04:00
Joey Hess
0da4150453
Merge branch 'master' of ssh://git-annex.branchable.com 2020-08-10 16:03:59 -04:00
Joey Hess
d48049bdd8
comment 2020-08-10 16:03:46 -04:00
kyle
5236f3c773 Added a comment 2020-08-10 19:48:03 +00:00
Joey Hess
c565267118
fix order 2020-08-10 15:38:31 -04:00
Joey Hess
70d2ba03ca
Merge branch 'master' of ssh://git-annex.branchable.com 2020-08-10 15:37:54 -04:00
Joey Hess
d53b88d4e4
comment 2020-08-10 15:37:25 -04:00
Joey Hess
a354263a19
document preferred content for import/export 2020-08-10 15:35:26 -04:00
Joey Hess
9358ff8bc1
document how to build standalone tarball 2020-08-10 15:17:04 -04:00
Joey Hess
120f4f33c0
comment 2020-08-10 15:06:40 -04:00
kyle
59a8698a20 Added a comment 2020-08-10 19:01:37 +00:00
Joey Hess
0e8ec45981
add missing command name 2020-08-10 15:00:58 -04:00
kyle
8beb30e84b Added a comment 2020-08-10 18:56:03 +00:00
Joey Hess
4b05293737
comment 2020-08-10 14:42:38 -04:00
arnaud@c1d1cc612a3921dc06a417301be08a3e125478c4
dd670639ac Added a comment 2020-08-10 18:42:15 +00:00
Joey Hess
8a94f2ccd5
response 2020-08-10 14:35:20 -04:00
Joey Hess
895e341e3f
comment 2020-08-10 14:14:15 -04:00
Joey Hess
ff7d8b45ac
tag moreinfo 2020-08-10 14:12:07 -04:00
Joey Hess
23dc611a0d
Merge branch 'master' of ssh://git-annex.branchable.com 2020-08-10 14:09:24 -04:00
Joey Hess
bfdbbd0601
comment 2020-08-10 14:08:09 -04:00
Joey Hess
cd49565b71
comment 2020-08-10 13:52:18 -04:00