Commit graph

13 commits

Author SHA1 Message Date
Joey Hess
3290a09a70
filter out control characters in warning messages
Converted warning and similar to use StringContainingQuotedPath. Most
warnings are static strings, some do refer to filepaths that need to be
quoted, and others don't need quoting.

Note that, since quote filters out control characters of even
UnquotedString, this makes all warnings safe, even when an attacker
sneaks in a control character in some other way.

When json is being output, no quoting is done, since json gets its own
quoting.

This does, as a side effect, make warning messages in json output not
be indented. The indentation is only needed to offset warning messages
underneath the display of the file they apply to, so that's ok.

Sponsored-by: Brett Eisenberg on Patreon
2023-04-10 15:55:44 -04:00
Yaroslav Halchenko
84b0a3707a
Apply codespell -w throughout 2023-03-17 15:14:58 -04:00
Joey Hess
63839532c9
remove uses of warningIO
It's not concurrent-output safe, and doesn't support
--json-error-messages.

Using Annex.makeRunner is a bit scary, because what if it's run in a
different thread from an active annex action? Normally the same Annex
state is not used concurrently in several threads, and it's not designed
to be fully concurrency safe. (Annex.Concurrent exists to deal with
that.) I think it will be ok in these simple cases though. Eg,
when buffering a warning message to json, Annex.changeState is used,
and it modifies the MVar in a concurrency safe way.

The only warningIO remaining is not a problem.
2020-12-02 14:57:43 -04:00
Joey Hess
7776677a5f
Fix hang on shutdown of external special remote using ASYNC protocol extension.
Reversion introduced in version 8.20201007, one release after the 1st
release with the extension.

Surprisingly, hClose can hang if another thread is reading from the
handle. This is because it uses takeMVar.

The use of cancel here does mean that, if receiveMessageAddonProcess
or Remote.External.AsyncExtension.receiveloop allocated some resource in
a non-async-exception safe way, they might not get a chance to clean it up.
They do not appear to, and anyway, this only happens when git-annex is
shutting down, so any recource that did leak would not be a problem.

This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
2020-11-30 13:04:02 -04:00
Joey Hess
ad64079b44
fix some warnings 2020-08-15 14:33:18 -04:00
Joey Hess
198b709561
switch to TMVars for thread safety when using the async extension
TVars were not updated atomically, which was ok when each thread got its
own External that was the only thing using these TVars. But, with the
async extension, several External instances can share the same var, so
it needs to be a TMVar to avoid read/write conflicts.

In particular, this makes PREPARE only be sent once.
2020-08-14 14:50:09 -04:00
Joey Hess
7da2d4dd2d
one jobid per thread
And, relay ERROR on to all listening threads.
2020-08-14 14:24:46 -04:00
Joey Hess
72561563d9
rethought the async protocol some more
Moving jobid generation to the git-annex side lets it be simplified a
lot.

Note that it will also be possible to generate one jobid per connection,
rather than a new job per request. That will make overflow not an issue,
and will avoid some work, and will simplify some of the code.
2020-08-13 20:18:06 -04:00
Joey Hess
59cbb42ee2
async proto fully tested and working
Including with a concurrent capable remote program.

However, this is not quite ready to merge, there's a TODO in the code.
2020-08-13 16:22:11 -04:00
Joey Hess
7546e686a2
async proto basically working
Simplified the protocol by removing END-ASYNC.

There's a STM crash when a non-async protocol message is sent, which
needs to be fixed.
2020-08-13 15:52:12 -04:00
Joey Hess
c9e8cafb98
further work on external async relay 2020-08-12 16:25:53 -04:00
Joey Hess
15706e6991
relayer receive loop is done
Receive loop looks right. Still need the send loop.

And, a complication is that some messages git-annex
sends need to be wrapped in REPLY_ASYNC, while others
do not. So will probably need to split externalSend
into two.
2020-08-12 15:56:58 -04:00
Joey Hess
06a4ab39fa
wip external remote async protocol extension 2020-08-12 15:17:53 -04:00