From 6e3c9bea2e33249efe4349365787f887e5ba059b Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Thu, 22 Sep 2022 14:35:20 -0400 Subject: [PATCH] drain transferrer read handle when shutting it down Fixes updating git index file after getting an unlocked file when annex.stalldetection is set. The transferrer may want to send additional protocol messages when it's shut down. Closing the read handle prevented it from doing that, and caused it to crash rather than cleanly shutting down. Draining the handle without processing the protocol seemed ok to do, because anything it outputs is going to be some side message displayed at shutdown. Displaying those once per transferrer process that is running seems unncessary. Sponsored-by: Dartmouth College's DANDI project --- Annex/TransferrerPool.hs | 8 ++++- CHANGELOG | 2 ++ ..._5c03731700b2a025755fb06c9a96f331._comment | 30 +++++++++++++++++++ 3 files changed, 39 insertions(+), 1 deletion(-) create mode 100644 doc/bugs/reports_file___34__modified__34___whenever_it_is_not/comment_13_5c03731700b2a025755fb06c9a96f331._comment diff --git a/Annex/TransferrerPool.hs b/Annex/TransferrerPool.hs index 513dc1f28e..b1046a9590 100644 --- a/Annex/TransferrerPool.hs +++ b/Annex/TransferrerPool.hs @@ -217,9 +217,15 @@ mkTransferrer signalactonsvar (RunTransferrer program params batchmaker) = do , transferrerWrite = writeh , transferrerHandle = ph , transferrerShutdown = do - hClose readh + -- The transferrer may write to stdout + -- as it's shutting down, so don't close + -- the readh right away. Instead, drain + -- anything sent to it. + drainer <- async $ void $ hGetContents readh hClose writeh void $ waitForProcess ph + wait drainer + hClose readh unregistersignalprop } diff --git a/CHANGELOG b/CHANGELOG index eebd73c927..e74887764c 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -29,6 +29,8 @@ git-annex (10.20220823) UNRELEASED; urgency=medium directory to somewhere else. (Introduced in version 10.20220525) * Improved handling of --time-limit when combined with -J + * Fix updating git index file after getting an unlocked file + when annex.stalldetection is set. -- Joey Hess Mon, 29 Aug 2022 15:03:04 -0400 diff --git a/doc/bugs/reports_file___34__modified__34___whenever_it_is_not/comment_13_5c03731700b2a025755fb06c9a96f331._comment b/doc/bugs/reports_file___34__modified__34___whenever_it_is_not/comment_13_5c03731700b2a025755fb06c9a96f331._comment new file mode 100644 index 0000000000..7574901278 --- /dev/null +++ b/doc/bugs/reports_file___34__modified__34___whenever_it_is_not/comment_13_5c03731700b2a025755fb06c9a96f331._comment @@ -0,0 +1,30 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 13""" + date="2022-09-22T18:16:22Z" + content=""" +Avoided the early stdout handle close, and that did fix this bug as +reported. + +The related problems I identified in comment #12 are still unfixed, so +leaving this open for now. + +I think what ought to be done to wrap this up is make restagePointerFile +record the files that need to be restaged in a log file. Then at shutdown, +git-annex can read the log file, and restage everything listed in it. +This will solve multiple problems: + +* When a previous git-annex process was interrupted after a get/drop of an + unlocked file, the file will be in the log, so git-annex can notice + that and handle the restaging. +* When a stalled `git-annex transferrer` is killed, the parent git-annex + will read the log and handle the restaging that it was not able to do. +* When multiple processes are trying to restage files at the same time, + an exclusive lock can be used to make only one of them run, and it can + handle restaging the files that the others have recorded in the log too. +* As a bonus, in the situations where git-annex is legitimately unable to + restage files, it can still record them to be restaged later. And the + "only a cosmetic problem" message can tell the user to run a single + simple git-annex command, rather than a complicated + `git update-index` command per file. +"""]]