Fix a hang when using git-annex with an old openssh 7.2p2

Which had some weird inheriting of ssh FDs by sshd.

Bug was introduced in git-annex version 7.20200202.7.

CHANGELOG

@@ -4,6 +4,9 @@ git-annex (8.20200720.2) UNRELEASED; urgency=medium
     like git-annex add with -J. Bug was introduced as part of a different FD
     leak fix in version 6.20160318.
   * Support building with dlist-1.0
+  * Fix a hang when using git-annex with an old openssh 7.2p2, which had
+    some weird inheriting of ssh FDs by sshd. Bug was introduced in
+    git-annex version 7.20200202.7.
 
  -- Joey Hess <id@joeyh.name>  Tue, 21 Jul 2020 12:58:30 -0400
 

Git/Config.hs

@@ -22,6 +22,7 @@ import Git.Types
 import qualified Git.Command
 import qualified Git.Construct
 import Utility.UserInfo
+import Utility.ThreadScheduler
 
 {- Returns a single git config setting, or a fallback value if not set. -}
 get :: ConfigKey -> ConfigValue -> Repo -> ConfigValue
@@ -214,10 +215,17 @@ fromPipe r cmd params st = tryNonAsync $ withCreateProcess p go
 		{ std_out = CreatePipe
 		, std_err = CreatePipe
 		}
-	go _ (Just hout) (Just herr) pid = do
+	go _ (Just hout) (Just herr) pid =
-		(val, err) <- concurrently 
+		withAsync (S.hGetContents herr) $ \errreader -> do
-			(S.hGetContents hout)
+			val <- S.hGetContents hout
-			(S.hGetContents herr)
+			-- In case the process exits while something else,
+			-- like a background process, keeps the stderr handle
+			-- open, don't block forever waiting for stderr.
+			-- A few seconds after finishing reading stdout
+			-- should get any error message.
+			err <- either id id <$> 
+				wait errreader
+					`race` (threadDelaySeconds (Seconds 2) >> return mempty)
 			forceSuccessProcess p pid
 			r' <- store val st r
 			return (r', val, err)

doc/bugs/Recent_hang_with_rsync_remote_with_older_systems___40__Xenial__44___Jessie__41__/comment_7_815bf1b8985a49a0fafbcddc7c96f9d6._comment

@@ -0,0 +1,12 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 7"""
+ date="2020-07-21T19:59:40Z"
+ content="""
+Reproducer script did not work for me.
+
+But in the meantime, 4c9ad1de4 has gotten into the built that script uses,
+so whatever about that commit fixes or masks the problem, it seems.
+
+(The other stall's reproducer script did still work.)
+"""]]

doc/bugs/SSH-based_git-annex-init_hang_on_older_systems___40__Xenial__44___Jessie__41__/comment_4_c480f7fad76759742bbdc4837d8c0c36._comment

@@ -0,0 +1,31 @@
+[[!comment format=mdwn
+ username="joey"
+ subject="""comment 4"""
+ date="2020-07-21T20:02:22Z"
+ content="""
+Reproduced this with the script.
+
+	1798550 pts/0    S+     0:00              |   \_ /root/git-annex.linux/exe/git --library-path /root/git-annex.linux//lib:/root/git-annex.linux//usr/lib/x86_64-linux-gnu/gconv:/root/git-annex.linux//usr/lib/x86_64-linux-gnu/audit:/root/git-annex.linux//etc/ld.so.conf.d:/root/git-annex.linux//lib64:/root/git-annex.linux//lib/x86_64-linux-gnu:/root/git-annex.linux//usr/lib/x86_64-linux-gnu: /root/git-annex.linux/shimmed/git/git -C clone annex init --debug
+	1798579 pts/0    Sl+    0:00              |       \_ /root/git-annex.linux/exe/git-annex --library-path /root/git-annex.linux//lib:/root/git-annex.linux//usr/lib/x86_64-linux-gnu/gconv:/root/git-annex.linux//usr/lib/x86_64-linux-gnu/audit:/root/git-annex.linux//etc/ld.so.conf.d:/root/git-annex.linux//lib64:/root/git-annex.linux//lib/x86_64-linux-gnu:/root/git-annex.linux//usr/lib/x86_64-linux-gnu: /root/git-annex.linux/shimmed/git-annex/git-annex init --debug
+	1798596 pts/0    S+     0:00              |           \_ /root/git-annex.linux/exe/git --library-path /root/git-annex.linux//lib:/root/git-annex.linux//usr/lib/x86_64-linux-gnu/gconv:/root/git-annex.linux//usr/lib/x86_64-linux-gnu/audit:/root/git-annex.linux//etc/ld.so.conf.d:/root/git-annex.linux//lib64:/root/git-annex.linux//lib/x86_64-linux-gnu:/root/git-annex.linux//usr/lib/x86_64-linux-gnu: /root/git-annex.linux/shimmed/git/git --git-dir=.git --work-tree=. --literal-pathspecs cat-file --batch
+	1798597 pts/0    S+     0:00              |           \_ /root/git-annex.linux/exe/git --library-path /root/git-annex.linux//lib:/root/git-annex.linux//usr/lib/x86_64-linux-gnu/gconv:/root/git-annex.linux//usr/lib/x86_64-linux-gnu/audit:/root/git-annex.linux//etc/ld.so.conf.d:/root/git-annex.linux//lib64:/root/git-annex.linux//lib/x86_64-linux-gnu:/root/git-annex.linux//usr/lib/x86_64-linux-gnu: /root/git-annex.linux/shimmed/git/git --git-dir=.git --work-tree=. --literal-pathspecs cat-file --batch-check=%(objectname) %(objecttype) %(objectsize)
+	1798598 pts/0    S+     0:00              |           \_ /root/git-annex.linux/exe/git --library-path /root/git-annex.linux//lib:/root/git-annex.linux//usr/lib/x86_64-linux-gnu/gconv:/root/git-annex.linux//usr/lib/x86_64-linux-gnu/audit:/root/git-annex.linux//etc/ld.so.conf.d:/root/git-annex.linux//lib64:/root/git-annex.linux//lib/x86_64-linux-gnu:/root/git-annex.linux//usr/lib/x86_64-linux-gnu: /root/git-annex.linux/shimmed/git/git --git-dir=.git --work-tree=. --literal-pathspecs hash-object -w --stdin-paths --no-filters
+	1798628 pts/0    Z+     0:00              |           \_ [ssh] <defunct>
+	1798267 ?        Ss     0:00              \_ /usr/sbin/sshd
+	1798629 ?        Ss     0:00              |   \_ sshd: root@notty
+
+I killed 1798629 and git-annex stopped hanging. That seems to confirm my
+theory that sshd has inherited the stderr handle that git-annex is waiting
+for input on. That process had these FDs:
+
+	lrwx------ 1 root root 64 Jul 21 16:04 0 -> /dev/null
+	lrwx------ 1 root root 64 Jul 21 16:04 1 -> /dev/null
+	lrwx------ 1 root root 64 Jul 21 16:04 2 -> /dev/null
+	lrwx------ 1 root root 64 Jul 21 16:04 3 -> socket:[15959002]
+	lr-x------ 1 root root 64 Jul 21 16:04 4 -> pipe:[15956866]
+	l-wx------ 1 root root 64 Jul 21 16:04 5 -> pipe:[15956866]
+
+So probably 4 or 5 was connected to git-annex on the other end.
+
+I've applied my patch, to fix this one.
+"""]]