incremental verify for byteRetriever special remotes

Several special remotes verify content while it is being retrieved,
avoiding a separate checksum pass. They are: S3, bup, ddar, and
gcrypt (with a local repository).

Not done when using chunking, yet.

Complicated by Retriever needing to change to be polymorphic. Which in turn
meant RankNTypes is needed, and also needed some code changes. The
change in Remote.External does not change behavior at all but avoids
the type checking failing because of a "rigid, skolem type" which
"would escape its scope". So I refactored slightly to make the type
checker's job easier there.

Unfortunately, directory uses fileRetriever (except when chunked),
so it is not amoung the improved ones. Fixing that would need a way for
FileRetriever to return a Verification. But, since the file retrieved
may be encrypted or chunked, it would be extra work to always
incrementally checksum the file while retrieving it. Hm.

Some other special remotes use fileRetriever, and so don't get incremental
verification, but could be converted to byteRetriever later. One is
GitLFS, which uses downloadConduit, which writes to the file, so could
verify as it goes. Other special remotes like web could too, but don't
use Remote.Helper.Special and so will need to be addressed separately.

Sponsored-by: Dartmouth College's DANDI project

Remote/External.hs

@@ -7,6 +7,7 @@
 
 {-# LANGUAGE OverloadedStrings #-}
 {-# LANGUAGE BangPatterns #-}
+{-# LANGUAGE RankNTypes #-}
 
 module Remote.External (remote) where
 
@@ -66,17 +67,19 @@ gen r u rc gc rs
 	| externaltype == "readonly" = do
 		c <- parsedRemoteConfig remote rc
 		cst <- remoteCost gc expensiveRemoteCost
-		mk c cst GloballyAvailable
-			readonlyStorer
-			retrieveUrl
-			readonlyRemoveKey
-			checkKeyUrl
+		let rmt = mk c cst GloballyAvailable
 			Nothing
 			(externalInfo externaltype)
 			Nothing
 			Nothing
 			exportUnsupported
 			exportUnsupported
+		return $ Just $ specialRemote c
+			readonlyStorer
+			retrieveUrl
+			readonlyRemoveKey
+			checkKeyUrl
+			rmt
 	| otherwise = do
 		c <- parsedRemoteConfig remote rc
 		external <- newExternal externaltype (Just u) c (Just gc)
@@ -103,20 +106,22 @@ gen r u rc gc rs
 		let cheapexportsupported = if exportsupported
 			then exportIsSupported
 			else exportUnsupported
-		mk c cst avail
-			(storeKeyM external)
-			(retrieveKeyFileM external)
-			(removeKeyM external)
-			(checkPresentM external)
+		let rmt = mk c cst avail
 			(Just (whereisKeyM external))
 			(getInfoM external)
 			(Just (claimUrlM external))
 			(Just (checkUrlM external))
 			exportactions
 			cheapexportsupported
+		return $ Just $ specialRemote c
+			(storeKeyM external)
+			(retrieveKeyFileM external)
+			(removeKeyM external)
+			(checkPresentM external)
+			rmt
   where
-	mk c cst avail tostore toretrieve toremove tocheckkey towhereis togetinfo toclaimurl tocheckurl exportactions cheapexportsupported = do
-		let rmt = Remote
+	mk c cst avail towhereis togetinfo toclaimurl tocheckurl exportactions cheapexportsupported =
+		Remote
 			{ uuid = u
 			, cost = cst
 			, name = Git.repoDescribe r
@@ -154,12 +159,6 @@ gen r u rc gc rs
 			, checkUrl = tocheckurl
 			, remoteStateHandle = rs
 			}
-		return $ Just $ specialRemote c
-			tostore
-			toretrieve
-			toremove
-			tocheckkey
-			rmt
 	externaltype = fromMaybe (giveup "missing externaltype") (remoteAnnexExternalType gc)
 
 externalSetup :: SetupStage -> Maybe UUID -> Maybe CredPair -> RemoteConfig -> RemoteGitConfig -> Annex (RemoteConfig, UUID)