add remote state logs

This allows a remote to store a piece of arbitrary state associated with a
key. This is needed to support Tahoe, where the file-cap is calculated from
the data stored in it, and used to retrieve a key later. Glacier also would
be much improved by using this.

GETSTATE and SETSTATE are added to the external special remote protocol.

Note that the state is left as-is even when a key is removed from a remote.
It's up to the remote to decide when it wants to clear the state.

The remote state log, $KEY.log.rmt, is a UUID-based log. However,
rather than using the old UUID-based log format, I created a new variant
of that format. The new varient is more space efficient (since it lacks the
"timestamp=" hack, and easier to parse (and the parser doesn't mess with
whitespace in the value), and avoids compatability cruft in the old one.

This seemed worth cleaning up for these new files, since there could be a
lot of them, while before UUID-based logs were only used for a few log
files at the top of the git-annex branch. The transition code has also
been updated to handle these new UUID-based logs.

This commit was sponsored by Daniel Hofer.

Remote/External.hs

@@ -19,6 +19,7 @@ import Crypto
 import Utility.Metered
 import Logs.Transfer
 import Logs.PreferredContent.Raw
+import Logs.RemoteState
 import Config.Cost
 import Annex.Content
 import Annex.UUID
@@ -235,6 +236,12 @@ handleRequest' lck external req mp responsehandler
 		expr <- fromMaybe "" . M.lookup (externalUUID external)
 			<$> preferredContentMapRaw
 		send $ VALUE expr
+	handleRemoteRequest (SETSTATE key state) =
+		setRemoteState (externalUUID external) key state
+	handleRemoteRequest (GETSTATE key) = do
+		state <- fromMaybe ""
+			<$> getRemoteState (externalUUID external) key
+		send $ VALUE state
 	handleRemoteRequest (VERSION _) =
 		sendMessage lck external $ ERROR "too late to send VERSION"
 

Remote/External/Types.hs

@@ -170,6 +170,8 @@ data RemoteRequest
 	| GETUUID
 	| SETWANTED PreferredContentExpression
 	| GETWANTED
+	| SETSTATE Key String
+	| GETSTATE Key
 	deriving (Show)
 
 instance Receivable RemoteRequest where
@@ -183,6 +185,8 @@ instance Receivable RemoteRequest where
 	parseCommand "GETUUID" = parse0 GETUUID
 	parseCommand "SETWANTED" = parse1 SETWANTED
 	parseCommand "GETWANTED" = parse0 GETWANTED
+	parseCommand "SETSTATE" = parse2 SETSTATE
+	parseCommand "GETSTATE" = parse1 GETSTATE
 	parseCommand _ = parseFail
 
 -- Responses to RemoteRequest.