avoid populating proxy's object file when storing on special remote

Now that storeKey can have a different object file passed to it, this
complication is not needed. This avoids a lot of strange situations,
and will also be needed if streaming is eventually supported.
This commit is contained in:
Joey Hess 2024-07-01 10:53:49 -04:00
parent 8b5fc94d50
commit dce3848ad8
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
2 changed files with 12 additions and 53 deletions

View file

@ -149,57 +149,27 @@ proxySpecialRemote protoversion r ihdl ohdl owaitv endv = go
a (toRawFilePath tmpdir P.</> keyFile k) a (toRawFilePath tmpdir P.</> keyFile k)
proxyput af k = do proxyput af k = do
-- In order to send to the special remote, the key will liftIO $ sendmessage $ PUT_FROM (Offset 0)
-- need to be inserted into the object directory. withproxytmpfile k $ \tmpfile -> do
-- It will be dropped again afterwards. Unless it's already let store = tryNonAsync (Remote.storeKey r k af (Just (decodeBS tmpfile)) nullMeterUpdate) >>= \case
-- present there. Right () -> liftIO $ sendmessage SUCCESS
ifM (inAnnex k)
( tryNonAsync (Remote.storeKey r k af Nothing nullMeterUpdate) >>= \case
Right () -> liftIO $ sendmessage ALREADY_HAVE
Left err -> liftIO $ propagateerror err Left err -> liftIO $ propagateerror err
, do
liftIO $ sendmessage $ PUT_FROM (Offset 0)
ifM receivedata
( do
tryNonAsync (Remote.storeKey r k af Nothing nullMeterUpdate) >>= \case
Right () -> do
depopulateobjectfile
liftIO $ sendmessage SUCCESS
Left err -> do
depopulateobjectfile
liftIO $ propagateerror err
, liftIO $ sendmessage FAILURE
)
)
where
receivedata = withproxytmpfile k $ \tmpfile ->
liftIO receivemessage >>= \case liftIO receivemessage >>= \case
Just (DATA (Len _)) -> do Just (DATA (Len _)) -> do
b <- liftIO receivebytestring b <- liftIO receivebytestring
liftIO $ L.writeFile (fromRawFilePath tmpfile) b liftIO $ L.writeFile (fromRawFilePath tmpfile) b
-- Signal that the whole bytestring -- Signal that the whole bytestring
-- has been stored. -- has been received.
liftIO $ atomically $ putTMVar owaitv () liftIO $ atomically $ putTMVar owaitv ()
if protoversion > ProtocolVersion 1 if protoversion > ProtocolVersion 1
then do then liftIO receivemessage >>= \case
liftIO receivemessage >>= \case Just (VALIDITY Valid) ->
Just (VALIDITY Valid) -> store
populateobjectfile tmpfile Just (VALIDITY Invalid) ->
Just (VALIDITY Invalid) -> return False return ()
_ -> giveup "protocol error" _ -> giveup "protocol error"
else populateobjectfile tmpfile else store
_ -> giveup "protocol error" _ -> giveup "protocol error"
populateobjectfile tmpfile =
getViaTmpFromDisk Remote.RetrievalAllKeysSecure Remote.DefaultVerify k af $ \dest -> do
unVerified $ do
liftIO $ renameFile
(fromRawFilePath tmpfile)
(fromRawFilePath dest)
return True
depopulateobjectfile = void $ tryNonAsync $
lockContentForRemoval k noop removeAnnex
proxyget offset af k = withproxytmpfile k $ \tmpfile -> do proxyget offset af k = withproxytmpfile k $ \tmpfile -> do
-- Don't verify the content from the remote, -- Don't verify the content from the remote,

View file

@ -34,17 +34,6 @@ For June's work on [[design/passthrough_proxy]], remaining todos:
* If GET from a proxied special remote sends an ERROR with a message * If GET from a proxied special remote sends an ERROR with a message
from the special remote, currently the user sees "protocol error". from the special remote, currently the user sees "protocol error".
* convert Remote.storeKey to take the path of the object file to send.
It's too ugly that PUT to a proxied special remote currently has to
temporarily populate the proxy's annex object file. There are too many
ways that could lead to surprising behavior, like an interrupted PUT
leaving it populated, or simulantaneous PUTs.
* PUT to a proxied special remote, in the case where the proxy contains
the key, and the special remote is not accessible, sends back ERROR
rather than PUT-FROM or ALREADY-HAVE. Verify that the client processes
that ok and displays it to the user.
* Streaming download from proxied special remotes. See design. * Streaming download from proxied special remotes. See design.
* Check annex.diskreserve when proxying for special remotes. * Check annex.diskreserve when proxying for special remotes.