webdav: store temp file in same collection as the final export location

This may work better in some webdav server that gets confused at
cross-collection renamed. I don't know, let's find out.

The only real downside of doing this is that the temp files are not all
in the top-level collection, in case an interrupted run leaves one
behind. But that does not seem especially significant.
This commit is contained in:
Joey Hess 2021-03-12 14:52:24 -04:00
parent 6481991208
commit 4f49c29d20
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
4 changed files with 64 additions and 1 deletions

View file

@ -212,7 +212,7 @@ storeExportDav :: DavHandleVar -> FilePath -> Key -> ExportLocation -> MeterUpda
storeExportDav hdl f k loc p = case exportLocation loc of
Right dest -> withDavHandle hdl $ \h -> runExport h $ \dav -> do
reqbody <- liftIO $ httpBodyStorer f p
storeHelper dav (keyTmpLocation k) dest reqbody
storeHelper dav (takeDirectory dest </> keyTmpLocation k) dest reqbody
Left err -> giveup err
retrieveExportDav :: DavHandleVar -> Key -> ExportLocation -> FilePath -> MeterUpdate -> Annex ()

View file

@ -0,0 +1,16 @@
[[!comment format=mdwn
username="joey"
subject="""comment 1"""
date="2021-03-12T18:14:59Z"
content="""
Unless this only happens on the one webdav server, my guess is
it involves the kind of weird way DAV handles collections.
In particular, the content is being written to a temp file,
which is in the webdav root, and then it runs:
inLocation src $ moveContentM (B8.fromString newurl)
where src = the webdav root. It may be that ignores the path in newurl and
just puts it into the same collection.
"""]]

View file

@ -0,0 +1,34 @@
[[!comment format=mdwn
username="joey"
subject="""comment 2"""
date="2021-03-12T18:27:26Z"
content="""
Well, I tried with the box.com webdav server, and did not reproduce the
problem there; exported files went into the proper subdirectories.
I think there's some chance that the webdav server you're trying to use is
just broken in its handling of moving from one collection to another.
Or perhaps the webdav spec can be interpreted multiple ways and this is
falling into an edge case.
Here's --debug of it working:
export box.com sub/t
[2021-03-12 14:29:33.044254918] getProps sub/t
[2021-03-12 14:29:35.56315446] putContent git-annex-webdav-tmp-SHA256E-s3--98ea6e4f216f2fb4b69fff9b3a44842c38686ca685f3f55dc48c5d3fb1107be4
[2021-03-12 14:29:37.751164566] delContent sub/t
[2021-03-12 14:29:38.695664147] getProps sub
[2021-03-12 14:29:40.638445948] moveContent git-annex-webdav-tmp-SHA256E-s3--98ea6e4f216f2fb4b69fff9b3a44842c38686ca685f3f55dc48c5d3fb1107be4 https://dav.box.com/dav/git-annex/sub/t
Does it look significantly different in your case?
We unfortunately haven't an easy way to get a HTTP trace, but this shows
effectively the api calls for the DAV libary, and the moveContent looks
like it would generate a http request like this:
MOVE /git-annex-webdav-tmp-SHA256E-s3...
Destination: https://dav.box.com/dav/git-annex/sub/t
Which seems fine, there are similar examples in the webdav spec
of moving from one directory to another.
"""]]

View file

@ -0,0 +1,13 @@
[[!comment format=mdwn
username="joey"
subject="""comment 3"""
date="2021-03-12T18:48:38Z"
content="""
If the problem does involve moving between collections, it could
avoid the problem by storing the temp file into the subdirectory in the first
place, and only renaming it to its final name once transferred.
I've implemented that and committed it along with this comment,
and it works tested against box.com again, can you try it with
the 4shared server?
"""]]