webdav: store temp file in same collection as the final export location
This may work better in some webdav server that gets confused at cross-collection renamed. I don't know, let's find out. The only real downside of doing this is that the temp files are not all in the top-level collection, in case an interrupted run leaves one behind. But that does not seem especially significant.
This commit is contained in:
parent
6481991208
commit
4f49c29d20
4 changed files with 64 additions and 1 deletions
|
@ -212,7 +212,7 @@ storeExportDav :: DavHandleVar -> FilePath -> Key -> ExportLocation -> MeterUpda
|
||||||
storeExportDav hdl f k loc p = case exportLocation loc of
|
storeExportDav hdl f k loc p = case exportLocation loc of
|
||||||
Right dest -> withDavHandle hdl $ \h -> runExport h $ \dav -> do
|
Right dest -> withDavHandle hdl $ \h -> runExport h $ \dav -> do
|
||||||
reqbody <- liftIO $ httpBodyStorer f p
|
reqbody <- liftIO $ httpBodyStorer f p
|
||||||
storeHelper dav (keyTmpLocation k) dest reqbody
|
storeHelper dav (takeDirectory dest </> keyTmpLocation k) dest reqbody
|
||||||
Left err -> giveup err
|
Left err -> giveup err
|
||||||
|
|
||||||
retrieveExportDav :: DavHandleVar -> Key -> ExportLocation -> FilePath -> MeterUpdate -> Annex ()
|
retrieveExportDav :: DavHandleVar -> Key -> ExportLocation -> FilePath -> MeterUpdate -> Annex ()
|
||||||
|
|
|
@ -0,0 +1,16 @@
|
||||||
|
[[!comment format=mdwn
|
||||||
|
username="joey"
|
||||||
|
subject="""comment 1"""
|
||||||
|
date="2021-03-12T18:14:59Z"
|
||||||
|
content="""
|
||||||
|
Unless this only happens on the one webdav server, my guess is
|
||||||
|
it involves the kind of weird way DAV handles collections.
|
||||||
|
|
||||||
|
In particular, the content is being written to a temp file,
|
||||||
|
which is in the webdav root, and then it runs:
|
||||||
|
|
||||||
|
inLocation src $ moveContentM (B8.fromString newurl)
|
||||||
|
|
||||||
|
where src = the webdav root. It may be that ignores the path in newurl and
|
||||||
|
just puts it into the same collection.
|
||||||
|
"""]]
|
|
@ -0,0 +1,34 @@
|
||||||
|
[[!comment format=mdwn
|
||||||
|
username="joey"
|
||||||
|
subject="""comment 2"""
|
||||||
|
date="2021-03-12T18:27:26Z"
|
||||||
|
content="""
|
||||||
|
Well, I tried with the box.com webdav server, and did not reproduce the
|
||||||
|
problem there; exported files went into the proper subdirectories.
|
||||||
|
|
||||||
|
I think there's some chance that the webdav server you're trying to use is
|
||||||
|
just broken in its handling of moving from one collection to another.
|
||||||
|
Or perhaps the webdav spec can be interpreted multiple ways and this is
|
||||||
|
falling into an edge case.
|
||||||
|
|
||||||
|
Here's --debug of it working:
|
||||||
|
|
||||||
|
export box.com sub/t
|
||||||
|
[2021-03-12 14:29:33.044254918] getProps sub/t
|
||||||
|
[2021-03-12 14:29:35.56315446] putContent git-annex-webdav-tmp-SHA256E-s3--98ea6e4f216f2fb4b69fff9b3a44842c38686ca685f3f55dc48c5d3fb1107be4
|
||||||
|
[2021-03-12 14:29:37.751164566] delContent sub/t
|
||||||
|
[2021-03-12 14:29:38.695664147] getProps sub
|
||||||
|
[2021-03-12 14:29:40.638445948] moveContent git-annex-webdav-tmp-SHA256E-s3--98ea6e4f216f2fb4b69fff9b3a44842c38686ca685f3f55dc48c5d3fb1107be4 https://dav.box.com/dav/git-annex/sub/t
|
||||||
|
|
||||||
|
Does it look significantly different in your case?
|
||||||
|
|
||||||
|
We unfortunately haven't an easy way to get a HTTP trace, but this shows
|
||||||
|
effectively the api calls for the DAV libary, and the moveContent looks
|
||||||
|
like it would generate a http request like this:
|
||||||
|
|
||||||
|
MOVE /git-annex-webdav-tmp-SHA256E-s3...
|
||||||
|
Destination: https://dav.box.com/dav/git-annex/sub/t
|
||||||
|
|
||||||
|
Which seems fine, there are similar examples in the webdav spec
|
||||||
|
of moving from one directory to another.
|
||||||
|
"""]]
|
|
@ -0,0 +1,13 @@
|
||||||
|
[[!comment format=mdwn
|
||||||
|
username="joey"
|
||||||
|
subject="""comment 3"""
|
||||||
|
date="2021-03-12T18:48:38Z"
|
||||||
|
content="""
|
||||||
|
If the problem does involve moving between collections, it could
|
||||||
|
avoid the problem by storing the temp file into the subdirectory in the first
|
||||||
|
place, and only renaming it to its final name once transferred.
|
||||||
|
|
||||||
|
I've implemented that and committed it along with this comment,
|
||||||
|
and it works tested against box.com again, can you try it with
|
||||||
|
the 4shared server?
|
||||||
|
"""]]
|
Loading…
Add table
Add a link
Reference in a new issue