compare urls irrespective of downloader
importfeed --force: Avoid creating duplicates of existing already downloaded files when yt-dlp or a special remote was used.
This commit is contained in:
parent
f1296e260e
commit
0c64cd30c2
3 changed files with 19 additions and 1 deletions
|
@ -2,6 +2,8 @@ git-annex (10.20240130) UNRELEASED; urgency=medium
|
||||||
|
|
||||||
* importfeed: Added --scrape option, which uses yt-dlp to screen scrape
|
* importfeed: Added --scrape option, which uses yt-dlp to screen scrape
|
||||||
the equivilant of an RSS feed.
|
the equivilant of an RSS feed.
|
||||||
|
* importfeed --force: Avoid creating duplicates of existing
|
||||||
|
already downloaded files when yt-dlp or a special remote was used.
|
||||||
|
|
||||||
-- Joey Hess <id@joeyh.name> Mon, 29 Jan 2024 15:59:33 -0400
|
-- Joey Hess <id@joeyh.name> Mon, 29 Jan 2024 15:59:33 -0400
|
||||||
|
|
||||||
|
|
|
@ -462,7 +462,7 @@ runDownload todownload url extension cache cv getter = do
|
||||||
in d </> show n ++ "_" ++ base
|
in d </> show n ++ "_" ++ base
|
||||||
tryanother = makeunique (n + 1) file
|
tryanother = makeunique (n + 1) file
|
||||||
alreadyexists = liftIO $ isJust <$> catchMaybeIO (R.getSymbolicLinkStatus (toRawFilePath f))
|
alreadyexists = liftIO $ isJust <$> catchMaybeIO (R.getSymbolicLinkStatus (toRawFilePath f))
|
||||||
checksameurl k = ifM (elem url <$> getUrls k)
|
checksameurl k = ifM (elem url . map fst . map getDownloader <$> getUrls k)
|
||||||
( return Nothing
|
( return Nothing
|
||||||
, tryanother
|
, tryanother
|
||||||
)
|
)
|
||||||
|
|
|
@ -0,0 +1,16 @@
|
||||||
|
[[!comment format=mdwn
|
||||||
|
username="joey"
|
||||||
|
subject="""comment 4"""
|
||||||
|
date="2024-02-02T19:27:41Z"
|
||||||
|
content="""
|
||||||
|
Oh, but it seems that `importfeed --force` already has code to handle this.
|
||||||
|
It checks if the file already exists and has the same url recorded as the url
|
||||||
|
being imported. For some reason in that case, it says it failed to import
|
||||||
|
the url, which seems a bit odd behavior to me, but in any case it doesn't
|
||||||
|
add a "2_" file.
|
||||||
|
|
||||||
|
That works for regular rss feeds, but does not work for yt-dlp urls.
|
||||||
|
|
||||||
|
The reason is the url mangling done for yt-dlp urls (and other special
|
||||||
|
remote urls). Fixed this bug.
|
||||||
|
"""]]
|
Loading…
Add table
Add a link
Reference in a new issue