compare urls irrespective of downloader

importfeed --force: Avoid creating duplicates of existing already
downloaded files when yt-dlp or a special remote was used.
This commit is contained in:
Joey Hess 2024-02-02 15:50:41 -04:00
parent f1296e260e
commit 0c64cd30c2
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
3 changed files with 19 additions and 1 deletions

View file

@ -2,6 +2,8 @@ git-annex (10.20240130) UNRELEASED; urgency=medium
* importfeed: Added --scrape option, which uses yt-dlp to screen scrape
the equivilant of an RSS feed.
* importfeed --force: Avoid creating duplicates of existing
already downloaded files when yt-dlp or a special remote was used.
-- Joey Hess <id@joeyh.name> Mon, 29 Jan 2024 15:59:33 -0400

View file

@ -462,7 +462,7 @@ runDownload todownload url extension cache cv getter = do
in d </> show n ++ "_" ++ base
tryanother = makeunique (n + 1) file
alreadyexists = liftIO $ isJust <$> catchMaybeIO (R.getSymbolicLinkStatus (toRawFilePath f))
checksameurl k = ifM (elem url <$> getUrls k)
checksameurl k = ifM (elem url . map fst . map getDownloader <$> getUrls k)
( return Nothing
, tryanother
)

View file

@ -0,0 +1,16 @@
[[!comment format=mdwn
username="joey"
subject="""comment 4"""
date="2024-02-02T19:27:41Z"
content="""
Oh, but it seems that `importfeed --force` already has code to handle this.
It checks if the file already exists and has the same url recorded as the url
being imported. For some reason in that case, it says it failed to import
the url, which seems a bit odd behavior to me, but in any case it doesn't
add a "2_" file.
That works for regular rss feeds, but does not work for yt-dlp urls.
The reason is the url mangling done for yt-dlp urls (and other special
remote urls). Fixed this bug.
"""]]