mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

add Utility.HtmlDetect

Browse source 
This will be used in youtube-dl integration, to tell when a html page has
been downloaded by addurl, in which case it is worth running youtube-dl
to see if it can extract media from it.

tagsoup is an almost free dependency, because yesod depends on it.
So, this only really adds a dep when git-annex is built without the
webapp.

I'd like this to as closely as possible match how browsers decide if a
page is html or not. Unfortunately, that is fairly heuristic, in order
to support malformed html. And, we don't want to falsely detect
something as html just because it has something that looks like a html
tag embedded somewhere in it. Probably any major video hosting site is
going to be serving html documents that at least start with a <html>
tag, so requiring that or a DOCTYPE should be good enough.

This commit was sponsored by Jeff Goeke-Smith on Patreon.
This commit is contained in:
Joey Hess 2017-11-28 12:50:30 -04:00
parent c37838d1b7
commit 57b4c5bdff
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
4 changed files with 46 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

1
debian/control vendored
View file

@ -25,6 +25,7 @@ Build-Depends:
libghc-dlist-dev,
libghc-uuid-dev,
libghc-aeson-dev,
libghc-tagsoup-dev,
libghc-unordered-containers-dev,
libghc-ifelse-dev,
libghc-bloomfilter-dev,