mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
git-annex/Utility
History
Joey Hess 57b4c5bdff
add Utility.HtmlDetect 
This will be used in youtube-dl integration, to tell when a html page has
been downloaded by addurl, in which case it is worth running youtube-dl
to see if it can extract media from it.

tagsoup is an almost free dependency, because yesod depends on it.
So, this only really adds a dep when git-annex is built without the
webapp.

I'd like this to as closely as possible match how browsers decide if a
page is html or not. Unfortunately, that is fairly heuristic, in order
to support malformed html. And, we don't want to falsely detect
something as html just because it has something that looks like a html
tag embedded somewhere in it. Probably any major video hosting site is
going to be serving html documents that at least start with a <html>
tag, so requiring that or a DOCTYPE should be good enough.

This commit was sponsored by Jeff Goeke-Smith on Patreon.
2017-11-28 13:03:11 -04:00
..
DirWatcher use unix-compat 0.5 on windows 2017-11-14 14:00:24 -04:00
LockFile remove temp file in failure case 2017-06-06 14:23:33 -04:00
LockPool Fix transfer log file locking problem when running concurrent transfers. 2017-05-25 17:40:23 -04:00
Process merge from propellor 2015-10-28 00:18:01 -04:00
Scheduled reorg quickcheck to a separate module 2015-11-17 15:49:22 -04:00
Touch move old ghc compat code into separate module; eliminate WITH_CLIBS 2016-02-15 11:47:33 -04:00
Applicative.hs update my email address and homepage url 2015-01-21 12:50:09 -04:00
AuthToken.hs unified AuthToken type between webapp and tor 2016-11-22 14:18:34 -04:00
Base64.hs avoid throwing exception when String is not encoded using the filesystem encoding 2015-08-12 10:57:48 -04:00
Batch.hs update my email address and homepage url 2015-01-21 12:50:09 -04:00
Bloom.hs fix bug in back-compat ifdef 2015-09-23 13:09:08 -04:00
CoProcess.hs Always use filesystem encoding for all file and handle reads and writes. 2016-12-24 14:46:31 -04:00
CopyFile.hs hard links on windows 2016-04-08 15:25:32 -04:00
Daemon.hs terminateProcessId renamed 2017-10-25 19:46:28 -04:00
Data.hs disable horrible tab warning, needed in every file that Setup.hs pulls in 2015-05-10 16:31:50 -04:00
DataUnits.hs adeiu, MissingH 2017-05-16 01:03:52 -04:00
DBus.hs Revert "When listing DBus services, also list activatable services." 2015-06-02 14:38:24 -04:00
Directory.hs use unix-compat 0.5 on windows 2017-11-14 14:00:24 -04:00
DirWatcher.hs disable closingTracked on OSX 2017-06-09 14:18:58 -04:00
DiskFree.hs build without disk-free-space on android 2016-03-08 02:45:10 -04:00
Dot.hs update my email address and homepage url 2015-01-21 12:50:09 -04:00
DottedVersion.hs Some optimisations to string splitting code. 2017-01-31 19:06:22 -04:00
Env.hs still can't express custom-setup deps 2017-11-14 14:59:51 -04:00
Exception.hs allow Utility.Exception to still be used when not building with cabal 2016-11-15 22:01:55 -04:00
ExternalSHA.hs Always use filesystem encoding for all file and handle reads and writes. 2016-12-24 14:46:31 -04:00
FileMode.hs use unix-compat 0.5 on windows 2017-11-14 14:00:24 -04:00
FileSize.hs matchexpression: New plumbing command to check if a preferred content expression matches some data. 2016-01-25 16:16:18 -04:00
FileSystemEncoding.hs adeiu, MissingH 2017-05-16 01:03:52 -04:00
Format.hs fix failing quickcheck properties 2017-06-17 16:48:00 -04:00
FreeDesktop.hs use System.Directory not Utility.Directory 2016-09-22 11:34:55 -04:00
Glob.hs adeiu, MissingH 2017-05-16 01:03:52 -04:00
Gpg.hs still can't express custom-setup deps 2017-11-14 14:59:51 -04:00
Hash.hs stop using MissingH for MD5 2017-05-15 21:36:03 -04:00
HtmlDetect.hs add Utility.HtmlDetect 2017-11-28 13:03:11 -04:00
HumanNumber.hs update my email address and homepage url 2015-01-21 12:50:09 -04:00
HumanTime.hs generalize parseDuration so it can be used in the ReadM monad 2015-07-08 16:08:26 -04:00
InodeCache.hs Fix build with QuickCheck 2.10. 2017-06-17 13:04:48 -04:00
libkqueue.c update my email address and homepage url 2015-01-21 12:50:09 -04:00
libkqueue.h fix prototype 2012-06-19 01:57:19 -04:00
LinuxMkLibs.hs adeiu, MissingH 2017-05-16 01:03:52 -04:00
LockFile.hs use lock pools throughout git-annex 2015-05-19 14:09:52 -04:00
LockPool.hs Fix shared lock file FD leak. 2016-03-01 15:31:39 -04:00
LogFile.hs update my email address and homepage url 2015-01-21 12:50:09 -04:00
Lsof.hs still can't express custom-setup deps 2017-11-14 14:59:51 -04:00
MagicWormhole.hs wormhole pairing appid flag day 2021-12-31 2017-02-03 15:06:40 -04:00
Matcher.hs improve comment 2015-09-15 13:12:21 -04:00
Metered.hs Improve progress display when watching file size, in cases where a transfer does not resume. 2017-05-25 14:30:18 -04:00
Misc.hs test: Avoid most situations involving failure to delete test directories 2017-08-14 16:29:47 -04:00
Monad.hs disable horrible tab warning, needed in every file that Setup.hs pulls in 2015-05-10 16:31:50 -04:00
Mounts.hs remove Utility.Mounts et al; moved to mountpoints package 2016-02-15 11:14:37 -04:00
Network.hs more {-# OPTIONS_GHC -fno-warn-tabs #-} ... Forcing people who have what is merely a difference of opinion to you to do this is a bit of an asshole move. Just saying. 2015-05-10 16:38:49 -04:00
NotificationBroadcaster.hs update my email address and homepage url 2015-01-21 12:50:09 -04:00
OptParse.hs I've been not documenting these import Preludes used to deal with the AMP transition 2015-09-15 11:32:47 -04:00
OSX.hs disable horrible tab warning, needed in every file that Setup.hs pulls in 2015-05-10 16:31:50 -04:00
Parallel.hs update my email address and homepage url 2015-01-21 12:50:09 -04:00
PartialPrelude.hs correct spelling mistakes 2017-02-12 17:30:23 -04:00
Path.hs finish fix for gitAnnexLink on windows 2017-10-26 12:01:16 -04:00
Percentage.hs update my email address and homepage url 2015-01-21 12:50:09 -04:00
PID.hs Revert "Revert "remove dep on Win32-extras"" 2017-11-13 12:55:23 -04:00
Process.hs simpler more generic processTranscript' 2017-02-15 16:02:10 -04:00
QuickCheck.hs Fix build with QuickCheck 2.10. 2017-06-17 13:04:48 -04:00
Quvi.hs Some optimisations to string splitting code. 2017-01-31 19:06:22 -04:00
Rsync.hs adeiu, MissingH 2017-05-16 01:03:52 -04:00
SafeCommand.hs adeiu, MissingH 2017-05-16 01:03:52 -04:00
Scheduled.hs adeiu, MissingH 2017-05-16 01:03:52 -04:00
Shell.hs clean up build warnings on Windows 2017-11-14 14:14:10 -04:00
SimpleProtocol.hs git-annex-shell, remotedaemon, git remote: Fix some memory DOS attacks. 2016-12-09 13:34:32 -04:00
Split.hs adeiu, MissingH 2017-05-16 01:03:52 -04:00
SRV.hs Fix build with dns-3.0. 2017-11-24 10:49:31 -04:00
SshConfig.hs redundant import 2015-09-22 12:31:54 -04:00
SshHost.hs avoid the dashed ssh hostname class of security holes 2017-08-17 22:11:31 -04:00
Su.hs clean up build warnings on Windows 2017-11-14 14:14:10 -04:00
SystemDirectory.hs Fix build with directory-1.3. 2016-12-20 15:23:59 -04:00
Tense.hs update my email address and homepage url 2015-01-21 12:50:09 -04:00
ThreadLock.hs update my email address and homepage url 2015-01-21 12:50:09 -04:00
ThreadScheduler.hs update my email address and homepage url 2015-01-21 12:50:09 -04:00
TList.hs update my email address and homepage url 2015-01-21 12:50:09 -04:00
Tmp.hs use unix-compat 0.5 on windows 2017-11-14 14:00:24 -04:00
Tor.hs Support all common locations of the torrc file. 2016-12-28 15:12:31 -04:00
Touch.hs another windows build fix 2016-03-05 15:08:37 -04:00
Tuple.hs adeiu, MissingH 2017-05-16 01:03:52 -04:00
Url.hs remove redundant pattern match 2017-09-24 16:17:58 -04:00
UserInfo.hs clean up build warnings on Windows 2017-11-14 14:14:10 -04:00
Verifiable.hs Removed dependency on haskell SHA library, instead using cryptohash >= 0.11.0. 2015-04-19 11:05:32 -04:00
WebApp.hs clean up build warnings on Windows 2017-11-14 14:14:10 -04:00
Yesod.hs fix build warning when building with yesod 1.2 and newer yesod-core 2015-08-03 15:42:44 -04:00