Use http-conduit for url downloads by default, annex.web-options enables curl

* For url downloads, git-annex now defaults to using a http library,
  rather than wget or curl. But, if annex.web-options is set, it will
  use curl. To use the .netrc file, run:
    git config annex.web-options --netrc
* git-annex no longer uses wget (and wget is no longer shipped with
  git-annex builds).

Note that curl is always run in silent mode, since the new API for
download has a MeterUpdate and doesn't make way for curl progress
output. It might be worth writing a parser for curl's progress output
to update the meter when using it, but I didn't bother with this edge
case for now.

This commit was supported by the NSF-funded DataLad project.

Command/AddUrl.hs

@@ -27,6 +27,7 @@ import Types.KeySource
 import Types.UrlContents
 import Annex.FileMatcher
 import Logs.Location
+import Messages.Progress
 import Utility.Metered
 import Utility.FileSystemEncoding
 import Utility.HtmlDetect
@@ -260,9 +261,8 @@ downloadWeb o url urlinfo file =
 	go =<< downloadWith' downloader urlkey webUUID url (AssociatedFile (Just file))
   where
 	urlkey = addSizeUrlKey urlinfo $ Backend.URL.fromUrl url Nothing
-	downloader f p = do
-		showOutput
-		downloadUrl urlkey p [url] f
+	downloader f p = metered (Just p) urlkey (pure Nothing) $ 
+		\_ p' -> downloadUrl urlkey p' [url] f
 	go Nothing = return Nothing
 	-- If we downloaded a html file, try to use youtube-dl to
 	-- extract embedded media.