git-annex/doc/special_remotes
Joey Hess 28720c795f
limit url downloads to whitelisted schemes
Security fix! Allowing any schemes, particularly file: and
possibly others like scp: allowed file exfiltration by anyone who had
write access to the git repository, since they could add an annexed file
using such an url, or using an url that redirected to such an url,
and wait for the victim to get it into their repository and send them a copy.

* Added annex.security.allowed-url-schemes setting, which defaults
  to only allowing http and https URLs. Note especially that file:/
  is no longer enabled by default.

* Removed annex.web-download-command, since its interface does not allow
  supporting annex.security.allowed-url-schemes across redirects.
  If you used this setting, you may want to instead use annex.web-options
  to pass options to curl.

With annex.web-download-command removed, nearly all url accesses in
git-annex are made via Utility.Url via http-client or curl. http-client
only supports http and https, so no problem there.
(Disabling one and not the other is not implemented.)

Used curl --proto to limit the allowed url schemes.

Note that this will cause git annex fsck --from web to mark files using
a disallowed url scheme as not being present in the web. That seems
acceptable; fsck --from web also does that when a web server is not available.

youtube-dl already disabled file: itself (probably for similar
reasons). The scheme check was also added to youtube-dl urls for
completeness, although that check won't catch any redirects it might
follow. But youtube-dl goes off and does its own thing with other
protocols anyway, so that's fine.

Special remotes that support other domain-specific url schemes are not
affected by this change. In the bittorrent remote, aria2c can still
download magnet: links. The download of the .torrent file is
otherwise now limited by annex.security.allowed-url-schemes.

This does not address any external special remotes that might download
an url themselves. Current thinking is all external special remotes will
need to be audited for this problem, although many of them will use
http libraries that only support http and not curl's menagarie.

The related problem of accessing private localhost and LAN urls is not
addressed by this commit.

This commit was sponsored by Brett Eisenberg on Patreon.
2018-06-16 11:57:50 -04:00
..
adb response 2018-05-09 16:20:07 -04:00
bittorrent
bup
directory
external limit url downloads to whitelisted schemes 2018-06-16 11:57:50 -04:00
glacier Added a comment: RequestTimeoutException when uploading to glacier 2017-02-09 20:45:42 +00:00
hook
ipfs Added a comment: IPFS Deduplication 2017-10-07 19:46:12 +00:00
rsync response 2016-12-13 12:48:11 -04:00
S3 very delayed response now that feature is added 2017-09-08 16:47:42 -04:00
tahoe
web limit url downloads to whitelisted schemes 2018-06-16 11:57:50 -04:00
webdav
xmpp
adb.mdwn exporttree support for adb special remote 2018-03-27 16:28:41 -04:00
bittorrent.mdwn
bup.mdwn
comment_1_961276c18e9353ca8e25cad53e7ec51f._comment
comment_2_97543acfa7434e332ebea5672e446317._comment
comment_3_9229776623c234204c8b164edff95da0._comment
comment_4_3bbda479d13f6bf393dcd59ed94ddeaa._comment
comment_5_f7000975d38077828ab11a99095b39eb._comment
comment_6_5d2bd7c1e1493d3c3784708a9b0bc001._comment
comment_7_af01ee5ce31b1490af565cb087d65277._comment
comment_8_3d4ffec566d68d601eafe8758a616756._comment
comment_9_26af468952f0403171370b56e127830a._comment
comment_10_e9881290486a1770bd260f8650ada9c6._comment
comment_11_e01b5cc5a0d81b071e93e27e7b91fe2a._comment
comment_12_13237170ef5b6646e0e25d3421af3fe5._comment
comment_13_1a36a0483a9db04d36e0234a192ebad8._comment
comment_14_a8419963dc024b1d9eb73807596012dc._comment
comment_15_95ccfdd22a2391daa99e0beb04adedd6._comment
comment_16_b9d238fb15ad7628e33c90b071e07bb0._comment
comment_17_cc21b81a8f809f6efa5f5b6332513fc3._comment
comment_18_3fe750118ff1edbe91a110b86fb5b662._comment
comment_19_6794eb52bd87c28ef1df3172aa7d5780._comment
comment_20_6b7242721f2f2c77b634568cb737e3e3._comment
comment_22_308afc586b86c66bbb3437d63864d9cb._comment
comment_23_0f5440e0e54cf7ac2a68b1ba115b0930._comment
comment_23_96ef232e13bc2dc102a667a06c856ee7._comment
comment_24_2c9eda62766c9d5000346a092fe5d0d8._comment
comment_25_d9f298f284d66fb0aff029eb01f1ce23._comment
comment_26_606c1bee71a265f9df3a8cf50fce9a21._comment
comment_27_b37c6ed2444bd7e8b4d7937abdfcbd1c._comment
comment_28_c7ab32e1e63fc114b3b2f56bab62eafa._comment
comment_29_8393a14f084d022986c8245ee01f4198._comment
comment_30_8e5b17431507ee2115b992e5156b749b._comment
comment_31_20ac13d009a4f451eb895ca16446ba88._comment
comment_32_8dea734fed26e5d9336a2da5bd81eabc._comment
comment_33_317c4d5edc9de8159c3b03a3e161e257._comment
comment_34_31256dd4b74d344aa49902adec4bcf02._comment
comment_35_5801755d10149f0d3971fef0b6e1b62f._comment
comment_36_48859c5ea8f63e37b7531ed0e07f4a54._comment
comment_37_9fa2adc324ba25428a681158e58a5300._comment S3: Support the special case endpoint needed for the cn-north-1 region. 2016-11-07 11:49:34 -04:00
ddar.mdwn
directory.mdwn implement exporttree=yes configuration 2017-09-04 13:09:38 -04:00
external.mdwn
gcrypt.mdwn
glacier.mdwn
hook.mdwn
ipfs.mdwn
rsync.mdwn Support exporttree=yes for rsync special remotes. 2018-02-28 13:36:20 -04:00
S3.mdwn S3 export (untested) 2017-09-08 15:46:24 -04:00
tahoe.mdwn
tor.mdwn add page for tor special remote 2016-12-07 15:44:52 -04:00
web.mdwn limit url downloads to whitelisted schemes 2018-06-16 11:57:50 -04:00
webdav.mdwn export to webdav 2017-09-12 14:10:09 -04:00
xmpp.mdwn add back share_with_a_friend_walkthrough, adapted for tor pairing 2016-12-24 15:46:02 -04:00