mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions
git-annex/Command
History
Joey Hess 28720c795f
limit url downloads to whitelisted schemes 
Security fix! Allowing any schemes, particularly file: and
possibly others like scp: allowed file exfiltration by anyone who had
write access to the git repository, since they could add an annexed file
using such an url, or using an url that redirected to such an url,
and wait for the victim to get it into their repository and send them a copy.

* Added annex.security.allowed-url-schemes setting, which defaults
  to only allowing http and https URLs. Note especially that file:/
  is no longer enabled by default.

* Removed annex.web-download-command, since its interface does not allow
  supporting annex.security.allowed-url-schemes across redirects.
  If you used this setting, you may want to instead use annex.web-options
  to pass options to curl.

With annex.web-download-command removed, nearly all url accesses in
git-annex are made via Utility.Url via http-client or curl. http-client
only supports http and https, so no problem there.
(Disabling one and not the other is not implemented.)

Used curl --proto to limit the allowed url schemes.

Note that this will cause git annex fsck --from web to mark files using
a disallowed url scheme as not being present in the web. That seems
acceptable; fsck --from web also does that when a web server is not available.

youtube-dl already disabled file: itself (probably for similar
reasons). The scheme check was also added to youtube-dl urls for
completeness, although that check won't catch any redirects it might
follow. But youtube-dl goes off and does its own thing with other
protocols anyway, so that's fine.

Special remotes that support other domain-specific url schemes are not
affected by this change. In the bittorrent remote, aria2c can still
download magnet: links. The download of the .torrent file is
otherwise now limited by annex.security.allowed-url-schemes.

This does not address any external special remotes that might download
an url themselves. Current thinking is all external special remotes will
need to be audited for this problem, although many of them will use
http libraries that only support http and not curl's menagarie.

The related problem of accessing private localhost and LAN urls is not
addressed by this commit.

This commit was sponsored by Brett Eisenberg on Patreon.
2018-06-16 11:57:50 -04:00
..
Add.hs add --json-error-messages (not yet implemented) 2018-02-19 14:32:15 -04:00
AddUnused.hs Avoid backtraces on expected failures when built with ghc 8; only use backtraces for unexpected errors. 2016-11-15 21:29:54 -04:00
AddUrl.hs limit url downloads to whitelisted schemes 2018-06-16 11:57:50 -04:00
Adjust.hs showStart variant for when there's no worktree file 2017-11-28 15:14:16 -04:00
Assistant.hs fold Build/SysConfig.hs into BuildInfo via include 2017-12-14 12:46:57 -04:00
Benchmark.hs change keys database to use IKey type with more efficient serialization 2016-01-12 14:01:50 -04:00
CalcKey.hs more lambda-case conversion 2017-12-05 15:00:50 -04:00
CheckPresentKey.hs more lambda-case conversion 2017-12-05 15:00:50 -04:00
Commit.hs remove 163 lines of code without changing anything except imports 2016-01-20 16:36:33 -04:00
Config.hs more lambda-case conversion 2017-12-05 15:00:50 -04:00
ConfigList.hs .noannex file 2017-12-13 14:34:32 -04:00
ContentLocation.hs Avoid backtraces on expected failures when built with ghc 8; only use backtraces for unexpected errors. 2016-11-15 21:29:54 -04:00
Copy.hs move: --safe/--unsafe and potential drop race fix 2018-04-09 16:20:10 -04:00
Dead.hs more lambda-case conversion 2017-12-05 15:00:50 -04:00
Describe.hs showStart variant for when there's no worktree file 2017-11-28 15:14:16 -04:00
DiffDriver.hs rename BlobType and add submodule to it 2018-05-14 14:45:41 -04:00
Direct.hs more lambda-case conversion 2017-12-05 15:00:50 -04:00
Drop.hs move: --safe/--unsafe and potential drop race fix 2018-04-09 16:20:10 -04:00
DropKey.hs add --json-error-messages (not yet implemented) 2018-02-19 14:32:15 -04:00
DropUnused.hs add gitAnnexTmpWorkDir and withTmpWorkDir 2017-11-29 13:53:39 -04:00
EnableRemote.hs change Remote.repo to Remote.getRepo 2018-06-04 15:30:26 -04:00
EnableTor.hs move protocol version stuff to the Net free monad 2018-03-12 15:20:51 -04:00
ExamineKey.hs add --json-error-messages (not yet implemented) 2018-02-19 14:32:15 -04:00
Expire.hs showStart variant for when there's no worktree file 2017-11-28 15:14:16 -04:00
Export.hs remove unused value 2018-06-14 12:32:36 -04:00
Find.hs add --json-error-messages (not yet implemented) 2018-02-19 14:32:15 -04:00
FindRef.hs add --json-error-messages (not yet implemented) 2018-02-19 14:32:15 -04:00
Fix.hs add --json-error-messages (not yet implemented) 2018-02-19 14:32:15 -04:00
Forget.hs showStart variant for when there's no worktree file 2017-11-28 15:14:16 -04:00
FromKey.hs showStart variant for when there's no worktree file 2017-11-28 15:14:16 -04:00
Fsck.hs fix migration bug and make fsck warn 2018-05-23 14:07:51 -04:00
FuzzTest.hs Fix several places where files in .git/annex/ were written with modes that did not take the core.sharedRepository config into account. 2018-01-02 17:25:25 -04:00
GCryptSetup.hs Avoid backtraces on expected failures when built with ghc 8; only use backtraces for unexpected errors. 2016-11-15 21:29:54 -04:00
Get.hs move: --safe/--unsafe and potential drop race fix 2018-04-09 16:20:10 -04:00
Group.hs showStart variant for when there's no worktree file 2017-11-28 15:14:16 -04:00
GroupWanted.hs showStart variant for when there's no worktree file 2017-11-28 15:14:16 -04:00
Help.hs remove 163 lines of code without changing anything except imports 2016-01-20 16:36:33 -04:00
Import.hs add --json-error-messages (not yet implemented) 2018-02-19 14:32:15 -04:00
ImportFeed.hs display addurl url not file 2018-04-13 01:37:46 -04:00
InAnnex.hs remove 163 lines of code without changing anything except imports 2016-01-20 16:36:33 -04:00
Indirect.hs showStart variant for when there's no worktree file 2017-11-28 15:14:16 -04:00
Info.hs Fix build with ghc 8.4+, which broke due to the Semigroup Monoid change 2018-05-30 12:28:43 -04:00
Init.hs .noannex file 2017-12-13 14:34:32 -04:00
InitRemote.hs showStart variant for when there's no worktree file 2017-11-28 15:14:16 -04:00
Inprogress.hs inprogress: Avoid showing failures for files not in progress. 2018-01-24 20:43:19 -04:00
List.hs add --json-error-messages (not yet implemented) 2018-02-19 14:32:15 -04:00
Lock.hs add --json-error-messages (not yet implemented) 2018-02-19 14:32:15 -04:00
LockContent.hs make sure that lockContentShared is always paired with an inAnnex check 2018-03-07 14:23:52 -04:00
Log.hs add --json-error-messages (not yet implemented) 2018-02-19 14:32:15 -04:00
LookupKey.hs lookupkey absolute path support 2017-12-08 15:35:02 -04:00
Map.hs Improve startup time for commands that do not operate on remotes 2018-01-09 16:22:07 -04:00
MatchExpression.hs matchexpression: Added --largefiles option to parse an annex.largefiles expression. 2016-02-03 16:58:36 -04:00
Merge.hs showStart variant for when there's no worktree file 2017-11-28 15:14:16 -04:00
MetaData.hs Fix mangling of --json output of utf-8 characters when not running in a utf-8 locale 2018-04-16 16:21:21 -04:00
Migrate.hs add --json-error-messages (not yet implemented) 2018-02-19 14:32:15 -04:00
Mirror.hs move: --safe/--unsafe and potential drop race fix 2018-04-09 16:20:10 -04:00
Move.hs remove the older move --force, which never behaved as documented and seems useless 2018-05-21 13:21:19 -04:00
Multicast.hs deal with unlocked files 2018-03-13 14:27:14 -04:00
NotifyChanges.hs make tor hidden service work when directory watching is not available 2016-12-09 16:40:47 -04:00
NumCopies.hs showStart variant for when there's no worktree file 2017-11-28 15:14:16 -04:00
P2P.hs move protocol version stuff to the Net free monad 2018-03-12 15:20:51 -04:00
P2PStdIO.hs GIT_ANNEX_SHELL_APPENDONLY 2018-05-25 13:17:56 -04:00
PostReceive.hs configuration to disable automatic merge conflict resolution 2017-06-01 12:51:01 -04:00
PreCommit.hs Avoid repeated checking that files passed on the command line exist. 2017-10-16 14:10:20 -04:00
Proxy.hs finally really add back custom-setup stanza 2017-12-31 16:36:39 -04:00
ReadPresentKey.hs Avoid backtraces on expected failures when built with ghc 8; only use backtraces for unexpected errors. 2016-11-15 21:29:54 -04:00
RecvKey.hs get, move, copy, mirror: Added --failed switch which retries failed copies/moves 2016-08-03 12:37:12 -04:00
RegisterUrl.hs showStart variant for when there's no worktree file 2017-11-28 15:14:16 -04:00
Reinit.hs .noannex file 2017-12-13 14:34:32 -04:00
Reinject.hs annex.securehashesonly 2017-02-27 13:33:59 -04:00
ReKey.hs deal with unlocked files 2018-03-13 14:27:14 -04:00
RemoteDaemon.hs remotedaemon: serve tor hidden service 2016-11-20 15:48:12 -04:00
Repair.hs remove 163 lines of code without changing anything except imports 2016-01-20 16:36:33 -04:00
Required.hs started converting to use optparse-applicative 2015-07-08 13:36:25 -04:00
ResolveMerge.hs showStart variant for when there's no worktree file 2017-11-28 15:14:16 -04:00
RmUrl.hs rekey: Added --batch mode. 2016-12-05 12:55:50 -04:00
Schedule.hs showStart variant for when there's no worktree file 2017-11-28 15:14:16 -04:00
Semitrust.hs convert all commands to work with optparse-applicative 2015-07-08 15:08:02 -04:00
SendKey.hs AssociatedFile newtype 2017-03-10 13:35:31 -04:00
SetKey.hs Avoid backtraces on expected failures when built with ghc 8; only use backtraces for unexpected errors. 2016-11-15 21:29:54 -04:00
SetPresentKey.hs setpresentkey: Added --batch support (for ronnypfa) 2018-05-27 14:56:14 -04:00
Smudge.hs add KeyVariety type 2017-02-24 15:16:56 -04:00
Status.hs Don't allow entering a view with staged or unstaged changes. 2018-05-14 16:51:06 -04:00
Sync.hs remove use of remoteGitConfig 2018-06-05 13:15:04 -04:00
Test.hs remove 163 lines of code without changing anything except imports 2016-01-20 16:36:33 -04:00
TestRemote.hs change Remote.repo to Remote.getRepo 2018-06-04 15:30:26 -04:00
TransferInfo.hs Fix reversion introduced in 6.20171214 that caused concurrent transfers to incorrectly fail with "transfer already in progress". 2018-03-14 18:55:34 -04:00
TransferKey.hs implement annex.retry et al 2018-03-29 13:04:07 -04:00
TransferKeys.hs implement annex.retry et al 2018-03-29 13:04:07 -04:00
Trust.hs showStart variant for when there's no worktree file 2017-11-28 15:14:16 -04:00
Unannex.hs add --json-error-messages (not yet implemented) 2018-02-19 14:32:15 -04:00
Undo.hs Avoid backtraces on expected failures when built with ghc 8; only use backtraces for unexpected errors. 2016-11-15 21:29:54 -04:00
Ungroup.hs showStart variant for when there's no worktree file 2017-11-28 15:14:16 -04:00
Uninit.hs Avoid repeated checking that files passed on the command line exist. 2017-10-16 14:10:20 -04:00
Unlock.hs avoid compiler warning 2018-03-15 13:21:32 -04:00
Untrust.hs convert all commands to work with optparse-applicative 2015-07-08 15:08:02 -04:00
Unused.hs avoid --all buffering list of all keys 2018-04-26 16:00:20 -04:00
Upgrade.hs .noannex file 2017-12-13 14:34:32 -04:00
VAdd.hs showStart variant for when there's no worktree file 2017-11-28 15:14:16 -04:00
VCycle.hs showStart variant for when there's no worktree file 2017-11-28 15:14:16 -04:00
Version.hs avoid unncessary version output differences in different contexts 2018-06-04 12:26:18 -04:00
VFilter.hs showStart variant for when there's no worktree file 2017-11-28 15:14:16 -04:00
Vicfg.hs finish fixing inverted Ord for TrustLevel 2018-04-13 15:17:54 -04:00
View.hs Don't allow entering a view with staged or unstaged changes. 2018-05-14 16:51:06 -04:00
VPop.hs showStart variant for when there's no worktree file 2017-11-28 15:14:16 -04:00
Wanted.hs showStart variant for when there's no worktree file 2017-11-28 15:14:16 -04:00
Watch.hs remove 163 lines of code without changing anything except imports 2016-01-20 16:36:33 -04:00
WebApp.hs fix webapp opening in termux 2018-04-25 14:38:42 -04:00
Whereis.hs Fix mangling of --json output of utf-8 characters when not running in a utf-8 locale 2018-04-16 16:21:21 -04:00