mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

rename annex.security.allowed-http-addresses

Browse source 
Renamed annex.security.allowed-http-addresses to
annex.security.allowed-ip-addresses because it is not really specific to
the http protocol, also limiting eg, git-annex's use of ftp and via
youtube-dl, several other protocols.

The old name for the config will still work.

If both old and new name are set, the new name will win.
This commit is contained in:
Joey Hess 2019-05-30 12:43:40 -04:00
parent 135404b33e
commit 1871295765
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
11 changed files with 38 additions and 25 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

14
Annex/Url.hs
View file

@ -11,7 +11,7 @@ module Annex.Url (
withUrlOptions, withUrlOptions,
getUrlOptions, getUrlOptions,
getUserAgent, getUserAgent,
httpAddressesUnlimited, ipAddressesUnlimited,
) where ) where
import Annex.Common import Annex.Common
@ -52,7 +52,7 @@ getUrlOptions = Annex.getState Annex.urloptions >>= \case
Just cmd -> lines <$> liftIO (readProcess "sh" ["-c", cmd]) Just cmd -> lines <$> liftIO (readProcess "sh" ["-c", cmd])
Nothing -> annexHttpHeaders <$> Annex.getGitConfig Nothing -> annexHttpHeaders <$> Annex.getGitConfig
checkallowedaddr = words . annexAllowedHttpAddresses <$> Annex.getGitConfig >>= \case checkallowedaddr = words . annexAllowedIPAddresses <$> Annex.getGitConfig >>= \case
["all"] -> do ["all"] -> do
-- Only allow curl when all are allowed, -- Only allow curl when all are allowed,
-- as its interface does not allow preventing -- as its interface does not allow preventing
@ -76,7 +76,7 @@ getUrlOptions = Annex.getState Annex.urloptions >>= \case
| isPrivateAddress addr = False | isPrivateAddress addr = False
| otherwise = True | otherwise = True
let connectionrestricted = addrConnectionRestricted let connectionrestricted = addrConnectionRestricted
("Configuration of annex.security.allowed-http-addresses does not allow accessing address " ++) ("Configuration of annex.security.allowed-ip-addresses does not allow accessing address " ++)
let r = Restriction let r = Restriction
{ addressRestriction = \addr -> { addressRestriction = \addr ->
if isallowed (addrAddress addr) if isallowed (addrAddress addr)
@ -88,13 +88,13 @@ getUrlOptions = Annex.getState Annex.urloptions >>= \case
case pr of case pr of
Nothing -> return () Nothing -> return ()
Just ProxyRestricted -> toplevelWarning True Just ProxyRestricted -> toplevelWarning True
"http proxy settings not used due to annex.security.allowed-http-addresses configuration" "http proxy settings not used due to annex.security.allowed-ip-addresses configuration"
manager <- liftIO $ U.newManager settings manager <- liftIO $ U.newManager settings
return (U.DownloadWithConduit, manager) return (U.DownloadWithConduit, manager)
httpAddressesUnlimited :: Annex Bool ipAddressesUnlimited :: Annex Bool
httpAddressesUnlimited = ipAddressesUnlimited =
("all" == ) . annexAllowedHttpAddresses <$> Annex.getGitConfig ("all" == ) . annexAllowedIPAddresses <$> Annex.getGitConfig
withUrlOptions :: (U.UrlOptions -> Annex a) -> Annex a withUrlOptions :: (U.UrlOptions -> Annex a) -> Annex a
withUrlOptions a = a =<< getUrlOptions withUrlOptions a = a =<< getUrlOptions

6
Annex/YoutubeDl.hs
View file

@ -31,13 +31,13 @@ import Control.Concurrent.Async
-- localhost or a private address. So, it's only allowed to download -- localhost or a private address. So, it's only allowed to download
-- content if the user has allowed access to all addresses. -- content if the user has allowed access to all addresses.
youtubeDlAllowed :: Annex Bool youtubeDlAllowed :: Annex Bool
youtubeDlAllowed = httpAddressesUnlimited youtubeDlAllowed = ipAddressesUnlimited
youtubeDlNotAllowedMessage :: String youtubeDlNotAllowedMessage :: String
youtubeDlNotAllowedMessage = unwords youtubeDlNotAllowedMessage = unwords
[ "This url is supported by youtube-dl, but" [ "This url is supported by youtube-dl, but"
, "youtube-dl could potentially access any address, and the" , "youtube-dl could potentially access any address, and the"
, "configuration of annex.security.allowed-http-addresses" , "configuration of annex.security.allowed-ip-addresses"
, "does not allow that. Not using youtube-dl." , "does not allow that. Not using youtube-dl."
] ]
@ -55,7 +55,7 @@ youtubeDlNotAllowedMessage = unwords
-- (Note that we can't use --output to specifiy the file to download to, -- (Note that we can't use --output to specifiy the file to download to,
-- due to <https://github.com/rg3/youtube-dl/issues/14864>) -- due to <https://github.com/rg3/youtube-dl/issues/14864>)
youtubeDl :: URLString -> FilePath -> Annex (Either String (Maybe FilePath)) youtubeDl :: URLString -> FilePath -> Annex (Either String (Maybe FilePath))
youtubeDl url workdir = ifM httpAddressesUnlimited youtubeDl url workdir = ifM ipAddressesUnlimited
( withUrlOptions $ youtubeDl' url workdir ( withUrlOptions $ youtubeDl' url workdir
, return $ Left youtubeDlNotAllowedMessage , return $ Left youtubeDlNotAllowedMessage
) )

4
CHANGELOG
View file

@ -20,6 +20,10 @@ git-annex (7.20190508) UNRELEASED; urgency=medium
bash. bash.
* When a remote is configured to be readonly, don't allow changing * When a remote is configured to be readonly, don't allow changing
what's exported to it. what's exported to it.
* Renamed annex.security.allowed-http-addresses to
annex.security.allowed-ip-addresses because it is not really specific
to the http protocol, also limiting eg, git-annex's use of ftp.
The old name for the config will still work.
-- Joey Hess <id@joeyh.name> Mon, 06 May 2019 13:52:02 -0400 -- Joey Hess <id@joeyh.name> Mon, 06 May 2019 13:52:02 -0400

8
Types/GitConfig.hs
View file

@ -98,7 +98,7 @@ data GitConfig = GitConfig
, annexRetry :: Maybe Integer , annexRetry :: Maybe Integer
, annexRetryDelay :: Maybe Seconds , annexRetryDelay :: Maybe Seconds
, annexAllowedUrlSchemes :: S.Set Scheme , annexAllowedUrlSchemes :: S.Set Scheme
, annexAllowedHttpAddresses :: String , annexAllowedIPAddresses :: String
, annexAllowUnverifiedDownloads :: Bool , annexAllowUnverifiedDownloads :: Bool
, annexMaxExtensionLength :: Maybe Int , annexMaxExtensionLength :: Maybe Int
, annexJobs :: Concurrency , annexJobs :: Concurrency
@ -172,8 +172,10 @@ extractGitConfig r = GitConfig
, annexAllowedUrlSchemes = S.fromList $ map mkScheme $ , annexAllowedUrlSchemes = S.fromList $ map mkScheme $
maybe ["http", "https", "ftp"] words $ maybe ["http", "https", "ftp"] words $
getmaybe (annex "security.allowed-url-schemes") getmaybe (annex "security.allowed-url-schemes")
, annexAllowedHttpAddresses = fromMaybe "" $ , annexAllowedIPAddresses = fromMaybe "" $
getmaybe (annex "security.allowed-http-addresses") getmaybe (annex "security.allowed-ip-addresses")
<|>
getmaybe (annex "security.allowed-http-addresses") -- old name
, annexAllowUnverifiedDownloads = (== Just "ACKTHPPT") $ , annexAllowUnverifiedDownloads = (== Just "ACKTHPPT") $
getmaybe (annex "security.allow-unverified-downloads") getmaybe (annex "security.allow-unverified-downloads")
, annexMaxExtensionLength = getmayberead (annex "maxextensionlength") , annexMaxExtensionLength = getmayberead (annex "maxextensionlength")

2
doc/git-annex-addurl.mdwn
View file

@ -13,7 +13,7 @@ Downloads each url to its own file, which is added to the annex.
When `youtube-dl` is installed, it can be used to check for a video When `youtube-dl` is installed, it can be used to check for a video
embedded in a web page at the url, and that is added to the annex instead. embedded in a web page at the url, and that is added to the annex instead.
(However, this is disabled by default as it can be a security risk. (However, this is disabled by default as it can be a security risk.
See the documentation of annex.security.allowed-http-addresses See the documentation of annex.security.allowed-ip-addresses
in [[git-annex]](1) for details.) in [[git-annex]](1) for details.)
Urls to torrent files (including magnet links) will cause the content of Urls to torrent files (including magnet links) will cause the content of

2
doc/git-annex-importfeed.mdwn
View file

@ -16,7 +16,7 @@ them.
When `youtube-dl` is installed, it can be used to download links in the feed. When `youtube-dl` is installed, it can be used to download links in the feed.
This allows importing e.g., YouTube playlists. This allows importing e.g., YouTube playlists.
(However, this is disabled by default as it can be a security risk. (However, this is disabled by default as it can be a security risk.
See the documentation of annex.security.allowed-http-addresses See the documentation of annex.security.allowed-ip-addresses
in [[git-annex]](1) for details.) in [[git-annex]](1) for details.)
To make the import process add metadata to the imported files from the feed, To make the import process add metadata to the imported files from the feed,

19
doc/git-annex.mdwn
View file

@ -1426,7 +1426,7 @@ Here are all the supported configuration settings.
Or to make curl use your ~/.netrc file, set it to "--netrc". Or to make curl use your ~/.netrc file, set it to "--netrc".
Setting this option makes git-annex use curl, but only Setting this option makes git-annex use curl, but only
when annex.security.allowed-http-addresses is configured in a when annex.security.allowed-ip-addresses is configured in a
specific way. See its documentation. specific way. See its documentation.
* `annex.youtube-dl-options` * `annex.youtube-dl-options`
@ -1469,10 +1469,11 @@ Here are all the supported configuration settings.
Some special remotes support their own domain-specific URL Some special remotes support their own domain-specific URL
schemes; those are not affected by this configuration setting. schemes; those are not affected by this configuration setting.
* `annex.security.allowed-http-addresses` * `annex.security.allowed-ip-addresses`
By default, git-annex only makes HTTP connections to public IP addresses; By default, git-annex only makes connections to public IP addresses;
it will refuse to use HTTP servers on localhost or on a private network. it will refuse to use HTTP and other servers on localhost or on a
private network.
This setting can override that behavior, allowing access to particular This setting can override that behavior, allowing access to particular
IP addresses. For example "127.0.0.1 ::1" allows access to localhost IP addresses. For example "127.0.0.1 ::1" allows access to localhost
@ -1480,13 +1481,19 @@ Here are all the supported configuration settings.
Think very carefully before changing this; there are security Think very carefully before changing this; there are security
implications. Anyone who can get a commit into your git-annex repository implications. Anyone who can get a commit into your git-annex repository
could `git annex addurl` an url on a private http server, possibly could `git annex addurl` an url on a private server, possibly
causing it to be downloaded into your repository and transferred to causing it to be downloaded into your repository and transferred to
other remotes, exposing its content. other remotes, exposing its content.
Note that, since the interfaces of curl and youtube-dl do not allow Note that, since the interfaces of curl and youtube-dl do not allow
these IP address restrictions to be enforced, curl and youtube-dl will these IP address restrictions to be enforced, curl and youtube-dl will
never be used unless annex.security.allowed-http-addresses=all. never be used unless annex.security.allowed-ip-addresses=all.
* `annex.security.allowed-http-addresses`
Old name for annex.security.allowed-ip-addresses.
If set, this is treated the same as having
annex.security.allowed-ip-addresses set.
* `annex.security.allow-unverified-downloads` * `annex.security.allow-unverified-downloads`

2
doc/news/security_fix_release.mdwn
View file

@ -21,7 +21,7 @@ security fixes:
* A related security fix prevents git-annex from connecting to http * A related security fix prevents git-annex from connecting to http
servers (and proxies) on localhost or private networks. This can servers (and proxies) on localhost or private networks. This can
be overridden, at your own risk, using annex.security.allowed-http-addresses. be overridden, at your own risk, using annex.security.allowed-ip-addresses.
* Setting annex.web-options no longer is enough to make curl be used, * Setting annex.web-options no longer is enough to make curl be used,
and youtube-dl is also no longer used by default. See the and youtube-dl is also no longer used by default. See the

2
doc/security/CVE-2018-10857_and_CVE-2018-10859.mdwn
View file

@ -58,7 +58,7 @@ decrypted file.
CVE-2018-10857 was fixed by making git-annex refuse to follow `file:///` urls CVE-2018-10857 was fixed by making git-annex refuse to follow `file:///` urls
and urls pointing to private/local IP addresses by default. Two new and urls pointing to private/local IP addresses by default. Two new
configuration settings, annex.security.allowed-url-schemes and configuration settings, annex.security.allowed-url-schemes and
annex.security.allowed-http-addresses, can relax this security policy, annex.security.allowed-ip-addresses, can relax this security policy,
and are intended for cases where the git-annex repository is kept and are intended for cases where the git-annex repository is kept
private and so the attack does not apply. private and so the attack does not apply.

2
doc/tips/downloading_podcasts.mdwn
View file

@ -85,7 +85,7 @@ manually. For a channel url like
feed is "https://www.youtube.com/feeds/videos.xml?channel_id=$foo" feed is "https://www.youtube.com/feeds/videos.xml?channel_id=$foo"
Use of youtube-dl is disabled by default as it can be a security risk. Use of youtube-dl is disabled by default as it can be a security risk.
See the documentation of annex.security.allowed-http-addresses See the documentation of annex.security.allowed-ip-addresses
in [[git-annex]] for details.) in [[git-annex]] for details.)
## metadata ## metadata

2
doc/tips/using_the_web_as_a_special_remote.mdwn
View file

@ -79,7 +79,7 @@ When you have youtube-dl installed, you can just
it is a video and download the video content for offline viewing. it is a video and download the video content for offline viewing.
(However, this is disabled by default as it can be a security risk. (However, this is disabled by default as it can be a security risk.
See the documentation of annex.security.allowed-http-addresses See the documentation of annex.security.allowed-ip-addresses
in [[git-annex]] for details.) in [[git-annex]] for details.)
Later, in another clone of the repository, you can run `git annex get` on Later, in another clone of the repository, you can run `git annex get` on