mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

git-annex version 6.20180626

Browse source 
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEEKKUAw1IH6rcvbA8l2xLbD/BfjzgFAlstCaQACgkQ2xLbD/Bf
 jzh5nxAAn7D9soTI0ex6AVDDo2CjOyTTDVrIcl2h5XizfuUD3ev5P0TR3BZmzpAb
 MI6uaZ8kxqZ/eGAsBTyH9PsV7QVYIdht9t89ytP4xWyTQiOgjyJeA6PnJl4zVK9z
 Y8Of3mlylaz+97+sndljpsvy/KHENrHI7HHd+qxAu7wKysJxG6fJB7CjremkjaCI
 zAwg3mIy72ZKyuR/8hL9puJN9fdfw1ulkzQR+he007e/HkurPCwgRAOYW/Aa2tpY
 Oigdb9a6/0nl/VnOS8ZyHrSPRrhLH9c4IBmsdC1Xt5NDVmID/sWgD9uPF9dsHSMF
 OM25QdSlJ5cSNg+/XCpmmhC9MjgKkuVNpZ/fWBaHFs6KYgGhtZcAayQdz5AmMS2N
 HTPWB1IxZiV5TQHQpLbdH/q3RfNtRq1G1tc24zpd/zdhzijeTM6D8n4No6LXNq8X
 7U0qcrp9TdLOpBCTf6Jrg/7qFaXddHoEW1e3KrsOmB0hlYHuNxfY4bs0+ROeXGOT
 00koezcbF8kEI0ekoDvJjtVqaUq+608YjJZ5v7dE0vbtTj0KGbl5EHwC9atUluCX
 MHyTDY89uq68g4HIDytL001ZLvE3EUGJc4jh3+OMDzuZSKB5uwJIIky+qIaQu34K
 QJrZuyAIY0sVFV6LUX9nwqTW6Nnx/bB+kZ6k0+gx+Lpf7pUpE+o=
 =kex4
 -----END PGP SIGNATURE-----
gpgsig -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCgAdFiEEKKUAw1IH6rcvbA8l2xLbD/BfjzgFAlsxnX4ACgkQ2xLbD/Bf
 jzjK1xAAnJ58ZxLyTYlCZRcKiR81UHS/Mk6+SDAjRIRbT0SsY+6gSP55XKjrcuOb
 Jatp+6cNNSgk2lBpn37mq+rYIqboFh9moDRK7JSh1mDHCVtIwdARGblFRfuwaWPi
 xHnu+Pj43+SP7OF+8qP8/kDM+js3iMS+0gvBBz8pQN/yJDROXii6u0eONOd7vbER
 iRY9QpJdj5lp3hjaWfXt5iJC0re0eOAY4eUSHPsFIASysShnn33dFPOZ2hbhRKjR
 unQHUVIUE+ehmW3w9qIqn+9v2kca7laGK11cvzYRpmu/9rrvpf+RF1h42S8822dP
 CKHvxDkBGbyqTA+F9/6zpU1i9/ARgHFDpScRcdq7ZJi9FbWabKDklHCsgxwrkdXb
 +FXgb7N5Sa4+eVDNUf4rxldtLPX53nrtZ3IqrGiCWApCvbysNyP5kE0nix02l9z2
 xzY2vlpicx7TOMoO9mZesSFNgRzuFAbbya/zDJrz+xfgSRYXRYg58yTpmhpTFvSI
 h3Fw6+MYvehvRdAweLtoQt2p/UV2MAWrTpNzFoqgf2OCQOiH97ACDHn8Yki9rnQi
 NuMsqv9WOYQs4SaygDZMKemgAxftf3uaXiBW0RzHHwwWnDjHhqsEioOvOhNNyZbz
 U3OjKrH1JZlkNHlIBQD4BsWGLlIct66ZTU3k2OxPEp+mpEG/Xi4=
 =p+cW
 -----END PGP SIGNATURE-----

Merge tag '6.20180626' - previously embargoed security release
This commit is contained in:
Joey Hess 2018-06-25 21:56:43 -04:00
commit 3160cadba3
83 changed files with 1909 additions and 648 deletions
Download patch file Download diff file Expand all files Collapse all files

3
doc/backends.mdwn
View file

@ -53,6 +53,9 @@ contents that were checked into a repository earlier, you should avoid
using the non-cryptographically-secure backends, and will need to use
signed git commits. See [[tips/using_signed_git_commits]] for details.
Retrieval of WORM and URL from many [[special_remotes]] is prohibited
for [[security_reasons|security/CVE-2018-10857_and_CVE-2018-10859]].
Note that the various 512 and 384 length hashes result in long paths,
which are known to not work on Windows. If interoperability on Windows is a
concern, avoid those.

3
doc/bugs/importfeed_does_not_work_with_socks_proxy.mdwn
View file

@ -3,3 +3,6 @@ It appears that `git annex importfeed` can not be used in with socks proxies bec
For `addurl`, I could configure the system to use a socks proxy by setting `git config annex.web-download-command "curl --silent --preproxy socks4a://localhost:1080 %url -o %file"`; for `importfeed`, I found no option to override the command used to fetch the URL, and `wget` [lacks SOCKS support](https://savannah.gnu.org/bugs/?func=detailitem&item_id=43576).
Please consider using `web-download-command` for `importfeed` too, introducing a dedicated option `web-get-command` (that would output to stdout rather than %file), or otherwise supporting operation behind a SOCKS proxy.
> Closing this, since the http library git-annex uses supports socks
> proxies via environment settings, as far as I know. [[done]] --[[Joey]]

214
doc/bugs/security_hole_private_data_exposure_via_addurl.mdwn Normal file
View file

@ -0,0 +1,214 @@
CVE-2018-10857
This is a security hole that allows exposure of
private data in files located outside the git-annex repository.
The attacker needs to have control over one of the remotes of the git-annex
repository. For example, they may provide a public git-annex repository
that the victim clones. Or the victim may have paired repositories with
them. Or, equivilantly, the attacker could have read access to the victim's
git-annex repository (eg on a server somewhere), and some channel to get
commits into it (eg a pull requests).
The attacker does `git-annex addurl --relaxed file:///etc/passwd` and commits
this to the repository in some out of the way place. Then they wait for the
victim to pull the change.
The easiest exploit is when the victim is running the assistant, or is
periodically doing `git annex sync --content`. The victim may also perform
the equivilant actions manually, not noticing they're operating on the
file.
In either case, git-annex gets the content of the annexed file, following
the file:// url. Then git-annex transfers the content back to the
attacker's repository.
It may also be possible to exploit scp:// sftp:// smb:// etc urls to get at
files on other computers that the user has access to as well as localhost.
I was not able to get curl to download from scp:// or sftp:// on debian
(unstable) but there may be configurations that allow it.
If the url is attached to a key using a cryptographic hash, then the
attacker would need to already know at least the hash of the content
to exploit this.
Sending that content back to them could be considered not a security
hole. Except, I can guess what content some file on your system might have,
and use this attack as an oracle to determine if I guessed right, and work
toward penetrating your system using that information.
So, best to not treat addurl with a hash any differently than
--relaxed and --fast when addressing this hole.
----
The fix must involve locking down the set of allowed in url schemes.
Better to have a whitelist than a blacklist. http and https seems like the
right default.
Some users do rely on file:// urls, and this is fine in some use cases,
eg when you're not merging changes from anyone else.
So this probably needs to be a git config of allowed url schemes,
with an appropriatly scary name, like `annex.security.allowed-url-schemes`.
Redirects from one url scheme to another could be usd to bypass such a
whitelist. curl's `--proto` also affects redirects. http-conduit
is limited to only http and will probably remain that way.
> done in [[!commit 28720c795ff57a55b48e56d15f9b6bcb977f48d9]] --[[Joey]]
----
The same kind of attack can be used to see the content of
localhost urls and other non-globally-available urls.
Redirects and DNS rebinding attacks mean that checking the IP address
of the hostname in the url is not good enough. It needs to check the IP
address that is actually connected to, for each http connection made,
including redirects.
There will need to be a config to relax checks, like
with an appropriatly scary name, like
`annex.security.allowed-http-addresses`. Users will want to enable
support for specific subnets, or perhaps allow all addresses.
When git-annex is configured to use curl, there seems to be no way
to prevent curl from following urls that redirect to localhost, other than
disabling redirections. And unless git-annex also pre-resolves the IP
address and rewrites it into the url passed to curl, DNS rebinding attacks
would still be possible. Also, one of the remaining reasons people enable
curl is to use a netrc file with passwords, and the content of
urls on those sites could also be exposed by this attack. So, it seems curl
should not be enabled by default and should have a big security warning if
it's supported at all. Probably ought to only enable it
when `annex.security.allowed-http-addresses=all`
http-client does not have hooks that can be used to find out what IP
address it's connecting to in a secure enough way.
See <https://github.com/snoyberg/http-client/issues/354>
Seems that building my own http Manager is the only way to go. By building
my own, I can do the IP address checks inside it when it's setting up
connections. And, the same manager can be passed to the S3 and WebDav libraries.
(The url scheme checks could also be moved into that Manager, to prevent
S3 redirect to file: url scenarios..)
> restricted http manager done and used in
> [[!commit b54b2cdc0ef1373fc200c0d28fded3c04fd57212]];
> curl also disabled by default
http proxies are another problem. They could be on the local network,
or even on localhost, and http-client does not provide a way to force
a http proxy to connect to an particular IP address (is that even possible?)
May have to disable use of http proxies unless
`annex.security.allowed-http-addresses=all`
Or better, find what http proxy is enabled and prevent using it if it's on
an IP not allowed there.
> done in [[!commit cc08135e659d3ca9ea157246433d8aa90de3baf7]]
----
The external special remote interface is another way to exploit this.
Since it bypasses git-annex's usual url download code, whatever fixes are
put in place there won't help external special remotes.
External special remotes that use GETURLS, typically in conjunction with
CLAIMURL and CHECKURL, and then themselves download the content of urls
in response to a TRANSFER RETRIEVE will have the same problems as
git-annex's url downloading.
An external special remote might also make a simple http request to a
key/value API to download a key, and follow a redirect to file:///
or http://localhost/.
If the key uses a cryptographic hash, git-annex verifies the content.
But, the attacker could have committed a key that doesn't
use a hash. Also, the attacker could use the hash check as an oracle,
to guess at the content of files.
If the external special remote is encrypted, the http content is passed
though gpg. So, whatever location an attacker redirects it to would also
have to be encrypted. gpg is not told what encryption key the content is
expected to be encrypted with. (Could it be told somehow for hybrid and
shared encryption which key to use? pubkey encryption of course uses
the regular gpg public key).
So if the attacker knows a file that the user has encrypted with any of
their gpg keys, they can provide that file, and hope it will be decrypted.
Note that this does not need a redirect to a local file or web server; the
attacker can make their web server serve up a gpg encrypted file.
This is a separate vulnerability and was assigned CVE-2018-10859.
So, content downloaded from encrypted special remotes (both internal and
external) must be rejected unless it can be verified with a hash. Then
content using WORM and URL keys would not be able to be downloaded from
them. Might as well also require a hash check for non-encrypted external
special remotes, to block the redirection attack. There could be a config
setting to say that the git-annex repository is not being shared with
untrusted third parties, and relax that check.
> done in [[!commit b657242f5d946efae4cc77e8aef95dd2a306cd6b]]
Could also tighten down the gpg decryption to only allow decrypting with
the provided symmetric key, as a further protection against CVE-2018-10859.
If this can be done, then only remotes with encryption=pubkey will
really need to reject WORM and URL keys, since encryption=shared
and encryption=hybrid use a symetric key that's only used to encrypt data
for that remote. Although opening those back up to WORM and URL would
allow the remote sending other content stored on it, to get the wrong
content decrypted. This seems unlikely to be a useful exploit in most
cases, but perhaps not all cases, so probably best to not relax the
rejection aven when doing this. It's still worth doing as a belt and braces
fix.
> AFAICS, gpg does not have a way to specify to decrypt with only a
> symmetric encryption key. It could be done by running gpg in an
> environment with an empty keyring, but gpg agent makes that difficult and
> it would be added complexity. Decided not to do it.
----
Built-in special remotes that use protocols on top of http, eg S3 and WebDAV,
don't use Utility.Url, could also be exploited, and will need to be fixed
separately.
> not affected for url schemes, because http-client only supports http,
> not file:/
> done for localhost/lan in [[!commit b54b2cdc0ef1373fc200c0d28fded3c04fd57212]]
youtube-dl
> already disabled file:/. Added a scheme check, but it won't block
> redirects to other schemes. But youtube-dl goes off and does its own thing with other
> protocols anyway, so that's fine.
>
> The youtube-dl generic extractor will download media files (including
> videos and photos) if passed an direct url to them. It does not seem to
> extract video etc from tags on html pages.
> git-annex first checks if a web page
> is html before pointing youtube-dl at it, to avoid using it to download
> direct urls to media files. But that would not prevent a DNS rebinding
> attack which made git-annex see the html page, and youtube-dl then see
> a media file on localhost.
>
> Also, the current code in htmlOnly
> runs youtube-dl if the html page download test fails to download
> anything.
>
> Also, in the course of a download, youtube-dl could chain to other urls,
> depending on how its extractor works. Those urls could redirect to
> a localhost/lan web server.
>
> So, youtube-dl needs to be disabled unless http address security checks
> are turned off.
>
> > done in [[!commit e62c4543c31a61186ebf2e4e0412df59fc8630c8]]
----
Both security holes are now fixed. [[done]] --[[Joey]]

27
doc/devblog/day_499__security_hole.mdwn Normal file
View file

@ -0,0 +1,27 @@
I'm writing this on a private branch, it won't be posted until a week from
now when the security hole is disclosed.
Security is not compositional. You can have one good feature, and add
another good feature, and the result is not two good features, but a new
security hole. In this case
[[bugs/security_hole_private_data_exposure_via_addurl]] (CVE-2018-10857).
And it can be hard to spot this kind of security hole, but then once it's
known it seems blindly obvious.
It came to me last night and by this morning I had decided the potential
impact was large enough to do a coordinated disclosure. Spent the first
half of the day thinking through ways to fix it that don't involve writing
my own http library. Then started getting in touch with all the
distributions' security teams. And then coded up a fairly complete fix for
the worst part of the security hole, although a secondary part is going to
need considerably more work.
It looks like the external special remotes are going to need at least some
security review too, and I'm still thinking that part of the problem over.
Exhausted.
Today's work was sponsored by Trenton Cronholm
[on Patreon](https://patreon.com/joeyh).
[[!meta date="Jun 15 2018 7:00 pm"]]

26
doc/devblog/day_500__security_hole_part_2.mdwn Normal file
View file

@ -0,0 +1,26 @@
Most of the day was spent staring at the http-client source code and trying
to find a way to add the IP address checks to it that I need to fully close
the security hole.
In the end, I did find a way, with the duplication of a couple dozen lines
of code from http-client. It will let the security fix be used with
libraries like aws and DAV that build on top of http-client, too.
While the code is in git-annex for now, it's fully disconnected and
would also be useful if a web browser were implemented in Haskell,
to implement same-origin restrictions while avoiding DNS rebinding attacks.
Looks like http proxies and curl will need to be disabled by default,
since this fix can't support either of them securely. I wonder how web
browsers deal with http proxies, DNS rebinding attacks and same-origin?
I can't think of a secure way.
Next I need a function that checks if an IP address is a link-local address
or a private network address. For both ipv4 and ipv6. Could not find
anything handy on hackage, so I'm gonna have to stare at some RFCs. Perhaps
this evening, for now, it's time to swim in the river.
Today's work was sponsored by Jake Vosloo
[on Patreon](https://patreon.com/joeyh)
[[!meta date="June 16 2018 4:00 pm"]]

13
doc/devblog/day_501__security_hole_part_3.mdwn Normal file
View file

@ -0,0 +1,13 @@
Got the IP address restrictions for http implemented. (Except for http
proxies.)
Unforunately as part of this, had to make youtube-dl and curl not be used
by default. The annex.security.allowed-http-addresses config has to be
opened up by the user in order to use those external commands, since they
can follow arbitrary redirects.
Also thought some more about how external special remotes might be
affected, and sent their authors' a heads-up.
[[!meta date="June 17 2018 4:00 pm"]]

24
doc/devblog/day_502__security_hole_part_4.mdwn Normal file
View file

@ -0,0 +1,24 @@
Spent several hours dealing with the problem of http proxies, which
bypassed the IP address checks added to prevent the security hole.
Eventually got it filtering out http proxies located on private IP
addresses.
Other than the question of what to do about external special remotes
that may be vulerable to related problems, it looks like the security
hole is all closed off in git-annex now.
Added a new page [[security]] with details of this and past security holes
in git-annex.
Several people I reached out to for help with special remotes have gotten
back to me, and we're discussing how the security hole may affect them and
what to do. Thanks especially to Robie Basak and Daniel Dent for their
work on security analysis.
Also prepared a minimal backport of the security fixes for the git-annex in
Debian stable, which will probably be more palatable to their security team
than the full 2000+ lines of patches I've developed so far.
The minimal fix is secure, but suboptimal; it prevents even safe urls from
being downloaded from the web special remote by default.
[[!meta date="June 18 2018 4:00 pm"]]

19
doc/devblog/day_503__security_hole_part_5.mdwn Normal file
View file

@ -0,0 +1,19 @@
Started testing that the security fix will build everywhere on
release day. This is being particularly painful for the android build,
which has very old libraries and needed http-client updated, with many
follow-on changes, and is not successfully building yet after 5 hours.
I really need to finish deprecating the android build.
Pretty exhausted from all this, and thinking what to do about
external special remotes, I elaborated on an idea that Daniel Dent had
raised in discussions about vulnerability, and realized that git-annex
has a second, worse vulnerability. This new one could be used to trick a
git-annex user into decrypting gpg encrypted data that they had
never stored in git-annex. The attacker needs to have control of both an
encrypted special remote and a git remote, so it's not an easy exploit to
pull off, but it's still super bad.
This week is going to be a lot longer than I thought, and it's already
feeling kind of endless..
[[!meta date="June 19 2018 8:00 pm"]]

10
doc/devblog/day_504__security_hole_part_6.mdwn Normal file
View file

@ -0,0 +1,10 @@
Was getting dangerously close to burnt out, or exhaustion leading to
mistakes, so yesterday I took the day off, aside from spending the morning
babysitting the android build every half hour. (It did finally succeed.)
Today, got back into it, and implemented a fix for CVE-2018-10859 and also
the one case of CVE-2018-10857 that had not been dealt with before.
This fix was really a lot easier than the previous fixes for
CVE-2018-10857.
Unfortunately this did mean not letting URL and WORM keys be downloaded
from many special remotes by default, which is going to be painful for some.

9
doc/git-annex-addurl.mdwn
View file

@ -10,9 +10,12 @@ git annex addurl `[url ...]`
Downloads each url to its own file, which is added to the annex.
When `youtube-dl` is installed, it's used to check for a video embedded in
a web page at the url, and that is added to the annex instead.
When `youtube-dl` is installed, it can be used to check for a video
embedded in a web page at the url, and that is added to the annex instead.
(However, this is disabled by default as it can be a security risk.
See the documentation of annex.security.allowed-http-addresses
in [[git-annex]](1) for details.)
Urls to torrent files (including magnet links) will cause the content of
the torrent to be downloaded, using `aria2c`.

5
doc/git-annex-importfeed.mdwn
View file

@ -13,8 +13,11 @@ content has not already been added to the repository before, so you can
delete, rename, etc the resulting files and repeated runs won't duplicate
them.
When `youtube-dl` is installed, it's used to download links in the feed.
When `youtube-dl` is installed, it can be used to download links in the feed.
This allows importing e.g., YouTube playlists.
(However, this is disabled by default as it can be a security risk.
See the documentation of annex.security.allowed-http-addresses
in [[git-annex]](1) for details.)
To make the import process add metadata to the imported files from the feed,
`git config annex.genmetadata true`

80
doc/git-annex.mdwn
View file

@ -1228,7 +1228,7 @@ Here are all the supported configuration settings.
Note that even when this is set to `false`, git-annex does verification
in some edge cases, where it's likely the case than an
object was downloaded incorrectly.
object was downloaded incorrectly, or when needed for security.
* `remote.<name>.annex-export-tracking`
@ -1356,12 +1356,16 @@ Here are all the supported configuration settings.
* `annex.web-options`
Setting this makes git-annex use curl to download urls
Options to pass to curl when git-annex uses it to download urls
(rather than the default built-in url downloader).
For example, to force IPv4 only, set it to "-4".
Or to make curl use your ~/.netrc file, set it to "--netrc".
Setting this option makes git-annex use curl, but only
when annex.security.allowed-http-addresses is configured in a
specific way. See its documentation.
* `annex.youtube-dl-options`
Options to pass to youtube-dl when using it to find the url to download
@ -1387,12 +1391,76 @@ Here are all the supported configuration settings.
If set, the command is run and each line of its output is used as a HTTP
header. This overrides annex.http-headers.
* `annex.web-download-command`
* `annex.security.allowed-url-schemes`
Use to specify a command to run to download a file from the web.
List of URL schemes that git-annex is allowed to download content from.
The default is "http https ftp".
In the command line, %url is replaced with the url to download,
and %file is replaced with the file that it should be saved to.
Think very carefully before changing this; there are security
implications. For example, if it's changed to allow "file" URLs, then
anyone who can get a commit into your git-annex repository could
`git-annex addurl` a pointer to a private file located outside that
repository, possibly causing it to be copied into your repository
and transferred on to other remotes, exposing its content.
Some special remotes support their own domain-specific URL
schemes; those are not affected by this configuration setting.
* `annex.security.allowed-http-addresses`
By default, git-annex only makes HTTP connections to public IP addresses;
it will refuse to use HTTP servers on localhost or on a private network.
This setting can override that behavior, allowing access to particular
IP addresses. For example "127.0.0.1 ::1" allows access to localhost
(both IPV4 and IPV6). To allow access to all IP addresses, use "all"
Think very carefully before changing this; there are security
implications. Anyone who can get a commit into your git-annex repository
could `git annex addurl` an url on a private http server, possibly
causing it to be downloaded into your repository and transferred to
other remotes, exposing its content.
Note that, since the interfaces of curl and youtube-dl do not allow
these IP address restrictions to be enforced, curl and youtube-dl will
never be used unless annex.security.allowed-http-addresses=all.
* `annex.security.allow-unverified-downloads`,
For security reasons, git-annex refuses to download content from
most special remotes when it cannot check a hash to verify
that the correct content was downloaded. This particularly impacts
downloading the content of URL or WORM keys, which lack hashes.
The best way to avoid problems due to this is to migrate files
away from such keys, before their content reaches a special remote.
See [[git-annex-migrate]](1).
When the content is only available from a special remote, you can
use this configuration to force git-annex to download it.
But you do so at your own risk, and it's very important you read and
understand the information below first!
Downloading unverified content from encrypted special remotes is
prevented, because the special remote could send some other encrypted
content than what you expect, causing git-annex to decrypt data that you
never checked into git-annex, and risking exposing the decrypted
data to any non-encrypted remotes you send content to.
Downloading unverified content from (non-encrypted)
external special remotes is prevented, because they could follow
http redirects to web servers on localhost or on a private network,
or in some cases to a file:/// url.
If you decide to bypass this security check, the best thing to do is
to only set it temporarily while running the command that gets the file.
The value to set the config to is "ACKTHPPT".
For example:
git -c annex.security.allow-unverified-downloads=ACKTHPPT annex get myfile
It would be a good idea to check that it downloaded the file you expected,
too.
* `annex.secure-erase-command`

2
doc/news.mdwn
View file

@ -1,7 +1,7 @@
[[!if test="news/*" then="""
This is where announcements of new releases, features, and other news is
posted. git-annex users are recommended to subscribe to this page's RSS
feed.
feed. Also, see [[security]] for security announcements.
[[!inline pages="./news/* and !./news/*/* and !*/Discussion" rootpage="news" show="30"]]

4
doc/security.mdwn Normal file
View file

@ -0,0 +1,4 @@
Information about all known security holes in git-annex, and their fixes.
Subscribe to the RSS feed to be kept up to date.
[[!inline pages="./security/* and !./security/*/* and !*/Discussion" rootpage="security" show="0"]]

10
doc/security/CVE-2014-6274.mdwn Normal file
View file

@ -0,0 +1,10 @@
CVE-2014-6274: Security fix for S3 and glacier when using embedcreds=yes with
encryption=pubkey or encryption=hybrid.
The creds embedded in the git repo were *not* encrypted.
git-annex enableremote will warn when used on a remote that has
this problem. For details, see [[upgrades/insecure_embedded_creds]].
Fixed in git-annex 5.20140919.
[[!meta date="Fri, 19 Sep 2014 12:53:42 -0400"]]

10
doc/security/CVE-2017-12976.mdwn Normal file
View file

@ -0,0 +1,10 @@
CVE-2017-12976: A hostname starting with a dash would get passed to ssh and be treated as
an option. This could be used by an attacker who provides a crafted
repository url to cause the victim to execute arbitrary code via
`-oProxyCommand`.
Fixed in git-annex 6.20170818
This is related to a git security hole, [CVE-2017-1000117](https://marc.info/?l=git&m=150238802328673&w=2).
[[!meta date="Fri, 18 Aug 2017 11:19:06 -0400"]]

92
doc/security/CVE-2018-10857_and_CVE-2018-10859.mdwn Normal file
View file

@ -0,0 +1,92 @@
CVE-2018-10857: Some uses of git-annex were vulnerable to a private data
exposure and exfiltration attack. It could expose the content of files
located outside the git-annex repository, or content from a private web
server on localhost or the LAN. Joey Hess discovered this attack.
CVE-2018-10859: A malicious server for a special remote could
trick git-annex into decrypting a file that was encrypted to the user's gpg
key. This attack could be used to expose encrypted data that was never
stored in git-annex. Daniel Dent discovered this attack in collaboration
with Joey Hess.
These security holes were fixed in git-annex 6.20180626. After upgrading
git-annex, you should restart any git-annex assistant processes.
Also, some external special remotes (plugins) were improved, as a second
line of defense against CVE-2018-10857:
* git-annex-remote-pcloud 0.0.2 (thanks to Tocho Tochev)
## attack descriptions
To perform these attacks, the attacker needs to have control over one of
the remotes of the victim's git-annex repository. For example, they may
provide a public git-annex repository that the victim clones. Or,
equivilantly, the attacker could have read access to the victim's git-annex
repository or a repository it pushes to, and some channel to get commits
into it (eg pull requests).
These exploits are most likely to succeed when the victim is running the
git-annex assistant, or is periodically running `git annex sync --content`.
To perform the private data exfiltration attack (CVE-2018-10857), the
attacker runs `git-annex addurl --relaxed file:///etc/passwd` and commits
this to the repository in some out of the way place. After the victim's
git repository receives that change, git-annex follows the attacker-provided
url to private data, which it stores in the git-annex repository.
From there it transfers the content to the git-annex repository that
the attacker has access to.
(As well as `file:///` urls, the attacker can use urls to private web
servers. The url can also be one that the attacker controls, that redirects
to such urls.)
To perform the gpg decryption attack (CVE-2018-10859), the attacker
additionally needs to have control of the server hosting an encrypted
special remote used by the victim's git-annex repository. The attacker uses
`git annex addurl --relaxed` with an innocuous url, and waits for the
user's git-annex to download it, and upload an (encrypted) copy to the
special remote they also control. At some later point, when the user
downloads the content from the special remote, the attacker instead sends
them the content of a gpg encrypted file that they wish to have decrypted
in its place. Finally, the attacker drops their own copy of the original
innocuous url, and waits for git-annex to send them the accidentially
decrypted file.
## git-annex security fixes
CVE-2018-10857 was fixed by making git-annex refuse to follow `file:///` urls
and urls pointing to private/local IP addresses by default. Two new
configuration settings, annex.security.allowed-url-schemes and
annex.security.allowed-http-addresses, can relax this security policy,
and are intended for cases where the git-annex repository is kept
private and so the attack does not apply.
CVE-2018-10859 was fixed by making git-annex refuse to download encrypted
content from special remotes, unless it knows the hash of the expected
content. When the attacker provides some other gpg encrypted content, it
will fail the hash check and be discarded.
External special remotes (plugins) that use HTTP/HTTPS could also be
attacked using the CVE-2018-10857 method, if the attacker additionally has
control of the server they connect to. To prevent such attacks,
git-annex refuses to download content from external special remotes unless
it can verify the hash of that content.
## impact on external special remotes
One variant of CVE-2018-10857 can exploit a vulnerable external special
remote, and could not be prevented by git-annex. (git-annex's own built-in
special remotes are not vulnerable to this attack.)
In this attack variant, the attacker guesses at the hash of a file stored
on the victim's private web server, and adds it to the git-annex
repository. The attacker also has control of the server hosting an
encrypted special remote used by the victim's git-annex repository. They
cause that server to redirect to the victim's web server. This allows the
attacker to verify if the victim's web server contains a file that the
attacker already knows the content of, assuming they can guess the URL to
it.
Developers of external special remotes are encouraged to prevent this
attack by not following such HTTP redirects.

12
doc/security/checksum_exposure_to_encrypted_special_remotes.mdwn Normal file
View file

@ -0,0 +1,12 @@
A bug exposed the checksum of annexed files to encrypted
special remotes, which are not supposed to have access to the checksum of
the un-encrypted file. This only occurred when resuming uploads to the
encrypted special remote, so it is considered a low-severity security hole.
For details, see [[!commit b890f3a53d936b5e40aa9acc5876cb98f18b9657]]
No CVE was assigned for this issue.
Fixed in git-annex 6.20160419
[[!meta date="Thu, 28 Apr 2016 09:31:14 -0400"]]

4
doc/special_remotes/external/git-annex-remote-torrent vendored
View file

@ -123,7 +123,7 @@ while read line; do
url="$2"
# List contents of torrent.
tmp=$(mktemp)
if ! runcmd curl -o "$tmp" "$url"; then
if ! runcmd curl --proto -all,http,https -o "$tmp" "$url"; then
echo CHECKURL-FAILURE
else
oldIFS="$IFS"
@ -166,7 +166,7 @@ while read line; do
echo TRANSFER-FAILURE RETRIEVE "$key" "no known torrent urls for this key"
else
tmp=$(mktemp)
if ! runcmd curl -o "$tmp" "$url"; then
if ! runcmd curl --proto -all,http,https -o "$tmp" "$url"; then
echo TRANSFER-FAILURE RETRIEVE "$key" "failed downloading torrent file from $url"
else
filenum="$(echo "$url" | sed 's/(.*#\(\d*\)/\1/')"

2
doc/special_remotes/web.mdwn
View file

@ -6,6 +6,6 @@ See [[tips/using_the_web_as_a_special_remote]] for usage examples.
Currently git-annex only supports downloading content from the web;
it cannot upload to it or remove content.
This special remote uses arbitrary urls on the web as the source for content.
This special remote uses urls on the web as the source for content.
git-annex can also download content from a normal git remote, accessible by
http.

3
doc/special_remotes/web/comment_2_333141cc9ec6c26ffd19aa95303a91e3._comment
View file

@ -5,4 +5,7 @@
date="2013-08-17T08:59:11Z"
content="""
When it says \"arbitrary urls\", it means it. The only requirement is that the url be well formed and that wget or whatever command you have it configured to use via annex.web-download-command knows how to download it.
Update 2018: That used to be the case, but it's now limited by default to
http and https urls.
"""]]

6
doc/tips/Using_Git-annex_as_a_web_browsing_assistant.mdwn
View file

@ -11,12 +11,6 @@ The first step is to install the Firefox plugin
[FlashGot](http://flashgot.net/). We will use it to provide the Firefox
shortcuts to add things to our annex.
We also need a normal download manager, if we want to get status updates as
the download is done. We'll need to configure git-annex to use it by
setting `annex.web-download-command` as Joey describes in his comment on
[[todo/wishlist: allow configuration of downloader for addurl]]. See the
manpage [[git-annex]] for more information on setting configuration.
Once we have installed all that, we need a script that has an interface
which FlashGot can treat as a downloader, but which calls git-annex to do
the actual downloading. Such a script is available from

8
doc/tips/Using_Git-annex_as_a_web_browsing_assistant/comment_1_74167f9fff400f148916003468c77de4._comment
View file

@ -1,8 +0,0 @@
[[!comment format=mdwn
username="http://joeyh.name/"
nickname="joey"
subject="comment 1"
date="2013-04-11T20:16:02Z"
content="""
As of my last commit, you don't really need a separate download manager. The webapp will now display urls that `git annex addurl` is downloading in among the other transfers.
"""]]

4
doc/tips/downloading_podcasts.mdwn
View file

@ -84,6 +84,10 @@ manually. For a channel url like
"https://www.youtube.com/channel/$foo", the
feed is "https://www.youtube.com/feeds/videos.xml?channel_id=$foo"
Use of youtube-dl is disabled by default as it can be a security risk.
See the documentation of annex.security.allowed-http-addresses
in [[git-annex]] for details.)
## metadata
As well as storing the urls for items imported from a feed, git-annex can

4
doc/tips/using_the_web_as_a_special_remote.mdwn
View file

@ -78,6 +78,10 @@ When you have youtube-dl installed, you can just
`git annex addurl http://youtube.com/foo` and it will detect that
it is a video and download the video content for offline viewing.
(However, this is disabled by default as it can be a security risk.
See the documentation of annex.security.allowed-http-addresses
in [[git-annex]] for details.)
Later, in another clone of the repository, you can run `git annex get` on
the file and it will also be downloaded with youtube-dl. This works
even if the video host has transcoded or otherwise changed the video

7
doc/todo/easy_way_to_reproduce_normal_download_command.mdwn
View file

@ -22,3 +22,10 @@ what about the other settings, is it okay to hardcode those?
maybe this would be easier if there would be an options override just
like rsync, but separate ones for curl and wget... --[[anarcat]]
> git-annex now only uses curl, and defaults to a built-in http downloader.
> The annex.web-download-command is no longer supported. annex.web-options
> can be used to pass options to curl.
>
> So, I don't think this todo is relevant anymore, closing [[done]].
> --[[Joey]]