limit url downloads to whitelisted schemes
Security fix! Allowing any schemes, particularly file: and
possibly others like scp: allowed file exfiltration by anyone who had
write access to the git repository, since they could add an annexed file
using such an url, or using an url that redirected to such an url,
and wait for the victim to get it into their repository and send them a copy.
* Added annex.security.allowed-url-schemes setting, which defaults
to only allowing http and https URLs. Note especially that file:/
is no longer enabled by default.
* Removed annex.web-download-command, since its interface does not allow
supporting annex.security.allowed-url-schemes across redirects.
If you used this setting, you may want to instead use annex.web-options
to pass options to curl.
With annex.web-download-command removed, nearly all url accesses in
git-annex are made via Utility.Url via http-client or curl. http-client
only supports http and https, so no problem there.
(Disabling one and not the other is not implemented.)
Used curl --proto to limit the allowed url schemes.
Note that this will cause git annex fsck --from web to mark files using
a disallowed url scheme as not being present in the web. That seems
acceptable; fsck --from web also does that when a web server is not available.
youtube-dl already disabled file: itself (probably for similar
reasons). The scheme check was also added to youtube-dl urls for
completeness, although that check won't catch any redirects it might
follow. But youtube-dl goes off and does its own thing with other
protocols anyway, so that's fine.
Special remotes that support other domain-specific url schemes are not
affected by this change. In the bittorrent remote, aria2c can still
download magnet: links. The download of the .torrent file is
otherwise now limited by annex.security.allowed-url-schemes.
This does not address any external special remotes that might download
an url themselves. Current thinking is all external special remotes will
need to be audited for this problem, although many of them will use
http libraries that only support http and not curl's menagarie.
The related problem of accessing private localhost and LAN urls is not
addressed by this commit.
This commit was sponsored by Brett Eisenberg on Patreon.
2018-06-15 20:52:24 +00:00
|
|
|
git-annex (6.20180622) upstream; urgency=high
|
|
|
|
|
|
|
|
|
|
A security fix has changed git-annex to only support http and https
|
|
|
|
|
URL schemes by default. You can enable other URL schemes, at your own risk,
|
|
|
|
|
using annex.security.allowed-url-schemes.
|
|
|
|
|
|
2018-06-17 17:56:17 +00:00
|
|
|
A related security fix prevents git-annex from connecting to http
|
|
|
|
|
servers on localhost or private networks. This can be overridden,
|
|
|
|
|
at your own risk, using annex.security.allowed-http-addresses.
|
2018-06-17 18:46:22 +00:00
|
|
|
|
|
|
|
|
Setting annex.web-options no longer is enough to make curl be used,
|
|
|
|
|
and youtube-dl is also no longer used by default. See the
|
|
|
|
|
documentation of annex.security.allowed-http-addresses for
|
|
|
|
|
details and how to enable them.
|
2018-06-17 17:56:17 +00:00
|
|
|
|
limit url downloads to whitelisted schemes
Security fix! Allowing any schemes, particularly file: and
possibly others like scp: allowed file exfiltration by anyone who had
write access to the git repository, since they could add an annexed file
using such an url, or using an url that redirected to such an url,
and wait for the victim to get it into their repository and send them a copy.
* Added annex.security.allowed-url-schemes setting, which defaults
to only allowing http and https URLs. Note especially that file:/
is no longer enabled by default.
* Removed annex.web-download-command, since its interface does not allow
supporting annex.security.allowed-url-schemes across redirects.
If you used this setting, you may want to instead use annex.web-options
to pass options to curl.
With annex.web-download-command removed, nearly all url accesses in
git-annex are made via Utility.Url via http-client or curl. http-client
only supports http and https, so no problem there.
(Disabling one and not the other is not implemented.)
Used curl --proto to limit the allowed url schemes.
Note that this will cause git annex fsck --from web to mark files using
a disallowed url scheme as not being present in the web. That seems
acceptable; fsck --from web also does that when a web server is not available.
youtube-dl already disabled file: itself (probably for similar
reasons). The scheme check was also added to youtube-dl urls for
completeness, although that check won't catch any redirects it might
follow. But youtube-dl goes off and does its own thing with other
protocols anyway, so that's fine.
Special remotes that support other domain-specific url schemes are not
affected by this change. In the bittorrent remote, aria2c can still
download magnet: links. The download of the .torrent file is
otherwise now limited by annex.security.allowed-url-schemes.
This does not address any external special remotes that might download
an url themselves. Current thinking is all external special remotes will
need to be audited for this problem, although many of them will use
http libraries that only support http and not curl's menagarie.
The related problem of accessing private localhost and LAN urls is not
addressed by this commit.
This commit was sponsored by Brett Eisenberg on Patreon.
2018-06-15 20:52:24 +00:00
|
|
|
The annex.web-download-command configuration has been removed,
|
|
|
|
|
use annex.web-options instead.
|
|
|
|
|
|
|
|
|
|
-- Joey Hess <id@joeyh.name> Fri, 15 Jun 2018 17:54:23 -0400
|
|
|
|
|
|
use P2P protocol for checkpresent, retrieve, and store
Note that, due to not using rsync to transfer files to ssh remotes
any longer, permissions and other file metadata of annexed files
will no longer be preserved when copying them to ssh remotes.
Other remotes never supported preserving that information, so
this is not considered a regression. Added NEWS item about this.
Another significant side effect of this is that, even when rsync is run to
retrieve a file, its progress display will no longer be shown, and
instead the native git-annex progress display will appear. It would be
possible to use the rsync process display when rsync is used (old
git-annex-shell and also retrieval from a local repository), but it
would have complicated the code unncessarily, and been inconsistent
behavior.
(I'd been thinking for a while about eliminating the rsync progress
display, since it's got some annoying verbosities, including display of
the key and the "(xfr#1, to-chk=0/1)" bit and was already somewhat
inconsistent.)
retrieveKeyFileCheap still uses rsync, since that ensures that it gets
the actual file content from the remote. Using the P2P protocol would
use the local content, as long as the local and remote size are the
same.
This commit was sponsored by John Pellman on Patreon.
2018-03-09 16:57:32 +00:00
|
|
|
git-annex (6.20180309) upstream; urgency=medium
|
|
|
|
|
|
2018-03-09 17:56:21 +00:00
|
|
|
Note that, due to not using rsync to transfer files over ssh
|
use P2P protocol for checkpresent, retrieve, and store
Note that, due to not using rsync to transfer files to ssh remotes
any longer, permissions and other file metadata of annexed files
will no longer be preserved when copying them to ssh remotes.
Other remotes never supported preserving that information, so
this is not considered a regression. Added NEWS item about this.
Another significant side effect of this is that, even when rsync is run to
retrieve a file, its progress display will no longer be shown, and
instead the native git-annex progress display will appear. It would be
possible to use the rsync process display when rsync is used (old
git-annex-shell and also retrieval from a local repository), but it
would have complicated the code unncessarily, and been inconsistent
behavior.
(I'd been thinking for a while about eliminating the rsync progress
display, since it's got some annoying verbosities, including display of
the key and the "(xfr#1, to-chk=0/1)" bit and was already somewhat
inconsistent.)
retrieveKeyFileCheap still uses rsync, since that ensures that it gets
the actual file content from the remote. Using the P2P protocol would
use the local content, as long as the local and remote size are the
same.
This commit was sponsored by John Pellman on Patreon.
2018-03-09 16:57:32 +00:00
|
|
|
any longer, permissions and other file metadata of annexed files
|
|
|
|
|
will no longer be preserved when copying them to and from ssh remotes.
|
|
|
|
|
Other remotes never supported preserving that information, so
|
|
|
|
|
this is not considered a regression.
|
|
|
|
|
|
|
|
|
|
-- Joey Hess <id@joeyh.name> Fri, 09 Mar 2018 13:22:47 -0400
|
|
|
|
|
|
|
|
|
|
git-annex (6.20170228) upstream; urgency=medium
|
2017-02-28 17:28:34 +00:00
|
|
|
|
|
|
|
|
This version of git-annex has mitigations for SHA1 hash collision
|
|
|
|
|
problems.
|
|
|
|
|
|
|
|
|
|
A new annex.securehashesonly configuration, when used in combination with
|
|
|
|
|
signed git commits, avoids potential hash collision problems in git-annex
|
|
|
|
|
repositories. For details, see this web page:
|
|
|
|
|
<https://git-annex.branchable.com/tips/using_signed_git_commits/>
|
|
|
|
|
|
|
|
|
|
-- Joey Hess <id@joeyh.name> Tue, 28 Feb 2017 13:28:50 -0400
|
|
|
|
|
|
use P2P protocol for checkpresent, retrieve, and store
Note that, due to not using rsync to transfer files to ssh remotes
any longer, permissions and other file metadata of annexed files
will no longer be preserved when copying them to ssh remotes.
Other remotes never supported preserving that information, so
this is not considered a regression. Added NEWS item about this.
Another significant side effect of this is that, even when rsync is run to
retrieve a file, its progress display will no longer be shown, and
instead the native git-annex progress display will appear. It would be
possible to use the rsync process display when rsync is used (old
git-annex-shell and also retrieval from a local repository), but it
would have complicated the code unncessarily, and been inconsistent
behavior.
(I'd been thinking for a while about eliminating the rsync progress
display, since it's got some annoying verbosities, including display of
the key and the "(xfr#1, to-chk=0/1)" bit and was already somewhat
inconsistent.)
retrieveKeyFileCheap still uses rsync, since that ensures that it gets
the actual file content from the remote. Using the P2P protocol would
use the local content, as long as the local and remote size are the
same.
This commit was sponsored by John Pellman on Patreon.
2018-03-09 16:57:32 +00:00
|
|
|
git-annex (6.20170101) upstream; urgency=medium
|
2016-12-27 20:36:05 +00:00
|
|
|
|
|
|
|
|
XMPP support has been removed from the assistant in this release.
|
|
|
|
|
|
|
|
|
|
If your repositories used XMPP to keep in sync, that will no longer
|
|
|
|
|
work, and you should enable some other remote to keep them in sync.
|
|
|
|
|
A ssh server is one way, or use the new Tor pairing feature.
|
|
|
|
|
|
|
|
|
|
-- Joey Hess <id@joeyh.name> Tue, 27 Dec 2016 16:37:46 -0400
|
|
|
|
|
|
use P2P protocol for checkpresent, retrieve, and store
Note that, due to not using rsync to transfer files to ssh remotes
any longer, permissions and other file metadata of annexed files
will no longer be preserved when copying them to ssh remotes.
Other remotes never supported preserving that information, so
this is not considered a regression. Added NEWS item about this.
Another significant side effect of this is that, even when rsync is run to
retrieve a file, its progress display will no longer be shown, and
instead the native git-annex progress display will appear. It would be
possible to use the rsync process display when rsync is used (old
git-annex-shell and also retrieval from a local repository), but it
would have complicated the code unncessarily, and been inconsistent
behavior.
(I'd been thinking for a while about eliminating the rsync progress
display, since it's got some annoying verbosities, including display of
the key and the "(xfr#1, to-chk=0/1)" bit and was already somewhat
inconsistent.)
retrieveKeyFileCheap still uses rsync, since that ensures that it gets
the actual file content from the remote. Using the P2P protocol would
use the local content, as long as the local and remote size are the
same.
This commit was sponsored by John Pellman on Patreon.
2018-03-09 16:57:32 +00:00
|
|
|
git-annex (4.20131002) upstream; urgency=low
|
2016-05-24 05:00:06 +00:00
|
|
|
|
|
|
|
|
The layout of gcrypt repositories has changed, and
|
|
|
|
|
if you created one you must manually upgrade it.
|
|
|
|
|
See /usr/share/doc/git-annex/html/upgrades/gcrypt.html
|
|
|
|
|
|
|
|
|
|
-- Joey Hess <joeyh@debian.org> Tue, 24 Sep 2013 13:55:23 -0400
|
|
|
|
|
|
use P2P protocol for checkpresent, retrieve, and store
Note that, due to not using rsync to transfer files to ssh remotes
any longer, permissions and other file metadata of annexed files
will no longer be preserved when copying them to ssh remotes.
Other remotes never supported preserving that information, so
this is not considered a regression. Added NEWS item about this.
Another significant side effect of this is that, even when rsync is run to
retrieve a file, its progress display will no longer be shown, and
instead the native git-annex progress display will appear. It would be
possible to use the rsync process display when rsync is used (old
git-annex-shell and also retrieval from a local repository), but it
would have complicated the code unncessarily, and been inconsistent
behavior.
(I'd been thinking for a while about eliminating the rsync progress
display, since it's got some annoying verbosities, including display of
the key and the "(xfr#1, to-chk=0/1)" bit and was already somewhat
inconsistent.)
retrieveKeyFileCheap still uses rsync, since that ensures that it gets
the actual file content from the remote. Using the P2P protocol would
use the local content, as long as the local and remote size are the
same.
This commit was sponsored by John Pellman on Patreon.
2018-03-09 16:57:32 +00:00
|
|
|
git-annex (3.20120123) upstream; urgency=low
|
2016-05-24 05:00:06 +00:00
|
|
|
|
|
|
|
|
There was a bug in the handling of directory special remotes that
|
|
|
|
|
could cause partial file contents to be stored in them. If you use
|
|
|
|
|
a directory special remote, you should fsck it, to avoid potential
|
|
|
|
|
data loss.
|
|
|
|
|
|
|
|
|
|
Example: git annex fsck --from mydirectory
|
|
|
|
|
|
|
|
|
|
-- Joey Hess <joeyh@debian.org> Thu, 19 Jan 2012 15:24:23 -0400
|
|
|
|
|
|
use P2P protocol for checkpresent, retrieve, and store
Note that, due to not using rsync to transfer files to ssh remotes
any longer, permissions and other file metadata of annexed files
will no longer be preserved when copying them to ssh remotes.
Other remotes never supported preserving that information, so
this is not considered a regression. Added NEWS item about this.
Another significant side effect of this is that, even when rsync is run to
retrieve a file, its progress display will no longer be shown, and
instead the native git-annex progress display will appear. It would be
possible to use the rsync process display when rsync is used (old
git-annex-shell and also retrieval from a local repository), but it
would have complicated the code unncessarily, and been inconsistent
behavior.
(I'd been thinking for a while about eliminating the rsync progress
display, since it's got some annoying verbosities, including display of
the key and the "(xfr#1, to-chk=0/1)" bit and was already somewhat
inconsistent.)
retrieveKeyFileCheap still uses rsync, since that ensures that it gets
the actual file content from the remote. Using the P2P protocol would
use the local content, as long as the local and remote size are the
same.
This commit was sponsored by John Pellman on Patreon.
2018-03-09 16:57:32 +00:00
|
|
|
git-annex (3.20110624) upstream; urgency=low
|
2016-05-24 05:00:06 +00:00
|
|
|
|
|
|
|
|
There has been another change to the git-annex data store.
|
|
|
|
|
Use `git annex upgrade` to migrate your repositories to the new
|
|
|
|
|
layout. See <http://git-annex.branchable.com/upgrades/> or
|
|
|
|
|
/usr/share/doc/git-annex/html/upgrades.html
|
|
|
|
|
|
|
|
|
|
The significant change this time is that the .git-annex/ directory
|
|
|
|
|
is gone; instead there is a git-annex branch that is automatically
|
|
|
|
|
maintained by git-annex, and encapsulates all its state nicely out
|
|
|
|
|
of your way.
|
|
|
|
|
|
|
|
|
|
You should make sure you include the git-annex branch when
|
|
|
|
|
git pushing and pulling.
|
|
|
|
|
|
|
|
|
|
-- Joey Hess <joeyh@debian.org> Tue, 21 Jun 2011 20:18:00 -0400
|
|
|
|
|
|
use P2P protocol for checkpresent, retrieve, and store
Note that, due to not using rsync to transfer files to ssh remotes
any longer, permissions and other file metadata of annexed files
will no longer be preserved when copying them to ssh remotes.
Other remotes never supported preserving that information, so
this is not considered a regression. Added NEWS item about this.
Another significant side effect of this is that, even when rsync is run to
retrieve a file, its progress display will no longer be shown, and
instead the native git-annex progress display will appear. It would be
possible to use the rsync process display when rsync is used (old
git-annex-shell and also retrieval from a local repository), but it
would have complicated the code unncessarily, and been inconsistent
behavior.
(I'd been thinking for a while about eliminating the rsync progress
display, since it's got some annoying verbosities, including display of
the key and the "(xfr#1, to-chk=0/1)" bit and was already somewhat
inconsistent.)
retrieveKeyFileCheap still uses rsync, since that ensures that it gets
the actual file content from the remote. Using the P2P protocol would
use the local content, as long as the local and remote size are the
same.
This commit was sponsored by John Pellman on Patreon.
2018-03-09 16:57:32 +00:00
|
|
|
git-annex (0.20110316) upstream; urgency=low
|
2016-05-24 05:00:06 +00:00
|
|
|
|
|
|
|
|
This version reorganises the layout of git-annex's files in your repository.
|
|
|
|
|
There is an upgrade process to convert a repository from the old git-annex
|
|
|
|
|
to this version. See <http://git-annex.branchable.com/upgrades/> or
|
|
|
|
|
/usr/share/doc/git-annex/html/upgrades.html
|
|
|
|
|
|
|
|
|
|
-- Joey Hess <joeyh@debian.org> Wed, 16 Mar 2011 15:49:15 -0400
|