Commit graph

154 commits

Author SHA1 Message Date
Joey Hess
b68d2a4b68
webapp: full wormhole pairing UI (untested)
This commit was sponsored by Riku Voipio.
2016-12-27 16:41:35 -04:00
Joey Hess
8484c0c197
Always use filesystem encoding for all file and handle reads and writes.
This is a big scary change. I have convinced myself it should be safe. I
hope!
2016-12-24 14:46:31 -04:00
Joey Hess
e08691b393
enable-tor: When run as a regular user, test a connection back to the hidden service over tor.
This way we know that after enable-tor, the tor hidden service is fully
published and working, and so there should be no problems with it at
pairing time.

It has to start up its own temporary listener on the hidden service. It
would be nice to have it start the remotedaemon running, so that extra
step is not needed afterwards. But, there may already be a remotedaemon
running, in communication with the assistant and we don't want to start
another one. I thought about trying to HUP any running remotedaemon, but
Windows does not make it easy to do that. In any case, having the user
start the remotedaemon themselves lets them know it needs to be running
to serve the hidden service.

This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
2016-12-24 12:50:23 -04:00
Joey Hess
22252e8e4c
Revert "close"
This reverts commit 3aaabc906b.

Commit contained incomplete work.
2016-12-24 12:07:15 -04:00
Joey Hess
3aaabc906b
close 2016-12-22 13:59:21 -04:00
Joey Hess
f7ca2b92fb
enable-tor: No longer needs to be run as root.
When run by not root, su's to root automatically.

This commit was sponsored by Brock Spratlen on Patreon.
2016-12-20 17:40:36 -04:00
Joey Hess
944a6503b9
relocate tor socket out of /etc
weasel explained that apparmor limits on what files tor can read do not
apply to sockets (because they're not files). And apparently the
problems I was seeing with hidden services not being accessible had to
do with onion address propigation and not the location of the socket
file.

remotedaemon looks up the HiddenServicePort in torrc, so if it was
previously configured with the socket in /etc, that will still work.

This commit was sponsored by Denis Dzyubenko on Patreon.
2016-12-20 16:24:46 -04:00
Joey Hess
8f3b2c206c
Debian: Suggest tor and magic-wormhole.
Suggests, not recommends, because tor is not for everyone.
2016-12-20 15:26:14 -04:00
Joey Hess
e312ec3750
Fix build with directory-1.3.
See https://github.com/haskell/directory/issues/66
2016-12-20 15:23:59 -04:00
Joey Hess
a171e576b2
rekey --force: Incorrectly marked the new key's content as being present in the local repo even when it was not. 2016-12-19 18:18:57 -04:00
Joey Hess
95c8b37544
Linux standalone: Improve generation of locale definition files, supporting locales such as, en_GB.UTF-8. 2016-12-19 17:03:52 -04:00
Joey Hess
ccde0932a5
p2p --pair with magic wormhole (untested)
It builds. I have not tried to run it yet. :)

This commit was sponsored by Jake Vosloo on Patreon.
2016-12-18 16:51:41 -04:00
Joey Hess
38f9337e16
Revert "p2p --link now defaults to setting up a bi-directional link"
This reverts commit 3037feb1bf.

On second thought, this was an overcomplication of what should be the
lowest-level primitive. Let's build bi-directional links at the pairing
level with eg magic wormhole.
2016-12-16 18:26:07 -04:00
Joey Hess
bd811d3853
p2p: Added --one-way option.
This commit was sponsored by Fernando Jimenez on Patreon.
2016-12-16 16:43:37 -04:00
Joey Hess
3037feb1bf
p2p --link now defaults to setting up a bi-directional link
Both the local and remote git repositories get remotes added
pointing at one-another.

Makes pairing twice as easy!

Security: The new LINK command in the protocol can be sent repeatedly,
but only by a peer who has authenticated with us. So, it's entirely safe to
add a link back to that peer, or to some other peer it knows about.
Anything we receive over such a link, the peer could send us over the
current connection.

There is some risk of being flooded with LINKs, and adding too many
remotes. To guard against that, there's a hard cap on the number of remotes
that can be set up this way. This will only be a problem if setting up
large p2p networks that have exceptional interconnectedness.

A new, dedicated authtoken is created when sending LINK.

This also allows, in theory, using a p2p network like tor, to learn about
links on other networks, like telehash.

This commit was sponsored by Bruno BEAUFILS on Patreon.
2016-12-16 16:38:06 -04:00
Joey Hess
e67a310da1
p2p: --link no longer takes a remote name, instead the --name option can be used. 2016-12-16 15:37:50 -04:00
Joey Hess
469bfa7ff3
Make all --batch input, as well as fromkey and registerurl stdin be processed without requiring it to be in the current encoding. 2016-12-13 15:35:04 -04:00
Joey Hess
48d9624a2d
Revert ServerAliveInterval
Revert ServerAliveInterval change in 6.20161111, which caused problems
with too many old versions of ssh and unusual ssh configurations.

It should have not been needed anyway since ssh is supposted to
have TCPKeepAlive enabled by default.
2016-12-13 12:12:38 -04:00
Joey Hess
59fead6da3
Pass annex.web-options to wget and curl after other options, so that eg --no-show-progress can be set by the user to disable the default --show-progress. 2016-12-13 11:56:23 -04:00
Joey Hess
d9490685fd
metadata --batch: Fix bug when conflicting metadata changes were made in the same batch run.
1 microsecond delay is ugly.. but, maintaining an queue of a list of timestamps
and taking a new one from the queue each time around, or maintaining a timestamp
counter, would probably be slower.
2016-12-13 11:07:49 -04:00
Joey Hess
a52c011581
Debian: Build webapp on armel. 2016-12-11 21:30:07 -04:00
Joey Hess
bb66e098b1
linux standalone builds should have "unable to decommit memory" bug fixed 2016-12-11 15:37:52 -04:00
Joey Hess
73a79147b1
releasing package git-annex version 6.20161210 2016-12-10 12:23:18 -04:00
Joey Hess
749623df86
fixed 2016-12-10 10:47:16 -04:00
Joey Hess
15be5c04a6
git-annex-shell, remotedaemon, git remote: Fix some memory DOS attacks.
The attacker could just send a very lot of data, with no \n and it would
all be buffered in memory until the kernel killed git-annex or perhaps OOM
killed some other more valuable process.

This is a low impact security hole, only affecting communication between
local git-annex and git-annex-shell on the remote system. (With either
able to be the attacker). Only those with the right ssh key can do it. And,
there are probably lots of ways to construct git repositories that make git
use a lot of memory in various ways, which would have similar impact as
this attack.

The fix in P2P/IO.hs would have been higher impact, if it had made it to a
released version, since it would have allowed DOSing the tor hidden
service without needing to authenticate.

(The LockContent and NotifyChanges instances may not be really
exploitable; since the line is read and ignored, it probably gets read
lazily and does not end up staying buffered in memory.)
2016-12-09 13:34:32 -04:00
Joey Hess
2fb6fd7434
Merge branch 'master' into tor 2016-12-07 14:32:25 -04:00
Joey Hess
f61508aed4
add: Stage modified non-large files when running in indirect mode.
(This was already done in v6 mode and direct mode.)
2016-12-05 14:10:21 -04:00
Joey Hess
82d01f5619
rekey: Added --batch mode.
Would have liked to make the Parser parse the file and key pairs, but it
seems that optparse-applicative is unable to handle eg:

	many ((,) <$> argument <*> argument)

This commit was sponsored by Thomas Hochstein on Patreon.
2016-12-05 12:55:50 -04:00
Joey Hess
e65c31e56b
changelog 2016-12-05 12:16:35 -04:00
Joey Hess
93852dd7e8
rmurl: --batch
* rmurl: Multiple pairs of files and urls can be provided on the
  command line.
* rmurl: Added --batch mode.

This commit was sponsored by Trenton Cronholm on Patreon.
2016-12-05 12:10:07 -04:00
Joey Hess
bfc8305814
implement p2p command 2016-11-30 14:35:24 -04:00
Joey Hess
24593aaa32
Merge branch 'master' into tor 2016-11-30 14:16:36 -04:00
Joey Hess
8354612131
prefer xdot over dot
* map: Run xdot if it's available in PATH. On OSX, the dot command
  does not support graphical display, while xdot does.
* Debian: xdot is a better interactive viewer than dot, so Suggest
  xdot, rather than graphviz.
2016-11-30 12:50:49 -04:00
Joey Hess
398345cb26
Merge branch 'master' into tor 2016-11-29 15:45:29 -04:00
Joey Hess
ae9f99f342
Relicense 5 source files that are not part of the webapp from AGPL to GPL.
Building w/o the webapp is not supposed to pull in any AGPLed files.

I appear to have written all the code in these files;
the only commit by anyone else is 64e844e1fe
and is a spelling fix that is not copyrightable.
2016-11-21 23:46:59 -04:00
Joey Hess
070fb9e624
Added git-remote-tor-annex, which allows git pull and push to the tor hidden service.
Almost working, but there's a bug in the relaying.

Also, made tor hidden service setup pick a random port, to make it harder
to port scan.

This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
2016-11-21 17:27:38 -04:00
Joey Hess
6e6d1a8c15
addurl: Fix bug in checking annex.largefiles expressions using largerthan, mimetype, and smallerthan; the first two always failed to match, and the latter always matched. 2016-11-21 11:30:53 -04:00
Joey Hess
74691ddf0e
remotedaemon: serve tor hidden service 2016-11-20 15:48:12 -04:00
Joey Hess
a101b8de37
remotedaemon: Fork to background by default. Added --foreground switch to enable old behavior.
Groundwork for tor hidden services, which the remotedaemon will serve.
2016-11-20 14:50:36 -04:00
Joey Hess
5680565122
releasing package git-annex version 6.20161118 2016-11-18 11:59:49 -04:00
Joey Hess
aedec5d08d
arm build uses 32kb page size
(Change was made in gitannexbuilder scripts not here.)
2016-11-16 18:05:42 -04:00
Joey Hess
2577f1c0a2
fsck --all --from was checking the content of files in the local repository, rather than on the special remote.
Straight up forgot to handle this case!

This commit was sponsored by Fernando Jimenez on Patreon.
2016-11-16 15:33:57 -04:00
Joey Hess
0a4479b8ec
Avoid backtraces on expected failures when built with ghc 8; only use backtraces for unexpected errors.
ghc 8 added backtraces on uncaught errors. This is great, but git-annex was
using error in many places for a error message targeted at the user, in
some known problem case. A backtrace only confuses such a message, so omit it.

Notably, commands like git annex drop that failed due to eg, numcopies,
used to use error, so had a backtrace.

This commit was sponsored by Ethan Aubin.
2016-11-15 21:29:54 -04:00
Joey Hess
556b2ded2b
sync: Pass --allow-unrelated-histories to git merge when used with git git 2.9.0 or newer.
This makes merging a remote into a freshly created direct mode repository
work the same as it works in indirect mode.

The git-annex branches would get merged in any case by a sync,
since that doesn't use git merge.

This might need to be revisited later to better mirror git's behavior.
2016-11-15 18:26:17 -04:00
Joey Hess
6416ae9c09
unbreak all the autobuilders
git-annex.cabal: Loosen bounds on persistent to allow 2.5, which on Debian
has been patched to work with esqueleto. This may break cabal's resolver on
non-Debian systems; if so, either use stack to build, or run cabal with
--constraint='persistent ==2.2.4.1' Hopefully this mess with esqueleto will
be resolved soon.

https://github.com/prowdsponsor/esqueleto/issues/137
2016-11-15 11:19:57 -04:00
Joey Hess
e544cf7a31
releasing package git-annex version 6.20161111 2016-11-11 14:47:31 -04:00
Joey Hess
d48f4caaef
Linux standalone: Avoid using hard links in the tarball so it can be untarred on eg, afs which does not support them. 2016-11-10 15:12:30 -04:00
Joey Hess
4643470537
webapp: Explicitly avoid checking for auth in static subsite requests.
Yesod didn't used to do auth checks for that, but this may have changed.
I don't have a way to reproduce the reported problem yet, but this change
certianly won't hurt anything.

This commit was sponsored by Thom May on Patreon.
2016-11-10 13:48:54 -04:00
Joey Hess
c44ac268be
OSX: Remove RPATHs from git-annex binary, which are not needed, slow down startup, and break the OSX Sierra linker.
ghc 8.0.2 may make this unncessary, but it's not in a stackage version yet,
so put in a workaround.

Note that the linux builds already delete the RPATHs for similar reasons.

This commit was sponsored by Josh Taylor on Patreon.
2016-11-07 14:22:14 -04:00
Joey Hess
5afc2eaa54
reinject --known: Avoid second, unncessary checksum of file. 2016-11-07 12:07:36 -04:00