Commit graph

499 commits

Author SHA1 Message Date
Joey Hess
c62d2ce5c5
avoid strictness problem 2019-01-22 19:37:49 -04:00
Joey Hess
5d58c54a8b
avoid writing to Build/Version when it already contains the same version
This will hopefully avoid ghc unncessarily rebuilding everything that
depends on it due to the file's timestamp changing.
2019-01-22 16:54:17 -04:00
Joey Hess
112bb82fc2
Windows: If 64 bit git is installed, use it when installing git-annex.
However, rsync still won't work with 64 bit git and
this is still not the documented way to install it.

So, if both 64 and 32 are installed, go with 32.

And if neither git can be found, default to 32.
2019-01-21 15:51:48 -04:00
Joey Hess
cbdb6671d6
remove old android app autobuild 2018-10-13 12:13:13 -04:00
Joey Hess
38d691a10f
removed the old Android app
Running git-annex linux builds in termux seems to work well enough that the
only reason to keep the Android app would be to support Android 4-5, which
the old Android app supported, and which I don't know if the termux method
works on (although I see no reason why it would not).
According to [1], Android 4-5 remains on around 29% of devices, down from
51% one year ago.

[1] https://www.statista.com/statistics/271774/share-of-android-platforms-on-mobile-devices-with-android-os/

This is a rather large commit, but mostly very straightfoward removal of
android ifdefs and patches and associated cruft.

Also, removed support for building with very old ghc < 8.0.1, and with
yesod < 1.4.3, and without concurrent-output, which were only being used
by the cross build.

Some documentation specific to the Android app (screenshots etc) needs
to be updated still.

This commit was sponsored by Brett Eisenberg on Patreon.
2018-10-13 01:41:11 -04:00
Joey Hess
0240775f32
adding arm64 build, and improved termux installation process
* Added arm64 Linux standalone build. (No autobuilder yet.)
* Improved termux installation process.

Added git-annex-install.sh script to avoid user needing to type as much in
termux. The scope of this script is limited; runshell handles the rest.

Runshell runs termux-fix-shebang on the shell scripts. The problem is
the bundled bin/sh script, deleting that script also works, but then the
others probably use the system Android /bin/sh, which could be old or
broken or not posix or whatever. Using termux sh to run the scripts is
better.

This commit was sponsored by Eric Drechsel on Patreon.
2018-10-11 13:32:00 -04:00
Joey Hess
c565340adc
stop using external hash programs, since cryptonite is faster
In 2013, I wrote "Cryptohash benchmarks 90 to 101% faster than external
hashers". Re-benchmarking today, I found cryptonite's sha256 consistently
outperformed coreutils by 10% for large files. Tested 10 mb, 100 mb, 1 gb
files with both sha256 and sha512. And for smaller files, the external
process startup time swamps the hash time.

Perhaps cryptonite has improved. Or it could just do better on my
current CPU Intel(R) Pentium(R) CPU 4410Y @ 1.50GHz). Anyway, even if cryptonite
is slower in some situations, seems likely it would only be marginally slower;
it's got the same class of highly optimised C code under the hood as coreutils.
The main difference between the two sha256 implementations seems to be
how much of the inner loop they unroll..

This commit was sponsored by Henrik Riomar on Patreon.
2018-08-28 18:10:58 -04:00
Joey Hess
f6b449eb3e
type signature 2018-08-04 14:21:30 -04:00
Joey Hess
a1b55056fc
Make Build/BuildVersion update embedded version number
This way, autobuilders can run it before building, in a tree where
git-annex was already built, and get the current git rev embedded in the
build.

Before, the version number only updated when the setup program was run,
which was up to cabal and not on every build.
2018-06-26 10:36:54 -04:00
Joey Hess
7ecbb6bc20
stop using git commit date as version
This turns out to have been prolimatic because building from a git tag
would make a git-annex that claimed a different version than the tag's
version.

The git rev is still included in the version, and people who want
to know the exact tree that was built can just use that, so there's no
real benefit to automatically advancing the date part of the version.
2018-06-26 10:11:28 -04:00
Joey Hess
87b329e451
remove code to update version in cabal file
I have not used this when making releases for a long time.
2018-06-26 10:04:42 -04:00
Joey Hess
74890e1457
set 6.20180626 as urgent upgrade
This causes the webapp to tell the user's it's an urgent upgrade, which
this security fix is.
2018-06-25 21:52:02 -04:00
Joey Hess
3f0d875b55
Include uname in standalone builds. 2018-06-16 10:02:05 -04:00
Joey Hess
a73d60785c
fix build 2018-04-27 12:59:09 -04:00
Joey Hess
c0c66e35b2
remove unused import 2018-04-22 13:42:04 -04:00
Joey Hess
9b8075e79f
fix build 2018-04-09 20:22:46 -04:00
Joey Hess
b47c23bb5c
fix build 2018-04-09 20:15:52 -04:00
Joey Hess
c34152777b
Use http-conduit for url downloads by default, annex.web-options enables curl
* For url downloads, git-annex now defaults to using a http library,
  rather than wget or curl. But, if annex.web-options is set, it will
  use curl. To use the .netrc file, run:
    git config annex.web-options --netrc
* git-annex no longer uses wget (and wget is no longer shipped with
  git-annex builds).

Note that curl is always run in silent mode, since the new API for
download has a MeterUpdate and doesn't make way for curl progress
output. It might be worth writing a parser for curl's progress output
to update the meter when using it, but I didn't bother with this edge
case for now.

This commit was supported by the NSF-funded DataLad project.
2018-04-06 17:36:20 -04:00
Joey Hess
5574e25c54
OSX app: Work around libz/libPng/ImageIO.framework version skew
By not bundling libz, assuming OSX includes a suitable libz.1.dylib.

This commit was supported by the NSF-funded DataLad project.
2018-03-22 11:53:10 -04:00
Joey Hess
3fa8245df6
fix build some more 2018-01-15 12:18:00 -04:00
Joey Hess
ec5d89e72b
fix build 2018-01-15 12:17:09 -04:00
Joey Hess
2b28f6130a
fix windows build 2018-01-10 13:23:10 -04:00
Joey Hess
25703e1413
finally really add back custom-setup stanza
Fourth or fifth try at this and finally found a way to make it work.

Absurd amount of busy-work forced on me by change in cabal's behavior.
Split up Utility modules that need posix stuff out of ones used by
Setup. Various other hacks around inability for Setup to use anything
that ifdefs a use of unix.

Probably lost a full day of my life to this.
This is how build systems make their users hate them. Just saying.
2017-12-31 16:36:39 -04:00
Joey Hess
308cd1383c
fold Build/SysConfig.hs into BuildInfo via include
This avoids warnings from stack about the module not being listed in the
cabal file. So, the generated file is also renamed to Build/SysConfig.

Note that the setup program seems to be cached despite these changes; I
had to cabal clean to get cabal to update it so that Build/SysConfig was
written.

This commit was sponsored by Jochen Bartl on Patreon.
2017-12-14 12:46:57 -04:00
Joey Hess
a59d1a0ee6
remove unused import 2017-11-21 12:46:20 -04:00
Joey Hess
1b6cbb63e9
still can't express custom-setup deps
They need unix on non-windows, for Utility.Env, which Build.Configure uses,
but cabal can't express that in a custom-setup stanza.

To avoid this problem, Utility.Env would need to be moved into
unix-compat..
2017-11-14 14:59:51 -04:00
Joey Hess
061f5ddd43
avoid Utility.Directory which needs Win32 on windows
custom-setup can't depend on Win32 due to cabal limitations
2017-11-14 14:53:36 -04:00
Joey Hess
50c0fb7336
unify stack build and stack install
For some reason, when stack install is run separately, it seems to be
installing a previous build from a different temp dir. WTF?
2017-10-26 10:10:02 -04:00
Joey Hess
96734ba350
allow building w/o rsync installed
rsync is needed to run, but may not be on the build machine; that's ok
2017-10-25 17:24:03 -04:00
Joey Hess
54e596894f
Windows: Remove wget from bundle; it needs libraries that are not included, and git for windows includes curl which git-annex will use instead.
wget was broken even in the previous old release of the windows bundle,
this is not new breakage. msys-idn-11.dll and probably more would be needed
to use it. git for windows includes msys-idn2-0.dll instead.
2017-10-25 13:51:31 -04:00
Joey Hess
edcce072a6
ln from stack install location to nsis location 2017-10-25 00:14:53 -04:00
Joey Hess
7224a58e5d
ding dong, the witch is dead
Which old witch? The EvilLinker witch!

This commit was sponsored by Thom May on Patreon.
2017-10-24 13:24:42 -04:00
Joey Hess
59a0308f5f
update windows build scripts to use stack
Not tested yet.

The EvilLinker workaround is removed. That got fixed in ghc 8.0.1,
(per https://ghc.haskell.org/trac/ghc/ticket/8596)
which will finally be used by the windows autobuilder now.

I have not deleted the EvilLinker yet (or closed its bugs).

This commit was sponsored by John Peloquin on Patreon.
2017-10-17 13:00:29 -04:00
Joey Hess
3bd3885dbb
fix OSX build 2017-06-09 14:57:11 -04:00
Joey Hess
a1730cd6af
adeiu, MissingH
Removed dependency on MissingH, instead depending on the split
library.

After laying groundwork for this since 2015, it
was mostly straightforward. Added Utility.Tuple and
Utility.Split. Eyeballed System.Path.WildMatch while implementing
the same thing.

Since MissingH's progress meter display was being used, I re-implemented
my own. Bonus: Now progress is displayed for transfers of files of
unknown size.

This commit was sponsored by Shane-o on Patreon.
2017-05-16 01:03:52 -04:00
Joey Hess
11d3219985
Linux standalone builds put the bundled ssh last in PATH, so any system ssh will be preferred over it.
This commit was sponsored by Denis Dzyubenko on Patreon.
2017-03-02 17:40:40 -04:00
Joey Hess
627dc6036e
remove unused import 2017-02-28 14:08:56 -04:00
Joey Hess
27eca014be
fix up Read instance incompatability caused by recent commit
9c4650358c changed the Read instance for
Key.

I've checked all uses of that instance (by removing it and seeing what
breaks), and they're all limited to the webapp, except one.
That is GitAnnexDistribution's Read instance.

So, 9c4650358c would have broken upgrades
of git-annex from downloads.kitenet.net. Once the .info files there got
updated for a new release, old releases would have failed to parse them
and never upgraded.

To fix this, I found a way to make the .info files that contain
GitAnnexDistribution values be readable by the old version of git-annex.

This commit was sponsored by Ewen McNeill.
2017-02-24 18:59:12 -04:00
Joey Hess
4a397b5313
Run wget with -nv instead of -q, so it will display HTTP errors.
This adds one extra line of output when a download is successful,
after the progress bar. I don't much like that, but wget does not provide a
way to show HTTP errors without it.
2017-02-20 15:25:02 -04:00
Joey Hess
113b10cdc9
simpler more generic processTranscript'
This allows using functions that generate CreateProcess and passing the
result to processTranscript', which is more flexible, and also simpler
than the old interface.

This commit was sponsored by Riku Voipio.
2017-02-15 16:02:10 -04:00
Joey Hess
3ecbde6376
fix OSX build 2017-02-13 12:41:28 -04:00
Joey Hess
9eb10caa27
Some optimisations to string splitting code.
Turns out that Data.List.Utils.split is slow and makes a lot of
allocations. Here's a much simpler single character splitter that behaves
the same (even in wacky corner cases) while running in half the time and
75% the allocations.

As well as being an optimisation, this helps move toward eliminating use of
missingh.

(Data.List.Split.splitOn is nearly as slow as Data.List.Utils.split and
allocates even more.)

I have not benchmarked the effect on git-annex, but would not be surprised
to see some parsing of eg, large streams from git commands run twice as
fast, and possibly in less memory.

This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
2017-01-31 19:06:22 -04:00
Joey Hess
e8a9b3cb7e
fix build 2016-12-28 14:31:43 -04:00
Joey Hess
8484c0c197
Always use filesystem encoding for all file and handle reads and writes.
This is a big scary change. I have convinced myself it should be safe. I
hope!
2016-12-24 14:46:31 -04:00
Joey Hess
ce2828a12f
finish fixing android build after tor merge 2016-12-12 13:27:39 -04:00
Joey Hess
f4b7caaa61
fix some more 2016-12-10 12:29:29 -04:00
Joey Hess
a88b5e7786
better fix 2016-12-10 12:13:41 -04:00
Joey Hess
dd715b5993
improve hask to work with TH for Free monads 2016-12-10 11:55:56 -04:00
Joey Hess
070fb9e624
Added git-remote-tor-annex, which allows git pull and push to the tor hidden service.
Almost working, but there's a bug in the relaying.

Also, made tor hidden service setup pick a random port, to make it harder
to port scan.

This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
2016-11-21 17:27:38 -04:00
Joey Hess
d2b49090b8
remove debug print 2016-11-15 21:52:10 -04:00
Joey Hess
d48f4caaef
Linux standalone: Avoid using hard links in the tarball so it can be untarred on eg, afs which does not support them. 2016-11-10 15:12:30 -04:00
Joey Hess
5485a8fe62
remove localedir file, fix build 2016-10-05 11:29:14 -04:00
Joey Hess
aacd9b190d
Linux standalone: Include locale files in the bundle, and generate locale definition files for the locales in use when starting runshell.
Currently only done for utf-8 locales because the charset can easily be
told for those. Other locales don't include the charset in their name.

The locale definition is generated under git-annex.linux/locales.
So, this only works if the user can write there.

If locale generation fails for any reason, it's silently skipped.

The git-annex-standalone.deb installs the bundle under /usr, so this locale
generation won't work for non-root users.
2016-10-04 16:37:43 -04:00
Joey Hess
c079811226
Linux standalone: Add back the LOCPATH=/dev/null hack to avoid the system locale-archive being read.
Version mismatches between the system locale-archive and the glibc in the
bundle have been observed to cause git crashes.

Unfortunately, this causes locales to not be used in the linux standalone
bundle, as was the case until version 6.20160419.

glibc hardcodes the path to /usr/lib/locale/locale-archive and does not
let an environment variable cause a different locale-archive file to be used.

The only other option to include locales in the bundle would be to include
exploded locale definition directories in the bundle for a number of
locales, generated by localedef. But these take at least 300 kb per locale,
and there are a great many locales; it would be hundreds of megabytes to
include them all.

(Hmm, we could include localdef in the bundle, and check LANG in runshell
and compile the locale directories on the fly. This would need
/usr/share/i18n/ and /usr/lib/locale-archive to be included in the bundle.
It's.. doable.)

I know this is going to once again cause users of the bundle to complain
that eg, ls doesn't show their unicode filenames right. Better than strange
crashes though.
2016-10-04 12:53:09 -04:00
Joey Hess
49b3ef88f7
Android: Fix disabling use of cp --reflink=auto, curl, sha224, and sha384.
This was originally done in a7ef05a9, but got lost in some change to the
Makefile. Use CROSS_COMPILE=Android to tell configure that it's configuring
for android instead of passing it a parameter.
2016-09-05 14:11:35 -04:00
Jim Paris
e2a55fdbae Don't escape leading dots in code blocks in manpage
The code block in git-annex-smudge(1) was misformatted.  Code blocks
start with tabs, so replace "\s" with " ".  Tested to not affect
anything except git-annex-smudge.1.
2016-08-16 11:51:16 -04:00
Joey Hess
43d9d757ee
fix build with old ghc 2016-06-13 15:25:05 -04:00
Joey Hess
c8dd196234
fix man page building 2016-06-02 16:54:58 -04:00
Yaroslav Halchenko
64e844e1fe
minor typo fixes throughout
problematic
flexibility
2016-06-02 11:22:18 -04:00
Joey Hess
f4489cc415
Remove Makefile from cabal tarball; man page building is now handled by a small haskell program.
This actually runs faster than building the man pages from the makefile
did. But the main purpose is to let Setup.hs import Build.Mans and so not
need the makefile.
2016-05-31 13:58:13 -04:00
Joey Hess
f79875ef3b
Updated cabal file explictly lists source files.
The tarball on hackage will include only the files needed for cabal install;
it is NOT the full git-annex source tree. While it's totally obnoxious that
cabal files need every file listed out when basic wildcard support could
avoid hundreds of lines, and have to be maintained when files are added,
this does get the tarball size back down to 1 mb.

This also stops stack from complaining that it found modules not listed in
the cabal file.

debian/changelog, debian/NEWS, debian/copyright: Converted to symlinks
to CHANGELOG, NEWS, and COPYRIGHT, which used to symlink to these instead.
This avoids needing to include debian/ in the hackage tarball.

Setup.hs: Build man pages at install time using make and mdwn2man.
If it fails, which it probably will on windows, just skip installing
them.
2016-05-24 01:28:07 -04:00
Joey Hess
b22409db38
avoid warnings about not exported System.Directory.isSymbolicLink 2016-04-28 15:18:11 -04:00
Joey Hess
5fe450514b
Fix build with directory-1.2.6.2.
It started exporting a isSymbolicLink which supports windows. But,
git-annex does no use symlinks on windows yet and this conflicts with the
function by the same name from unix-compat, so hide it.
2016-04-28 13:18:44 -04:00
Joey Hess
71e5bebc45
remove warning 2016-04-27 15:17:02 -04:00
Joey Hess
665d7ac6e0
fix build with old ghc that lacks MIN_VERSION_GLASGOW_HASKELL macro 2016-04-27 14:32:05 -04:00
Joey Hess
f6c6963ed4
only filter out mdwn2man warning, not other warnings 2016-04-20 14:29:54 -04:00
Joey Hess
f947781fa0
Avoid setting LOCPATH in linux standalone builds now that ghc has been fixed to not hang when it cannot find locale files. 2016-04-20 12:40:11 -04:00
Joey Hess
7a77985f37
hack to get escaped quotes right on git-annex-shell man page 2016-04-04 17:11:35 -04:00
Joey Hess
8bed51ae95
use clamav to check builds for viruses before adding them
Had to workaround various problems in clamscan. Increased its max filesize
a lot, because it's too small to check git-annex. Manual unpacking seemed
to be needed for dmg and tar.gz.
2016-03-07 15:54:27 -04:00
Joey Hess
ab5f7b05f5
Fix OSX dmg to include libraries needed by bundled gpg, lost in last release. 2016-03-07 12:55:01 -04:00
Joey Hess
ae0a10618b
fix build more 2016-02-23 12:48:18 -04:00
Joey Hess
d04af5fb68
fix build error 2016-02-23 12:30:11 -04:00
Joey Hess
63e6f08e0b
avoid trying to include gpg in bundle on Windows 2016-02-23 11:33:07 -04:00
Joey Hess
398d93d4d2
Linux and OSX standalone builds put the bundled gpg last in PATH, so any system gpg will be preferred over it. 2016-02-19 16:19:19 -04:00
Joey Hess
05e7cdce71
fix warning with ghc 7.10 2016-01-26 19:12:33 -04:00
Joey Hess
737e45156e
remove 163 lines of code without changing anything except imports 2016-01-20 16:36:33 -04:00
Joey Hess
6a5e241a74
throw exception if library copy fails 2016-01-14 17:56:21 -04:00
Joey Hess
9240c02fd1
fix build of DistributionUpdate 2016-01-14 15:55:37 -04:00
Joey Hess
927e1a067e
fix import warnings 2016-01-14 10:30:54 -04:00
Joey Hess
4ca9fa4d39
fix build warning with AMP ghc 2015-12-28 13:16:46 -04:00
Joey Hess
11ceee93b4
fix build warning 2015-12-28 12:41:36 -04:00
Joey Hess
e5323f4414
got logic backwards.. 2015-11-03 10:51:21 -04:00
Joey Hess
0717ad4440
Avoid installing desktop file and program file if cabal install git-annex is run as root, since that is not a systemwide install, but to /root, and so generating a systemwide desktop file is not right. 2015-11-03 10:38:08 -04:00
Joey Hess
933fef6ae0 Merge branch 'winprocfix' 2015-10-04 15:46:25 -04:00
Joey Hess
1c98029bb0 add i386-ancient build 2015-09-23 13:36:45 -04:00
Joey Hess
5aa5e92da7 git for windows includes curl, so avoid bundling it 2015-09-11 14:01:42 -04:00
Joey Hess
1355a241ea now fully working even when git is not in path
The vbs launchers now are passed the path where git-annex was installed to,
so they will work when it's not in path.
2015-09-11 13:18:21 -04:00
Joey Hess
c7a6296ef3 get git-annex working even if user doesn't add git to path
This is pretty complicated, but I have both "git-annex" and "git annex"
working both in the git bash shell even with git not added to path.
And, when git's added to path, both work from MS-DOS prompt window too.

I think that the webapp startup does still need git in path, so
instructions will keep saying to do that. But, users often disregard them,
and hopefully this will reduce support traffic.

Also, switched the wget from the cygwin one to the msys2 one, avoiding the
complication of needing to bundle any cygwin dlls.
2015-09-11 12:44:27 -04:00
Joey Hess
6158036e23 Switched to using git for Windows, rather than msysgit.
Using msysgit with git-annex is no longer supported.

At the same time, I'm updating the rsync.exe in my downloads repository
with the one from msys2.

Note that rsync is currently still being ldded and installed in Git/cmd/
like the other cygwin programs. The ldd fails and this failure is ignored.
It would be better to special case it to go in Git/usr/bin/, so that the
user can't run rsync in a dos prompt window, which doesn't work, as it needs
additional libs. However, as far as git-annex running rsync running ssh,
it works ok in this location.

Removed the ssh.cmd and ssh-keygen.cmd; these are not needed with git for
windows. Keeping them would let ssh be run manually from a dos prompt
window, but that's not really a goal.
2015-09-10 19:16:30 -04:00
Joey Hess
330f671a21 remove bup from bundledprograms
It never belonged there.
2015-09-10 12:37:05 -04:00
Joey Hess
19dbe2a611 webapp: Fix support for entering password when setting up a ssh remote. 2015-09-03 11:03:08 -07:00
Joey Hess
4ea33b09c9 update for ghc 7.10 2015-09-01 15:02:37 -07:00
Joey Hess
86e638567a Fix Windows build to work with ghc 7.10
It was failing at link time, some problem with terminatePID.
Re-implemented that to not use a C wrapper function, which cleared up the
problem. Removed old EvilLinker hack with must have been related to the
same problem.

Note that I have not tested this with older ghc's. In
f11f7520b5 I mention having tried this
approach before, and getting segfaults.. So, who knows. It seems to work
fine with ghc 7.10 at least.
2015-09-01 14:51:14 -07:00
Joey Hess
1de67fa46e remove unused import 2015-08-24 14:26:41 -07:00
Joey Hess
485465107a Makefile: Pass LDFLAGS, CFLAGS, and CPPFLAGS through ghc and on to ld, cc, and cpp.
As a result of the Makefile changes, the Debian package is built
with various hardening options. Although their benefit to a largely
haskell program is unknown.
2015-08-19 13:53:57 -04:00
Joey Hess
6204dcbd7b fix formatting of git-annex(1) synopsis 2015-07-09 11:22:37 -04:00
Joey Hess
eb33569f9d remove Params constructor from Utility.SafeCommand
This removes a bit of complexity, and should make things faster
(avoids tokenizing Params string), and probably involve less garbage
collection.

In a few places, it was useful to use Params to avoid needing a list,
but that is easily avoided.

Problems noticed while doing this conversion:

	* Some uses of Params "oneword" which was entirely unnecessary
	  overhead.
	* A few places that built up a list of parameters with ++
	  and then used Params to split it!

Test suite passes.
2015-06-01 13:52:23 -04:00
Joey Hess
799dc8d329 avoid unused variable 2015-05-27 17:02:23 -04:00
Joey Hess
38c8b757d8 munge NAME section of man pages to make lintian happy 2015-05-22 16:06:21 -04:00
Joey Hess
6168537c34 require RELEAEE_BUILD to be 1, not any value 2015-05-11 09:44:01 -04:00