Commit graph

80 commits

Author SHA1 Message Date
Joey Hess
5c105a8540
test borg special remote
Sponsored-by: Boyd Stephen Smith Jr. on Patreon
2021-09-03 13:18:07 -04:00
Joey Hess
e17342b2a0
Run cp -a with --no-preserve=xattr, to avoid problems with copied xattrs
Including them breaking permissions setting on some NFS servers.

Sponsored-by: Dartmouth College's Datalad project
2021-08-27 13:09:34 -04:00
Joey Hess
898a677bce
fix build on windows
Sponsored-by: Dartmouth College's Datalad project
2021-07-01 12:52:33 -04:00
Joey Hess
9cb5ed2446
avoid using cp --reflink=auto on windows
Sponsored-by: Dartmouth College's Datalad project
2021-06-28 12:51:40 -04:00
Joey Hess
fef3cd055d
Removed support for git versions older than 2.1
debian oldoldstable has 2.1, and that's what i386ancient uses. It would be
better to require git 2.2, which is needed to use adjusted branches, but
can't do that w/o losing support for some old linux kernels or a
complicated git backport.
2019-09-11 16:14:43 -04:00
Joey Hess
21ff5e1e5a
CoW probing
Improved probing when CoW copies can be made between files on the same
drive. Now supports CoW between BTRFS subvolumes. And, falls back to rsync
instead of using cp when CoW won't work, eg copies between repos on the
same EXT4 filesystem.

Rather than trying cp --reflink=always for each file copied to a remote,
it's tried once and if it fails it falls back to using rsync thereafter
for the lifetime of the Remote object. That avoids overhead of calling cp
which while small, will add up over a large number of files.

This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
2019-07-17 14:19:08 -04:00
Joey Hess
0dc26cd6f1
remove android cross build leftover
Android cross builds are no longer done; I missed removing this
before.

This commit was sponsored by Brock Spratlen on Patreon.
2019-07-17 13:31:37 -04:00
Joey Hess
c565340adc
stop using external hash programs, since cryptonite is faster
In 2013, I wrote "Cryptohash benchmarks 90 to 101% faster than external
hashers". Re-benchmarking today, I found cryptonite's sha256 consistently
outperformed coreutils by 10% for large files. Tested 10 mb, 100 mb, 1 gb
files with both sha256 and sha512. And for smaller files, the external
process startup time swamps the hash time.

Perhaps cryptonite has improved. Or it could just do better on my
current CPU Intel(R) Pentium(R) CPU 4410Y @ 1.50GHz). Anyway, even if cryptonite
is slower in some situations, seems likely it would only be marginally slower;
it's got the same class of highly optimised C code under the hood as coreutils.
The main difference between the two sha256 implementations seems to be
how much of the inner loop they unroll..

This commit was sponsored by Henrik Riomar on Patreon.
2018-08-28 18:10:58 -04:00
Joey Hess
a1b55056fc
Make Build/BuildVersion update embedded version number
This way, autobuilders can run it before building, in a tree where
git-annex was already built, and get the current git rev embedded in the
build.

Before, the version number only updated when the setup program was run,
which was up to cabal and not on every build.
2018-06-26 10:36:54 -04:00
Joey Hess
87b329e451
remove code to update version in cabal file
I have not used this when making releases for a long time.
2018-06-26 10:04:42 -04:00
Joey Hess
c0c66e35b2
remove unused import 2018-04-22 13:42:04 -04:00
Joey Hess
c34152777b
Use http-conduit for url downloads by default, annex.web-options enables curl
* For url downloads, git-annex now defaults to using a http library,
  rather than wget or curl. But, if annex.web-options is set, it will
  use curl. To use the .netrc file, run:
    git config annex.web-options --netrc
* git-annex no longer uses wget (and wget is no longer shipped with
  git-annex builds).

Note that curl is always run in silent mode, since the new API for
download has a MeterUpdate and doesn't make way for curl progress
output. It might be worth writing a parser for curl's progress output
to update the meter when using it, but I didn't bother with this edge
case for now.

This commit was supported by the NSF-funded DataLad project.
2018-04-06 17:36:20 -04:00
Joey Hess
25703e1413
finally really add back custom-setup stanza
Fourth or fifth try at this and finally found a way to make it work.

Absurd amount of busy-work forced on me by change in cabal's behavior.
Split up Utility modules that need posix stuff out of ones used by
Setup. Various other hacks around inability for Setup to use anything
that ifdefs a use of unix.

Probably lost a full day of my life to this.
This is how build systems make their users hate them. Just saying.
2017-12-31 16:36:39 -04:00
Joey Hess
308cd1383c
fold Build/SysConfig.hs into BuildInfo via include
This avoids warnings from stack about the module not being listed in the
cabal file. So, the generated file is also renamed to Build/SysConfig.

Note that the setup program seems to be cached despite these changes; I
had to cabal clean to get cabal to update it so that Build/SysConfig was
written.

This commit was sponsored by Jochen Bartl on Patreon.
2017-12-14 12:46:57 -04:00
Joey Hess
1b6cbb63e9
still can't express custom-setup deps
They need unix on non-windows, for Utility.Env, which Build.Configure uses,
but cabal can't express that in a custom-setup stanza.

To avoid this problem, Utility.Env would need to be moved into
unix-compat..
2017-11-14 14:59:51 -04:00
Joey Hess
061f5ddd43
avoid Utility.Directory which needs Win32 on windows
custom-setup can't depend on Win32 due to cabal limitations
2017-11-14 14:53:36 -04:00
Joey Hess
96734ba350
allow building w/o rsync installed
rsync is needed to run, but may not be on the build machine; that's ok
2017-10-25 17:24:03 -04:00
Joey Hess
4a397b5313
Run wget with -nv instead of -q, so it will display HTTP errors.
This adds one extra line of output when a download is successful,
after the progress bar. I don't much like that, but wget does not provide a
way to show HTTP errors without it.
2017-02-20 15:25:02 -04:00
Joey Hess
49b3ef88f7
Android: Fix disabling use of cp --reflink=auto, curl, sha224, and sha384.
This was originally done in a7ef05a9, but got lost in some change to the
Makefile. Use CROSS_COMPILE=Android to tell configure that it's configuring
for android instead of passing it a parameter.
2016-09-05 14:11:35 -04:00
Joey Hess
b22409db38
avoid warnings about not exported System.Directory.isSymbolicLink 2016-04-28 15:18:11 -04:00
Joey Hess
5fe450514b
Fix build with directory-1.2.6.2.
It started exporting a isSymbolicLink which supports windows. But,
git-annex does no use symlinks on windows yet and this conflicts with the
function by the same name from unix-compat, so hide it.
2016-04-28 13:18:44 -04:00
Joey Hess
5c7cdbae46 more {-# OPTIONS_GHC -fno-warn-tabs #-} ... Forcing people who have what is merely a difference of opinion to you to do this is a bit of an asshole move. Just saying. 2015-05-10 16:38:49 -04:00
Joey Hess
5871f7abd2 put in workaround for strange version of git on the autobuilder 2015-04-14 14:44:19 -04:00
Joey Hess
a92737c8b9 Fix build failure when wget is not installed. 2015-01-15 15:42:15 -04:00
Joey Hess
c64ede23cd Use wget -q --show-progress for less verbose wget output, when built with wget 1.16. 2014-12-16 14:04:40 -04:00
Joey Hess
6eb5c3f479 Do not preserve permissions and acls when copying files from one local git repository to another. Timestamps are still preserved as long as cp --preserve=timestamps is supported.
This avoids cp -a overriding the default mode acls that the user might have
set in a git repository.

With GNU cp, this behavior change should not be a breaking change, because
git-anex also uses rsync sometimes in the same situation, and has only ever
preserved timestamps when using rsync.

Systems without GNU cp will no longer use cp -a, but instead just cp.
So, timestamps will no longer be preserved. Preserving timestamps when
copying between repos is not guaranteed anyway.

Closes: #729757
2014-08-26 17:10:25 -07:00
Joey Hess
3a7ae6f144 indentation 2014-07-13 22:33:27 -04:00
Joey Hess
a615817ab1 add Build/BuildVersion, for use by autobuilders 2014-06-18 14:29:39 -04:00
Joey Hess
dc0a8b7618 Revert "add debugging for odd OOM or hang on 5 architectures, all where configure runs sha224sum"
This reverts commit 238a935f9c.

The OOM was cabal's way of telling us it couldn't resolve dependencies.
2014-04-22 20:23:37 -04:00
Joey Hess
238a935f9c add debugging for odd OOM or hang on 5 architectures, all where configure runs sha224sum 2014-04-20 19:55:08 -04:00
Joey Hess
b04f62a962 make configure -Wall clean 2014-03-10 16:59:38 -04:00
Joey Hess
7ac37a7854 Probe for quvi version at run time.
Overhead: git annex addurl runs quvi --version once.
And more bloat to Annex state..
2014-02-28 14:54:02 -04:00
Joey Hess
5a5adc44f5 Refuse to build with git older than 1.7.1.1, which is needed for git checkout -B 2014-01-13 15:17:48 -04:00
Joey Hess
22d6019bae add back sha commands on OSX; SIGILL problem fixed 2014-01-06 16:11:21 -04:00
Joey Hess
1118247394 OSX: Avoid using external hash programs, since SIGILL is not fun 2014-01-01 16:26:43 -04:00
Joey Hess
0d91432442 test for nocache at configure time 2013-12-01 15:12:32 -04:00
Joey Hess
e2f50f5110 Added support for quvi 0.9. Slightly suboptimal due to limitations in its interface compared with the old version. 2013-11-24 23:44:30 -04:00
Joey Hess
0e8e684d6c no-op change to ensure configure is re-run on autobuilders to pick up UPGRADE_LOCATION 2013-11-22 15:25:42 -04:00
Joey Hess
e2f17e9da3 upgrade alerts
The webapp will check twice a day, when the network is connected, to see if
it can download a distributon upgrade file. If a newer version is found,
display an upgrade alert.

This will need the autobuilders to set UPGRADE_LOCATION to the url
it can be downloaded from when building git-annex. Only builds with that
set need automatic upgrade alerts.

Currently, the upgrade page just requests the user manually download
and upgrade it. But, all the info is provided to do automated upgrades
in the future.

Note that urls used will need to all be https.

This commit was sponsored by Dirk Kraft.
2013-11-21 17:49:56 -04:00
Joey Hess
faf032c55a avoid using "gcrypt" for git-remote-gcrypt; there's also a library by that name 2013-10-21 18:45:19 -04:00
Joey Hess
c80bc53960 cronner builds, should work (untested)
I probably need to improve handling of the PleaseTerminate exception to
kill the fsck process. Also, if fsck finds bad files, something needs
to requeue downloads of them. Otherwise, this should work, but is probably
quite buggy since I have only tested the pure code over the past 2 days.
2013-10-08 18:13:08 -04:00
Joey Hess
990924703f factor out Build.Version 2013-10-06 17:48:38 -04:00
Joey Hess
a9cb7d4744 add git-remote-gcrypt to bundle when available 2013-09-13 12:05:47 -04:00
Joey Hess
46b6d75274 Youtube support! (And 53 other video hosts)
When quvi is installed, git-annex addurl automatically uses it to detect
when an page is a video, and downloads the video file.

web special remote: Also support using quvi, for getting files,
or checking if files exist in the web.

This commit was sponsored by Mark Hepburn. Thanks!
2013-08-22 18:50:43 -04:00
Joey Hess
672cfc3923 better git version checking 2013-08-02 18:32:26 -04:00
Joey Hess
53d52d57c1 check in configure if ionice -c3 works
On old systems, it may need to be run as root.
2013-06-21 13:43:04 -04:00
Joey Hess
d901ba1781 assistant --autostart: Automatically ionices the daemons it starts. 2013-06-21 13:23:20 -04:00
Joey Hess
d5ba9cb728 Allow building with gpg2. 2013-05-19 17:59:58 -04:00
Joey Hess
39ac8d18ed Sanitize debian changelog version before putting it into cabal file. Closes: #708619 2013-05-17 11:24:18 -04:00
Joey Hess
dc66b1f27d Merge branch 'master' into windows
Conflicts:
	Annex/Environment.hs
	Build/Configure.hs
	Git/Construct.hs
	Utility/FileMode.hs
2013-05-14 15:37:24 -04:00