Commit graph

748 commits

Author SHA1 Message Date
Joey Hess
b90ddbc383
enable-tor: Use pkexec to run command as root when gksu and kdesu are not available.
gksu is no longer in debian, even stable

kdesu in debian is not installed in PATH any longer, though the executable
is still present under /usr/lib

pkexec is packagekit's replacement for those older commands.
2019-09-30 15:19:01 -04:00
Joey Hess
f2737a5fbe
enable-tor: Run kdesu with -c option. 2019-09-30 15:14:05 -04:00
Joey Hess
2b55a2b882
remotedaemon: Don't list --stop in help since it's not supported.
Also, move out of plumbing section. When using tor, the remotedaemon is
part of the user's workflow, as it runs the tor hidden service.
2019-09-30 14:40:46 -04:00
Joey Hess
090898a138
adjust --lock: This enters an adjusted branch where files are locked.
Straightforward, except for the issue of how to reverse LockAdjustment.

With --unlock, a commit that modifies/adds unlocked files gets reverse
adjusted to use locked files. That's fairly reasonable, I think.

But reversing --lock by unlocking all modified files feels wrong. Maybe
that's just because repositories typically seem to still have mostly
locked files in them (unless one is in an adjusted unlocked branch of
course!)

It may be that eventually how to reverse both will need to be configurable,
I don't know.
2019-09-27 14:23:25 -04:00
Joey Hess
9628ae2e67
Close sqlite databases more robustly.
Had a report of close throwing ErrorBusy on CIFS.

Retrying up to 16 seconds is a balance between hopefully waiting long
enough for the problem to clear up and waiting so long that git-annex seems
to hang.

The new dependency is free; persistent depends on unliftio-core.
2019-09-26 12:25:21 -04:00
Joey Hess
8af791d769
Test: Use more robust directory removal method.
I just had a test that crashed at cleanup on linux with:

.t/gpgtest/12/S.gpg-agent.browser: removeDirectoryRecursive:removeContentsRecursive:removePathRecursive:removeContentsRecursive:removePathRecursive:removeContentsRecursive:removePathRecursive:getSymbolicLinkStatus: does not exist (No such file or directory)
sleeping 10 seconds and will retry directory cleanup
git-annex: .t/gpgtest/14/S.gpg-agent.browser: removeDirectoryRecursive:removeContentsRecursive:removePathRecursive:removeContentsRecursive:removePathRecursive:removeContentsRecursive:removePathRecursive:getSymbolicLinkStatus: does not exist (No such file or directory)

removePathForcibly is supposed to be more robust to things in the directory vanishing while it's running, etc.
Will probably avoid such crashes.

It was added to directory-1.2.7, which comes with ghc since 8.0.2.
Since base >= 4.11.1.0 means ghc 8.4.4, I expect all builds will have it,
but I ifdefed it to be sure.
2019-09-24 16:59:37 -04:00
Joey Hess
6ae0a44c64
git-lfs: Added support for http basic auth 2019-09-24 14:46:20 -04:00
Joey Hess
de564df8b3
git-lfs: Only do endpoint discovery once when concurrency is enabled
This avoids some extra work, but I don't think it was possible for two ssh
endpoint discoveries run concurrently to both prompt for the ssh password;
Annex.Ssh itself deals with concurrency.

This is mostly groundwork for http password prompting.
2019-09-24 13:01:51 -04:00
Joey Hess
b13a350556
added --unlocked and --locked 2019-09-19 12:33:13 -04:00
Joey Hess
fda1bdd679
Added --mimetype and --mimeencoding file matching options.
Already had these for largefiles matching, but I forgot to add them as
command-line options.
2019-09-19 12:09:59 -04:00
Joey Hess
ab739242a3
releasing package git-annex version 7.20190912 2019-09-13 12:53:40 -04:00
Joey Hess
a8fea1644d
docs for git-annex-standalone rpm 2019-09-13 12:18:36 -04:00
Joey Hess
4508198507
building a standalone rpm from the standalone tarball
This allows the rpm to be built anywhere the necessary build deps are
available (including on debian) and the resulting package will work on as
broad a range of rpm distributions as the libc/kernel supports.

The DistributionUpdate changes to use the new script have not yet been
tested.
2019-09-13 11:53:17 -04:00
Joey Hess
4a4e08e123
release prep 2019-09-12 13:53:22 -04:00
Joey Hess
fef3cd055d
Removed support for git versions older than 2.1
debian oldoldstable has 2.1, and that's what i386ancient uses. It would be
better to require git 2.2, which is needed to use adjusted branches, but
can't do that w/o losing support for some old linux kernels or a
complicated git backport.
2019-09-11 16:14:43 -04:00
Joey Hess
061231621e
Merge branch 'master' into v7-default 2019-09-10 16:06:43 -04:00
Joey Hess
94c75d2bd9
init: Fix a reversion that broke initialization on systems that need to use pid locking
This brings back .git/annex/misctmp, but only for init. If an init
is interrupted while probing using that temp directory, the files it left
will get deleted 1 week later by a subsequent git-annex run.
2019-09-10 13:37:07 -04:00
Joey Hess
0af7ebdc2a
info: Display trust level when getting info on a uuid, same as on a remote. 2019-09-01 16:48:46 -04:00
Joey Hess
f845195354
Added annex.autoupgraderepository configuration
Can be set to false to prevent any automatic repository upgrades.

Also, removed direct mode specific upgrade code in Annex.Init, and made
needsUpgrade always include the name/path of the repo, so if
there's a problem it's clear what repo has the problem.

And, made needsUpgrade catch any exceptions that might occur during the
upgrade, so it can display a more useful error message than just the
exception.
2019-09-01 13:42:26 -04:00
Joey Hess
3f0eef4baa
v7 for all repositories
* Default to v7 for new repositories.
* Automatically upgrade v5 repositories to v7.
2019-08-30 14:09:14 -04:00
Joey Hess
1558e03014
Refuse to upgrade direct mode repositories when git is older than 2.22
That git fixed a memory leak that could cause an OOM during the upgrade.

Most git-annex builds have a new enough git already.
OSX git was upgraded with brew.

Linux i386ancient build's git was too old. Upgrading it to a fixed
git didn't work (due to the newer git not working with the old ssh,
https://bugs.chromium.org/p/git/issues/detail?id=7 )

Choices to deal with that were:

* Somehow make direct mode upgrade work with the old git, avoiding its
  OOM problem. One way would be to switch the repo to indirect mode
  first, and so upgrade to a repo with locked files. Not good when
  the filesystem does not support symlinks.
* backport the OOM fix from git 2.22
  (And do what about the version number so git-annex knows it's fixed?)
* backport openssh (and possibly more stuff)
* move the i386ancient build to at least Debian stretch (still backporting git)
  But this will make it no longer work with some of the ancient kernels it
  targets.

Of those, backporting the OOM fix seemed the best approach. Put "oomfix"
in the git version number to indicate it.

I have not automated building the git backport, so here's the patch I
used:

diff -ur orig/git-2.1.4/convert.c git-2.1.4/convert.c
--- orig/git-2.1.4/convert.c	2014-12-18 18:42:18.000000000 +0000
+++ git-2.1.4/convert.c	2019-08-29 20:05:04.371872338 +0100
@@ -404,7 +404,7 @@
 	if (start_async(&async))
 		return 0;	/* error was already reported */

-	if (strbuf_read(&nbuf, async.out, len) < 0) {
+	if (strbuf_read(&nbuf, async.out, 0) < 0) {
 		error("read from external filter %s failed", cmd);
 		ret = 0;
 	}
diff -ur orig/git-2.1.4/GIT-VERSION-GEN git-2.1.4/GIT-VERSION-GEN
--- orig/git-2.1.4/GIT-VERSION-GEN	2014-12-18 18:42:18.000000000 +0000
+++ git-2.1.4/GIT-VERSION-GEN	2019-08-29 20:06:39.132743228 +0100
@@ -1,7 +1,7 @@
 #!/bin/sh

 GVF=GIT-VERSION-FILE
-DEF_VER=v2.1.4
+DEF_VER=v2.1.4.oomfix

 LF='
 '
diff -ur orig/git-2.1.4/configure git-2.1.4/configure
--- orig/git-2.1.4/configure	2014-12-18 18:42:19.000000000 +0000
+++ git-2.1.4/configure	2019-08-29 20:27:45.896380015 +0100
@@ -580,8 +580,8 @@
 # Identity of this package.
 PACKAGE_NAME='git'
 PACKAGE_TARNAME='git'
-PACKAGE_VERSION='2.1.4'
-PACKAGE_STRING='git 2.1.4'
+PACKAGE_VERSION='2.1.4.oomfix'
+PACKAGE_STRING='git 2.1.4.oomfix'
 PACKAGE_BUGREPORT='git@vger.kernel.org'
 PACKAGE_URL=''

diff -ur orig/git-2.1.4/version git-2.1.4/version
--- orig/git-2.1.4/version	2014-12-18 18:42:19.000000000 +0000
+++ git-2.1.4/version	2019-08-29 20:06:17.572545210 +0100
@@ -1 +1 @@
-2.1.4
+2.1.4.oomfix
2019-08-29 15:24:41 -04:00
Joey Hess
4f59ac05b6
info: remove "repository mode"
info: Removed the "repository mode" from its output (including the --json
output) since with the removal of direct mode, there is no repository mode.
2019-08-29 14:12:22 -04:00
Joey Hess
d6e1f09ed2
init: Catch more exceptions when testing locking. 2019-08-29 12:19:07 -04:00
Joey Hess
586db7f06d
Avoid making a commit when upgrading from direct mode to v7
Three reasons:

* Committing as part of an upgrade is very unusual and unexpected.
* The commit was failing with a weird error message when done during an
  automatic upgrade.
* Let me remove more of that sweet^Whorrible direct mode code.
2019-08-26 16:35:44 -04:00
Joey Hess
adb89ee71b
update test suite for removal of direct mode
Removed that pass and all the complications of checking direct mode's
edge cases.
2019-08-26 15:07:10 -04:00
Joey Hess
20741b1eb4
Automatically convert direct mode repositories to v7 with adjusted unlocked branches
* Automatically convert direct mode repositories to v7 with adjusted
  unlocked branches and set annex.thin.
* init: When run on a crippled filesystem with --version=5,
  will error out, since version 7 is needed for adjusted unlocked branch.
* direct: This command always errors out as direct mode is no longer
  supported.
* indirect: This command has become a deprecated noop.
* proxy: This command is deprecated because it was only needed in direct
  mode. (But it continues to work.)

Also removed mentions of direct mode throughough the documentation.

I have not removed all the direct mode code yet.
2019-08-26 15:05:25 -04:00
Joey Hess
5877a15d7b
fix hard links when upgrading from direct mode
When upgrading a direct mode repo to v7 with adjusted unlocked branches,
fix a bug that prevented annex.thin from taking effect for the files in
working tree.

The hard links used to be ok, but commit 8e22114735 accidentially
broke them. It repopulates the worktree file, which is already a hard link,
and when it's creating the new file, the link count is already 2, and so it
doesn't make a hard link then.
2019-08-26 13:54:39 -04:00
Joey Hess
2fd27c6df5
assistant: When creating a new repository use v7 adjusted branches with annex.thin
Rather than direct mode, which this is a small step on the path to
removing.

Init on a crippled filesystem already used v7 adjusted branches,
and like that, this doesn't pose any interoperability issues with old
versions of git-annex that clone the same repo, because files are only
unlocked on the adjusted branch.
2019-08-26 12:54:14 -04:00
Joey Hess
c650389118
info: error out when file matching options used on non-directory
When file matching options are specified when getting info of
something other than a directory, they won't have any effect, so error out
to avoid confusion.

This commit was sponsored by mo on Patreon.
2019-08-24 13:20:19 -04:00
Joey Hess
972fd11f4e
releasing package git-annex version 7.20190819 2019-08-19 12:26:45 -04:00
Joey Hess
7f97575941
Makefile: Changed default zsh completion location to zsh default fpath.
Systems such as Debian that have overridden the default fpath will need to
set ZSH_COMPLETIONS_PATH.

I feel that Debian is causing unncessary complexity by making this change,
and have filed a bug report about it.

This also means that when git-annex is installed with PREFIX=/usr/local
it will use /usr/local/share/zsh/site-functions which works with probably
all versions of zsh.
2019-08-16 14:08:56 -04:00
Joey Hess
5fcaaf77db
Make git-annex-standalone.deb include the shell completions again
Was lost when the install-completions target was added.
2019-08-16 13:47:48 -04:00
Joey Hess
fa62c32233
Fix intermittent failure of the test suite
Its repeated opening and writing to the sqlite database somehow caused
inode cache information to occasionally be lost.

This loses code coverage, since running git-annex as a child process
prevents tracking what parts of the code are exercised. I have not looked
at the code coverage in a long time. It would probably be possible to
collect code coverage for the child procesess and merge it together.
2019-08-16 11:11:55 -04:00
Joey Hess
708fc6567f
S3: Fix encoding when generating public urls of S3 objects.
This code feels worryingly stringily typed, but using URI does not help
because the uriPath still has to be constructed with the right
uri-encoding.
2019-08-15 12:56:46 -04:00
Joey Hess
dc672863c3
init: Install working hook scripts when run on a crippled filesystem and on Windows 2019-08-13 15:14:17 -04:00
Joey Hess
b87ea12b6b
git-annex merge branch
* merge: When run with a branch parameter, merges from that branch.
  This is especially useful when using an adjusted branch, because
  it applies the same adjustment to the branch before merging it.
2019-08-09 13:21:15 -04:00
Joey Hess
b90ee6dc52
test: Add pass using adjusted unlocked branch
On second thought, the extra time running the test suite is worth it.
It will be gained back once we finally get rid of direct mode.

There are two failing tests, same two that have been failing on windows
(though the failure does not look identical). So this should also spare me
the Windows VM while fixing.
2019-08-09 11:34:10 -04:00
Joey Hess
298812a353
use separate main repo dir for each test suite pass
This way a failure to clean up the main repo dir from a previous pass
can't result in reusing that repo, which won't be configured right for the
current pass.
2019-08-08 14:29:28 -04:00
Joey Hess
70b71bf660
have init --version fail when repo is already initialized with other version
init: When the repo is already initialized, and --version requests a
different version, error out rather than silently not changing the version.
2019-08-08 14:13:02 -04:00
Joey Hess
3adc251f9d
Build with silently-1.2.5.1 on Windows; the old one used "NUL" which is not supported with recent versions of ghc. 2019-08-07 17:42:16 -04:00
Joey Hess
30ca02928c
Windows installer: Always install to 64 bit program files directory, since it needs 64 bit git now
I saw the installer not defaulting to any installation directory,
and I had to manually enter C:\Program Files\Git

Maybe it was choosing gitInstallDir32, and that was empty? Or the
conditional somehow failed. Simplifying so it will hopefully work again.
2019-08-07 14:05:03 -04:00
Joey Hess
bf5dd723d3
Fix querying git for object type when operating on a file containing newlines
This typo would make "git cat-file cat-file" fail, and the way it's used,
I think it broke querying all info from filenames containing newlines,
because the other queries are only run when it succeeds.
2019-08-07 13:35:42 -04:00
Joey Hess
fb7d92457f
support using gcrypt with git-lfs special remote 2019-08-05 13:43:45 -04:00
Joey Hess
8401b09e32
Allow setting up a gcrypt special remote with encryption=shared
It was documented to work, but seems it has been broken for a
while/forever.
2019-08-05 12:41:05 -04:00
Joey Hess
d1a0c7b16f
make --in=here fast
Use the same optimisation for --in=here as has always been used for --in=.
rather than the slow code path that unncessarily queries the git-annex
branch.

It looks like when "here" got added as an alias for "." back in 2012, I
forgot about this place.

Also sped up some very unlikely ways of referring to the current
repository.

Note that, this could in some rare corner case cause a behavior
change, if the git-annex branch and inAnnex disagree about whether content
is present in the local repository. But --in=. already behaved
that way, and the truth on the ground should win also.
2019-08-01 00:29:47 -04:00
Joey Hess
018b5b8173
Support building with socks-0.6 and persistant-template-2.7
persistent-template now needs UndecidableInstances.

socks changed defaultSocksConf to take a SockAddr.
2019-07-30 12:50:48 -04:00
Joey Hess
9fd37e65d0
prep release 2019-07-30 12:47:33 -04:00
Joey Hess
426053cb6c
Corrected some license statements
In 40ecf58d4b I changed the license of code I
wrote from GPL to AGPL. But, two files containing code I wrote combined
with code by others were updated to say their license is AGPL, while in
fact part of it was (the code I wrote) but part remained under the original
license (the code written by others).

Remote/Ddar.hs is now changed entirely back to GPL 3.

Annex/DirHashes.hs stays AGPL, but I broke out Utility/MD5.hs with the code
not written by me, and corrected its license statement to GPL-2, which
is the actual version of the GPL included with the code in its original
distribution at http://www.cs.ox.ac.uk/people/ian.lynagh/md5/
2019-07-28 14:27:33 -04:00
Joey Hess
875c7b5cc9
windows long filenames should be fixed now by new ghc 2019-07-22 09:44:09 -04:00
Joey Hess
ff85adba76
remove bundled rsync from windows build
rsync is only needed for rsync special remotes and git-annex-shell from
Debian oldstable. Since the library situation on windows for rsync required
a particular 32 bit build of git for it to work, and may also somehow need
git-annex to be 32 bit build, it's better to not include it.

This commit was sponsored by Jake Vosloo on Patreon.
2019-07-22 09:37:42 -04:00