Commit graph

878 commits

Author SHA1 Message Date
Joey Hess
5555697ae6
Enable building with git-annex benchmark by default
Only turning it off when the criterion library is not installed.

Not enabled for osx or i386ancient yet since that will need some
invesitgation to update their respective stack.yaml files.
2020-10-02 13:57:10 -04:00
Joey Hess
1a785d05c0
releasing package git-annex version 8.20200908 2020-09-08 14:20:47 -04:00
Joey Hess
6ea511beb4
Removed the S3 and WebDAV build flags
So these special remotes are always supported.

IIRC these build flags were added because the dep chains were a bit too
long, or perhaps because the libraries were not available in Debian stable,
or something like that. That was long ago, those reasons no longer apply,
and users get confused when builtin special remotes are not available, so
it seems best to remove the build flags now.

If this does cause a problem it can be reverted of course..

This commit was sponsored by Jochen Bartl on Patreon.
2020-09-08 12:42:59 -04:00
Joey Hess
8656afd3e1
rename http special remote to httpalso
"http" was too generic and easy to confuse with web. The new name makes
clear it's used in addition to some other remote. And other protocols
can use the same naming scheme.
2020-09-02 10:41:53 -04:00
Joey Hess
571ec900ac
Added http special remote, which is useful for accessing other remotes that publish content stored in them via http/https.
With automatic layout learning!
2020-09-01 15:16:35 -04:00
Joey Hess
b24ba92231
refactor out Annex.PidLock 2020-08-26 12:29:13 -04:00
Joey Hess
06a4ab39fa
wip external remote async protocol extension 2020-08-12 15:17:53 -04:00
Joey Hess
bcbdada8bf
fixed 2020-08-10 13:12:55 -04:00
Joey Hess
f75be32166
external backends wip
It's able to start them up, the only thing not implemented is generating
and verifying keys. And, the key translation for HasExt.
2020-07-29 15:23:18 -04:00
Joey Hess
555fe669e1
refactoring in preparation for external backends 2020-07-29 12:00:27 -04:00
Joey Hess
abd56fb019
Fix a bug in find --batch in the previous version. 2020-07-20 19:50:53 -04:00
Joey Hess
af901d1366
releasing package git-annex version 8.20200720 2020-07-20 14:41:12 -04:00
Joey Hess
087b7ee66a
Revert "data type that starts off using a set but converts to a bloom filter when large"
This reverts commit 7e2c4ed216.

I was not able to use this in the end..
See comment in the previous commit.
2020-07-01 20:12:19 -04:00
Joey Hess
7e2c4ed216
data type that starts off using a set but converts to a bloom filter when large
This adds a dep on hashable, but it's a free dependency, since
unordered-containers already pulled it in.

Using unordered-containers for the set seems to make sense, since it
hashes and bloom filter hashes too. (Though different hashes.)
I dunno, never quite know if I should use unordered-containers or containers.
2020-07-01 14:06:12 -04:00
Joey Hess
104b3a9c6a
Build with the http-client-restricted library when available
Otherwise use the vendored copy as before.

The library is in Debian testing but not stable. Once it reaches
stable, the vendored copy can be removed.

Did not add it to debian/control because IIRC that's used to build
git-annex on stable too, possibly. However, the Debian maintainer will
probably want to make the package depend on libghc-http-client-restricted-dev

This commit was sponsored by Ilya Shlyakhter on Patreon.
2020-06-22 11:31:31 -04:00
Joey Hess
01eb863a14
Build with the git-lfs library when available
Otherwise use the vendored copy as before.

The library is in Debian testing but not stable. Once it reaches
stable, the vendored copy can be removed.

Did not add it to debian/control because IIRC that's used to build
git-annex on stable too, possibly. However, the Debian maintainer will
probably want to make the package depend on libghc-git-lfs-dev.

This commit was sponsored by Ilya Shlyakhter on Patreon.
2020-06-22 11:21:25 -04:00
Joey Hess
6ef62cb3c7
fix unused import warning
Network.HTTP.Client exports makeConnection since 0.5.3.

Debian stable has a newer version than 0.5.3, so bumping the
min version seems better than adding an ifdef.
2020-06-22 10:55:37 -04:00
Joey Hess
48a88d822d
releasing package git-annex version 8.20200617 2020-06-17 15:59:34 -04:00
Joey Hess
660d8d3a87
simpler way to do this
Remove old code that can be trivially implemented using async in a much
nicer way (that is async exception safe).

I've audited all forkOS calls (except for ones in the assistant),
and this was the last remaining one that is not async exception safe.
The rest look ok to me.
2020-06-05 14:18:06 -04:00
Joey Hess
c429bbf2bd
remove workaround for old versions of process
ghc 8.4.4 has process 1.6.3, which was the first version to include
getPid.
2020-06-03 16:03:08 -04:00
Joey Hess
156e728b56
bump process version
Want to use eg withCreateProcess.

The base constraint already implied a ghc version bundled with process
1.6 or newer.
2020-06-03 12:09:41 -04:00
Joey Hess
484a74f073
auto-init autoenable=yes
Try to enable special remotes configured with autoenable=yes when git-annex
auto-initialization happens in a new clone of an existing repo. Previously,
git-annex init had to be explicitly run to enable them. That was a bit of a
wart of a special case for users to need to keep in mind.

Special remotes cannot display anything when autoenabled this way, to avoid
interfering with the output of git-annex query commands.

Any error messages will be hidden, and if it fails, nothing is displayed.
The user will realize the remote isn't enable when they try to use it,
and can run git-annex init manually then to try the autoenable again and
see what failed.

That seems like a reasonable approach, and it's less complicated than
communicating something across a pipe in order to display it as a side
message. Other reason not to do that is that, if the first command the
user runs is one like git-annex find that has machine readable output,
any message about autoenable failing would need to not be displayed anyway.
So better to not display a failure message ever, for consistency.

(Had to split out Remote.List.Util to avoid an import cycle.)
2020-05-27 12:40:35 -04:00
Joey Hess
01513da127
releasing package git-annex version 8.20200522 2020-05-22 12:07:59 -04:00
Joey Hess
6952060665
addurl --preserve-filename and a few related changes
* addurl --preserve-filename: New option, uses server-provided filename
  without any sanitization, but with some security checking.

  Not yet implemented for remotes other than the web.

* addurl, importfeed: Avoid adding filenames with leading '.', instead
  it will be replaced with '_'.

  This might be considered a security fix, but a CVE seems unwattanted.
  It was possible for addurl to create a dotfile, which could change
  behavior of some program. It was also possible for a web server to say
  the file name was ".git" or "foo/.git". That would not overrwrite the
  .git directory, but would cause addurl to fail; of course git won't
  add "foo/.git".

sanitizeFilePath is too opinionated to remain in Utility, so moved it.

The changes to mkSafeFilePath are because it used sanitizeFilePath.
In particular:

	isDrive will never succeed, because "c:" gets munged to "c_"
	".." gets sanitized now
	".git" gets sanitized now
	It will never be null, because sanitizeFilePath keeps the length
	the same, and splitDirectories never returns a null path.

Also, on the off chance a web server suggests a filename of "",
ignore that, rather than trying to save to such a filename, which would
fail in some way.
2020-05-08 16:22:55 -04:00
Joey Hess
2d51dd2e8c
changes required by cabal-version 1.10
Extensions got renamed.

Default-Language is required. I had to put Haskell98 because there are
subtle differences between 98 and 2010 and git-annex has always been
built with the default, which was 98 when there was a default. I don't
know how to establish that git-annex will behave the same under 2010.
2020-05-04 15:49:14 -04:00
Joey Hess
a5830c3f6e
bump cabal-version
hackage now requires 1.10 or newer
2020-05-04 15:45:10 -04:00
Joey Hess
e72ab75633
releasing package git-annex version 8.20200501 2020-05-01 17:41:36 -04:00
Joey Hess
04352ed9c5
check-ignore resource pool
Much like check-attr before.
2020-04-21 11:25:28 -04:00
Joey Hess
cee6b344b4
cat-file resource pool
Avoid running a large number of git cat-file child processes when run with
a large -J value.

This implementation takes care to avoid adding any overhead to git-annex
when run without -J. When run with -J, there is a small bit of added
overhead, to manipulate the resource pool. That optimisation added a
fair bit of complexity.
2020-04-20 15:19:31 -04:00
Joey Hess
fe9cf1256e
move remoteList into dupState
This does mean that RemoteDaemon.Transport.Tor's call runs it, otherwise
no change, but this is groundwork for doing more such expensive actions
in dupState.
2020-04-17 14:36:45 -04:00
Joey Hess
bcc0ec5b99
fix runtime crash on incomplete pattern match in lambda
This was very susprising to me that it was not caught by -Wall, so I
enabled -Wincomplete-uni-patterns to catch such things. It found a
second one just lines above, but no others anywhere.
2020-04-13 16:03:21 -04:00
Joey Hess
2caf579718
cache annex index filename for 1.5% speedup to queries 2020-04-10 13:37:04 -04:00
Joey Hess
dbad6c5c39
releasing package git-annex version 8.20200330 2020-03-30 13:46:24 -04:00
Joey Hess
14a4a9f4cd
releasing package git-annex version 8.20200309 2020-03-09 17:08:16 -04:00
Joey Hess
bfa015ae4e
Merge branch 'v7' 2020-02-26 18:49:36 -04:00
Joey Hess
f2a50a9944
version 2020-02-26 18:42:34 -04:00
Joey Hess
029c883713
Merge branch 'master' into v8 2020-02-19 14:32:11 -04:00
Joey Hess
cd8a208b8c
releasing package git-annex version 7.20200219 2020-02-19 12:45:30 -04:00
Joey Hess
46bf2a259b
releasing package git-annex version 7.20200204 2020-02-04 14:33:03 -04:00
Joey Hess
818d140748
oh I lost my version bump here
gonna update the release tag due to dumb joke
2020-02-02 17:03:00 -04:00
Joey Hess
c20fe23079
remove deleted module 2020-01-15 13:08:39 -04:00
Joey Hess
71f78fe45d
wip separate RemoteConfig parsing
Remote now contains a ParsedRemoteConfig. The parsing happens when the
Remote is constructed, rather than when individual configs are used.

This is more efficient, and it lets initremote/enableremote
reject configs that have unknown fields or unparsable values.

It also allows for improved type safety, as shown in
Remote.Helper.Encryptable where things that used to match on string
configs now match on data types.

This is a work in progress, it does not build yet.

The main risk in this conversion is forgetting to add a field to
RemoteConfigParser. That will prevent using that field with
initremote/enableremote, and will prevent remotes that already are set
up from seeing that configuration. So will need to check carefully that
every field that getRemoteConfigValue is called on has been added to
RemoteConfigParser.

(One such case I need to remember is that credPairRemoteField needs to be
included in the RemoteConfigParser.)
2020-01-13 12:39:21 -04:00
Joey Hess
71ecfbfccf
be stricter about rejecting invalid configurations for remotes
This is a first step toward that goal, using the ProposedAccepted type
in RemoteConfig lets initremote/enableremote reject bad parameters that
were passed in a remote's configuration, while avoiding enableremote
rejecting bad parameters that have already been stored in remote.log

This does not eliminate every place where a remote config is parsed and a
default value is used if the parse false. But, I did fix several
things that expected foo=yes/no and so confusingly accepted foo=true but
treated it like foo=no. There are still some fields that are parsed with
yesNo but not not checked when initializing a remote, and there are other
fields that are parsed in other ways and not checked when initializing a
remote.

This also lays groundwork for rejecting unknown/typoed config keys.
2020-01-10 14:52:48 -04:00
Joey Hess
b68a8d8968
use conversion functions from filepath-bytestring (again)
This reverts commit 3a04af7927.
2020-01-04 20:18:40 -04:00
Joey Hess
dfb25ecc29
simplfy unix-compat dep 2020-01-02 12:18:38 -04:00
Joey Hess
2cea674d1e
Merge branch 'master' into v8 2020-01-01 14:26:43 -04:00
Joey Hess
985373f8e7
releasing package git-annex version 7.20191230 2019-12-30 14:49:31 -04:00
Joey Hess
6d10fecff4
adjust filepath-bytestring deps
cabal needs to support the old one for debian currently, but stack can
require the newer one
2019-12-30 11:43:15 -04:00
Joey Hess
3a04af7927
temporary revert "use conversion functions from filepath-bytestring"
This reverts commit 75c40279c1.

Debian unstable is one version too old, so this can be de-reverted in a
bit.
2019-12-27 19:29:09 -04:00
Joey Hess
f6c18f6940
Merge branch 'bs' into sqlite-bs 2019-12-18 15:14:44 -04:00