Commit graph

1310 commits

Author SHA1 Message Date
Joey Hess
0dd5691951
update notes 2019-10-10 16:12:59 -04:00
Joey Hess
df5b0ffab3
inherit other fields
I think this is all that need to be inherited.
2019-10-10 16:11:21 -04:00
Joey Hess
c3975ff3b4
sameas RemoteConfig inheritance
I found a way to avoid inheritance complicating anything outside of
Logs.Remote. It seems fine to require all inherited values to be
inherited and not set in the sameas remote's config. Since inherited
values will be used for stuff like encryption and perhaps chunking, which
control the actual content stored on the remote, it seems likely that
there will not be any reason to need them to vary between two remotes
that access the same underlying data store.

The newer version of containers is free; the minimum ghc version is
bundled with a newer version than that.
2019-10-10 15:58:22 -04:00
Joey Hess
59908586f4
rename RemoteConfigKey to RemoteConfigField
And some associated renames.
I was going to have some values named fooKeyKey otherwise..
2019-10-10 15:44:05 -04:00
Joey Hess
92ff30df70
set annex-config-uuid when RemoteConfig contains a sameas-uuid
Initremote sets that, so after both initremote and enableremote,
the git config will be set.

Any remote that does not use Annex.SpecialRemote won't set
annex-config-uuid. But that's only Remote.Git, which doesn't use
RemoteConfig anyway.
2019-10-10 12:58:59 -04:00
Joey Hess
97b499a4dc
use sameas-name and sameas-uuid for sameas remotes
initremote --sameas=remotename sets sameas-name and sameas-uuid

Using sameas-name rather than name prevents old git-annex initremote
from enabling a sameas remote by name, since it would not handle it
correctly.
2019-10-10 12:32:05 -04:00
Joey Hess
3066bdb1fb
fix annex.largefiles largerthan/smallerthan bug
Fix bug in handling of annex.largefiles that use largerthan/smallerthan.
When adding a modified file, it incorrectly used the file size of the old
version of the file, not the current size.

That was the only largefiles limit that didn't directly look at the file on
disk already. Added a new type to keep straight the two different ways such
a limit can be matched. I kind of wanted to extend MatchingFile or FileInfo
to indicate that the matcher is supposed to operate on files from disk or
annex, but it turned out to be too complex to implement it that way.

This also changes the LimitAnnexFiles case when lookupFileKey does not find
a key. It used to fall back to statting the file, now it always returns
False. I doubt the old code could really get to that point, but if it
somehow does, it's better for preferred content matching to be consistent.
2019-09-30 17:15:08 -04:00
Joey Hess
9f27d03945
fix a typo that didn't matter so far 2019-09-27 14:08:16 -04:00
Joey Hess
fda1bdd679
Added --mimetype and --mimeencoding file matching options.
Already had these for largefiles matching, but I forgot to add them as
command-line options.
2019-09-19 12:09:59 -04:00
Joey Hess
53fd746705
avoid some build warnings on windows 2019-09-12 14:11:19 -04:00
Joey Hess
fef3cd055d
Removed support for git versions older than 2.1
debian oldoldstable has 2.1, and that's what i386ancient uses. It would be
better to require git 2.2, which is needed to use adjusted branches, but
can't do that w/o losing support for some old linux kernels or a
complicated git backport.
2019-09-11 16:14:43 -04:00
Joey Hess
061231621e
Merge branch 'master' into v7-default 2019-09-10 16:06:43 -04:00
Joey Hess
94c75d2bd9
init: Fix a reversion that broke initialization on systems that need to use pid locking
This brings back .git/annex/misctmp, but only for init. If an init
is interrupted while probing using that temp directory, the files it left
will get deleted 1 week later by a subsequent git-annex run.
2019-09-10 13:37:07 -04:00
Joey Hess
f845195354
Added annex.autoupgraderepository configuration
Can be set to false to prevent any automatic repository upgrades.

Also, removed direct mode specific upgrade code in Annex.Init, and made
needsUpgrade always include the name/path of the repo, so if
there's a problem it's clear what repo has the problem.

And, made needsUpgrade catch any exceptions that might occur during the
upgrade, so it can display a more useful error message than just the
exception.
2019-09-01 13:42:26 -04:00
Joey Hess
8f0af15020
one missed thing for automatic v5 -> v7 upgrades 2019-08-30 17:35:10 -04:00
Joey Hess
3f0eef4baa
v7 for all repositories
* Default to v7 for new repositories.
* Automatically upgrade v5 repositories to v7.
2019-08-30 14:09:14 -04:00
Joey Hess
d6e1f09ed2
init: Catch more exceptions when testing locking. 2019-08-29 12:19:07 -04:00
Joey Hess
4e30b06ffb
remove unused import 2019-08-28 15:38:29 -04:00
Joey Hess
e804f48f82
remove a few more isDirect tests 2019-08-28 11:53:10 -04:00
Joey Hess
863db16e53
remove unused 2019-08-27 16:13:49 -04:00
Joey Hess
9b1331881c
reorg remaining direct mode code
Only used for upgrading, so put it under there.
2019-08-27 14:05:38 -04:00
Joey Hess
e395ba2cb0
remove unused code 2019-08-27 13:57:17 -04:00
Joey Hess
da6f4d8887
remove direct mode support from Annex.Content
No longer used. The only possible user of it would be code in
Upgrade.V5, so I verified that the parts of Annex.Content it used were
not used to manipulate direct mode files.
2019-08-27 13:14:06 -04:00
Joey Hess
770b8ff926
clearer message when direct mode upgrade fails
When a remote is being upgraded, the message looked as if the local
repo was where the problem was. So include the path of the repo.
2019-08-27 12:23:34 -04:00
Joey Hess
586db7f06d
Avoid making a commit when upgrading from direct mode to v7
Three reasons:

* Committing as part of an upgrade is very unusual and unexpected.
* The commit was failing with a weird error message when done during an
  automatic upgrade.
* Let me remove more of that sweet^Whorrible direct mode code.
2019-08-26 16:35:44 -04:00
Joey Hess
689d1fcc92
remove most remnants of direct mode
A few remain, as needed for upgrades, and for accessing objects from
remotes that are direct mode repos that have not been converted yet.
2019-08-26 16:27:48 -04:00
Joey Hess
20741b1eb4
Automatically convert direct mode repositories to v7 with adjusted unlocked branches
* Automatically convert direct mode repositories to v7 with adjusted
  unlocked branches and set annex.thin.
* init: When run on a crippled filesystem with --version=5,
  will error out, since version 7 is needed for adjusted unlocked branch.
* direct: This command always errors out as direct mode is no longer
  supported.
* indirect: This command has become a deprecated noop.
* proxy: This command is deprecated because it was only needed in direct
  mode. (But it continues to work.)

Also removed mentions of direct mode throughough the documentation.

I have not removed all the direct mode code yet.
2019-08-26 15:05:25 -04:00
Joey Hess
f6fb4b8cdb
avoid side message when doing automatic upgrade to v7
An automatic upgrade is supposed to be silent.
2019-08-26 13:54:52 -04:00
Joey Hess
5877a15d7b
fix hard links when upgrading from direct mode
When upgrading a direct mode repo to v7 with adjusted unlocked branches,
fix a bug that prevented annex.thin from taking effect for the files in
working tree.

The hard links used to be ok, but commit 8e22114735 accidentially
broke them. It repopulates the worktree file, which is already a hard link,
and when it's creating the new file, the link count is already 2, and so it
doesn't make a hard link then.
2019-08-26 13:54:39 -04:00
Joey Hess
1e02360283
remove only case 2019-08-26 13:28:28 -04:00
Joey Hess
2fd27c6df5
assistant: When creating a new repository use v7 adjusted branches with annex.thin
Rather than direct mode, which this is a small step on the path to
removing.

Init on a crippled filesystem already used v7 adjusted branches,
and like that, this doesn't pose any interoperability issues with old
versions of git-annex that clone the same repo, because files are only
unlocked on the adjusted branch.
2019-08-26 12:54:14 -04:00
Joey Hess
b599e8e6ac
move module only used by assistant 2019-08-26 12:32:45 -04:00
Joey Hess
bb16a26109
use headExists
Turns out that 7be690f326 broke the
test suite on the i386ancient builder. There, git show-ref --verify HEAD
fails with "'HEAD' - not a valid ref". Apparently git 2.1.4 didn't
support that.

headExists works there and does the same thing.
2019-08-19 11:12:19 -04:00
Joey Hess
f845636e30
correct license to AGPL
This code was already AGPL, except for the bit split out
to Utility/MD5.hs in commit 426053cb6c.
That commit accidentially updated the license of this file from AGPL
to GPL.

Thanks to Sean Whitton for spotting this.
2019-08-17 14:08:07 -04:00
Joey Hess
e4a8366162
fix edge case failure in prop_view_roundtrips
"./" made it fail, because that gets eliminated
2019-08-16 11:35:32 -04:00
Joey Hess
dc672863c3
init: Install working hook scripts when run on a crippled filesystem and on Windows 2019-08-13 15:14:17 -04:00
Joey Hess
868942e19b
fix unused module import warnings when building on windows 2019-08-08 12:18:53 -04:00
Joey Hess
8ba4de2d9c
remove unused import 2019-07-30 12:16:41 -04:00
Joey Hess
5080a7be1e
fix build 2019-07-29 12:41:45 -04:00
Joey Hess
426053cb6c
Corrected some license statements
In 40ecf58d4b I changed the license of code I
wrote from GPL to AGPL. But, two files containing code I wrote combined
with code by others were updated to say their license is AGPL, while in
fact part of it was (the code I wrote) but part remained under the original
license (the code written by others).

Remote/Ddar.hs is now changed entirely back to GPL 3.

Annex/DirHashes.hs stays AGPL, but I broke out Utility/MD5.hs with the code
not written by me, and corrected its license statement to GPL-2, which
is the actual version of the GPL included with the code in its original
distribution at http://www.cs.ox.ac.uk/people/ian.lynagh/md5/
2019-07-28 14:27:33 -04:00
Joey Hess
4c5a489f3e
avoid build warning when built w/o magic-mime 2019-07-22 11:03:26 -04:00
Joey Hess
7fd650355e
merge from http-client-restricted
I made some improvements to its API after splitting it out of git-annex,
so merge those back in.

This is groundwork for removing the embedded copy of it and depending on
it.

Also moved the managerResponseTimeout disabling to Annex.Url as it's
git-annex specific.

This commit was sponsored by Ethan Aubin on Patreon.
2019-07-17 16:48:50 -04:00
Joey Hess
7be690f326
check headRef not Branch.current
Support running v7 upgrade in a repo where there is no branch checked out,
but HEAD is set directly to some other ref.

This commit was sponsored by Jack Hill on Patreon.
2019-07-16 12:36:29 -04:00
Joey Hess
9a5ddda511
remove many old version ifdefs
Drop support for building with ghc older than 8.4.4, and with older
versions of serveral haskell libraries than will be included in Debian 10.

The only remaining version ifdefs in the entire code base are now a couple
for aws!

This commit should only be merged after the Debian 10 release.
And perhaps it will need to wait longer than that; it would make
backporting new versions of  git-annex to Debian 9 (stretch) which
has been actively happening as recently as this year.

This commit was sponsored by Ilya Shlyakhter.
2019-07-05 15:09:37 -04:00
Joey Hess
26c54d6ea3
make metered more generic
Allow it to be used when the Key is not known.
2019-06-25 12:33:36 -04:00
Joey Hess
8355dba5cc
plumb MeterUpdate into getKey
No behavior changes, but this shows everywhere that a progress meter
could be displayed when hashing a file to add to the annex.

Many of the places don't make sense to display a progress meter though,
eg when importing the copy of the file probably swamps the hashing of
the file.
2019-06-25 11:43:24 -04:00
Joey Hess
84e729fda5
fix init default description reversion
init: Fix a reversion in the last release that prevented automatically
generating and setting a description for the repository.

Seemed best to factor out uuidDescMapRaw that does not
have the default mempty descrition behavior.

I don't much like that behavior, but I know things depend on it.
One thing in particular is `git annex info` which lists the uuids and
descriptions; if the current repo has been initialized in some way that
means it does not have a description, it would not show up w/o that.

(Not only repos created due to this bug might lack that. For example a repo
that was marked dead and had --drop-dead delete its git-annex branch info,
and then came back from the dead would similarly not be in the uuid.log.
Also there have been other versions of git-annex that didn't set a default
description; for years there was no default description.)
2019-06-20 20:30:24 -04:00
Joey Hess
ba433bdc85
refactor 2019-06-19 20:19:38 -04:00
Joey Hess
26f0f8b20f
optimisation
Avoid an unncessary STM transaction. This will happen when the worker
pool is not completely full of the new stage, which is the common case.

In the uncommon case, this adds only a tiny bit of overhead for the
extra traversal of the worker pool. And the thread is going to block
for some time anyway.
2019-06-19 20:13:19 -04:00
Joey Hess
37d505dd6b
avoid STM deadlock
When all worker threads are running and enteringStage is called,
it waits for an idle slot. If all off the other threads then call it in
turn, a deadlock occurrs.

This is the same problem I didn't actually fix in
5a9842d7ed.

Fixed by doing two separate STM transactions, the first replaces its
active thread with an idle thread, and the second waits for another idle
thread. That guarantees there will eventually be an idle thread to find.

The changes to WorkerPool were necessary because it can't add an idle
thread containing the Annex state and go on to run an action using that
same state, so I had to remove the Annex state from IdleWorker.
2019-06-19 18:15:25 -04:00