Commit graph

220 commits

Author SHA1 Message Date
Joey Hess
55b178a6ba
minor cleanup 2017-03-10 15:03:33 -04:00
Joey Hess
71a05b0d25
use ActionItem rather than String
This changes fsck -A warnings to include the name of the key,
which is a bit redundant in one case, but was missing in another case.
2017-03-10 14:13:10 -04:00
Joey Hess
c8e1e3dada
AssociatedFile newtype
To prevent any further mistakes like 301aff34c4

This commit was sponsored by Francois Marier on Patreon.
2017-03-10 13:35:31 -04:00
Joey Hess
f90e2d0893
fix fsck bug introduced in 301aff34c4
Got two Maybe FilePaths crossed. Test suite caught it.
Slightly improved types to avoid this mistake.
2017-03-10 12:11:00 -04:00
Joey Hess
301aff34c4
fsck -q: When a file has bad content, include the name of the file in the warning message.
This commit was sponsored by Alexander Thompson on Patreon.
2017-03-08 15:15:20 -04:00
Joey Hess
942e0174b3
make fsck check annex.securehashesonly, and new tip for working around SHA1 collisions with git-annex
This commit was sponsored by andrea rota.
2017-02-27 13:55:15 -04:00
Joey Hess
9c4650358c
add KeyVariety type
Where before the "name" of a key and a backend was a string, this makes
it a concrete data type.

This is groundwork for allowing some varieties of keys to be disabled
in file2key, so git-annex won't use them at all.

Benchmarks ran in my big repo:

old git-annex info:

real	0m3.338s
user	0m3.124s
sys	0m0.244s

new git-annex info:

real	0m3.216s
user	0m3.024s
sys	0m0.220s

new git-annex find:

real	0m7.138s
user	0m6.924s
sys	0m0.252s

old git-annex find:

real	0m7.433s
user	0m7.240s
sys	0m0.232s

Surprising result; I'd have expected it to be slower since it now parses
all the key varieties. But, the parser is very simple and perhaps
sharing KeyVarieties uses less memory or something like that.

This commit was supported by the NSF-funded DataLad project.
2017-02-24 15:16:56 -04:00
Joey Hess
2577f1c0a2
fsck --all --from was checking the content of files in the local repository, rather than on the special remote.
Straight up forgot to handle this case!

This commit was sponsored by Fernando Jimenez on Patreon.
2016-11-16 15:33:57 -04:00
Joey Hess
0a4479b8ec
Avoid backtraces on expected failures when built with ghc 8; only use backtraces for unexpected errors.
ghc 8 added backtraces on uncaught errors. This is great, but git-annex was
using error in many places for a error message targeted at the user, in
some known problem case. A backtrace only confuses such a message, so omit it.

Notably, commands like git annex drop that failed due to eg, numcopies,
used to use error, so had a backtrace.

This commit was sponsored by Ethan Aubin.
2016-11-15 21:29:54 -04:00
Joey Hess
1a0e2c9901
get, move, copy, mirror: Added --failed switch which retries failed copies/moves
Note that get --from foo --failed will get things that a previous get --from bar
tried and failed to get, etc. I considered making --failed only retry
transfers from the same remote, but it was easier, and seems more useful,
to not have the same remote requirement.

Noisy due to some refactoring into Types/
2016-08-03 12:37:12 -04:00
Joey Hess
d13194b230
--branch, stage 2
Show branch:file that is being operated on.

I had to make ActionItem a type and not a type class because
withKeyOptions' passed two different types of values when using the type
class, and I could not get the type checker to accept that.
2016-07-20 15:23:43 -04:00
Joey Hess
5642daa651
fsck: Fix a reversion in direct mode fsck of a file that is present when the location log thinks it is not. Reversion introduced in version 5.20151208. 2016-07-12 13:41:03 -04:00
Joey Hess
ae65aecb0b
fsck: When a key is not previously known in the location log, record something so that reinject --known will work. 2016-05-10 13:20:45 -04:00
Joey Hess
9169234c34
fix overindent 2016-05-10 13:08:24 -04:00
Joey Hess
bd516af734
fsck: Warn when core.sharedRepository is set and an annex object file's write bit is not set and cannot be set due to the file being owned by a different user.
Made all Annex.Perms file mode changing functions ignore errors when
core.sharedRepository is set, because the file might be owned by someone
else. I don't fancy getting bug reports about crashes due to set modes in
this configuration, which is a very foot-shooty configuration in the first
place.

The fsck warning is necessary because old repos kept files mode 444, which
doesn't allow locking them, and so if the mode remains 444 due to the file
being owned by someone else, the user should be told about it.
2016-04-14 15:36:53 -04:00
Joey Hess
b7c8bf5274
Preserve execute bits of unlocked files in v6 mode.
When annex.thin is set, adding an object will add the execute bits to the
work tree file, and this does mean that the annex object file ends up
executable.

This doesn't add any complexity that wasn't already present, because git
annex add of an executable file has always ingested it so that the annex
object ends up executable.

But, since an annex object file can be executable or not, when populating
an unlocked file from one, the executable bit is always added or removed
to match the mode of the pointer file.
2016-04-14 14:47:08 -04:00
Joey Hess
ec198fec83
fsck: When the only copy of a file is in a dead repository, mention the repository. 2016-02-19 15:12:11 -04:00
Joey Hess
885e54df0a
fsck: Populate unlocked files in v6 repositories whose content is present in annex/objects but didn't reach the work tree.
This also handles fixing up after cf260d9a15
2016-02-14 17:27:50 -04:00
Joey Hess
675321264f
fsck: Detect and fix missing associated file mappings in v6 repositories.
This also handles fixing up after the bad data written by
cf260d9a15.
2016-02-14 17:09:54 -04:00
Joey Hess
74bbdfa888
files with only 1 linkCount may still be unlocked
When on crippled filesystem, or without annex.thin set.
2016-02-14 17:04:09 -04:00
Joey Hess
5b51db7645
clean up 2016-02-14 16:52:43 -04:00
Joey Hess
737e45156e
remove 163 lines of code without changing anything except imports 2016-01-20 16:36:33 -04:00
Joey Hess
87f0708f88
persistent-sqlite is now a hard build dependency, since v6 repository mode needs it. 2015-12-26 13:00:52 -04:00
Joey Hess
99f1d7991d
recent fsck changes caused ugly message when object was not present 2015-12-15 16:10:48 -04:00
Joey Hess
0ddcaae9c1
changes for v6 broke fsck in direct mode 2015-12-15 14:27:20 -04:00
Joey Hess
e7183d83d3
fsck for v6 unlocked files
This only adds 1 stat to each file fscked for locked files, so
added overhead is minimal.

For unlocked files it has to access the database to see if a file
is modified.
2015-12-11 16:07:54 -04:00
Joey Hess
abd66c7089
fsck: Failed to honor annex.diskreserve when checking a remote. 2015-12-11 13:50:27 -04:00
Joey Hess
4b02af57b6
display a message in the unlikely scenario of fsking a dead repository 2015-11-10 14:44:58 -04:00
Joey Hess
cd7929034a
fsck: When fscking a dead repo, avoid incorrect "fixing location log" message.
keyLocations doesn't return locations in dead repos, but if we're fscking a
dead repo, we want to look at what locations are actually logged for it.
2015-11-10 13:59:04 -04:00
Joey Hess
3d0f41518d
parallel fsck (yes, these changes are all it takes now!) 2015-11-04 16:28:14 -04:00
Joey Hess
2def1d0a23 other 80% of avoding verification when hard linking to objects in shared repo
In c6632ee5c8, it actually only handled
uploading objects to a shared repository. To avoid verification when
downloading objects from a shared repository, was a lot harder.

On the plus side, if the process of downloading a file from a remote
is able to verify its content on the side, the remote can indicate this
now, and avoid the extra post-download verification.

As of yet, I don't have any remotes (except Git) using this ability.
Some more work would be needed to support it in special remotes.

It would make sense for tahoe to implicitly verify things downloaded from it;
as long as you trust your tahoe server (which typically runs locally),
there's cryptographic integrity. OTOH, despite bup being based on shas,
a bup repo under an attacker's control could have the git ref used for an
object changed, and so a bup repo shouldn't implicitly verify. Indeed,
tahoe seems unique in being trustworthy enough to implicitly verify.
2015-10-02 14:35:12 -04:00
Joey Hess
cad3349001 rename fsckKey to verifyKeyContent
No behavior changes.
2015-10-01 13:29:17 -04:00
Joey Hess
3f47d1b351 Improve bash completion, so it completes names of remotes and backends in appropriate places.
Not necessarily everywhere, but a lot of the most often used places.

Re the use of .Internal, see
https://github.com/pcapriotti/optparse-applicative/issues/155
2015-09-14 13:19:04 -04:00
Joey Hess
0b7a8b72bb Fix building without database.
Ben Boeckel had a patch, but..
Actually, that was not the only place that used ScheduleIncremental when
built w/o database. Since the data type doesn't need database stuff,
I've instead fixed this build problem by exposing the
ScheduleIncremental constructor to database-less builds.
2015-08-23 15:39:29 -07:00
Joey Hess
9dfe03dbcd Improve shutdown due to --time-limit, especially for fsck
* Perform a clean shutdown when --time-limit is reached.
  This includes running queued git commands, and cleanup actions normally
  run when a command is finished.
* fsck: Commit incremental fsck database when --time-limit is reached.
  Previously, some of the last files fscked did not make it into the
  database when using --time-limit.

Note that this changes Annex.addCleanup hooks, to run after --time-limit
expires. Fsck was using such a hook to clean up after a
--incremental-schedule, and that shouldn't run when --time-limit exipires
it. So, instead, moved that cleanup code to be run by cleanupIncremental.
Resulted in some data type juggling.
2015-07-31 16:01:54 -04:00
Joey Hess
1fb9ab342b Support building without persistent database on for systems that lack TH. This removes support for incremental fsck. 2015-07-25 17:37:09 -04:00
Joey Hess
6a4f2087be finished converting all the main options 2015-07-10 13:23:06 -04:00
Joey Hess
a7f58634b8 wip 2015-07-09 16:05:45 -04:00
Joey Hess
8ad927dbc6 converted copy and move
Got a little tricky..
2015-07-09 15:23:14 -04:00
Joey Hess
032e6485fa use Alternative for parsing KeyOptions 2015-07-09 12:44:03 -04:00
Joey Hess
94e703e8b8 use Alternative when parsing mutually exclusive fsck options 2015-07-09 12:26:25 -04:00
Joey Hess
c1c64ec76c formatting 2015-07-09 10:42:28 -04:00
Joey Hess
d8d1499229 finalOpt is the same as optional 2015-07-09 01:02:27 -04:00
Joey Hess
60806dd191 wip 2015-07-08 17:59:06 -04:00
Joey Hess
6a88c7c101 converted fsck's options to optparse-applicative
Global options and seeking and key options are still to be done.
2015-07-08 16:58:54 -04:00
Joey Hess
6e5c1f8db3 convert all commands to work with optparse-applicative
Still no options though.
2015-07-08 15:08:02 -04:00
Joey Hess
a2ba701056 started converting to use optparse-applicative
This is a work in progress. It compiles and is able to do basic command
dispatch, including git autocorrection, while using optparse-applicative
for the core commandline parsing.

* Many commands are temporarily disabled before conversion.
* Options are not wired in yet.
* cmdnorepo actions don't work yet.

Also, removed the [Command] list, which was only used in one place.
2015-07-08 13:36:25 -04:00
Joey Hess
5123a512d6 add a hint about marking a key dead 2015-06-09 15:12:40 -04:00
Joey Hess
6eefc5db65 fsck: Ignore keys that are known to be dead when running in --all mode or a in a bare repo. Otherwise, still reports files with lost contents, even if the content is dead. 2015-06-09 14:08:57 -04:00
Joey Hess
a812d598ef Take space that will be used by running downloads into account when checking annex.diskreserve. 2015-05-12 15:20:22 -04:00