Joey Hess
6a88c7c101
converted fsck's options to optparse-applicative
...
Global options and seeking and key options are still to be done.
2015-07-08 16:58:54 -04:00
Joey Hess
7d9072d1db
remove git-annex help options display
...
Common options will be displayed in the --help for a command by
optparse-applicative.
2015-07-08 15:41:54 -04:00
Joey Hess
92d8f80bff
support cmdnorepo actions, also using getopt-applicative there
2015-07-08 15:39:05 -04:00
Joey Hess
6e5c1f8db3
convert all commands to work with optparse-applicative
...
Still no options though.
2015-07-08 15:08:02 -04:00
Joey Hess
3125da54f6
display cmdparamdesc in optparse-applicative usage messages
...
Since optparse-applicative display "FOO" as "[FOO]", the paramOptional
modifier which wrapped it in square brackets was removed from most places.
2015-07-08 13:39:11 -04:00
Joey Hess
a2ba701056
started converting to use optparse-applicative
...
This is a work in progress. It compiles and is able to do basic command
dispatch, including git autocorrection, while using optparse-applicative
for the core commandline parsing.
* Many commands are temporarily disabled before conversion.
* Options are not wired in yet.
* cmdnorepo actions don't work yet.
Also, removed the [Command] list, which was only used in one place.
2015-07-08 13:36:25 -04:00
Joey Hess
24800b1bf1
Only look at reflogs for relevant branches, not for git-annex branches
...
This speeds it up quite a bit.. May still be too slow in large repos.
2015-07-07 17:36:30 -04:00
Joey Hess
600717417f
update comment
2015-07-07 17:23:02 -04:00
Joey Hess
b11d2f5a8a
unused: --used-refspec can now be configured to look at refs in the reflog. This provides a way to not consider old versions of files to be unused after they have reached a specified age, when the old refs in the reflog expire.
...
May be slow.
2015-07-07 17:13:50 -04:00
Joey Hess
a51b98cdd5
sync: When annex.autocommit=false, avoid making any commit of local changes, while still merging with remote to the extent possible.
2015-07-07 16:36:11 -04:00
Joey Hess
81ad277a85
add: Stage symlinks the same as git add would, even if they are not a link to annexed content.
2015-07-07 16:15:30 -04:00
Joey Hess
60fc25e9d3
merge: Avoid creating the synced/master branch.
...
The branch needs to be created when merging from the remote in sync,
since we diff between it and the remote's sync branch. But git annex merge
should not be creating sync branches.
2015-07-07 15:53:27 -04:00
Joey Hess
625303226d
import: Fix failure of cross-device import on Windows.
...
As well as import, 2 other places ran "mv" manually, so changed them to use
moveFile as well.
2015-07-07 14:48:23 -04:00
Joey Hess
1529add61a
Brought back the setkey plumbing command that was removed in 2011, since we found a use case for it. Note that the command's syntax was changed for consistency.
2015-07-02 17:44:25 -04:00
Joey Hess
1f35141aff
assistant --autostart: First any daemons that are already running, which might be left over from a previous login session and so unable to use the ssh agent of a new login session.
2015-07-02 14:16:53 -04:00
Joey Hess
833cf5fff9
log: Fix reversion introduced in version 5.20150528 that broke this command.
2015-07-02 12:53:38 -04:00
Joey Hess
f77d485915
move comment
2015-06-16 19:07:14 -04:00
Joey Hess
be9d9cb5ad
avoid building unused bloomfilter when run without --all
2015-06-16 19:04:20 -04:00
Joey Hess
5b801fcad9
on second thought, sync --content --unused is probably not useful, remove
2015-06-16 19:01:06 -04:00
Joey Hess
adba0595bd
use bloom filter in second pass of sync --all --content
...
This is needed because when preferred content matches on files,
the second pass would otherwise want to drop all keys. Using a bloom filter
avoids this, and in the case of a false positive, a key will be left
undropped that preferred content would allow dropping. Chances of that
happening are a mere 1 in 1 million.
2015-06-16 18:50:13 -04:00
Joey Hess
a0a8127956
instance Hashable Key for bloomfilter
2015-06-16 18:37:41 -04:00
Joey Hess
8b74aec3ea
Increased the default annex.bloomaccuracy from 1000 to 10000000
...
This makes git annex unused use around 48 mb more memory than it did before,
but the massive increase in accuracy makes this worthwhile for all but the
smallest systems.
Also, I want to use the bloom filter for sync --all --content, to avoid
dropping files that the preferred content doesn't want, and 1/1000
false positives would be far too many in that use case, even if it were
acceptable for unused.
Actual memory use numbers:
1000: 21.06user 3.42system 0:26.40elapsed 92%CPU (0avgtext+0avgdata 501552maxresident)k
1000000: 21.41user 3.55system 0:26.84elapsed 93%CPU (0avgtext+0avgdata 549496maxresident)k
10000000: 21.84user 3.52system 0:27.89elapsed 90%CPU (0avgtext+0avgdata 549920maxresident)k
Based on these numbers, 10 million seemed a better pick than 1 million.
2015-06-16 18:12:00 -04:00
Joey Hess
29c03145e6
sync: Add support for --all and --unused.
2015-06-16 16:50:03 -04:00
Joey Hess
99a1113461
switch code to using associated files
2015-06-16 15:07:03 -04:00
Joey Hess
67f7f1b1cb
info: Added json output for "backend usage", "numcopies stats", "repositories containing these files", and "transfers in progress".
2015-06-16 13:50:28 -04:00
Joey Hess
829007d629
Improve url parsing to handle some urls containing illegal [] characters in their paths.
...
Ie, "https://archive.org/download/zoom-2/Zoom - Release 2 (1996)(Active Software)[!].iso"
2015-06-14 13:39:44 -04:00
Joey Hess
5123a512d6
add a hint about marking a key dead
2015-06-09 15:12:40 -04:00
Joey Hess
f8ab3bc449
dead --key: Can be used to mark a key as dead.
2015-06-09 14:52:05 -04:00
Joey Hess
6eefc5db65
fsck: Ignore keys that are known to be dead when running in --all mode or a in a bare repo. Otherwise, still reports files with lost contents, even if the content is dead.
2015-06-09 14:08:57 -04:00
Joey Hess
de3bd11a2c
import --clean-duplicates: Fix bug that didn't count local or trusted repo's copy of a file as one of the necessary copies to allow removing it from the import location.
2015-06-03 13:15:38 -04:00
Joey Hess
d28e8fbfd5
get --incomplete: New option to resume any interrupted downloads.
2015-06-02 14:20:38 -04:00
Joey Hess
eb33569f9d
remove Params constructor from Utility.SafeCommand
...
This removes a bit of complexity, and should make things faster
(avoids tokenizing Params string), and probably involve less garbage
collection.
In a few places, it was useful to use Params to avoid needing a list,
but that is easily avoided.
Problems noticed while doing this conversion:
* Some uses of Params "oneword" which was entirely unnecessary
overhead.
* A few places that built up a list of parameters with ++
and then used Params to split it!
Test suite passes.
2015-06-01 13:52:23 -04:00
Joey Hess
36b9c9ca5f
fromkey, registerurl: Improve handling of urls that happen to also be parsable as strange keys.
2015-05-30 02:08:49 -04:00
Joey Hess
77c43a388e
fromkey, registerurl: Allow urls to be specified instead of keys, and generate URL keys.
...
This is especially useful because the caller doesn't need to generate valid
url keys, which involves some escaping of characters, and may involve
taking a md5sum of the url if it's too long.
2015-05-22 22:41:36 -04:00
Joey Hess
823bb8031b
add annex.used-refspec
2015-05-14 15:44:08 -04:00
Joey Hess
86699ff861
unused: Add --used option, which can specify a set of refs to consider used, rather than the default of considering all refs used.
2015-05-14 15:31:38 -04:00
Joey Hess
61ccf95004
Avoid accumulating transfer failure log files unless the assistant is being used.
...
Only the assistant uses these, and only the assistant cleans them up, so
make only git annex transferkeys write them,
There is one behavior change from this. If glacier is being used, and a
manual git annex get --from glacier fails because the file isn't available
yet, the assistant will no longer later see that failed transfer file and
retry the get. Hope no-one depended on that old behavior.
2015-05-12 15:53:38 -04:00
Joey Hess
a812d598ef
Take space that will be used by running downloads into account when checking annex.diskreserve.
2015-05-12 15:20:22 -04:00
Joey Hess
e27b97d364
Merge branch 'master' into concurrentprogress
...
Conflicts:
Command/Fsck.hs
Messages.hs
Remote/Directory.hs
Remote/Git.hs
Remote/Helper/Special.hs
Types/Remote.hs
debian/changelog
git-annex.cabal
2015-05-12 13:23:22 -04:00
Joey Hess
4b2e3210ea
drop: Now supports --all, --unused, and --key.
2015-05-12 13:00:06 -04:00
Joey Hess
db5d831d07
import: Refuse to import files that are within the work tree, as that does not make sense and could cause data loss.
2015-05-11 12:57:47 -04:00
Joey Hess
26ac0753c1
more FlexibleContexts
2015-05-10 15:54:58 -04:00
Joey Hess
6c2d5b5e41
more time-1.5 fixes
2015-05-10 15:36:58 -04:00
Joey Hess
6cf62a9bde
support time-1.5.0
...
This no longer uses old-locale's defaultTimeLocale, but provides one
of its own.
Factored out a Logs.TimeStamp.
2015-05-10 15:21:35 -04:00
Joey Hess
f395ca7a41
Improve quvi 0.4 output parsing to handle cases wher there is no known filename extension. This is currently the case when using quvi with youtube. In this case, the extension ".m" will be used.
2015-05-08 13:39:00 -04:00
Joey Hess
469242ac4d
fsck: Ignore error recording the fsck in the activity log, which can happen when running fsck in a read-only repository. Closes : #698559 (fsck can still need to write to the repository if it find problems, but a successful fsck can be done read-only)
2015-05-06 14:45:20 -04:00
Joey Hess
eeb0359a2e
contentlocation, examinekey, lookupkey: Added --batch mode option.
2015-05-06 13:44:53 -04:00
Joey Hess
437d9db41d
assistant: Added --autostop to complement --autostart.
2015-05-01 13:53:45 -04:00
Joey Hess
efb37e7c78
Improve behavior when a git-annex command is told to operate on a file that doesn't exist. It will now continue to other files specified after that on the command line, and only error out at the end.
2015-04-30 15:28:17 -04:00
Joey Hess
9f3e51dd51
move nubbing into function whose algo needs a nubbed list
2015-04-30 14:11:59 -04:00
Joey Hess
607eed0de2
improve messages
2015-04-30 14:10:28 -04:00
Joey Hess
ac6b492711
import: Before removing a duplicate file in --deduplicate or --clean-duplicates mode, verify that enough copies of its content still exist.
2015-04-30 14:04:36 -04:00
Joey Hess
38c458b407
refactor
2015-04-30 14:02:56 -04:00
Joey Hess
9e10b5ca9c
nub list after adding ocal uuid to it
...
This potentially fixes a numcopies counting bug when dropping --from a
remote, and the local repository is trusted. The local repo would end up in
the list twice, so it would verify one less copy than it was supposed to.
2015-04-30 13:13:28 -04:00
Joey Hess
d8ad1d5503
import: Don't stop entire import when one file fails due to being gitignored or conflicting with something in the work tree.
2015-04-29 13:56:41 -04:00
Joey Hess
2e54251c18
import: Check for gitignored files before moving them into the tree. (Needs git 1.8.4 or newer.)
2015-04-29 13:46:12 -04:00
Joey Hess
cfbeb1e7b7
Fix bogus failure of fsck --fast.
2015-04-27 17:40:21 -04:00
Joey Hess
ca2cf63e34
clean up imports
2015-04-19 10:47:07 -04:00
Joey Hess
addc82dab7
removed all uses of undefined from code base
...
It's a code smell, can lead to hard to diagnose error messages.
2015-04-19 00:38:29 -04:00
Joey Hess
978906c616
forgot to add these
2015-04-18 16:09:09 -04:00
Joey Hess
ef2202fd94
required: New command, like wanted, but for required content.
...
Also refactored some code to reduce duplication.
2015-04-18 16:04:35 -04:00
Joey Hess
8d685768d3
fsck --from remote: Avoid downloading a key if it would go over the annex.diskreserve limit.
2015-04-18 14:23:42 -04:00
Joey Hess
8489057e8d
fsck --from remote: When bad content is found in the remote, and the local repo does not have a copy of the content, preserve the bad content in .git/annex/bad/ to avoid further data loss.
2015-04-18 14:13:07 -04:00
Joey Hess
a2902cdaaf
add filename to progress bar, and display ok/failed at end
...
This needed plumbing an AssociatedFile through retrieveKeyFileCheap.
2015-04-14 16:35:10 -04:00
Joey Hess
86a2f9dc4d
Merge branch 'master' into concurrentprogress
...
Conflicts:
debian/changelog
2015-04-14 15:35:15 -04:00
Joey Hess
2be4834822
info: Added --bytes option.
2015-04-12 14:08:40 -04:00
Joey Hess
b14ec45aa8
info dir: Added information about repositories that contain files in the specified directory.
...
This is a nearly free feature; it piggybacks on the location log lookups
done for the numcopies stats. So, the only extra overhead is updating
the map of repository sizes.
However, I had to switch to Data.Map.Strict, which needs containers 0.5.
If backporting to wheezy, will probably need to revert this commit.
2015-04-12 12:54:41 -04:00
Joey Hess
8077ccbd54
get, move, copy, mirror: Concurrent downloads and uploads are now supported!
...
This works, and seems fairly robust. Clean get of 20 files at -J3. At -J10,
there are some messages about ssh multiplexing, probably due to a race
spinning up the ssh connection cacher. But, it manages to get all the files
ok regardless.
The progress bars are a scrambled mess though, due to bugs in
ascii-progress, which I've already filed. Particularly this one:
https://github.com/yamadapc/haskell-ascii-progress/issues/8
2015-04-10 17:08:07 -04:00
Joey Hess
40aae2dd11
unused import
2015-04-10 13:16:04 -04:00
Joey Hess
ce0a82f493
contentlocationn: New plumbing command.
2015-04-09 15:34:47 -04:00
Joey Hess
3695094454
importfeed: Error out when passed a non-url.
2015-04-09 13:09:06 -04:00
Joey Hess
aa31af5594
importfeed: Fix feed download when curl is used.
...
This bug used to only afflict file:/// urls, but became more exposed
by b8f0b7309f
2015-04-09 13:02:57 -04:00
Joey Hess
84dc5629c6
info: Display repository mode: bare when in a bare (non-direct mode) repo.
2015-04-09 12:28:26 -04:00
Joey Hess
d5a25bd567
proxy: Made it work when run in a new repository before initial commit.
2015-04-09 12:24:42 -04:00
Joey Hess
3c2cb25698
wording
2015-04-08 16:16:42 -04:00
Joey Hess
4da371af1e
add: If annex.largefiles is set and does not match a file that's being added, the file will be checked into git rather than being added to the annex. Previously, git annex add skipped over such files; this new behavior is more useful in direct mode.
2015-04-08 16:14:23 -04:00
Joey Hess
279fa44cd0
fromkey, registerurl: When reading from stdin, allow the filename and url, respectively, to contain whitespace.
2015-04-07 00:58:51 -04:00
Joey Hess
62ed5409c6
expire --no-act
2015-04-05 15:02:55 -04:00
Joey Hess
e3ea9d21fc
info remove: show trust level
2015-04-05 13:51:01 -04:00
Joey Hess
9445556c97
rethought distributed fsck; instead add activity.log and expire command
...
This is much more space efficient!
2015-04-05 12:50:02 -04:00
Joey Hess
1c91024978
rename bothHandles -> ioHandles
2015-04-03 15:35:18 -04:00
Jean Jordaan
500cf3e37e
Steer towards deduplication
2015-04-03 14:27:34 +07:00
Joey Hess
7a44dacb74
version: Add --raw
2015-04-02 15:37:39 -04:00
Joey Hess
8b6013dad4
remove unused param
2015-04-01 23:14:51 -04:00
Joey Hess
656fc1c881
fsck: Added --distributed and --expire options, for distributed fsck.
2015-04-01 17:53:16 -04:00
Joey Hess
42bbed7ce5
import: --deduplicate and --cleanduplicates now output the keys corresponding to duplicated files they process.
2015-03-31 15:36:02 -04:00
Joey Hess
6b55974723
addurl --file: When used with a special remote that claims urls and checks their contents, don't override the user's provided filename with filenames that the special remote suggests. Also, don't allow adding the url if the special remote says it contains multiple files.
2015-03-31 15:20:29 -04:00
Joey Hess
9e25cbde20
importfeed: Avoid downloading a redundant item from a feed whose guid has been downloaded before, even when the url has changed.
...
To support this, always store itemid in metadata; before this was only done
when annex.genmetadata was set.
2015-03-31 13:30:13 -04:00
Joey Hess
cd6b62f35e
--auto is no longer a global option; only get, drop, and copy accept it.
...
Not a behavior change unless you were passing it to a command that ignored it.
2015-03-25 17:06:14 -04:00
Joey Hess
c955fb2736
a few command usage fixes
2015-03-25 12:09:42 -04:00
Joey Hess
3cc7c03721
Man pages for individual commands now available, and can be opened using "git annex help <command>"
2015-03-23 17:50:03 -04:00
Joey Hess
daec4b007a
splitting up the man page
...
Common command man pages all split out and often expanded.
A few sections split out into their own pages.
Still need to do all the other commands..
2015-03-23 15:36:10 -04:00
Joey Hess
c233f98564
migrate: --force will force migration of keys already using the destination backend. Useful in rare cases.
2015-03-23 12:11:16 -04:00
Joey Hess
e6158130c6
checkpresentkey: New plumbing command to check if a key can be verified to be present on a remote.
2015-03-20 11:44:46 -04:00
Joey Hess
50ef4105e3
readpresentkey: New plumbing command for checking location log.
2015-03-20 11:22:27 -04:00
Joey Hess
abfe3c09b2
registerurl: New plumbing command for mass-adding urls to keys.
2015-03-15 14:37:33 -04:00
Joey Hess
b24bb6b435
fromkey: Add stdin mode.
2015-03-15 14:07:43 -04:00
Joey Hess
fa180c1ba1
fromkey --force: Skip test that the key has its content in the annex.
2015-03-15 13:51:58 -04:00
Joey Hess
504dda82a4
addurl: Added --raw option, which bypasses special handling of quvi, bittorrent etc urls.
2015-03-05 14:46:08 -04:00
Joey Hess
a6db10d565
sync: Fix committing when in a direct mode repo that has no HEAD ref.
...
Seen for example, a newly checked out git submodule. In this case,
.git/HEAD is a raw sha, rather than the usual reference to a ref.
Removed currentSha in passing, since it was a more roundabout way of
doing what headSha does, and headSha is more robust.
2015-03-04 15:25:35 -04:00
