mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions
25742 commits 13 branches 445 tags 110 MiB
Commit graph

1491 commits

Author SHA1 Message Date
Joey Hess
730cc3feb5 wire tasty's option parser into the main program option parser 
This makes bash completion work for git-annex test, and is
generally cleaner.
 2015-07-13 13:20:10 -04:00
Joey Hess
fd086c5752 convert WebApp; avoid duplicate arg parsing for no repo mode 2015-07-13 11:53:42 -04:00
Joey Hess
b95a48fe45 converted Assistant and Watch; avoid duplicate arg parsing for no repo mode 2015-07-13 11:42:42 -04:00
Joey Hess
084f8d9ac7 convert Import 2015-07-13 11:15:21 -04:00
Joey Hess
9c1a42873c converted ImportFeed 2015-07-13 11:06:41 -04:00
Joey Hess
aff4b9a3dc converted addurl 2015-07-13 10:57:49 -04:00
Joey Hess
3639b4f8ea converted Log 2015-07-13 10:44:51 -04:00
Joey Hess
3d8a8e9792 convert Dead, and allow multiple --key options 2015-07-13 10:26:54 -04:00
Joey Hess
adec382bc2 converted MetaData, eliminating a global value from Annex state .. beautiful 2015-07-12 09:25:43 -04:00
Joey Hess
fdcb54d4f2 converted ContentLocation, ExampleKey, LookupKey 2015-07-11 20:43:45 -04:00
Joey Hess
c6375a9158 converted Status 2015-07-11 10:42:46 -04:00
Joey Hess
215f636bb3 converted Info 2015-07-11 10:41:52 -04:00
Joey Hess
9ad20c2869 converted Forget and TestRemote 2015-07-11 00:42:32 -04:00
Joey Hess
c70c841d30 converted Mirror 2015-07-10 21:05:34 -04:00
Joey Hess
e4b3701dfe convert List 2015-07-10 20:46:48 -04:00
Joey Hess
a0b2fcc663 converted ViCfg (trivially) 2015-07-10 20:40:30 -04:00
Joey Hess
02f6d9b33a convert FindRef 2015-07-10 20:38:11 -04:00
Joey Hess
c94e219fea convert Whereis 2015-07-10 16:32:33 -04:00
Joey Hess
8eb0a440b9 convert Expire 2015-07-10 16:26:23 -04:00
Joey Hess
decfababe9 convert DropUnused 2015-07-10 16:15:31 -04:00
Joey Hess
160d4b9fe0 convert Unused, and remove some dead code for old style option parsing 2015-07-10 16:05:56 -04:00
Joey Hess
4064dd4c82 convert version command, and make --raw work when not in a git repo 2015-07-10 13:49:46 -04:00
Joey Hess
6a4f2087be finished converting all the main options 2015-07-10 13:23:06 -04:00
Joey Hess
5cc882a35e implement withGlobalOptions, and convert Find 2015-07-10 12:47:35 -04:00
Joey Hess
820b92abab wip 
Current status:

* building again, but several commands are commented out
* still need to implement global options, file matching options, etc
 2015-07-09 19:03:24 -04:00
Joey Hess
a7f58634b8 wip 2015-07-09 16:05:45 -04:00
Joey Hess
8ad927dbc6 converted copy and move 
Got a little tricky..
 2015-07-09 15:23:14 -04:00
Joey Hess
032e6485fa use Alternative for parsing KeyOptions 2015-07-09 12:44:03 -04:00
Joey Hess
94e703e8b8 use Alternative when parsing mutually exclusive fsck options 2015-07-09 12:26:25 -04:00
Joey Hess
8a9d2a6e9d let optparse-applicative handle the usage display when run w/o command or bad command 
Still generating the list of commands myself, to get it sorted into
sections and with short synopses.
 2015-07-09 11:52:47 -04:00
Joey Hess
c1c64ec76c formatting 2015-07-09 10:42:28 -04:00
Joey Hess
d8d1499229 finalOpt is the same as optional 2015-07-09 01:02:27 -04:00
Joey Hess
60806dd191 wip 2015-07-08 17:59:06 -04:00
Joey Hess
6a88c7c101 converted fsck's options to optparse-applicative 
Global options and seeking and key options are still to be done.
 2015-07-08 16:58:54 -04:00
Joey Hess
7d9072d1db remove git-annex help options display 
Common options will be displayed in the --help for a command by
optparse-applicative.
 2015-07-08 15:41:54 -04:00
Joey Hess
92d8f80bff support cmdnorepo actions, also using getopt-applicative there 2015-07-08 15:39:05 -04:00
Joey Hess
6e5c1f8db3 convert all commands to work with optparse-applicative 
Still no options though.
 2015-07-08 15:08:02 -04:00
Joey Hess
3125da54f6 display cmdparamdesc in optparse-applicative usage messages 
Since optparse-applicative display "FOO" as "[FOO]", the paramOptional
modifier which wrapped it in square brackets was removed from most places.
 2015-07-08 13:39:11 -04:00
Joey Hess
a2ba701056 started converting to use optparse-applicative 
This is a work in progress. It compiles and is able to do basic command
dispatch, including git autocorrection, while using optparse-applicative
for the core commandline parsing.

* Many commands are temporarily disabled before conversion.
* Options are not wired in yet.
* cmdnorepo actions don't work yet.

Also, removed the [Command] list, which was only used in one place.
 2015-07-08 13:36:25 -04:00
Joey Hess
24800b1bf1 Only look at reflogs for relevant branches, not for git-annex branches 
This speeds it up quite a bit.. May still be too slow in large repos.
 2015-07-07 17:36:30 -04:00
Joey Hess
600717417f update comment 2015-07-07 17:23:02 -04:00
Joey Hess
b11d2f5a8a unused: --used-refspec can now be configured to look at refs in the reflog. This provides a way to not consider old versions of files to be unused after they have reached a specified age, when the old refs in the reflog expire. 
May be slow.
 2015-07-07 17:13:50 -04:00
Joey Hess
a51b98cdd5 sync: When annex.autocommit=false, avoid making any commit of local changes, while still merging with remote to the extent possible. 2015-07-07 16:36:11 -04:00
Joey Hess
81ad277a85 add: Stage symlinks the same as git add would, even if they are not a link to annexed content. 2015-07-07 16:15:30 -04:00
Joey Hess
60fc25e9d3 merge: Avoid creating the synced/master branch. 
The branch needs to be created when merging from the remote in sync,
since we diff between it and the remote's sync branch. But git annex merge
should not be creating sync branches.
 2015-07-07 15:53:27 -04:00
Joey Hess
625303226d import: Fix failure of cross-device import on Windows. 
As well as import, 2 other places ran "mv" manually, so changed them to use
moveFile as well.
 2015-07-07 14:48:23 -04:00
Joey Hess
1529add61a Brought back the setkey plumbing command that was removed in 2011, since we found a use case for it. Note that the command's syntax was changed for consistency. 2015-07-02 17:44:25 -04:00
Joey Hess
1f35141aff assistant --autostart: First any daemons that are already running, which might be left over from a previous login session and so unable to use the ssh agent of a new login session. 2015-07-02 14:16:53 -04:00
Joey Hess
833cf5fff9 log: Fix reversion introduced in version 5.20150528 that broke this command. 2015-07-02 12:53:38 -04:00
Joey Hess
f77d485915 move comment 2015-06-16 19:07:14 -04:00
Joey Hess
be9d9cb5ad avoid building unused bloomfilter when run without --all 2015-06-16 19:04:20 -04:00
Joey Hess
5b801fcad9 on second thought, sync --content --unused is probably not useful, remove 2015-06-16 19:01:06 -04:00
Joey Hess
adba0595bd use bloom filter in second pass of sync --all --content 
This is needed because when preferred content matches on files,
the second pass would otherwise want to drop all keys. Using a bloom filter
avoids this, and in the case of a false positive, a key will be left
undropped that preferred content would allow dropping. Chances of that
happening are a mere 1 in 1 million.
 2015-06-16 18:50:13 -04:00
Joey Hess
a0a8127956 instance Hashable Key for bloomfilter 2015-06-16 18:37:41 -04:00
Joey Hess
8b74aec3ea Increased the default annex.bloomaccuracy from 1000 to 10000000 
This makes git annex unused use around 48 mb more memory than it did before,
but the massive increase in accuracy makes this worthwhile for all but the
smallest systems.

Also, I want to use the bloom filter for sync --all --content, to avoid
dropping files that the preferred content doesn't want, and 1/1000
false positives would be far too many in that use case, even if it were
acceptable for unused.

Actual memory use numbers:

1000: 21.06user 3.42system 0:26.40elapsed 92%CPU (0avgtext+0avgdata 501552maxresident)k
1000000: 21.41user 3.55system 0:26.84elapsed 93%CPU (0avgtext+0avgdata 549496maxresident)k
10000000: 21.84user 3.52system 0:27.89elapsed 90%CPU (0avgtext+0avgdata 549920maxresident)k

Based on these numbers, 10 million seemed a better pick than 1 million.
 2015-06-16 18:12:00 -04:00
Joey Hess
29c03145e6 sync: Add support for --all and --unused. 2015-06-16 16:50:03 -04:00
Joey Hess
99a1113461 switch code to using associated files 2015-06-16 15:07:03 -04:00
Joey Hess
67f7f1b1cb info: Added json output for "backend usage", "numcopies stats", "repositories containing these files", and "transfers in progress". 2015-06-16 13:50:28 -04:00
Joey Hess
829007d629 Improve url parsing to handle some urls containing illegal [] characters in their paths. 
Ie, "https://archive.org/download/zoom-2/Zoom - Release 2 (1996)(Active Software)[!].iso"
 2015-06-14 13:39:44 -04:00
Joey Hess
5123a512d6 add a hint about marking a key dead 2015-06-09 15:12:40 -04:00
Joey Hess
f8ab3bc449 dead --key: Can be used to mark a key as dead. 2015-06-09 14:52:05 -04:00
Joey Hess
6eefc5db65 fsck: Ignore keys that are known to be dead when running in --all mode or a in a bare repo. Otherwise, still reports files with lost contents, even if the content is dead. 2015-06-09 14:08:57 -04:00
Joey Hess
de3bd11a2c import --clean-duplicates: Fix bug that didn't count local or trusted repo's copy of a file as one of the necessary copies to allow removing it from the import location. 2015-06-03 13:15:38 -04:00
Joey Hess
d28e8fbfd5 get --incomplete: New option to resume any interrupted downloads. 2015-06-02 14:20:38 -04:00
Joey Hess
eb33569f9d remove Params constructor from Utility.SafeCommand 
This removes a bit of complexity, and should make things faster
(avoids tokenizing Params string), and probably involve less garbage
collection.

In a few places, it was useful to use Params to avoid needing a list,
but that is easily avoided.

Problems noticed while doing this conversion:

	* Some uses of Params "oneword" which was entirely unnecessary
	  overhead.
	* A few places that built up a list of parameters with ++
	  and then used Params to split it!

Test suite passes.
 2015-06-01 13:52:23 -04:00
Joey Hess
36b9c9ca5f fromkey, registerurl: Improve handling of urls that happen to also be parsable as strange keys. 2015-05-30 02:08:49 -04:00
Joey Hess
77c43a388e fromkey, registerurl: Allow urls to be specified instead of keys, and generate URL keys. 
This is especially useful because the caller doesn't need to generate valid
url keys, which involves some escaping of characters, and may involve
taking a md5sum of the url if it's too long.
 2015-05-22 22:41:36 -04:00
Joey Hess
823bb8031b add annex.used-refspec 2015-05-14 15:44:08 -04:00
Joey Hess
86699ff861 unused: Add --used option, which can specify a set of refs to consider used, rather than the default of considering all refs used. 2015-05-14 15:31:38 -04:00
Joey Hess
61ccf95004 Avoid accumulating transfer failure log files unless the assistant is being used. 
Only the assistant uses these, and only the assistant cleans them up, so
make only git annex transferkeys write them,

There is one behavior change from this. If glacier is being used, and a
manual git annex get --from glacier fails because the file isn't available
yet, the assistant will no longer later see that failed transfer file and
retry the get. Hope no-one depended on that old behavior.
 2015-05-12 15:53:38 -04:00
Joey Hess
a812d598ef Take space that will be used by running downloads into account when checking annex.diskreserve. 2015-05-12 15:20:22 -04:00
Joey Hess
e27b97d364 Merge branch 'master' into concurrentprogress 
Conflicts:
	Command/Fsck.hs
	Messages.hs
	Remote/Directory.hs
	Remote/Git.hs
	Remote/Helper/Special.hs
	Types/Remote.hs
	debian/changelog
	git-annex.cabal
 2015-05-12 13:23:22 -04:00
Joey Hess
4b2e3210ea drop: Now supports --all, --unused, and --key. 2015-05-12 13:00:06 -04:00
Joey Hess
db5d831d07 import: Refuse to import files that are within the work tree, as that does not make sense and could cause data loss. 2015-05-11 12:57:47 -04:00
Joey Hess
26ac0753c1 more FlexibleContexts 2015-05-10 15:54:58 -04:00
Joey Hess
6c2d5b5e41 more time-1.5 fixes 2015-05-10 15:36:58 -04:00
Joey Hess
6cf62a9bde support time-1.5.0 
This no longer uses old-locale's defaultTimeLocale, but provides one
of its own.

Factored out a Logs.TimeStamp.
 2015-05-10 15:21:35 -04:00
Joey Hess
f395ca7a41 Improve quvi 0.4 output parsing to handle cases wher there is no known filename extension. This is currently the case when using quvi with youtube. In this case, the extension ".m" will be used. 2015-05-08 13:39:00 -04:00
Joey Hess
469242ac4d fsck: Ignore error recording the fsck in the activity log, which can happen when running fsck in a read-only repository. Closes: #698559 (fsck can still need to write to the repository if it find problems, but a successful fsck can be done read-only) 2015-05-06 14:45:20 -04:00
Joey Hess
eeb0359a2e contentlocation, examinekey, lookupkey: Added --batch mode option. 2015-05-06 13:44:53 -04:00
Joey Hess
437d9db41d assistant: Added --autostop to complement --autostart. 2015-05-01 13:53:45 -04:00
Joey Hess
efb37e7c78 Improve behavior when a git-annex command is told to operate on a file that doesn't exist. It will now continue to other files specified after that on the command line, and only error out at the end. 2015-04-30 15:28:17 -04:00
Joey Hess
9f3e51dd51 move nubbing into function whose algo needs a nubbed list 2015-04-30 14:11:59 -04:00
Joey Hess
607eed0de2 improve messages 2015-04-30 14:10:28 -04:00
Joey Hess
ac6b492711 import: Before removing a duplicate file in --deduplicate or --clean-duplicates mode, verify that enough copies of its content still exist. 2015-04-30 14:04:36 -04:00
Joey Hess
38c458b407 refactor 2015-04-30 14:02:56 -04:00
Joey Hess
9e10b5ca9c nub list after adding ocal uuid to it 
This potentially fixes a numcopies counting bug when dropping --from a
remote, and the local repository is trusted. The local repo would end up in
the list twice, so it would verify one less copy than it was supposed to.
 2015-04-30 13:13:28 -04:00
Joey Hess
d8ad1d5503 import: Don't stop entire import when one file fails due to being gitignored or conflicting with something in the work tree. 2015-04-29 13:56:41 -04:00
Joey Hess
2e54251c18 import: Check for gitignored files before moving them into the tree. (Needs git 1.8.4 or newer.) 2015-04-29 13:46:12 -04:00
Joey Hess
cfbeb1e7b7 Fix bogus failure of fsck --fast. 2015-04-27 17:40:21 -04:00
Joey Hess
ca2cf63e34 clean up imports 2015-04-19 10:47:07 -04:00
Joey Hess
addc82dab7 removed all uses of undefined from code base 
It's a code smell, can lead to hard to diagnose error messages.
 2015-04-19 00:38:29 -04:00
Joey Hess
978906c616 forgot to add these 2015-04-18 16:09:09 -04:00
Joey Hess
ef2202fd94 required: New command, like wanted, but for required content. 
Also refactored some code to reduce duplication.
 2015-04-18 16:04:35 -04:00
Joey Hess
8d685768d3 fsck --from remote: Avoid downloading a key if it would go over the annex.diskreserve limit. 2015-04-18 14:23:42 -04:00
Joey Hess
8489057e8d fsck --from remote: When bad content is found in the remote, and the local repo does not have a copy of the content, preserve the bad content in .git/annex/bad/ to avoid further data loss. 2015-04-18 14:13:07 -04:00
Joey Hess
a2902cdaaf add filename to progress bar, and display ok/failed at end 
This needed plumbing an AssociatedFile through retrieveKeyFileCheap.
 2015-04-14 16:35:10 -04:00
Joey Hess
86a2f9dc4d Merge branch 'master' into concurrentprogress 
Conflicts:
	debian/changelog
 2015-04-14 15:35:15 -04:00
Joey Hess
2be4834822 info: Added --bytes option. 2015-04-12 14:08:40 -04:00
Joey Hess
b14ec45aa8 info dir: Added information about repositories that contain files in the specified directory. 
This is a nearly free feature; it piggybacks on the location log lookups
done for the numcopies stats. So, the only extra overhead is updating
the map of repository sizes.

However, I had to switch to Data.Map.Strict, which needs containers 0.5.
If backporting to wheezy, will probably need to revert this commit.
 2015-04-12 12:54:41 -04:00