mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions
git-annex/CmdLine/GitAnnex.hs

275 lines
7.6 KiB
Haskell
Raw Normal View History

make test suite link in git-annex's commands and run directly this way, test coverage works
2011-01-06 20:26:57 -04:00
{- git-annex main program
-
restage: New git-annex command, handles restaging unlocked files This is much easier and less failure-prone than having the user run git update-index --refresh themselves. Sponsored-by: Dartmouth College's DANDI project
2022-09-23 16:29:28 -04:00
- Copyright 2010-2022 Joey Hess <id@joeyh.name>
make test suite link in git-annex's commands and run directly this way, test coverage works
2011-01-06 20:26:57 -04:00
-
update licenses from GPL to AGPL This does not change the overall license of the git-annex program, which was already AGPL due to a number of sources files being AGPL already. Legally speaking, I'm adding a new license under which these files are now available; I already released their current contents under the GPL license. Now they're dual licensed GPL and AGPL. However, I intend for all my future changes to these files to only be released under the AGPL license, and I won't be tracking the dual licensing status, so I'm simply changing the license statement to say it's AGPL. (In some cases, others wrote parts of the code of a file and released it under the GPL; but in all cases I have contributed a significant portion of the code in each file and it's that code that is getting the AGPL license; the GPL license of other contributors allows combining with AGPL code.)
2019-03-13 15:48:14 -04:00
- Licensed under the GNU AGPL version 3 or higher.
make test suite link in git-annex's commands and run directly this way, test coverage works
2011-01-06 20:26:57 -04:00
-}
Allow optionally configuring git-annex with -fEKG to enable awesome remote monitoring interfaceat http://localhost:4242/
2013-11-06 16:33:45 -04:00
{-# LANGUAGE CPP, OverloadedStrings #-}
Version build dependency on STM, and allow building without it, which disables the watch command.
2012-06-26 09:15:47 -04:00
reorganize some files and imports
2014-01-26 16:25:55 -04:00
module CmdLine.GitAnnex where
make test suite link in git-annex's commands and run directly this way, test coverage works
2011-01-06 20:26:57 -04:00
Add support for core.worktree, and fix support for GIT_WORK_TREE and GIT_DIR. The environment needs to override git-config. Changed when git config is read, and avoid rereading it once it's been read. chdir for both worktree settings.
2012-05-18 18:20:53 -04:00
import qualified Git.CurrentRepo
simplify
2011-01-07 02:15:10 -04:00
import CmdLine
make test suite link in git-annex's commands and run directly this way, test coverage works
2011-01-06 20:26:57 -04:00
import Command
sync, assistant, remotedaemon: Use ssh connection caching for git pushes and pulls. For sync, saves 1 ssh connection per remote. For remotedaemon, the same ssh connection that is already open to run git-annex-shell notifychanges is reused to pull from the remote. Only potential problem is that this also enables connection caching when the assistant syncs with a ssh remote. Including the sync it does when a network connection has just come up. In that case, cached ssh connections are likely to be stale, and so using them would hang. Until I'm sure such problems have been dealt with, this commit needs to stay on the remotecontrol branch, and not be merged to master. This commit was sponsored by Alexandre Dupas.
2014-04-12 15:59:34 -04:00
import Utility.Env
import Annex.Ssh
multicast: New command, uses uftp to multicast annexed files, for eg a classroom setting. This commit was supported by the NSF-funded DataLad project.
2017-03-30 19:32:58 -04:00
import Annex.Multicast
wire tasty's option parser into the main program option parser This makes bash completion work for git-annex test, and is generally cleaner.
2015-07-13 13:19:20 -04:00
import Types.Test
new improved benchmark command that can benchmark anything git-annex does
2019-01-04 13:43:53 -04:00
import Types.Benchmark
make test suite link in git-annex's commands and run directly this way, test coverage works
2011-01-06 20:26:57 -04:00
convert Whereis
2015-07-10 16:32:33 -04:00
import qualified Command.Help
make test suite link in git-annex's commands and run directly this way, test coverage works
2011-01-06 20:26:57 -04:00
import qualified Command.Add
import qualified Command.Unannex
import qualified Command.Drop
import qualified Command.Move
import qualified Command.Copy
import qualified Command.Get
converted copy and move Got a little tricky..
2015-07-09 15:23:14 -04:00
import qualified Command.Fsck
converted ContentLocation, ExampleKey, LookupKey
2015-07-11 20:43:45 -04:00
import qualified Command.LookupKey
calckey: New plumbing command, calculates the key that would be used to refer to a file
2016-04-20 13:49:42 -04:00
import qualified Command.CalcKey
converted ContentLocation, ExampleKey, LookupKey
2015-07-11 20:43:45 -04:00
import qualified Command.ContentLocation
import qualified Command.ExamineKey
matchexpression: New plumbing command to check if a preferred content expression matches some data.
2016-01-25 16:16:18 -04:00
import qualified Command.MatchExpression
make test suite link in git-annex's commands and run directly this way, test coverage works
2011-01-06 20:26:57 -04:00
import qualified Command.FromKey
registerurl: New plumbing command for mass-adding urls to keys.
2015-03-15 14:37:33 -04:00
import qualified Command.RegisterUrl
unregisterurl: New command Implemented by generalizing registerurl. Without the implicit batch mode of registerurl since that is only a backwards compatability thing (see commit 1d1054faa6989acb716d620266a0a0a8a2ec407e).
2021-03-01 14:28:24 -04:00
import qualified Command.UnregisterUrl
Brought back the setkey plumbing command that was removed in 2011, since we found a use case for it. Note that the command's syntax was changed for consistency.
2015-07-02 17:44:25 -04:00
import qualified Command.SetKey
make test suite link in git-annex's commands and run directly this way, test coverage works
2011-01-06 20:26:57 -04:00
import qualified Command.DropKey
rename helper More consistent name with TransferrerPool
2020-12-09 13:21:20 -04:00
import qualified Command.Transferrer
add transferkey command Used by the assistant, rather than copy, this is faster because it avoids using git ls-files, avoids checking the location log redundantly, and runs in oneshot mode, avoiding making a commit to the git-annex branch for every file transferred.
2012-08-24 17:23:58 -04:00
import qualified Command.TransferKey
implement transferkeys plumbing command
2013-03-19 16:58:36 -04:00
import qualified Command.TransferKeys
setpresentkey: A new plumbing-level command.
2014-12-29 15:16:40 -04:00
import qualified Command.SetPresentKey
readpresentkey: New plumbing command for checking location log.
2015-03-20 11:22:27 -04:00
import qualified Command.ReadPresentKey
checkpresentkey: New plumbing command to check if a key can be verified to be present on a remote.
2015-03-20 11:44:46 -04:00
import qualified Command.CheckPresentKey
rekey: New plumbing level command, can be used to change the keys used for files en masse.
2012-02-16 16:36:35 -04:00
import qualified Command.ReKey
adjusted branches, proof of concept "git annex adjust" may be a temporary interface, but works for a proof of concept. It is pretty fast at creating the adjusted branch. The main overhead is injecting pointer files. It might be worth optimising that by reusing the symlink target as the pointer file content. When I tried to do that, the problem was that the clean filter doesn't use that same format, and so git thought files had changed. Could be dealt with, perhaps make the clean filter use symlink format for pointer files when on an adjusted branch? But the real overhead is in checking out the branch, when git runs the smudge filter once per file. That is perhaps too slow to be usable, although it may only affect initial checkout of the branch, and not updates. TBD.
2016-02-25 16:11:13 -04:00
import qualified Command.Adjust
converted MetaData, eliminating a global value from Annex state .. beautiful
2015-07-12 09:25:43 -04:00
import qualified Command.MetaData
add git annex view command (And a vpop command, which is still a bit buggy.) Still need to do vadd and vrm, though this also adds their documentation. Currently not very happy with the view log data serialization. I had to lose the TDFA regexps temporarily, so I can have Read/Show instances of View. I expect the view log format will change in some incompatable way later, probably adding last known refs for the parent branch to View or something like that. Anyway, it basically works, although it's a bit slow looking up the metadata. The actual git branch construction is about as fast as it can be using the current git plumbing. This commit was sponsored by Peter Hogg.
2014-02-18 17:38:23 -04:00
import qualified Command.View
add vadd command
2014-02-18 20:01:56 -04:00
import qualified Command.VAdd
add tip about metadata driven views (and more flexible view filtering) While writing this documentation, I realized that there needed to be a way to stay in a view like tag=* while adding a filter like tag=work that applies to the same field. So, there are really two ways a view can be refined. It can have a new "field=explicitvalue" filter added to it, which does not change the "shape" of the view, but narrows the files it shows. Or, it can have a new view added, which adds another level of subdirectories. So, added a vfilter command, which takes explicit values to add to the filter, and rejects changes that would change the shape of the view. And, made vadd only accept changes that change the shape of the view. And, changed the View data type slightly; now components that can match multiple metadata values can be visible, or not visible. This commit was sponsored by Stelian Iancu.
2014-02-19 15:10:18 -04:00
import qualified Command.VFilter
add git annex view command (And a vpop command, which is still a bit buggy.) Still need to do vadd and vrm, though this also adds their documentation. Currently not very happy with the view log data serialization. I had to lose the TDFA regexps temporarily, so I can have Read/Show instances of View. I expect the view log format will change in some incompatable way later, probably adding last known refs for the parent branch to View or something like that. Anyway, it basically works, although it's a bit slow looking up the metadata. The actual git branch construction is about as fast as it can be using the current git plumbing. This commit was sponsored by Peter Hogg.
2014-02-18 17:38:23 -04:00
import qualified Command.VPop
add vcycle command
2014-02-18 20:16:28 -04:00
import qualified Command.VCycle
better command name
2011-10-31 15:18:41 -04:00
import qualified Command.Reinject
make test suite link in git-annex's commands and run directly this way, test coverage works
2011-01-06 20:26:57 -04:00
import qualified Command.Fix
import qualified Command.Init
add describe subcommand
2011-03-03 17:21:00 -04:00
import qualified Command.Describe
started on initremote
2011-03-28 23:22:31 -04:00
import qualified Command.InitRemote
To enable an existing special remote, the new enableremote command must be used. The initremote command now is used only to create new special remotes.
2013-04-26 18:22:44 -04:00
import qualified Command.EnableRemote
configremote New command, currently limited to changing autoenable= setting of a special remote. It will probably never be used for more than that given the limitations on it. Sponsored-by: Brock Spratlen on Patreon
2023-04-18 15:30:49 -04:00
import qualified Command.ConfigRemote
added renameremote command
2019-04-15 13:05:44 -04:00
import qualified Command.RenameRemote
git-annex enable-tor command Tor unfortunately does not come out of the box configured to let hidden services register themselves on the fly via the ControlPort. And, changing the config to enable the ControlPort and a particular type of auth for it may break something already using the ControlPort, or lessen the security of the system. So, this leaves only one option to us: Add a hidden service to the torrc. git-annex enable-tor does so, and picks an unused high port for tor to listen on for connections to the hidden service. It's up to the caller to somehow pick a local port to listen on that won't be used by something else. That may be difficult to do.. This commit was sponsored by Jochen Bartl on Patreon.
2016-11-14 13:26:34 -04:00
import qualified Command.EnableTor
multicast: New command, uses uftp to multicast annexed files, for eg a classroom setting. This commit was supported by the NSF-funded DataLad project.
2017-03-30 19:32:58 -04:00
import qualified Command.Multicast
convert Expire
2015-07-10 16:26:23 -04:00
import qualified Command.Expire
add repair command
2013-10-23 12:21:59 -04:00
import qualified Command.Repair
convert Unused, and remove some dead code for old style option parsing
2015-07-10 16:05:56 -04:00
import qualified Command.Unused
convert DropUnused
2015-07-10 16:15:31 -04:00
import qualified Command.DropUnused
addunused: New command, the opposite of dropunused, it relinks unused content into the git repository.
2012-05-02 14:59:05 -04:00
import qualified Command.AddUnused
make test suite link in git-annex's commands and run directly this way, test coverage works
2011-01-06 20:26:57 -04:00
import qualified Command.Unlock
import qualified Command.Lock
import qualified Command.PreCommit
post-recive hook to make updateInstead work in direct mode and adjusted branches * Added post-recieve hook, which makes updateInstead work with direct mode and adjusted branches. * init: Set up the post-receive hook. This commit was sponsored by Fernando Jimenez on Patreon.
2017-02-17 14:04:43 -04:00
import qualified Command.PostReceive
skeleton of filter-branch command, with option parser
2021-05-14 10:59:48 -04:00
import qualified Command.FilterBranch
implement withGlobalOptions, and convert Find
2015-07-10 12:47:35 -04:00
import qualified Command.Find
findkeys: New command, very similar to git-annex find but operating on keys I've long been asked for `git-annex find --all` or something like that, but pushed back on it because I feel that the command is analagous to find(1) and so it would be surprising for it to list keys rather than files. So instead, add a new findkeys subcommand. Note that the use of withKeyOptions is rather strange because usually that is used to fall back to --all rather than listing files, but here it's made to default to --all like behavior and never list files. A performance thing that could be improved is that withKeyOptions always reads and caches location logs. But findkeys with no options does not need them, so it could be made faster. That caching does speed up options like --in though. This is really just a subset of a more general performance thing that --all reads location logs sometimes unncessarily. Anyway, it needs to read the location log in order to checkDead, and it seems good that findkeys does skip dead keys. Also, cleaned up comments on git-annex-find man page asking for --all option. Sponsored-by: Dartmouth College's DANDI project
2023-01-17 14:42:29 -04:00
import qualified Command.FindKeys
convert FindRef
2015-07-10 20:38:11 -04:00
import qualified Command.FindRef
convert Whereis
2015-07-10 16:32:33 -04:00
import qualified Command.Whereis
whereused implemented except --historical Sponsored-by: Jack Hill on Patreon
2021-07-14 14:25:52 -04:00
import qualified Command.WhereUsed
convert List
2015-07-10 20:46:48 -04:00
import qualified Command.List
converted Log
2015-07-13 10:44:51 -04:00
import qualified Command.Log
add merge subcommand
2011-06-22 18:46:45 -04:00
import qualified Command.Merge
resolvemerge: New plumbing command that runs the automatic merge conflict resolver.
2014-07-11 16:45:18 -04:00
import qualified Command.ResolveMerge
converted Info
2015-07-11 10:41:52 -04:00
import qualified Command.Info
converted Status
2015-07-11 10:42:46 -04:00
import qualified Command.Status
Added inprogress command for accessing files as they are being downloaded. Chose to make this only handle files actively being downloaded, not temp files for downloads that were interrupted or files that have been fully downloaded. This commit was sponsored by Ole-Morten Duesund on Patreon.
2017-12-28 11:46:39 -04:00
import qualified Command.Inprogress
New migrate subcommand can be used to switch files to using a different backend, safely and with no duplication of content.
2011-01-08 15:54:14 -04:00
import qualified Command.Migrate
make test suite link in git-annex's commands and run directly this way, test coverage works
2011-01-06 20:26:57 -04:00
import qualified Command.Uninit
reinit: New command that can initialize a new reposotory using the configuration of a previously known repository. Useful if a repository got deleted and you want to clone it back the way it was.
2014-04-15 20:13:35 -04:00
import qualified Command.Reinit
global numcopies setting * numcopies: New command, sets global numcopies value that is seen by all clones of a repository. * The annex.numcopies git config setting is deprecated. Once the numcopies command is used to set the global number of copies, any annex.numcopies git configs will be ignored. * assistant: Make the prefs page set the global numcopies. This global numcopies setting is needed to let preferred content expressions operate on numcopies. It's also convenient, because typically if you want git-annex to preserve N copies of files in a repo, you want it to do that no matter which repo it's running in. Making it global avoids needing to warn the user about gotchas involving inconsistent annex.numcopies settings. (See changes to doc/numcopies.mdwn.) Added a new variety of git-annex branch log file, that holds only 1 value. Will probably be useful for other stuff later. This commit was sponsored by Nicolas Pouillard.
2014-01-20 16:47:56 -04:00
import qualified Command.NumCopies
mincopies This is conceptually very simple, just making a 1 that was hard coded be exposed as a config option. The hard part was plumbing all that, and dealing with complexities like reading it from git attributes at the same time that numcopies is read. Behavior change: When numcopies is set to 0, git-annex used to drop content without requiring any copies. Now to get that (highly unsafe) behavior, mincopies also needs to be set to 0. It seemed better to remove that edge case, than complicate mincopies by ignoring it when numcopies is 0. This commit was sponsored by Denis Dzyubenko on Patreon.
2021-01-06 14:11:08 -04:00
import qualified Command.MinCopies
make test suite link in git-annex's commands and run directly this way, test coverage works
2011-01-06 20:26:57 -04:00
import qualified Command.Trust
import qualified Command.Untrust
implement 3 level trust storage in trust.log
2011-01-26 15:37:16 -04:00
import qualified Command.Semitrust
convert Dead, and allow multiple --key options
2015-07-13 10:26:54 -04:00
import qualified Command.Dead
group, ungroup: New commands to indicate groups of repositories.
2012-10-01 15:12:04 -04:00
import qualified Command.Group
The "git annex content" command is renamed to "git annex wanted".
2013-10-28 14:08:38 -04:00
import qualified Command.Wanted
groupwanted: New command to set the groupwanted preferred content expression.
2015-02-06 15:12:42 -04:00
import qualified Command.GroupWanted
required: New command, like wanted, but for required content. Also refactored some code to reduce duplication.
2015-04-18 16:04:25 -04:00
import qualified Command.Required
add schedule command Mostly because it gives me an excuse and a hook to document the schedule expression format.
2013-10-13 15:40:38 -04:00
import qualified Command.Schedule
group, ungroup: New commands to indicate groups of repositories.
2012-10-01 15:12:04 -04:00
import qualified Command.Ungroup
config: New command for storing configuration in the git-annex branch. Any config names can be set using this; git-annex commands will only look at specific ones that make sense and are worth the overhead of querying the branch. This might also be useful for storing whatever other config-type stuff the user might want to shove into the git-annex branch. This commit was sponsored by Jochen Bartl on Patreon.
2017-01-30 16:41:29 -04:00
import qualified Command.Config
converted ViCfg (trivially)
2015-07-10 20:40:30 -04:00
import qualified Command.Vicfg
sync: New command that synchronises the local repository and default remote, by running git commit, pull, and push for you.
2011-12-09 20:27:22 -04:00
import qualified Command.Sync
git-annex pull and push Split out two new commands, git-annex pull and git-annex push. Those plus a git commit are equivilant to git-annex sync. In a sense, git-annex sync conflates 3 things, and it would have been better to have push and pull from the beginning and not sync. Although note that git-annex sync --content is faster than a pull followed by a push, because it only has to walk the tree once, look at preferred content once, etc. So there is some value in git-annex sync in speed, as well as user convenience. And it would be hard to split out pull and push from sync, as far as the implementaton goes. The implementation inside sync was easy, just adjust SyncOptions so it does the right thing. Note that the new commands default to syncing content, unless annex.synccontent is explicitly set to false. I'd like sync to also do that, but that's a hard transition to make. As a start to that transition, I added a note to git-annex-sync.mdwn that it may start to do so in a future version of git-annex. But a real transition would necessarily involve displaying warnings when sync is used without --content, and time. Sponsored-by: Kevin Mueller on Patreon
2023-05-16 16:37:30 -04:00
import qualified Command.Pull
import qualified Command.Push
converted Mirror
2015-07-10 21:05:34 -04:00
import qualified Command.Mirror
converted addurl
2015-07-13 10:57:49 -04:00
import qualified Command.AddUrl
converted ImportFeed
2015-07-13 11:06:41 -04:00
import qualified Command.ImportFeed
rmurl: New command, removes one of the recorded urls for a file.
2013-04-22 17:18:53 -04:00
import qualified Command.RmUrl
convert Import
2015-07-13 11:15:21 -04:00
import qualified Command.Import
initial export command Very basic operation works, but of course this is only the beginning. This commit was sponsored by Nick Daly on Patreon.
2017-08-29 14:58:38 -04:00
import qualified Command.Export
new map subcommand, basically working Still todo: - add repos from uuid.log that were not directly found - group repos into their respective hosts - display inaccessible repos and broken remote connections in red - anonymize the url display somewhat, so the maps can be shared - use uuid info to tell when two apparently different repos are actually the same repo accessed in different ways
2011-02-03 18:55:12 -04:00
import qualified Command.Map
added direct and indirect commands
2012-12-13 15:44:56 -04:00
import qualified Command.Direct
import qualified Command.Indirect
add explicit upgrade command
2011-03-16 15:48:26 -04:00
import qualified Command.Upgrade
converted Forget and TestRemote
2015-07-11 00:42:32 -04:00
import qualified Command.Forget
implement p2p command
2016-11-30 14:35:24 -04:00
import qualified Command.P2P
proxy: for all your direct mode repository munging needs This allows bypassing the direct mode guard in a safe way to do all sorts of things including git revert, git mv, git checkout ... This commit was sponsored by the WikiMedia Foundation.
2014-11-12 15:41:15 -04:00
import qualified Command.Proxy
diffdriver: New git-annex command, to make git external diff drivers work with annexed files. Closes https://github.com/datalad/datalad/issues/18
2014-11-24 16:14:01 -04:00
import qualified Command.DiffDriver
skeleton smudge/clean filters
2015-12-04 13:02:56 -04:00
import qualified Command.Smudge
add git-annex filter-process filter-process: New command that can make git add/checkout faster when there are a lot of unlocked annexed files or non-annexed files, but that also makes git add of large annexed files slower. Use it by running: git config filter.annex.process 'git-annex filter-process' Fully tested and working, but I have not benchmarked it at all. And, incremental hashing is not done when git add uses it, so extra work is done in that case. Sponsored-by: Mark Reidenbach on Patreon
2021-11-04 15:02:36 -04:00
import qualified Command.FilterProcess
restage: New git-annex command, handles restaging unlocked files This is much easier and less failure-prone than having the user run git update-index --refresh themselves. Sponsored-by: Dartmouth College's DANDI project
2022-09-23 16:29:28 -04:00
import qualified Command.Restage
converted Forget and TestRemote
2015-07-11 00:42:32 -04:00
import qualified Command.Undo
Add version command to show git-annex version as well as repository version information.
2011-03-19 14:33:24 -04:00
import qualified Command.Version
enable remotedaemon when assistant is not enabled Needed to serve tor hidden services.
2016-11-29 15:41:28 -04:00
import qualified Command.RemoteDaemon
Version build dependency on STM, and allow building without it, which disables the watch command.
2012-06-26 09:15:47 -04:00
#ifdef WITH_ASSISTANT
converted Assistant and Watch; avoid duplicate arg parsing for no repo mode
2015-07-13 11:42:42 -04:00
import qualified Command.Watch
import qualified Command.Assistant
git annex webapp now opens a browser to the webapp Also, starts the assistant if it wasn't already running.
2012-07-25 23:13:01 -04:00
#ifdef WITH_WEBAPP
convert WebApp; avoid duplicate arg parsing for no repo mode
2015-07-13 11:53:42 -04:00
import qualified Command.WebApp
git annex webapp now opens a browser to the webapp Also, starts the assistant if it wasn't already running.
2012-07-25 23:13:01 -04:00
#endif
Version build dependency on STM, and allow building without it, which disables the watch command.
2012-06-26 09:15:47 -04:00
#endif
testremote: New command to test uploads/downloads to a remote. This only performs some basic tests so far; no testing of chunking or resuming. Also, the existing encryption type of the remote is used; it would be good later to derive an encrypted and a non-encrypted version of the remote and test them both. This commit was sponsored by Joseph Liu.
2014-08-01 15:09:49 -04:00
import qualified Command.Test
fuzz tester
2013-05-23 19:00:46 -04:00
import qualified Command.FuzzTest
converted Forget and TestRemote
2015-07-11 00:42:32 -04:00
import qualified Command.TestRemote
add database benchmark The benchmark shows that the database access is quite fast indeed! And, it scales linearly to the number of keys, with one exception, getAssociatedKey. Based on this benchmark, I don't think I need worry about optimising for cases where all files are locked and the database is mostly empty. In those cases, database access will be misses, and according to this benchmark, should add only 50 milliseconds to runtime. (NB: There may be some overhead to getting the database opened and locking the handle that this benchmark doesn't see.) joey@darkstar:~/src/git-annex>./git-annex benchmark setting up database with 1000 setting up database with 10000 benchmarking keys database/getAssociatedFiles from 1000 (hit) time 62.77 μs (62.70 μs .. 62.85 μs) 1.000 R² (1.000 R² .. 1.000 R²) mean 62.81 μs (62.76 μs .. 62.88 μs) std dev 201.6 ns (157.5 ns .. 259.5 ns) benchmarking keys database/getAssociatedFiles from 1000 (miss) time 50.02 μs (49.97 μs .. 50.07 μs) 1.000 R² (1.000 R² .. 1.000 R²) mean 50.09 μs (50.04 μs .. 50.17 μs) std dev 206.7 ns (133.8 ns .. 295.3 ns) benchmarking keys database/getAssociatedKey from 1000 (hit) time 211.2 μs (210.5 μs .. 212.3 μs) 1.000 R² (0.999 R² .. 1.000 R²) mean 211.0 μs (210.7 μs .. 212.0 μs) std dev 1.685 μs (334.4 ns .. 3.517 μs) benchmarking keys database/getAssociatedKey from 1000 (miss) time 173.5 μs (172.7 μs .. 174.2 μs) 1.000 R² (0.999 R² .. 1.000 R²) mean 173.7 μs (173.0 μs .. 175.5 μs) std dev 3.833 μs (1.858 μs .. 6.617 μs) variance introduced by outliers: 16% (moderately inflated) benchmarking keys database/getAssociatedFiles from 10000 (hit) time 64.01 μs (63.84 μs .. 64.18 μs) 1.000 R² (1.000 R² .. 1.000 R²) mean 64.85 μs (64.34 μs .. 66.02 μs) std dev 2.433 μs (547.6 ns .. 4.652 μs) variance introduced by outliers: 40% (moderately inflated) benchmarking keys database/getAssociatedFiles from 10000 (miss) time 50.33 μs (50.28 μs .. 50.39 μs) 1.000 R² (1.000 R² .. 1.000 R²) mean 50.32 μs (50.26 μs .. 50.38 μs) std dev 202.7 ns (167.6 ns .. 252.0 ns) benchmarking keys database/getAssociatedKey from 10000 (hit) time 1.142 ms (1.139 ms .. 1.146 ms) 1.000 R² (1.000 R² .. 1.000 R²) mean 1.142 ms (1.140 ms .. 1.144 ms) std dev 7.142 μs (4.994 μs .. 10.98 μs) benchmarking keys database/getAssociatedKey from 10000 (miss) time 1.094 ms (1.092 ms .. 1.096 ms) 1.000 R² (1.000 R² .. 1.000 R²) mean 1.095 ms (1.095 ms .. 1.097 ms) std dev 4.277 μs (2.591 μs .. 7.228 μs)
2016-01-12 13:01:44 -04:00
import qualified Command.Benchmark
make test suite link in git-annex's commands and run directly this way, test coverage works
2011-01-06 20:26:57 -04:00
new improved benchmark command that can benchmark anything git-annex does
2019-01-04 13:43:53 -04:00
cmds :: Parser TestOptions -> TestRunner -> MkBenchmarkGenerator -> [Command]
remove --backend from global options --backend is no longer a global option, and is only accepted by commands that actually need it. Three commands that used to support backend but don't any longer are watch, webapp, and assistant. It would be possible to make them support it, but I doubt anyone used the option with these. And in the case of webapp and assistant, the option was handled inconsistently, only taking affect when the command is run with an existing git-annex repo, not when it creates a new one. Also, renamed GlobalOption etc to AnnexOption. Because there are many options of this type that are not actually global (any more) and get added to commands that need them. Sponsored-by: Kevin Mueller on Patreon
2022-06-29 13:28:08 -04:00
cmds testoptparser testrunner mkbenchmarkgenerator = map addGitAnnexCommonOptions $
convert Whereis
2015-07-10 16:32:33 -04:00
[ Command.Help.cmd
, Command.Add.cmd
doh't use "def" for command definitions, it conflicts with Data.Default.def
2014-10-14 14:20:10 -04:00
, Command.Get.cmd
, Command.Drop.cmd
, Command.Move.cmd
, Command.Copy.cmd
converted copy and move Got a little tricky..
2015-07-09 15:23:14 -04:00
, Command.Fsck.cmd
doh't use "def" for command definitions, it conflicts with Data.Default.def
2014-10-14 14:20:10 -04:00
, Command.Unlock.cmd
started converting to use optparse-applicative This is a work in progress. It compiles and is able to do basic command dispatch, including git autocorrection, while using optparse-applicative for the core commandline parsing. * Many commands are temporarily disabled before conversion. * Options are not wired in yet. * cmdnorepo actions don't work yet. Also, removed the [Command] list, which was only used in one place.
2015-07-08 12:33:27 -04:00
, Command.Unlock.editcmd
doh't use "def" for command definitions, it conflicts with Data.Default.def
2014-10-14 14:20:10 -04:00
, Command.Lock.cmd
, Command.Sync.cmd
git-annex pull and push Split out two new commands, git-annex pull and git-annex push. Those plus a git commit are equivilant to git-annex sync. In a sense, git-annex sync conflates 3 things, and it would have been better to have push and pull from the beginning and not sync. Although note that git-annex sync --content is faster than a pull followed by a push, because it only has to walk the tree once, look at preferred content once, etc. So there is some value in git-annex sync in speed, as well as user convenience. And it would be hard to split out pull and push from sync, as far as the implementaton goes. The implementation inside sync was easy, just adjust SyncOptions so it does the right thing. Note that the new commands default to syncing content, unless annex.synccontent is explicitly set to false. I'd like sync to also do that, but that's a hard transition to make. As a start to that transition, I added a note to git-annex-sync.mdwn that it may start to do so in a future version of git-annex. But a real transition would necessarily involve displaying warnings when sync is used without --content, and time. Sponsored-by: Kevin Mueller on Patreon
2023-05-16 16:37:30 -04:00
, Command.Pull.cmd
, Command.Push.cmd
converted Mirror
2015-07-10 21:05:34 -04:00
, Command.Mirror.cmd
converted addurl
2015-07-13 10:57:49 -04:00
, Command.AddUrl.cmd
converted ImportFeed
2015-07-13 11:06:41 -04:00
, Command.ImportFeed.cmd
doh't use "def" for command definitions, it conflicts with Data.Default.def
2014-10-14 14:20:10 -04:00
, Command.RmUrl.cmd
convert Import
2015-07-13 11:15:21 -04:00
, Command.Import.cmd
initial export command Very basic operation works, but of course this is only the beginning. This commit was sponsored by Nick Daly on Patreon.
2017-08-29 14:58:38 -04:00
, Command.Export.cmd
doh't use "def" for command definitions, it conflicts with Data.Default.def
2014-10-14 14:20:10 -04:00
, Command.Init.cmd
, Command.Describe.cmd
, Command.InitRemote.cmd
, Command.EnableRemote.cmd
configremote New command, currently limited to changing autoenable= setting of a special remote. It will probably never be used for more than that given the limitations on it. Sponsored-by: Brock Spratlen on Patreon
2023-04-18 15:30:49 -04:00
, Command.ConfigRemote.cmd
added renameremote command
2019-04-15 13:05:44 -04:00
, Command.RenameRemote.cmd
git-annex enable-tor command Tor unfortunately does not come out of the box configured to let hidden services register themselves on the fly via the ControlPort. And, changing the config to enable the ControlPort and a particular type of auth for it may break something already using the ControlPort, or lessen the security of the system. So, this leaves only one option to us: Add a hidden service to the torrc. git-annex enable-tor does so, and picks an unused high port for tor to listen on for connections to the hidden service. It's up to the caller to somehow pick a local port to listen on that won't be used by something else. That may be difficult to do.. This commit was sponsored by Jochen Bartl on Patreon.
2016-11-14 13:26:34 -04:00
, Command.EnableTor.cmd
multicast: New command, uses uftp to multicast annexed files, for eg a classroom setting. This commit was supported by the NSF-funded DataLad project.
2017-03-30 19:32:58 -04:00
, Command.Multicast.cmd
doh't use "def" for command definitions, it conflicts with Data.Default.def
2014-10-14 14:20:10 -04:00
, Command.Reinject.cmd
, Command.Unannex.cmd
, Command.Uninit.cmd
, Command.Reinit.cmd
, Command.PreCommit.cmd
post-recive hook to make updateInstead work in direct mode and adjusted branches * Added post-recieve hook, which makes updateInstead work with direct mode and adjusted branches. * init: Set up the post-receive hook. This commit was sponsored by Fernando Jimenez on Patreon.
2017-02-17 14:04:43 -04:00
, Command.PostReceive.cmd
doh't use "def" for command definitions, it conflicts with Data.Default.def
2014-10-14 14:20:10 -04:00
, Command.NumCopies.cmd
mincopies This is conceptually very simple, just making a 1 that was hard coded be exposed as a config option. The hard part was plumbing all that, and dealing with complexities like reading it from git attributes at the same time that numcopies is read. Behavior change: When numcopies is set to 0, git-annex used to drop content without requiring any copies. Now to get that (highly unsafe) behavior, mincopies also needs to be set to 0. It seemed better to remove that edge case, than complicate mincopies by ignoring it when numcopies is 0. This commit was sponsored by Denis Dzyubenko on Patreon.
2021-01-06 14:11:08 -04:00
, Command.MinCopies.cmd
doh't use "def" for command definitions, it conflicts with Data.Default.def
2014-10-14 14:20:10 -04:00
, Command.Trust.cmd
, Command.Untrust.cmd
, Command.Semitrust.cmd
convert Dead, and allow multiple --key options
2015-07-13 10:26:54 -04:00
, Command.Dead.cmd
doh't use "def" for command definitions, it conflicts with Data.Default.def
2014-10-14 14:20:10 -04:00
, Command.Group.cmd
, Command.Wanted.cmd
groupwanted: New command to set the groupwanted preferred content expression.
2015-02-06 15:12:42 -04:00
, Command.GroupWanted.cmd
required: New command, like wanted, but for required content. Also refactored some code to reduce duplication.
2015-04-18 16:04:25 -04:00
, Command.Required.cmd
doh't use "def" for command definitions, it conflicts with Data.Default.def
2014-10-14 14:20:10 -04:00
, Command.Schedule.cmd
, Command.Ungroup.cmd
config: New command for storing configuration in the git-annex branch. Any config names can be set using this; git-annex commands will only look at specific ones that make sense and are worth the overhead of querying the branch. This might also be useful for storing whatever other config-type stuff the user might want to shove into the git-annex branch. This commit was sponsored by Jochen Bartl on Patreon.
2017-01-30 16:41:29 -04:00
, Command.Config.cmd
converted ViCfg (trivially)
2015-07-10 20:40:30 -04:00
, Command.Vicfg.cmd
converted ContentLocation, ExampleKey, LookupKey
2015-07-11 20:43:45 -04:00
, Command.LookupKey.cmd
calckey: New plumbing command, calculates the key that would be used to refer to a file
2016-04-20 13:49:42 -04:00
, Command.CalcKey.cmd
converted ContentLocation, ExampleKey, LookupKey
2015-07-11 20:43:45 -04:00
, Command.ContentLocation.cmd
, Command.ExamineKey.cmd
matchexpression: New plumbing command to check if a preferred content expression matches some data.
2016-01-25 16:16:18 -04:00
, Command.MatchExpression.cmd
doh't use "def" for command definitions, it conflicts with Data.Default.def
2014-10-14 14:20:10 -04:00
, Command.FromKey.cmd
forgot one thing
2015-03-15 14:55:44 -04:00
, Command.RegisterUrl.cmd
unregisterurl: New command Implemented by generalizing registerurl. Without the implicit batch mode of registerurl since that is only a backwards compatability thing (see commit 1d1054faa6989acb716d620266a0a0a8a2ec407e).
2021-03-01 14:28:24 -04:00
, Command.UnregisterUrl.cmd
Brought back the setkey plumbing command that was removed in 2011, since we found a use case for it. Note that the command's syntax was changed for consistency.
2015-07-02 17:44:25 -04:00
, Command.SetKey.cmd
doh't use "def" for command definitions, it conflicts with Data.Default.def
2014-10-14 14:20:10 -04:00
, Command.DropKey.cmd
rename helper More consistent name with TransferrerPool
2020-12-09 13:21:20 -04:00
, Command.Transferrer.cmd
doh't use "def" for command definitions, it conflicts with Data.Default.def
2014-10-14 14:20:10 -04:00
, Command.TransferKey.cmd
, Command.TransferKeys.cmd
setpresentkey: A new plumbing-level command.
2014-12-29 15:16:40 -04:00
, Command.SetPresentKey.cmd
readpresentkey: New plumbing command for checking location log.
2015-03-20 11:22:27 -04:00
, Command.ReadPresentKey.cmd
checkpresentkey: New plumbing command to check if a key can be verified to be present on a remote.
2015-03-20 11:44:46 -04:00
, Command.CheckPresentKey.cmd
doh't use "def" for command definitions, it conflicts with Data.Default.def
2014-10-14 14:20:10 -04:00
, Command.ReKey.cmd
adjusted branches, proof of concept "git annex adjust" may be a temporary interface, but works for a proof of concept. It is pretty fast at creating the adjusted branch. The main overhead is injecting pointer files. It might be worth optimising that by reusing the symlink target as the pointer file content. When I tried to do that, the problem was that the clean filter doesn't use that same format, and so git thought files had changed. Could be dealt with, perhaps make the clean filter use symlink format for pointer files when on an adjusted branch? But the real overhead is in checking out the branch, when git runs the smudge filter once per file. That is perhaps too slow to be usable, although it may only affect initial checkout of the branch, and not updates. TBD.
2016-02-25 16:11:13 -04:00
, Command.Adjust.cmd
converted MetaData, eliminating a global value from Annex state .. beautiful
2015-07-12 09:25:43 -04:00
, Command.MetaData.cmd
doh't use "def" for command definitions, it conflicts with Data.Default.def
2014-10-14 14:20:10 -04:00
, Command.View.cmd
, Command.VAdd.cmd
, Command.VFilter.cmd
, Command.VPop.cmd
, Command.VCycle.cmd
, Command.Fix.cmd
convert Expire
2015-07-10 16:26:23 -04:00
, Command.Expire.cmd
doh't use "def" for command definitions, it conflicts with Data.Default.def
2014-10-14 14:20:10 -04:00
, Command.Repair.cmd
convert Unused, and remove some dead code for old style option parsing
2015-07-10 16:05:56 -04:00
, Command.Unused.cmd
convert DropUnused
2015-07-10 16:15:31 -04:00
, Command.DropUnused.cmd
doh't use "def" for command definitions, it conflicts with Data.Default.def
2014-10-14 14:20:10 -04:00
, Command.AddUnused.cmd
skeleton of filter-branch command, with option parser
2021-05-14 10:59:48 -04:00
, Command.FilterBranch.cmd
implement withGlobalOptions, and convert Find
2015-07-10 12:47:35 -04:00
, Command.Find.cmd
findkeys: New command, very similar to git-annex find but operating on keys I've long been asked for `git-annex find --all` or something like that, but pushed back on it because I feel that the command is analagous to find(1) and so it would be surprising for it to list keys rather than files. So instead, add a new findkeys subcommand. Note that the use of withKeyOptions is rather strange because usually that is used to fall back to --all rather than listing files, but here it's made to default to --all like behavior and never list files. A performance thing that could be improved is that withKeyOptions always reads and caches location logs. But findkeys with no options does not need them, so it could be made faster. That caching does speed up options like --in though. This is really just a subset of a more general performance thing that --all reads location logs sometimes unncessarily. Anyway, it needs to read the location log in order to checkDead, and it seems good that findkeys does skip dead keys. Also, cleaned up comments on git-annex-find man page asking for --all option. Sponsored-by: Dartmouth College's DANDI project
2023-01-17 14:42:29 -04:00
, Command.FindKeys.cmd
convert FindRef
2015-07-10 20:38:11 -04:00
, Command.FindRef.cmd
convert Whereis
2015-07-10 16:32:33 -04:00
, Command.Whereis.cmd
whereused implemented except --historical Sponsored-by: Jack Hill on Patreon
2021-07-14 14:25:52 -04:00
, Command.WhereUsed.cmd
convert List
2015-07-10 20:46:48 -04:00
, Command.List.cmd
converted Log
2015-07-13 10:44:51 -04:00
, Command.Log.cmd
doh't use "def" for command definitions, it conflicts with Data.Default.def
2014-10-14 14:20:10 -04:00
, Command.Merge.cmd
, Command.ResolveMerge.cmd
converted Info
2015-07-11 10:41:52 -04:00
, Command.Info.cmd
converted Status
2015-07-11 10:42:46 -04:00
, Command.Status.cmd
Added inprogress command for accessing files as they are being downloaded. Chose to make this only handle files actively being downloaded, not temp files for downloads that were interrupted or files that have been fully downloaded. This commit was sponsored by Ole-Morten Duesund on Patreon.
2017-12-28 11:46:39 -04:00
, Command.Inprogress.cmd
doh't use "def" for command definitions, it conflicts with Data.Default.def
2014-10-14 14:20:10 -04:00
, Command.Migrate.cmd
, Command.Map.cmd
, Command.Direct.cmd
, Command.Indirect.cmd
, Command.Upgrade.cmd
converted Forget and TestRemote
2015-07-11 00:42:32 -04:00
, Command.Forget.cmd
implement p2p command
2016-11-30 14:35:24 -04:00
, Command.P2P.cmd
proxy: for all your direct mode repository munging needs This allows bypassing the direct mode guard in a safe way to do all sorts of things including git revert, git mv, git checkout ... This commit was sponsored by the WikiMedia Foundation.
2014-11-12 15:41:15 -04:00
, Command.Proxy.cmd
diffdriver: New git-annex command, to make git external diff drivers work with annexed files. Closes https://github.com/datalad/datalad/issues/18
2014-11-24 16:14:01 -04:00
, Command.DiffDriver.cmd
skeleton smudge/clean filters
2015-12-04 13:02:56 -04:00
, Command.Smudge.cmd
add git-annex filter-process filter-process: New command that can make git add/checkout faster when there are a lot of unlocked annexed files or non-annexed files, but that also makes git add of large annexed files slower. Use it by running: git config filter.annex.process 'git-annex filter-process' Fully tested and working, but I have not benchmarked it at all. And, incremental hashing is not done when git add uses it, so extra work is done in that case. Sponsored-by: Mark Reidenbach on Patreon
2021-11-04 15:02:36 -04:00
, Command.FilterProcess.cmd
restage: New git-annex command, handles restaging unlocked files This is much easier and less failure-prone than having the user run git update-index --refresh themselves. Sponsored-by: Dartmouth College's DANDI project
2022-09-23 16:29:28 -04:00
, Command.Restage.cmd
converted Forget and TestRemote
2015-07-11 00:42:32 -04:00
, Command.Undo.cmd
doh't use "def" for command definitions, it conflicts with Data.Default.def
2014-10-14 14:20:10 -04:00
, Command.Version.cmd
enable remotedaemon when assistant is not enabled Needed to serve tor hidden services.
2016-11-29 15:41:28 -04:00
, Command.RemoteDaemon.cmd
Version build dependency on STM, and allow building without it, which disables the watch command.
2012-06-26 09:15:47 -04:00
#ifdef WITH_ASSISTANT
converted Assistant and Watch; avoid duplicate arg parsing for no repo mode
2015-07-13 11:42:42 -04:00
, Command.Watch.cmd
, Command.Assistant.cmd
git annex webapp now opens a browser to the webapp Also, starts the assistant if it wasn't already running.
2012-07-25 23:13:01 -04:00
#ifdef WITH_WEBAPP
convert WebApp; avoid duplicate arg parsing for no repo mode
2015-07-13 11:53:42 -04:00
, Command.WebApp.cmd
add xmppgit command; roughed out xmpp push protocol and design
2012-11-06 00:52:35 -04:00
#endif
Version build dependency on STM, and allow building without it, which disables the watch command.
2012-06-26 09:15:47 -04:00
#endif
wire tasty's option parser into the main program option parser This makes bash completion work for git-annex test, and is generally cleaner.
2015-07-13 13:19:20 -04:00
, Command.Test.cmd testoptparser testrunner
doh't use "def" for command definitions, it conflicts with Data.Default.def
2014-10-14 14:20:10 -04:00
, Command.FuzzTest.cmd
converted Forget and TestRemote
2015-07-11 00:42:32 -04:00
, Command.TestRemote.cmd
new improved benchmark command that can benchmark anything git-annex does
2019-01-04 13:43:53 -04:00
, Command.Benchmark.cmd $
mkbenchmarkgenerator $ cmds testoptparser testrunner (\_ _ -> return noop)
make test suite link in git-annex's commands and run directly this way, test coverage works
2011-01-06 20:26:57 -04:00
]
remove --backend from global options --backend is no longer a global option, and is only accepted by commands that actually need it. Three commands that used to support backend but don't any longer are watch, webapp, and assistant. It would be possible to make them support it, but I doubt anyone used the option with these. And in the case of webapp and assistant, the option was handled inconsistently, only taking affect when the command is run with an existing git-annex repo, not when it creates a new one. Also, renamed GlobalOption etc to AnnexOption. Because there are many options of this type that are not actually global (any more) and get added to commands that need them. Sponsored-by: Kevin Mueller on Patreon
2022-06-29 13:28:08 -04:00
addGitAnnexCommonOptions :: Command -> Command
addGitAnnexCommonOptions c = c { cmdannexoptions = gitAnnexCommonOptions ++ cmdannexoptions c }
move global options handling closer to Command definitions
2021-02-02 15:55:45 -04:00
new improved benchmark command that can benchmark anything git-annex does
2019-01-04 13:43:53 -04:00
run :: Parser TestOptions -> TestRunner -> MkBenchmarkGenerator -> [String] -> IO ()
run testoptparser testrunner mkbenchmarkgenerator args = go envmodes
support being run by ssh as ssh-askpass replacement To use, set GIT_ANNEX_SSHASKPASS to point to a fifo or regular file (FIFO is better, avoids touching disk or multiple readers) that contains the password. Then set SSH_ASKPASS=git-annex, and when ssh runs it, it will tell ssh the password. This is not yet used..
2014-04-29 18:08:10 -04:00
where
addon commands Seems only fair, that, like git runs git-annex, git-annex runs git-annex-foo. Implementation relies on O.forwardOptions, so that any options are passed through to the addon program. Note that this includes options before the subcommand, eg: git-annex -cx=y foo Unfortunately, git-annex eats the --help/-h options. This is because it uses O.hsubparser, which injects that option into each subcommand. Seems like this should be possible to avoid somehow, to let commands display their own --help, instead of the dummy one git-annex displays. The two step searching mirrors how git works, it makes finding git-annex-foo fast when "git annex foo" is run, but will also support fuzzy matching, once findAllAddonCommands gets implemented. This commit was sponsored by Dr. Land Raider on Patreon.
2021-02-02 16:32:25 -04:00
go [] = dispatch True True args
new improved benchmark command that can benchmark anything git-annex does
2019-01-04 13:43:53 -04:00
(cmds testoptparser testrunner mkbenchmarkgenerator)
move global options handling closer to Command definitions
2021-02-02 15:55:45 -04:00
[] Git.CurrentRepo.get
let optparse-applicative handle the usage display when run w/o command or bad command Still generating the list of commands myself, to get it sorted into sections and with short synopses.
2015-07-09 11:49:52 -04:00
"git-annex"
"manage files with git, without checking their contents in"
support being run by ssh as ssh-askpass replacement To use, set GIT_ANNEX_SSHASKPASS to point to a fifo or regular file (FIFO is better, avoids touching disk or multiple readers) that contains the password. Then set SSH_ASKPASS=git-annex, and when ssh runs it, it will tell ssh the password. This is not yet used..
2014-04-29 18:08:10 -04:00
go ((v, a):rest) = maybe (go rest) a =<< getEnv v
envmodes =
propigate ssh-options everywhere ssh caching is used * sync: Use the ssh-options git config when doing git pull and push. * remotedaemon: Use the ssh-options git config. Note that the rename env var means that if a new git-annex calls an old one for git-annex ssh, or a new calls an old, nothing much will go wrong; just ssh caching won't happen.
2015-02-12 16:12:32 -04:00
[ (sshOptionsEnv, runSshOptions args)
support being run by ssh as ssh-askpass replacement To use, set GIT_ANNEX_SSHASKPASS to point to a fifo or regular file (FIFO is better, avoids touching disk or multiple readers) that contains the password. Then set SSH_ASKPASS=git-annex, and when ssh runs it, it will tell ssh the password. This is not yet used..
2014-04-29 18:08:10 -04:00
, (sshAskPassEnv, runSshAskPass)
multicast: New command, uses uftp to multicast annexed files, for eg a classroom setting. This commit was supported by the NSF-funded DataLad project.
2017-03-30 19:32:58 -04:00
, (multicastReceiveEnv, runMulticastReceive args)
support being run by ssh as ssh-askpass replacement To use, set GIT_ANNEX_SSHASKPASS to point to a fifo or regular file (FIFO is better, avoids touching disk or multiple readers) that contains the password. Then set SSH_ASKPASS=git-annex, and when ssh runs it, it will tell ssh the password. This is not yet used..
2014-04-29 18:08:10 -04:00
]
Reference in a new issue Copy permalink