Commit graph

93 commits

Author SHA1 Message Date
Joey Hess
a58c90ccf4
skeleton of filter-branch command, with option parser 2021-05-14 10:59:48 -04:00
Joey Hess
eb594c710e
unregisterurl: New command
Implemented by generalizing registerurl. Without the implicit batch mode
of registerurl since that is only a backwards compatability thing
(see commit 1d1054faa6).
2021-03-01 14:28:24 -04:00
Joey Hess
aec2cf0abe
addon commands
Seems only fair, that, like git runs git-annex, git-annex runs
git-annex-foo.

Implementation relies on O.forwardOptions, so that any options are passed
through to the addon program. Note that this includes options before the
subcommand, eg: git-annex -cx=y foo

Unfortunately, git-annex eats the --help/-h options.
This is because it uses O.hsubparser, which injects that option into each
subcommand. Seems like this should be possible to avoid somehow, to let
commands display their own --help, instead of the dummy one git-annex
displays.

The two step searching mirrors how git works, it makes finding
git-annex-foo fast when "git annex foo" is run, but will also support fuzzy
matching, once findAllAddonCommands gets implemented.

This commit was sponsored by Dr. Land Raider on Patreon.
2021-02-02 16:32:49 -04:00
Joey Hess
e78d2c9642
move global options handling closer to Command definitions 2021-02-02 15:55:45 -04:00
Joey Hess
cc89699457
mincopies
This is conceptually very simple, just making a 1 that was hard coded be
exposed as a config option. The hard part was plumbing all that, and
dealing with complexities like reading it from git attributes at the
same time that numcopies is read.

Behavior change: When numcopies is set to 0, git-annex used to drop
content without requiring any copies. Now to get that (highly unsafe)
behavior, mincopies also needs to be set to 0. It seemed better to
remove that edge case, than complicate mincopies by ignoring it when
numcopies is 0.

This commit was sponsored by Denis Dzyubenko on Patreon.
2021-01-06 14:15:19 -04:00
Joey Hess
677003a6df
rename helper
More consistent name with TransferrerPool
2020-12-09 13:24:24 -04:00
Joey Hess
05c0543e8e
move new interface to git-annex transfer
This is to avoid breakage when upgrading or downgrading git-annex with a
process running that uses the interface. It's better to keep the
compatability code for a few years than worry about such breakage.

This commit was sponsored by Brett Eisenberg on Patreon.
2020-12-09 12:33:56 -04:00
Joey Hess
c0c38e986d
added renameremote command 2019-04-15 13:49:03 -04:00
Joey Hess
40ecf58d4b
update licenses from GPL to AGPL
This does not change the overall license of the git-annex program, which
was already AGPL due to a number of sources files being AGPL already.

Legally speaking, I'm adding a new license under which these files are
now available; I already released their current contents under the GPL
license. Now they're dual licensed GPL and AGPL. However, I intend
for all my future changes to these files to only be released under the
AGPL license, and I won't be tracking the dual licensing status, so I'm
simply changing the license statement to say it's AGPL.

(In some cases, others wrote parts of the code of a file and released it
under the GPL; but in all cases I have contributed a significant portion
of the code in each file and it's that code that is getting the AGPL
license; the GPL license of other contributors allows combining with
AGPL code.)
2019-03-13 15:48:14 -04:00
Joey Hess
5ba14b5095
build cleanrly when benchmark flag is not enabled 2019-01-05 08:09:28 -04:00
Joey Hess
11d6e2e260
new improved benchmark command that can benchmark anything git-annex does 2019-01-04 13:46:36 -04:00
Joey Hess
83109affd1
remove leftovers from removed TestSuite build flag
Test suite is always built, so this can be simplified.
2018-11-19 12:39:16 -04:00
Joey Hess
67338fd7ac
Added inprogress command for accessing files as they are being downloaded.
Chose to make this only handle files actively being downloaded, not temp
files for downloads that were interrupted or files that have been fully
downloaded.

This commit was sponsored by Ole-Morten Duesund on Patreon.
2017-12-28 11:46:39 -04:00
Joey Hess
79857d7e9f
Removed the testsuite build flag
Test suite is always included.

Building with this flag disabled has actually been broken for some time,
since Command.TestRemote uses tasty. Fewer build flags are better, so good
time to drop it.

This commit was sponsored by Thomas Hochstein on Patreon.
2017-12-20 12:25:03 -04:00
Joey Hess
9f3630f4e0
initial export command
Very basic operation works, but of course this is only the beginning.

This commit was sponsored by Nick Daly on Patreon.
2017-08-29 15:10:01 -04:00
Joey Hess
c3970f6c1a
multicast: New command, uses uftp to multicast annexed files, for eg a classroom setting.
This commit was supported by the NSF-funded DataLad project.
2017-03-30 19:35:30 -04:00
Joey Hess
d074532aff
post-recive hook to make updateInstead work in direct mode and adjusted branches
* Added post-recieve hook, which makes updateInstead work with direct
  mode and adjusted branches.
* init: Set up the post-receive hook.

This commit was sponsored by Fernando Jimenez on Patreon.
2017-02-17 14:04:43 -04:00
Joey Hess
339464e847
config: New command for storing configuration in the git-annex branch.
Any config names can be set using this; git-annex commands will only look
at specific ones that make sense and are worth the overhead of querying the
branch.

This might also be useful for storing whatever other config-type stuff the
user might want to shove into the git-annex branch.

This commit was sponsored by Jochen Bartl on Patreon.
2017-01-30 16:46:38 -04:00
Joey Hess
ab66bbfeb6
Merge branch 'master' into no-xmpp 2016-12-24 15:01:55 -04:00
Joey Hess
bfc8305814
implement p2p command 2016-11-30 14:35:24 -04:00
Joey Hess
87b325de71
enable remotedaemon when assistant is not enabled
Needed to serve tor hidden services.
2016-11-29 15:41:28 -04:00
Joey Hess
d58148031b
remove xmpp support
I've long considered the XMPP support in git-annex a wart.
It's nice to remove it.

(This also removes the NetMessager, which was only used for XMPP, and the
daemonstatus's desynced list (likewise).)

Existing XMPP remotes should be ignored by git-annex.

This commit was sponsored by Brock Spratlen on Patreon.
2016-11-14 14:53:08 -04:00
Joey Hess
07ad19f421
git-annex enable-tor command
Tor unfortunately does not come out of the box configured to let hidden
services register themselves on the fly via the ControlPort.

And, changing the config to enable the ControlPort and a particular type
of auth for it may break something already using the ControlPort, or
lessen the security of the system.

So, this leaves only one option to us: Add a hidden service to the
torrc. git-annex enable-tor does so, and picks an unused high port for
tor to listen on for connections to the hidden service.

It's up to the caller to somehow pick a local port to listen on
that won't be used by something else. That may be difficult to do..

This commit was sponsored by Jochen Bartl on Patreon.
2016-11-14 13:48:35 -04:00
Joey Hess
8b36d54319
Remove the EKG build flag, since Gentoo for some reason decided to enable this flag, depsite it not being intended for production use and so enabled by default. 2016-07-06 15:09:56 -04:00
Joey Hess
1bfea9b3e5
calckey: New plumbing command, calculates the key that would be used to refer to a file 2016-04-20 13:50:26 -04:00
Joey Hess
0a1b02ce04
adjusted branches, proof of concept
"git annex adjust" may be a temporary interface, but works for a proof of
concept.

It is pretty fast at creating the adjusted branch. The main overhead is
injecting pointer files. It might be worth optimising that by reusing the
symlink target as the pointer file content. When I tried to do that,
the problem was that the clean filter doesn't use that same format, and so
git thought files had changed. Could be dealt with, perhaps make the clean
filter use symlink format for pointer files when on an adjusted branch?

But the real overhead is in checking out the branch, when git runs the
smudge filter once per file. That is perhaps too slow to be usable,
although it may only affect initial checkout of the branch, and not
updates. TBD.
2016-02-25 16:23:24 -04:00
Joey Hess
f051b51645
remove 3 build flags
* Removed the webapp-secure build flag, rolling it into the webapp build
  flag.
* Removed the quvi and tahoe build flags, which only adds aeson to
  the core dependencies.
* Removed the feed build flag, which only adds feed to the core
  dependencies.

Build flags have cost in both code complexity and also make Setup configure
have to work harder to find a usable set of build flags when some
dependencies are missing.
2016-01-26 08:14:57 -04:00
Joey Hess
d3ba9fe5c8
matchexpression: New plumbing command to check if a preferred content expression matches some data. 2016-01-25 16:16:18 -04:00
Joey Hess
f9c5aa84e0
add database benchmark
The benchmark shows that the database access is quite fast indeed!
And, it scales linearly to the number of keys, with one exception,
getAssociatedKey.

Based on this benchmark, I don't think I need worry about optimising
for cases where all files are locked and the database is mostly empty.
In those cases, database access will be misses, and according to this
benchmark, should add only 50 milliseconds to runtime.

(NB: There may be some overhead to getting the database opened and locking
the handle that this benchmark doesn't see.)

joey@darkstar:~/src/git-annex>./git-annex benchmark
setting up database with 1000
setting up database with 10000
benchmarking keys database/getAssociatedFiles from 1000 (hit)
time                 62.77 μs   (62.70 μs .. 62.85 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 62.81 μs   (62.76 μs .. 62.88 μs)
std dev              201.6 ns   (157.5 ns .. 259.5 ns)

benchmarking keys database/getAssociatedFiles from 1000 (miss)
time                 50.02 μs   (49.97 μs .. 50.07 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 50.09 μs   (50.04 μs .. 50.17 μs)
std dev              206.7 ns   (133.8 ns .. 295.3 ns)

benchmarking keys database/getAssociatedKey from 1000 (hit)
time                 211.2 μs   (210.5 μs .. 212.3 μs)
                     1.000 R²   (0.999 R² .. 1.000 R²)
mean                 211.0 μs   (210.7 μs .. 212.0 μs)
std dev              1.685 μs   (334.4 ns .. 3.517 μs)

benchmarking keys database/getAssociatedKey from 1000 (miss)
time                 173.5 μs   (172.7 μs .. 174.2 μs)
                     1.000 R²   (0.999 R² .. 1.000 R²)
mean                 173.7 μs   (173.0 μs .. 175.5 μs)
std dev              3.833 μs   (1.858 μs .. 6.617 μs)
variance introduced by outliers: 16% (moderately inflated)

benchmarking keys database/getAssociatedFiles from 10000 (hit)
time                 64.01 μs   (63.84 μs .. 64.18 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 64.85 μs   (64.34 μs .. 66.02 μs)
std dev              2.433 μs   (547.6 ns .. 4.652 μs)
variance introduced by outliers: 40% (moderately inflated)

benchmarking keys database/getAssociatedFiles from 10000 (miss)
time                 50.33 μs   (50.28 μs .. 50.39 μs)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 50.32 μs   (50.26 μs .. 50.38 μs)
std dev              202.7 ns   (167.6 ns .. 252.0 ns)

benchmarking keys database/getAssociatedKey from 10000 (hit)
time                 1.142 ms   (1.139 ms .. 1.146 ms)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 1.142 ms   (1.140 ms .. 1.144 ms)
std dev              7.142 μs   (4.994 μs .. 10.98 μs)

benchmarking keys database/getAssociatedKey from 10000 (miss)
time                 1.094 ms   (1.092 ms .. 1.096 ms)
                     1.000 R²   (1.000 R² .. 1.000 R²)
mean                 1.095 ms   (1.095 ms .. 1.097 ms)
std dev              4.277 μs   (2.591 μs .. 7.228 μs)
2016-01-12 13:07:03 -04:00
Joey Hess
723e4e31a1
merge clean into smudge command
The git filter config can be used to map the single git-annex command to
the 2 actions, and this avoids "git annex clean" being used for this thing,
it might have a better use for that name later.
2015-12-04 15:32:47 -04:00
Joey Hess
20ca89dfa3
skeleton smudge/clean filters 2015-12-04 13:03:39 -04:00
Joey Hess
730cc3feb5 wire tasty's option parser into the main program option parser
This makes bash completion work for git-annex test, and is
generally cleaner.
2015-07-13 13:20:10 -04:00
Joey Hess
fd086c5752 convert WebApp; avoid duplicate arg parsing for no repo mode 2015-07-13 11:53:42 -04:00
Joey Hess
b95a48fe45 converted Assistant and Watch; avoid duplicate arg parsing for no repo mode 2015-07-13 11:42:42 -04:00
Joey Hess
084f8d9ac7 convert Import 2015-07-13 11:15:21 -04:00
Joey Hess
9c1a42873c converted ImportFeed 2015-07-13 11:06:41 -04:00
Joey Hess
aff4b9a3dc converted addurl 2015-07-13 10:57:49 -04:00
Joey Hess
3639b4f8ea converted Log 2015-07-13 10:44:51 -04:00
Joey Hess
3d8a8e9792 convert Dead, and allow multiple --key options 2015-07-13 10:26:54 -04:00
Joey Hess
adec382bc2 converted MetaData, eliminating a global value from Annex state .. beautiful 2015-07-12 09:25:43 -04:00
Joey Hess
fdcb54d4f2 converted ContentLocation, ExampleKey, LookupKey 2015-07-11 20:43:45 -04:00
Joey Hess
c6375a9158 converted Status 2015-07-11 10:42:46 -04:00
Joey Hess
215f636bb3 converted Info 2015-07-11 10:41:52 -04:00
Joey Hess
9ad20c2869 converted Forget and TestRemote 2015-07-11 00:42:32 -04:00
Joey Hess
c70c841d30 converted Mirror 2015-07-10 21:05:34 -04:00
Joey Hess
e4b3701dfe convert List 2015-07-10 20:46:48 -04:00
Joey Hess
a0b2fcc663 converted ViCfg (trivially) 2015-07-10 20:40:30 -04:00
Joey Hess
02f6d9b33a convert FindRef 2015-07-10 20:38:11 -04:00
Joey Hess
c94e219fea convert Whereis 2015-07-10 16:32:33 -04:00
Joey Hess
8eb0a440b9 convert Expire 2015-07-10 16:26:23 -04:00