Commit graph

4032 commits

Author SHA1 Message Date
nobodyinperson
8da345b1fb 2023-01-12 10:10:37 +00:00
Joey Hess
8a305e5fa3
respect urlinclude/urlexclude of other web special remotes
When a web special remote does not have urlinclude/urlexclude
configured, make it respect the configuration of other web special
remotes and avoid using urls that match the config of another.

Note that the other web special remote does not have to be enabled.
That seems ok, it would have been extra work to check for only ones that
are enabled.

The implementation does mean that the web special remote re-parses
its own config once at startup, as well as re-parsing the configs of any
other web special remotes. This should be a very small slowdown
unless there are lots of web special remotes.

Sponsored-by: Dartmouth College's DANDI project
2023-01-10 14:58:53 -04:00
Joey Hess
0fc476f16e
comments 2023-01-10 11:52:11 -04:00
yarikoptic
79be0eea2f Added a comment: Concern 2023-01-10 13:17:30 +00:00
nobodyinperson
4a13131ce7 Added a comment: {copy,move} with both --to and --from would be great! 2023-01-10 09:38:39 +00:00
yarikoptic
dfaccf0b21 Added a comment 2023-01-09 22:04:11 +00:00
yarikoptic
f33dda3e7a initial report on copy --from --to 2023-01-09 21:51:18 +00:00
Joey Hess
6fa166e1fc
web: Add urlinclude and urlexclude configuration settings
Sponsored-by: Dartmouth College's DANDI project
2023-01-09 17:16:53 -04:00
Joey Hess
8d06930c88
web special remote is no longer a singleton
Allow initremote of additional special remotes with type=web, in addition
to the default web special remote.

When --sameas=web is used, these provide additional names for the web
special remote, and may also have their own additional configuration
(once there is any for the web special remote) and cost.

Sponsored-by: Dartmouth College's DANDI project
2023-01-09 15:49:20 -04:00
satra
a5d3328a6f Added a comment: use list order for cost 2023-01-06 19:36:07 +00:00
yarikoptic
c7aa99808c initial todo on prioritization of URLs 2023-01-06 17:27:34 +00:00
yarikoptic
975cb5edf1 Added a comment: importtree readonly remote folder? 2023-01-06 14:21:36 +00:00
Joey Hess
4d90053e17
remove old closed bugs and todo items to speed up wiki updates and reduce size
Remove closed bugs and todos that were last edited or commented before 2022.

Except for ones tagged projects/* since projects like datalad want to keep
around records of old deleted bugs longer.

Command line used:

	for f in $(grep -l '|done\]\]' -- ./*.mdwn); do if ! grep -q "projects/" "$f"; then d="$(echo "$f" | sed 's/.mdwn$//')"; if [ -z "$(git log --since=01-01-2022 --pretty=oneline -- "$f")" -a -z "$(git log --since=01-01-2022 --pretty=oneline -- "$d")" ]; then git rm -- "./$f" ; git rm -rf "./$d"; fi; fi; done
	for f in $(grep -l '\[\[done\]\]' -- ./*.mdwn); do if ! grep -q "projects/" "$f"; then d="$(echo "$f" | sed 's/.mdwn$//')"; if [ -z "$(git log --since=01-01-2022 --pretty=oneline -- "$f")" -a -z "$(git log --since=01-01-2022 --pretty=oneline -- "$d")" ]; then git rm -- "./$f" ; git rm -rf "./$d"; fi; fi; done
2023-01-05 15:09:30 -04:00
nobodyinperson
45e71edc17 Added a comment 2023-01-02 22:58:08 +00:00
nobodyinperson
6e2f35f77d Added a comment 2023-01-02 21:58:26 +00:00
Joey Hess
adb122adcf
todo 2023-01-02 15:58:47 -04:00
Joey Hess
c326e5100f
comment 2023-01-02 15:08:24 -04:00
nobodyinperson
2c40c30dd0 Added a comment 2022-12-29 10:49:27 +00:00
Joey Hess
0b76a32629
comment 2022-12-27 17:03:47 -04:00
nobodyinperson
c14e15fd1a Added a comment: Maybe add a very slow KDF first? 2022-12-27 20:26:45 +00:00
Joey Hess
0be6cad7a8
idea 2022-12-27 13:48:53 -04:00
Joey Hess
65ddd13cf7
close 2022-12-26 13:31:09 -04:00
nobodyinperson
82ae848dd0 Added a comment 2022-12-23 10:05:27 +00:00
nobodyinperson
8ec93b6e2f Added a comment 2022-12-23 09:39:58 +00:00
Joey Hess
bda406ff90
Merge branch 'master' of ssh://git-annex.branchable.com 2022-12-22 14:34:16 -04:00
Joey Hess
2fa7656627
switch to readMaybe to handle values with leading number followed by non-number
readish ignores a trailing string after a number, but to support values
like "YYYY:MM:DD" which it makes sense to compare lexographically,
require the whole string to be parsed as a number in order to enable
numeric comparison.

Sponsored-by: Max Thoursie on Patreon
2022-12-22 14:33:47 -04:00
nobodyinperson
11873d00cd Added a comment 2022-12-22 11:39:41 +00:00
Joey Hess
eb8e0594bb
use status --ignore-submodules in configureSmudgeFilter
Speed up git-annex upgrade (from v5) and init in a repository that has
submodules. Setting the config does not affect the submodules, so avoid
the work of getting status in them, which may involve using the smudge
filter etc.

Sponsored-By: the NIH-funded NICEMAN (ReproNim TR&D3) project
2022-12-20 16:02:42 -04:00
Joey Hess
0b2dd374d8
--anything and --nothing
Added --anything (and --nothing). Eg, git-annex find --anything will list
all annexed files whether or not the content is present. This is slightly
faster and clearer than --include=* or --exclude=*

While I can't imagine how --nothing will be used, preferred content
expressions already had anything and nothing, so might as well support both
as matching options as well.

Sponsored-by: Dartmouth College's Datalad project
2022-12-20 15:44:09 -04:00
Joey Hess
5cbfb74391
belatedly added comment made somewhat earlier 2022-12-20 15:02:09 -04:00
yarikoptic
d3532ebe2b initial TODO for making it possible to use find/findref without penalty 2022-12-15 01:24:55 +00:00
yarikoptic
2e62601db3 fixing formatting 2022-12-12 23:11:21 +00:00
yarikoptic
e1133e9de8 initial report on annex upgrade too curious about submodules 2022-12-12 23:03:33 +00:00
Joey Hess
aa6919737c
--metadata lexicographical comparisons
Change --metadata comparisons < > <= and >= to fall back to lexicographical
comparisons when one or both values being compared are not numbers.

Sponsored-by: Erik Bjäreholt on Patreon
2022-12-12 13:33:24 -04:00
nobodyinperson
ea0a8f5551 2022-12-02 12:00:35 +00:00
nobodyinperson
c0df47c6ac 2022-12-02 11:57:40 +00:00
nobodyinperson
ae9ef1f8fe 2022-12-02 11:56:02 +00:00
Joey Hess
4ae920e0f2
close dup with comment 2022-11-21 15:13:21 -04:00
lena.wildervanck@4b6aac156870f72a36b090e210e4747f702b69cb
06793aee26 Add my name for better comunication 2022-11-18 22:05:07 +00:00
lena.wildervanck@4b6aac156870f72a36b090e210e4747f702b69cb
58eedf062d Add another reflink support request 2022-11-18 22:02:20 +00:00
Joey Hess
9313bf0eac
close as git-annex already makes reflinks when supported 2022-11-14 13:29:27 -04:00
Lena Wildervanck
a140e16fb6 add macos reflink to comment 2022-11-13 20:20:03 +01:00
lena.wildervanck@4b6aac156870f72a36b090e210e4747f702b69cb
896a299a34 Added a comment 2022-11-13 19:03:49 +00:00
lena.wildervanck@4b6aac156870f72a36b090e210e4747f702b69cb
092b118099 2022-11-13 15:42:14 +00:00
Joey Hess
e03cf52504
done 2022-11-04 16:20:51 -04:00
Joey Hess
3149a1e2fe
More robust handling of ErrorBusy when writing to sqlite databases
While ErrorBusy and other exceptions were caught and the write retried for
up to 10 seconds, it was still possible for git-annex to eventually
give up and error out without writing to the database. Now it will retry
as long as necessary.

This does mean that, if one git-annex process is suspended just as sqlite
has locked the database for writing, another git-annex that tries to write
it it might get stuck retrying forever. But, that could already happen when
opening the sqlite database, which retries forever on ErrorBusy. This is an
area where git-annex is known to not behave well, there's a todo about the
general case of it.

Sponsored-by: Dartmouth College's Datalad project
2022-10-17 15:56:19 -04:00
Joey Hess
d5cd1de280
update and open a todo about something I'm pondering 2022-10-12 15:53:56 -04:00
Joey Hess
b4305315b2
S3: pass fileprefix into getBucket calls
S3: Speed up importing from a large bucket when fileprefix= is set by only
asking for files under the prefix.

getBucket still returns the files with the prefix included, so the rest of
the fileprefix stripping still works unchanged.

Sponsored-by: Dartmouth College's DANDI project
2022-10-10 17:37:26 -04:00
Joey Hess
1328be2013
applied a patch 2022-09-30 14:04:10 -04:00
asakurareiko@f3d908c71c009580228b264f63f21c7274df7476
3c1230a8c0 2022-09-30 15:41:51 +00:00