Commit graph

4123 commits

Author SHA1 Message Date
Joey Hess
fdac66ae10
sanitize control characters in main thread fatal exceptions
Sponsored-by: Noam Kremen on Patreon
2023-04-12 14:21:53 -04:00
Joey Hess
c50aa21d5f
init: Avoid autoenabling special remotes that have control characters in their names
I'm on the fence about this. Notice that pulling from a git remote can
pull branches that have escape sequences in their names. Git will
display those as-is. Arguably git should try harder to avoid that.

But, names of remotes are usually up to the local user, and autoenable
changes that, and so it makes sense that git chooses to display control
characters in names of remotes, and so autoenable needs to guard against
it.

Sponsored-by: Graham Spencer on Patreon
2023-04-12 12:37:12 -04:00
Joey Hess
27915817b0
update 2023-04-11 16:31:31 -04:00
Joey Hess
8b6c7bdbcc
filter out control characters in all other Messages
This does, as a side effect, make long notes in json output not
be indented. The indentation is only needed to offset them
underneath the display of the file they apply to, so that's ok.

Sponsored-by: Brock Spratlen on Patreon
2023-04-11 12:58:01 -04:00
Joey Hess
a0e6fa18eb
eliminate showStart showStartOther
These were not handling control characters and are redundant.

Sponsored-by: Jack Hill on Patreon
2023-04-10 16:28:58 -04:00
Joey Hess
3290a09a70
filter out control characters in warning messages
Converted warning and similar to use StringContainingQuotedPath. Most
warnings are static strings, some do refer to filepaths that need to be
quoted, and others don't need quoting.

Note that, since quote filters out control characters of even
UnquotedString, this makes all warnings safe, even when an attacker
sneaks in a control character in some other way.

When json is being output, no quoting is done, since json gets its own
quoting.

This does, as a side effect, make warning messages in json output not
be indented. The indentation is only needed to offset warning messages
underneath the display of the file they apply to, so that's ok.

Sponsored-by: Brett Eisenberg on Patreon
2023-04-10 15:55:44 -04:00
Joey Hess
cd544e548b
filter out control characters in error messages
giveup changed to filter out control characters. (It is too low level to
make it use StringContainingQuotedPath.)

error still does not, but it should only be used for internal errors,
where the message is not attacker-controlled.

Changed a lot of existing error to giveup when it is not strictly an
internal error.

Of course, other exceptions can still be thrown, either by code in
git-annex, or a library, that include some attacker-controlled value.
This does not guard against those.

Sponsored-by: Noam Kremen on Patreon
2023-04-10 13:50:51 -04:00
Joey Hess
2ba1559a8e
git style quoting for ActionItemOther
Added StringContainingQuotedPath, which is used for ActionItemOther.

In the process, checked every ActionItemOther for those containing
filenames, and made them use quoting.

Sponsored-by: Graham Spencer on Patreon
2023-04-08 16:30:01 -04:00
Joey Hess
c5b017e55b
full emulation of git filename escaping
Not yet used, but the plan is to make git-annex use this when displaying
filenames similar to how git does.

Sponsored-by: Lawrence Brogan on Patreon
2023-04-07 17:17:31 -04:00
Joey Hess
218cb6ab56
expand 2023-04-07 13:13:16 -04:00
Joey Hess
c417336fbb
todo 2023-04-05 19:37:21 -04:00
Joey Hess
2b940f7725
registerurl, unregisterurl: Added --remote option
This serves two purposes. --remote=web bypasses other special remotes that
claim the url, same as addurl --raw. And, specifying some other remote
allows making sure that an url is claimed by the remote you expect,
which makes then using setpresentkey not be fragile.

Sponsored-By: the NIH-funded NICEMAN (ReproNim TR&D3) project
2023-04-05 15:54:41 -04:00
Joey Hess
c39d72ac78
comments 2023-04-05 15:00:39 -04:00
yarikoptic
e4c93a233e Added a comment 2023-04-05 01:03:38 +00:00
Joey Hess
e37b968383
comment 2023-04-04 14:09:02 -04:00
Joey Hess
bac1a990a6
comments 2023-04-04 13:52:23 -04:00
yarikoptic
d9549d4744 initial todo for adding --remote to registerurl 2023-03-31 22:15:26 +00:00
Joey Hess
18d326cb6f
external protocol VERSION 2
Support VERSION 2 in the external special remote protocol, which is
identical to VERSION 1, but avoids external remote programs neededing to
work around the above bug. External remote program that support
exporttree=yes are recommended to be updated to send VERSION 2.

Sponsored-by: Kevin Mueller on Patreon
2023-03-28 17:00:08 -04:00
Joey Hess
bb8c550d0d
comment 2023-03-28 12:14:23 -04:00
gerzoyayde@85d55f6dec266753698f42d1c8e06917ef6674a3
a91f8070e7 Added a comment: annex.bwlimit and jobs 2023-03-28 00:00:44 +00:00
Joey Hess
2b5fa091e2
annex.maxextensionlength for view
view: Support annex.maxextensionlength when generating filenames for the
view branch.

Note that refining an existing view will reuse the extension length that was
configured when initially constructing the view. This is necessarily the case
because it reuses the filenames.

Also view files used to have all extensions at the end, no matter how
many there were. Since annex.maxextensionlength's documentation includes
that it's limited to 2 extensions, I made it consistent with that.

Sponsored-by: k0ld on Patreon
2023-03-24 14:01:38 -04:00
Xyem
bf32e6137d Added a comment 2023-03-23 22:26:25 +00:00
Joey Hess
85702b2b38
comment 2023-03-23 16:45:03 -04:00
Xyem
328e9672db Added a comment 2023-03-23 17:05:24 +00:00
Joey Hess
47c010155f
todo that I decided not to do, recorded for posterity 2023-03-14 12:25:22 -04:00
Joey Hess
8c0051930e
comment 2023-03-10 11:48:37 -04:00
yarikoptic
db8d50dbeb Added a comment 2023-03-10 02:52:08 +00:00
yarikoptic
996868f552 original idea on some "native" support for ephemeral datalad clones. 2023-03-10 02:50:38 +00:00
Joey Hess
bded62378a
close 2023-03-01 12:34:40 -04:00
Joey Hess
70d89456d2
close 2023-03-01 12:22:38 -04:00
Joey Hess
9fcaf27cba
done with adjusted view branches!
Well, perhaps it could be documented better, but it's a compositional
feature so users who need it will probably try it and be happy to find
that it works.
2023-02-27 15:55:31 -04:00
Joey Hess
a5a23f0e9a
comment 2023-02-27 11:57:07 -04:00
nobodyinperson
f9b4b318c2 Added a comment 2023-02-23 22:59:49 +00:00
Joey Hess
595f5b8b1a
comment 2023-02-23 14:46:44 -04:00
Atemu
cfb396becf Added a comment 2023-02-23 08:59:28 +00:00
nobodyinperson
a268dbf401 2023-02-22 21:57:19 +00:00
Joey Hess
da61d564f1
fix view reversion caused by optimisation
view: Fix a reversion in 10.20230214 that omitted a file from a view when
the file had no metadata set, but the view only used path fields.

Sponsored-by: Jack Hill on Patreon
2023-02-16 15:18:17 -04:00
Joey Hess
1c2e5b4519
close bug as dup and update todo item with implementation plan 2023-02-16 14:51:00 -04:00
jkniiv
5565f6b61b a todo on: explicit metadata needed for view inclusion or not 2023-02-15 19:07:25 +00:00
Joey Hess
2b73662291
comment 2023-02-14 12:18:00 -04:00
nobodyinperson
2e80d964de Added a comment: 👍 This made it easier for the Thunar plugin! 2023-02-14 09:35:32 +00:00
Joey Hess
826b225ca8
Sped up view branch construction by 50%
A benchmark in my sound repository with `git-annex view feedtitle=*`
took 2:52 wall clock time before and 1:58 after. Though it still only used
130% of CPU.

This is the same kind of optimisation that is in seekFilteredKeys, though
that precaches location logs while this streams the metadata logs direct
to parsing them.

seekFilteredKeys contains more streaming, to find the annexed files, and
this could be further sped up with similar streaming.

Sponsored-by: Nicholas Golder-Manning on Patreon
2023-02-13 13:29:57 -04:00
Joey Hess
551d5abeb5
close todo and open new one 2023-02-10 15:51:38 -04:00
Joey Hess
9e59ebea52
comment 2023-02-10 14:42:14 -04:00
nobodyinperson
11b250352c Added a comment: long filenames in view 2023-02-08 22:11:30 +00:00
nobodyinperson
6e7e9813ee Added a comment: git annex sync when in view 2023-02-08 20:17:12 +00:00
Joey Hess
5f9bf51438
sync in view branch updates the view branch
* sync: When run in a view branch, refresh the view branch to reflect any
    changes that have been made to the parent branch or metadata.

This is basically working, but probably needs some more work to deal with
all the edge cases of things sync does.

Sponsored-by: Lawrence Brogan on Patreon
2023-02-08 15:37:28 -04:00
Joey Hess
dcaea65e39
comment 2023-02-08 12:41:54 -04:00
Joey Hess
21e8d201f3
link to related todo 2023-02-08 12:35:29 -04:00
Joey Hess
ece8f393f8
fix spelling 2023-02-08 12:21:53 -04:00