Commit graph

639 commits

Author SHA1 Message Date
Joey Hess
6c2e84f6ec
Bump aws build dependency to 0.24.1
That's the version in Debian stable now. And this removes a lot of ifdefs.

Also I'm pretty sure a recent commit broke building with older versions of
aws, although that could be fixed with sufficent testing.
2025-08-13 15:32:39 -04:00
Joey Hess
3b1702e658
probe AWS datacenter
S3: When initremote is given the name of a bucket that already exists,
automatically set datacenter to the right value, rather than needing it to
be explicitly set.

This needs aws-0.23. But, initremote stores the datacenter value, so
a remote set up this way can be used with git-annex built with an older aws.

This is not done when signature=anonymous, because in that case,
using AWS.defaultRegion works fine for accessing buckets on other
datacenters.

It feels a bit round-about to need to do this probing. But without it,
the problem seems to be that, with a v4 signature, the location constraint
is included in the Authorization header. When that is the wrong location,
AWS S3 rejects it. I do wonder though if there is an easier way that I
am currently missing.

Sponsored-by: Dartmouth College's DANDI project
2025-08-13 15:23:31 -04:00
Joey Hess
215640096f
S3: Default to signature=v4 when using an AWS endpoint
* S3: Default to signature=v4 when using an AWS endpoint, since some
  AWS regions need v4 and all support it. When host= is used to specify
  a different S3 host, the default remains signature=v2.
* webapp: Support setting up S3 buckets in regions that need v4
  signatures.

For the webapp, went ahead and added all current S3 regions
(except govcloud, which  is not usable by everyone).

Sponsored-by: Dartmouth College's DANDI project
2025-08-13 13:18:35 -04:00
Joey Hess
8d78682387
layout 2025-08-01 13:45:39 -04:00
Joey Hess
3b6b3416d9
add example git-annex-p2p-unix-sockets program and end-user docs 2025-08-01 13:44:00 -04:00
Joey Hess
7ccf2ff188
comment 2025-06-23 13:34:00 -04:00
ruka
7302c5f329 Added a comment: Ignoring files on directory special remote 2025-06-15 14:41:28 +00:00
beryllium@5bc3c32eb8156390f96e363e4ba38976567425ec
81b93443bc Added a comment 2025-06-10 12:01:15 +00:00
beryllium@5bc3c32eb8156390f96e363e4ba38976567425ec
c2b079a893 Added a comment 2025-06-09 13:09:25 +00:00
Joey Hess
8607bc6903
response 2025-05-13 10:54:12 -04:00
Basile.Pinsard
7bf6e1df8a Added a comment: Setup dirhash parameter for httpalso to avoid hammering server 2025-05-02 17:53:54 +00:00
Joey Hess
89fccc15a8
page for mask remotes
documentation only so far
2025-04-09 13:53:54 -04:00
Joey Hess
218efe79e3
fix name of config 2025-04-01 09:35:54 -04:00
octvs@17a99a7aaeb0c0e0a2375e14807b138740ba34e9
75ad64b27c removed 2025-03-24 17:56:57 +00:00
octvs@17a99a7aaeb0c0e0a2375e14807b138740ba34e9
3cd3be80af Added a comment: failing with Operation not petmitted 2025-03-24 17:56:46 +00:00
octvs@17a99a7aaeb0c0e0a2375e14807b138740ba34e9
b6a0ac99df Added a comment: failing with Operation not petmitted 2025-03-24 17:56:39 +00:00
Joey Hess
e505ade963
add compute tip 2025-03-12 13:43:50 -04:00
Joey Hess
ea9a79ae81
comment 2025-03-11 12:53:32 -04:00
Joey Hess
02c894958b
Merge branch 'master' of ssh://git-annex.branchable.com 2025-03-11 12:42:10 -04:00
Joey Hess
0477a8d098
add INPUT-REQUIRED
Used by git-annex-compute-singularity to make addcomputed --fast work.

Also, simplified git-annex-compute-singularity; there is no need to hard
link the container into place. singularity does not care about the
extension of the container, so can just pass it the annex object file.
2025-03-11 11:46:31 -04:00
yarikoptic
89534f2890 Added a comment: just thinking out loud 2025-03-11 15:15:15 +00:00
yarikoptic
cb9c79c26c Added a comment 2025-03-11 15:09:20 +00:00
Joey Hess
a9df446d5d
expand 2025-03-10 17:35:34 -04:00
Joey Hess
106373c53b
response 2025-03-10 16:46:55 -04:00
Joey Hess
24b6f50b79
Merge branch 'master' of ssh://git-annex.branchable.com 2025-03-10 16:42:24 -04:00
Joey Hess
e0b7653495
added git-annex-compute-singularity
And implemented SANDBOX, which it needs.
2025-03-10 16:41:26 -04:00
Joey Hess
f59c0d1f07
make usage an error 2025-03-10 14:13:32 -04:00
yarikoptic
1e6324c179 Added a comment: Any way to annotate what are input files? 2025-03-08 14:51:20 +00:00
Joey Hess
9d6c052c27
symlink, don't hardlink
hardlink can cause problems with unlocked files
2025-03-07 17:15:54 -04:00
Joey Hess
45d7f3ca4b
disconnect stdio for wasm binaries 2025-03-07 17:15:21 -04:00
Joey Hess
18be4910d8
use pwd and quote it
Seems more portable and safe
2025-03-07 16:06:37 -04:00
Joey Hess
5ef1c44e07
case 2025-03-07 16:03:35 -04:00
Joey Hess
10e36759bf
layout 2025-03-07 16:03:09 -04:00
Joey Hess
dcd7c207a8
layout 2025-03-07 16:02:43 -04:00
Joey Hess
2391c2802a
add git-annex-compute-wasmedge 2025-03-07 16:02:11 -04:00
Joey Hess
ed51924211
redirect command stdout to stderr
Otherwise it will be interpreted as compute program protocol
2025-03-07 16:01:27 -04:00
Joey Hess
2c6dce83de
make OUTPUT subdirs
Simplifies compute programs.
2025-03-07 14:57:12 -04:00
Joey Hess
81ce4264df
compute: add response to OUTPUT
This allows rejecting output filenames that are outside the repository,
and also handles converting eg "-foo" to "./-foo" to prevent a command
that it's passed to interpreting the output filename as a dashed option.
2025-03-07 14:47:34 -04:00
Joey Hess
1f59545ad0
improve 2025-03-06 14:54:05 -04:00
Joey Hess
138421449e
add git-annex-compute-imageconvert 2025-03-06 14:47:22 -04:00
Joey Hess
52f51d065a
rename config to annex.security.allowed-compute-programs
And require for enable as well as autoenable.

It seemed asking for trouble for `git-annex enable foo` to use whatever
compute program is stored in the git config, without verifying that the
user wants that program to be used.

Note that it would be good to allow `git-annex enable foo program=...`
to be used without the program being in the git config. Not implemented yet
though.
2025-03-03 16:12:03 -04:00
Joey Hess
f32d2aecce
autoenable security for compute special remote
Added annex.security.autoenable-compute-programs and only allow
autoenabling special remotes that use compute programs on that list.

The reason this is needed is a user might have some compute programs
that are less safe to use than others. They might want to use an unsafe
one only with one repository, where they are the only committer or other
committers are trusted. They might be ok with others being used by any
repository, and if so they can add them to the list.

Another reason would be a user who has installed a compute program by
accident. Eg, it might be included with git-annex at some point, or
pulled in by some dependency. That user doesn't necessarily want that
compute program to be used in an autoenabled special remote.
2025-03-03 15:52:56 -04:00
Joey Hess
eed522a0f8
addcomputed inherits extra initremote parameters
This is limited because the remote config is a field/value map. So order
is not preserved, and when 2 parameters have the same field name, only
the last one will be passed.
2025-02-26 09:45:35 -04:00
Joey Hess
b5319ec575
documentation for compute remote and associated commands
None of this is implemented yet.
2025-02-19 14:29:18 -04:00
Joey Hess
d394f0b020
git-lfs apiurl parameter
git-lfs: Added an optional apiurl parameter.

This needs version 1.2.5 of the haskell git-lfs library to be used.
stack.yaml updated to use that.

Note that git-annex enableremote can be used to add apiurl= to an existing
git-lfs special remote. To allow unsetting the apiurl and instead use
the probed url, support enableremote with apiurl set to an empty string.

Sponsored-by: Luke T. Shumaker
2025-02-18 14:11:21 -04:00
Joey Hess
cbb6df35aa
merge in doc changes from master 2025-01-29 18:57:25 -04:00
octvs@17a99a7aaeb0c0e0a2375e14807b138740ba34e9
682fb1ada1 Added a comment: failing with Operation not petmitted 2025-01-08 08:25:04 +00:00
Joey Hess
5d2aaafa6c
git-remote-annex enableremote to support readonly webdav
* Allow enableremote of an existing webdav special remote that has
  read-only access.
* git-remote-annex: Use enableremote rather than initremote.
2025-01-07 15:57:38 -04:00
Basile.Pinsard
338281d193 Added a comment: Sync git-annex metadata subset with S3 metadata. 2024-12-18 19:35:00 +00:00
mike@2d6d71f56ce2a992244350475251df87c26fe351
d4f5956582 Added a comment: Re: corruption using git-annex-remote-rclone 2024-11-05 15:43:10 +00:00