Commit graph

46967 commits

Author SHA1 Message Date
Joey Hess
9db30d60e1
improve error when unable to reverse adjust merge
Give the user a hint about what to do, as well as what they should have
done to avoid the problem.
2025-08-20 13:35:31 -04:00
Joey Hess
fd89e611b2
error out when another branch has been manually merged into the adjusted branch
This avoids losing the merge commit when re-running git-annex adjust in the
adjusted branch.

It also makes git-annex sync error out, rather than displaying a warning
and exiting successfully.

Sponsored-by: Leon Schuermann on Patreon
2025-08-20 13:28:03 -04:00
Joey Hess
16463f8f02
Merge branch 'master' of ssh://git-annex.branchable.com 2025-08-20 12:31:35 -04:00
jcjgraf
9d1f47a45b Added a comment: Reproduce Issue 2025-08-18 19:02:35 +00:00
stv0g
5d7a7e2562 Added a comment: encryption=credsonly 2025-08-18 17:07:40 +00:00
Joey Hess
8991c75d04
bug 2025-08-15 18:11:19 -04:00
Joey Hess
ce8b32de07
update 2025-08-15 11:54:42 -04:00
Joey Hess
0b388c0924
warn and refuse to autoenable a special remote when name is in use
Improve behavior when there are special remotes configured with
autoenable=yes with names that conflict with other remotes.

The use of remoteList' is to avoid using the cached remote list in the case
where there are two special remotes both configured to autoenable and both
with the same name. Once the 1st is autoenabled, this makes reload the
remote list and so see the 1st, and so refuse to autoenable the second.

This adds a little bit of overhead, but it should be sufficiently small not
to need optimising.

Sponsored-by: Dartmouth College's OpenNeuro project
2025-08-14 11:05:42 -04:00
Joey Hess
5323e0c20b
followup 2025-08-14 10:15:29 -04:00
Joey Hess
b5808c6413
close 2025-08-14 09:56:15 -04:00
Joey Hess
6b77c28186
simplify 2025-08-14 09:22:44 -04:00
Joey Hess
0924a45cc4
info: Added --show option
To pick which parts of the info to calculate and display.

Sponsored-by: Dartmouth College's DANDI project
2025-08-13 16:49:21 -04:00
Joey Hess
bcc86c92e2
comment 2025-08-13 16:04:46 -04:00
Joey Hess
8c51fc5fdf
Merge branch 'master' of ssh://git-annex.branchable.com 2025-08-13 15:33:42 -04:00
Joey Hess
7e3a176f0c
close 2025-08-13 15:33:39 -04:00
Joey Hess
6c2e84f6ec
Bump aws build dependency to 0.24.1
That's the version in Debian stable now. And this removes a lot of ifdefs.

Also I'm pretty sure a recent commit broke building with older versions of
aws, although that could be fixed with sufficent testing.
2025-08-13 15:32:39 -04:00
Joey Hess
3b1702e658
probe AWS datacenter
S3: When initremote is given the name of a bucket that already exists,
automatically set datacenter to the right value, rather than needing it to
be explicitly set.

This needs aws-0.23. But, initremote stores the datacenter value, so
a remote set up this way can be used with git-annex built with an older aws.

This is not done when signature=anonymous, because in that case,
using AWS.defaultRegion works fine for accessing buckets on other
datacenters.

It feels a bit round-about to need to do this probing. But without it,
the problem seems to be that, with a v4 signature, the location constraint
is included in the Authorization header. When that is the wrong location,
AWS S3 rejects it. I do wonder though if there is an easier way that I
am currently missing.

Sponsored-by: Dartmouth College's DANDI project
2025-08-13 15:23:31 -04:00
Joey Hess
fed5e00a18
fix default region reversion
Commit 215640096f caused the default
region for S3 to change to us-east-2. This was due to regionInfo having
an undocumented property that the first item in the list is for the
default region.

Avoid relying on regionInfo for defaultRegion.

Sponsored-by: Dartmouth College's DANDI project
2025-08-13 14:19:36 -04:00
yarikoptic
ee7e1450b1 wishlist of faster/specific info for a remote 2025-08-13 17:49:20 +00:00
Joey Hess
215640096f
S3: Default to signature=v4 when using an AWS endpoint
* S3: Default to signature=v4 when using an AWS endpoint, since some
  AWS regions need v4 and all support it. When host= is used to specify
  a different S3 host, the default remains signature=v2.
* webapp: Support setting up S3 buckets in regions that need v4
  signatures.

For the webapp, went ahead and added all current S3 regions
(except govcloud, which  is not usable by everyone).

Sponsored-by: Dartmouth College's DANDI project
2025-08-13 13:18:35 -04:00
Joey Hess
edc1d92059
document "anonymous" in ValueDesc 2025-08-13 12:51:27 -04:00
Joey Hess
ebf0bfe6d8
comment 2025-08-13 12:50:05 -04:00
Joey Hess
0ffd07165f
Merge branch 'master' of ssh://git-annex.branchable.com 2025-08-12 14:26:29 -04:00
Michael
330c0af0de 2025-08-11 03:37:34 +00:00
Joey Hess
0bc278c16d
add https://pypi.org/project/git-annex-remote-internxt/ 2025-08-08 16:10:47 -04:00
Joey Hess
11daee00ae
Merge branch 'master' of ssh://git-annex.branchable.com 2025-08-08 16:10:02 -04:00
evgeni
b286b4f365 Added a comment: repodata is empty again :( 2025-08-08 12:49:42 +00:00
Joey Hess
020b47c98a
stack.yaml: Update to lts-24.2. 2025-08-04 14:28:47 -04:00
nobodyinperson
f40b4026be Add YouTube link 2025-08-04 08:22:12 +00:00
branchable.git-annex@cd7b2a7420674f7ca1d0ff0a450d466c47cf6684
dab5c679b7 Updated the S3-special-remote example and prose based on recent B2 experience 2025-08-03 11:27:56 +00:00
Joey Hess
786fa037c8
comment 2025-08-01 13:57:50 -04:00
Joey Hess
655750cffb
reword 2025-08-01 13:55:38 -04:00
Joey Hess
3b2f09d6a2
link to new page 2025-08-01 13:52:29 -04:00
Joey Hess
a690c02361
add section on security 2025-08-01 13:51:25 -04:00
Joey Hess
398664d160
inline didn't work due to extension 2025-08-01 13:46:53 -04:00
Joey Hess
8d78682387
layout 2025-08-01 13:45:39 -04:00
Joey Hess
3b6b3416d9
add example git-annex-p2p-unix-sockets program and end-user docs 2025-08-01 13:44:00 -04:00
Joey Hess
c15dad6040
improve AuthToken display in P2P --debug
Using an empty string to obscure the AuthToken made it impossible to
tell if one was really being presented or not.
2025-08-01 12:58:03 -04:00
Joey Hess
05e945fdf9
Merge branch 'master' into genericp2p 2025-08-01 12:10:33 -04:00
Joey Hess
4958d74163
Merge branch 'master' of ssh://git-annex.branchable.com 2025-07-31 15:21:22 -04:00
Joey Hess
59bcfc838e
update design doc with changes from genericp2p branch
That branch is basically ready to merge, but needs more testing in a
chicken and egg situation.
2025-07-31 15:20:28 -04:00
yarikoptic
6578d323a1 Added a comment 2025-07-31 19:07:57 +00:00
Joey Hess
7e0802e951
avoid broken pipe zombies
hClose crashes if the pipe is broken, preventing waiting for the process
2025-07-31 14:57:51 -04:00
Joey Hess
f1781d01d8
remotedaemon support for generic P2P transports
RemoteDaemon.Transport.Tor was refactored into this, and most of the
code is reused between them.

getSocketFile does not yet deal with repositories on crippled
filesystems that don't support sockets. Annex.Ssh detects that and
allows the user to set an environment variable, and something similar
could be done here.

And it does not deal with a situation where there is no path to the
socket file that is not too long. In that situation it would crash out
I suppose. Probably though, remotedaemon is ran from the top of the
repo, and in that case the path is just ".git/annex/p2p/<md5>" so nice
and short.

This seems to mostly work. But I don't yet have a working git-annex-p2p-
command to test it with.

And with my not quite working git-annex-p2p-foo test script, running
remotedaemon results in an ever-growing number of zombie processes
that it's not waiting on.
2025-07-31 14:45:32 -04:00
Joey Hess
7643c716cd
changed design for p2p generic socket
Having the git-annex-p2p-<netname> command output the socket filename
left git-annex scrambling to listen to it in order to not miss incoming
connections. And if the command uses something like socat UNIX-CONNECT,
that expects the socket to be accepting connections and errors out when
it's not, that would be a problem.

Rather than complicating the protocol with git-annex needing to send
back a message when it's listening to the socket, simplified it by
having git-annex provide the socket path to the command.

This does mean that, if a P2P network has its own place it expects to
find a socket file, the git-annex-p2p-<netname> command would need to
somehow arrange for it to use the git-annex socket path. A symlink would
be one way to handle that situation.
2025-07-31 13:18:30 -04:00
Yann Büchau
4c652de8ee
Add yann's TdF talk about life in git annex 2025-07-31 18:25:31 +02:00
Joey Hess
86e9e88530
add git-remote-p2p-annex
Added git-remote-p2p-annex, which allows git pull and push to P2P networks
provided by external commands.

This is a refactor of git-remote-tor-annex, and should just work. Except
possibly for quirks with the address parsing. I've checked that the address
parsing basically works.

One thing I don't understand is why git-remote-tor-annex removes "/*" from
the end of the address. The git history does not provide any hints. So I
didn't make git-remote-p2p-annex do the same. Maybe that is needed in some
situation? But, a P2P address could contain "/", so removing it would be a
problem. I can't see anything in gitremote-helpers(7) about why the url
might get such a thing added to the end of it. My guess is that is not
needed for tor either (but does no harm there since onion addresses never
contain "/").

At this point, the implementation of generic P2P transports needs only
remotedaemon support.
2025-07-30 15:25:56 -04:00
Joey Hess
7403aeb95f
use Annex.ExternalAddonProcess for P2P.Generic processes
These are another sort of external addon process, and this makes several
things work including shell scripts on windows. And it makes for nicer
error messages when the command is not in the path.

Note that the refactored startExternalAddonProcess used by this
does not use propGitEnv to set git environment variables in the
environment. Unlike startExternalAddonProcessProtocol which does.
This is because it runs in IO and does not have access to that
information. But also, I don't think that P2P.Generic processes need
that.
2025-07-30 14:46:37 -04:00
Joey Hess
d3fbda13e4
p2p --enable
p2p: Added --enable option, which can be used to enable P2P networks
provided by external commands git-annex-p2p-<netname>

Made git-annex p2p --enable tor behave the same as git-annex enable-tor,
to make tor a bit less of a special case. However, it canot be run as root,
since it cannot take the user id parameter.
2025-07-30 14:08:59 -04:00
Joey Hess
4fb9b7cb67
support P2PAnnex in connectPeer
This is probably enough to support accessing remotes using p2p-annex:: urls.
Not tested yet of course since there is not yet support for serving the
other side of such a connection, or for setting up such a connection.

P2P.Generic has an implementation of the whole interface to the
git-annex-p2p-<netname> commands.
2025-07-30 13:23:23 -04:00