Merge branch 'master' into httpproto

doc/todo/Peer_to_peer_connection_purely_over_magic-wormhole.mdwn

@@ -0,0 +1,7 @@
+Currently peer to peer communication seems to be possible only over tor (requiring root privileges to setup). It would be great to have an alternative connection method that can easily be used as an unprivileged user as well.
+
+Magic-wormhole has an experimental feature called "dilation" (<https://magic-wormhole.readthedocs.io/en/latest/api.html#dilation>) which can be used to open a direct bidirectional TCP connection between two systems only using the usual magic-wormhole codes (which can be generated once and re-used, so essentially like a pre-shared key stored on each side).
+
+There is a project called fowl (<https://github.com/meejah/fowl>) that uses this feature to port-forward over such a tunnel, which could be used for this purpose or serve as a reference for how to use this feature in git-annex. This implementation has some issues, but I think the approach has potential.
+
+It would be great if `git annex remotedaemon` (I suppose? I am not too well-versed on the internals) could optionally be configured to establish such a tunnel to remotes and use it for communication. Or maybe this is already possible to implement from outside of git-annex and I just need a hint on how to do that?

doc/todo/Peer_to_peer_connection_purely_over_magic-wormhole/comment_1_c1e4564192cb1887ff17443a327556d7._comment

@@ -0,0 +1,8 @@
+[[!comment format=mdwn
+ username="nobodyinperson"
+ avatar="http://cdn.libravatar.org/avatar/736a41cd4988ede057bae805d000f4f5"
+ subject="Also Serveo.net"
+ date="2024-07-19T15:21:18Z"
+ content="""
+Don't know your use case excatly, but just today I stumbled upon [Serveo.net](https://serveo.net), which facilitates no-login, no-install reverse SSH tunnels, with with you can effectively break out of NAT'ed networks without root privileges. Together with `autossh` for persistence, maybe that can help you? Although I guess for forwarding to a privileged port 22 you would still need root privileges, meh...
+"""]]

doc/todo/Peer_to_peer_connection_purely_over_magic-wormhole/comment_2_a457473353e2b8acf2afa86cec6ef9be._comment

@@ -0,0 +1,17 @@
+[[!comment format=mdwn
+ username="m.risse@77eac2c22d673d5f10305c0bade738ad74055f92"
+ nickname="m.risse"
+ avatar="http://cdn.libravatar.org/avatar/59541f50d845e5f81aff06e88a38b9de"
+ subject="comment 2"
+ date="2024-07-21T12:38:12Z"
+ content="""
+Forwarding to port 22 shouldn't require root, e.g. the mentioned fowl tunnel could also be used to tunnel a local port to a remote ssh server on port 22. You just cannot listen on a local privileged port, but that shouldn't be a problem.
+
+There are a bunch of \"tunnelers\" like serveo, e.g. ngrok and zrok, but the disadvantage of that is that it still requires a running ssh server.
+
+My imagined use-case would be something like two phones or laptops behind NAT without tor or a ssh daemon. I think with magic-wormhole's dilation feature it would be possible to make it so that you could run `git annex remotedaemon` or `git annex assistant` on one or both devices (after pairing) and have them communicate without any further setup required.
+
+Since magic-wormhole is already used for pairing it wouldn't even be a new dependency.
+
+Maybe this is already implementable from outside git-annex as a custom git-remote though, I'd have to take a look at what git-remote-tor-annex is really doing...
+"""]]

doc/todo/Read-only_support_for_webdav.mdwn

@@ -0,0 +1,7 @@
+This is in response to https://git-annex.branchable.com/special_remotes/webdav/#comment-cd53cf0276427cc924ae66553985ec5c where `httpalso` is recommended as an approach to get read-only access to a `webdav` remote.
+
+A use case not possible with that approach is *authenticated* read-only access. According to http://git-annex.branchable.com/special_remotes/httpalso/#comment-4f41f401d4b0133d2ef12912b9217e44 this is not supported right now, but could be added.
+
+Weighing the two approaches (read-only `webdav` vs authenticated `httpalso`), it appears that only the read-only `webdav` is compatible with [git-remote-annex](https://git-annex.branchable.com/git-remote-annex/), because a user would need to declare *one* special remote (configuration) to use for cloning.
+
+It would be great to have authenticated, read-only access to webdav shares. Thanks in advance for considering!

doc/todo/use_copy__95__file__95__range_for_get_and_copy.mdwn

@@ -0,0 +1,16 @@
+I was looking into why `annex get` and `annex copy` between local clones on NFS mounts don't utilize NFS4.2's server-side copy feature,
+which would be pretty relevant for a setup like ours (institutional compute cluster; big datalad datasets).
+This seems to boil down to not calling `copy_file_range`. However, `cp` generally does call `copy_file_range`, so that seemed confusing.
+Turns out the culprit is `--reflink=always` which does not work as expected on ZFS. `--reflink=auto` does, though.
+A summary of how that is, can be found here: https://github.com/openzfs/zfs/pull/13392#issuecomment-1742172842
+
+I am not sure why annex insists on `always` rather than `auto`, so not sure whether the solution actually would be to change that.
+Reading old issues it seems the reason was to let annex handle the fallback, which kinda is the problem in case of ZFS.
+Changing (back?) to `auto` may be an issue in other cases - I don't know. If annex' fallback when `cp --reflink=always` fails would end up calling `copy_file_range`, that would still solve the issue, though, as NFS would then end up performing a server-side copy rather than transferring big files back and forth.
+
+Just to be clear: It's specifically ZFS via NFS on linux that's the issue here.
+
+
+P.S.: Didn't want to call this a bug, mostly b/c the "real bug" isn't in annex exactly (see link above), so putting it here.
+
+[[!meta author=ben]]