Merge branch 'master' of ssh://git-annex.branchable.com

This commit is contained in:
Joey Hess 2025-01-06 16:42:12 -04:00
commit 3b280faac7
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
13 changed files with 410 additions and 0 deletions

View file

@ -0,0 +1,9 @@
[[!comment format=mdwn
username="brendan.ward@a2e11ad27f6b2fa2c556aea6811496e0d95dd0da"
nickname="brendan.ward"
avatar="http://cdn.libravatar.org/avatar/bc7899f1aa523365cdfadc2ba3f7913d"
subject="comment 1"
date="2025-01-04T04:34:02Z"
content="""
The same bug report was made about three months ago, as seen [here](https://git-annex.branchable.com/bugs/Android_Installation_Error/).
"""]]

View file

@ -0,0 +1,9 @@
[[!comment format=mdwn
username="brendan.ward@a2e11ad27f6b2fa2c556aea6811496e0d95dd0da"
nickname="brendan.ward"
avatar="http://cdn.libravatar.org/avatar/bc7899f1aa523365cdfadc2ba3f7913d"
subject="comment 2"
date="2025-01-04T04:35:23Z"
content="""
The same bug report was made about three months ago, as seen [here](https://git-annex.branchable.com/bugs/Android_Installation_Error/).
"""]]

View file

@ -0,0 +1,9 @@
[[!comment format=mdwn
username="brendan.ward@a2e11ad27f6b2fa2c556aea6811496e0d95dd0da"
nickname="brendan.ward"
avatar="http://cdn.libravatar.org/avatar/bc7899f1aa523365cdfadc2ba3f7913d"
subject="comment 3"
date="2025-01-04T04:39:20Z"
content="""
The same bug report was made about three months ago, as seen [here](../Android_Installation_Error).
"""]]

View file

@ -0,0 +1,9 @@
[[!comment format=mdwn
username="brendan.ward@a2e11ad27f6b2fa2c556aea6811496e0d95dd0da"
nickname="brendan.ward"
avatar="http://cdn.libravatar.org/avatar/bc7899f1aa523365cdfadc2ba3f7913d"
subject="comment 4"
date="2025-01-04T04:41:14Z"
content="""
The same bug report was made about three months ago, as seen here: [[Android Installation Error]].
"""]]

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="Brendan"
avatar="http://cdn.libravatar.org/avatar/bc7899f1aa523365cdfadc2ba3f7913d"
subject="Same bug"
date="2025-01-04T04:46:11Z"
content="""
The same bug was posted about 3 months ago in this report: [[Android Installation Error]].
"""]]

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="Brendan"
avatar="http://cdn.libravatar.org/avatar/bc7899f1aa523365cdfadc2ba3f7913d"
subject="Same bug"
date="2025-01-04T04:46:29Z"
content="""
The same bug was posted about 3 months ago.
"""]]

View file

@ -0,0 +1,77 @@
[[!comment format=mdwn
username="jkniiv"
avatar="http://cdn.libravatar.org/avatar/05fd8b33af7183342153e8013aa3713d"
subject="comment 2"
date="2025-01-06T08:52:04Z"
content="""
> AFAIK, git-remote-annex is not installed on windows. I assume you set up the link to git-annex yourself.
That's right. Knowing that some of the functionality of git-remote-annex was already present I made
a symlink in a directory in my `$env:PATH` with `cmd /c mklink git-remote-annex ..\path-to\git-annex.exe`
(I have developer mode active in Windows settings) and that made git-remote-annex available to me/git-annex.
> This is a puzzling problem to me. I don't know anything about windows readonly attributes. But I don't think git-annex would ever set them.
>
>Indeed, it never freezes content on windows at all. That can be seen in the debug output you posted, where it does say it's \"thawing content\", but never \"freezing content\".
That is puzzling. Could the thawing do the wrong/opposite thing in some cases? Also, remember
that anything that does `chmod u-w` by way of Git Bash's own shell (sh.exe/bash.exe), e.g. via `sh -c 'chmod u-w file'`, will effect
a Windows readonly attribute on that file (thanks to MSYS2 behind the scenes):
```
e:\git-annex-tests\test-git-remote-annex\annex-c [adjusted/master(unlocked)]> touch ddd
e:\git-annex-tests\test-git-remote-annex\annex-c [adjusted/master(unlocked) +1 ~0 -0 !]> ls
Directory: e:\git-annex-tests\test-git-remote-annex\annex-c
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a--- 1.6.2024 23:28 5 a-1
-a--- 1.6.2024 23:28 5 b-2
-a--- 1.6.2024 23:28 7 c-3
-a--- 6.1.2025 10:31 0 ddd
e:\git-annex-tests\test-git-remote-annex\annex-c [adjusted/master(unlocked) +1 ~0 -0 !]> sh -c 'chmod u-w ddd'
e:\git-annex-tests\test-git-remote-annex\annex-c [adjusted/master(unlocked) +1 ~0 -0 !]> ls
Directory: e:\git-annex-tests\test-git-remote-annex\annex-c
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a--- 1.6.2024 23:28 5 a-1
-a--- 1.6.2024 23:28 5 b-2
-a--- 1.6.2024 23:28 7 c-3
-ar-- 6.1.2025 10:31 0 ddd
e:\git-annex-tests\test-git-remote-annex\annex-c [adjusted/master(unlocked) +1 ~0 -0 !]> rm ddd
Remove-Item: You do not have sufficient access rights to perform this operation or the item is hidden, system, or read only.
e:\git-annex-tests\test-git-remote-annex\annex-c [adjusted/master(unlocked) +1 ~0 -0 !]> sh -c 'type chmod'
chmod is /usr/bin/chmod
e:\git-annex-tests\test-git-remote-annex\annex-c [adjusted/master(unlocked) +1 ~0 -0 !]> sh -c 'type mount'
mount is /usr/bin/mount
e:\git-annex-tests\test-git-remote-annex\annex-c [adjusted/master(unlocked) +1 ~0 -0 !]> sh -c mount
C:/scoop/apps/git/2.47.1 on / type ntfs (binary,noacl,auto)
C:/scoop/apps/git/2.47.1/usr/bin on /bin type ntfs (binary,noacl,auto)
C:/Users/jkniiv/AppData/Local/Temp on /tmp type ntfs (binary,noacl,posix=0,usertemp)
C: on /c type ntfs (binary,noacl,posix=0,user,noumount,auto)
D: on /d type ntfs (binary,noacl,posix=0,user,noumount,auto)
E: on /e type ntfs (binary,noacl,posix=0,user,noumount,auto)
```
But I guess git-annex isn't calling out to external `chmod` in these cases.
>
>If you had somehow configured a freeze hook that set the readonly attribute, it would run it on windows. I suppose you would have thought to mention if that was the case though.
Nope. No freeze/thaw hooks set. I'm not that brave nor cognizant of how to script them in a fool proof manner. :)
```
e:\git-annex-tests\test-git-remote-annex\annex-c [adjusted/master(unlocked)]> git config get annex.freezecontent-command
e:\git-annex-tests\test-git-remote-annex\annex-c [adjusted/master(unlocked)]> git config get annex.thawcontent-command
```
>
>Also rather puzzling is that this is a temp object file, and not a .git/annex/objects/ file. So the failure is apparently happening in the middle of downloading the GITBUNDLE object, before it gets moved to that location. But the same code is run at that point as by any download of any git-annex object.
"""]]

View file

@ -0,0 +1,78 @@
[[!comment format=mdwn
username="jkniiv"
avatar="http://cdn.libravatar.org/avatar/05fd8b33af7183342153e8013aa3713d"
subject="comment 3"
date="2025-01-06T08:54:05Z"
content="""
> AFAIK, git-remote-annex is not installed on windows. I assume you set up the link to git-annex yourself.
That's right. Knowing that some of the functionality of git-remote-annex was already present I made
a symlink in a directory in my `$env:PATH` with `cmd /c mklink git-remote-annex ..\path-to\git-annex.exe`
(I have developer mode active in Windows settings) and that made git-remote-annex available to me/git-annex.
> This is a puzzling problem to me. I don't know anything about windows readonly attributes. But I don't think git-annex would ever set them.
>
>Indeed, it never freezes content on windows at all. That can be seen in the debug output you posted, where it does say it's \"thawing content\", but never \"freezing content\".
That is puzzling. Could the thawing do the wrong/opposite thing in some cases? Also, remember
that anything that does `chmod u-w` by way of Git Bash's own shell (sh.exe/bash.exe), e.g. via `sh -c 'chmod u-w file'`, will effect
a Windows readonly attribute on that file (thanks to MSYS2 behind the scenes):
```
e:\git-annex-tests\test-git-remote-annex\annex-c [adjusted/master(unlocked)]> touch ddd
e:\git-annex-tests\test-git-remote-annex\annex-c [adjusted/master(unlocked) +1 ~0 -0 !]> ls
Directory: e:\git-annex-tests\test-git-remote-annex\annex-c
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a--- 1.6.2024 23:28 5 a-1
-a--- 1.6.2024 23:28 5 b-2
-a--- 1.6.2024 23:28 7 c-3
-a--- 6.1.2025 10:31 0 ddd
e:\git-annex-tests\test-git-remote-annex\annex-c [adjusted/master(unlocked) +1 ~0 -0 !]> sh -c 'chmod u-w ddd'
e:\git-annex-tests\test-git-remote-annex\annex-c [adjusted/master(unlocked) +1 ~0 -0 !]> ls
Directory: e:\git-annex-tests\test-git-remote-annex\annex-c
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a--- 1.6.2024 23:28 5 a-1
-a--- 1.6.2024 23:28 5 b-2
-a--- 1.6.2024 23:28 7 c-3
-ar-- 6.1.2025 10:31 0 ddd
e:\git-annex-tests\test-git-remote-annex\annex-c [adjusted/master(unlocked) +1 ~0 -0 !]> rm ddd
Remove-Item: You do not have sufficient access rights to perform this operation or the item is hidden, system, or read only.
e:\git-annex-tests\test-git-remote-annex\annex-c [adjusted/master(unlocked) +1 ~0 -0 !]> sh -c 'type chmod'
chmod is /usr/bin/chmod
e:\git-annex-tests\test-git-remote-annex\annex-c [adjusted/master(unlocked) +1 ~0 -0 !]> sh -c 'type mount'
mount is /usr/bin/mount
e:\git-annex-tests\test-git-remote-annex\annex-c [adjusted/master(unlocked) +1 ~0 -0 !]> sh -c mount
C:/scoop/apps/git/2.47.1 on / type ntfs (binary,noacl,auto)
C:/scoop/apps/git/2.47.1/usr/bin on /bin type ntfs (binary,noacl,auto)
C:/Users/jkniiv/AppData/Local/Temp on /tmp type ntfs (binary,noacl,posix=0,usertemp)
C: on /c type ntfs (binary,noacl,posix=0,user,noumount,auto)
D: on /d type ntfs (binary,noacl,posix=0,user,noumount,auto)
E: on /e type ntfs (binary,noacl,posix=0,user,noumount,auto)
```
But I guess git-annex isn't calling out to external `chmod` in these cases.
>
>If you had somehow configured a freeze hook that set the readonly attribute, it would run it on windows. I suppose you would have thought to mention if that was the case though.
Nope. No freeze/thaw hooks set. I'm not that brave nor cognizant of how to script them in a fool proof manner. :)
```
e:\git-annex-tests\test-git-remote-annex\annex-c [adjusted/master(unlocked)]> git config get annex.freezecontent-command
e:\git-annex-tests\test-git-remote-annex\annex-c [adjusted/master(unlocked)]> git config get annex.thawcontent-command
```
>
>Also rather puzzling is that this is a temp object file, and not a .git/annex/objects/ file. So the failure is apparently happening in the middle of downloading the GITBUNDLE object, before it gets moved to that location. But the same code is run at that point as by any download of any git-annex object.
"""]]

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="Doable8234"
avatar="http://cdn.libravatar.org/avatar/b0d5fea745f92c3b8cc8ecc3dafa6278"
subject="comment 1"
date="2025-01-06T05:00:47Z"
content="""
I did a simple search and it looks like it's already been asked and phrased much better [here](https://git-annex.branchable.com/todo/git-annex-cat/). It's in the todo forum and tagged with `needsthought`. I'm going to add a comment with some of my thoughts and hopefully have a more fruitful discussion there.
"""]]

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="Doable8234"
avatar="http://cdn.libravatar.org/avatar/b0d5fea745f92c3b8cc8ecc3dafa6278"
subject="comment 2"
date="2025-01-06T05:02:21Z"
content="""
Looking back, it looks like this has already been asked and phrased much better [here](https://git-annex.branchable.com/todo/git-annex-cat/). It's the TODO section and marked as `needsthought`. I'm going to add a comment to that instead of here and hopefully have a more fruitful discussion.
"""]]

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="Doable8234"
avatar="http://cdn.libravatar.org/avatar/b0d5fea745f92c3b8cc8ecc3dafa6278"
subject="comment 3"
date="2025-01-06T05:03:22Z"
content="""
Looking back, it looks like this has already been asked and phrased much better [here](https://git-annex.branchable.com/todo/git-annex-cat/). It's the TODO section and marked as `needsthought`. I'm going to add a comment to that instead of here and hopefully have a more fruitful discussion.
"""]]

View file

@ -0,0 +1,119 @@
After spending some time in troubleshooting the situation that `git-annex` for some reason claims that `origin` remote does not support git-annex, I have ran into [11 year old discussion](https://git-annex.branchable.com/forum/ssh__95__exchange__95__identification__58___read__58___Connection_reset_by_peer/#comment-1776b4d3b175c77cdb44fc8656e3352c) which boils down to
> It would be a good idea to `git config annex.sshcaching` false in repositories stored on encfs.
As AFAIK encfs is generally working ok (seems to test ok with bleeding edge [ref](https://github.com/datalad/git-annex/issues/216)), and overall if there is something wrong, it better be fixed and not "avoided", I think that git-annex should either inform user about necessity to not do internal caching within the same repository (e.g. to have it configured to be shared somewhere under ~/.cache/` instead or a like) if file system does not support the ssh caching, or even somehow automagically handle the case, but it should not mislead in its reporting stating that remote does not support git-annex whenever it does.
With this minimalistic reproducer where origin on a regular filesystem and a clone is under `encfs`, we receive a misleading error:
<details>
<summary>reproducer</summary>
```
#!/bin/bash
export PS4='> '
set -x
set -eu
cd "$(mktemp -d ${TMPDIR:-/tmp}/dl-XXXXXXX)"
topd=$PWD
mkdir -p denc{r,}; encfs --standard --extpass="echo 123" $PWD/denc{r,}
cleanup () {
# to possible kill/troubleshoot
if ! fusermount -u $topd; then
lsof $topd/denc
echo "ERROR: failed to unmount"
exit 1
fi
}
trap cleanup SIGINT SIGHUP SIGABRT EXIT
mkdir d
(cd d; git init; git annex init; echo 1 > 1.dat; git annex add 1.dat; git commit -m 1 1.dat)
echo "cloning locally and ssh works ok, testing"
git clone localhost:$topd/d d-clone
git -C d-clone annex wanted origin
echo "but would not be happy under encfs with misleading error:"
cd denc
git clone localhost:$topd/d d-clone
git -C d-clone annex wanted origin
```
</details>
```
> git -C d-clone annex wanted origin
Unable to parse git config from origin
Remote origin does not have git-annex installed; setting annex-ignore
This could be a problem with the git-annex installation on the remote. Please make sure that git-annex-shell is available in PATH when you ssh into the remote. Once you have fixed the git-annex installation, run: git annex enableremote origin
git-annex: cannot determine uuid for origin (perhaps you need to run "git annex sync"?)
```
which is simply due to the fact that git-annex does not only unable to parse, it is unable to connect. But if so, IMHO ideally it should avoid claiming anything about git annex installation there.
After spending some time in troubleshooting the situation that `git-annex` for some reason claims that `origin` remote does not support git-annex, I have ran into [11 year old discussion](https://git-annex.branchable.com/forum/ssh__95__exchange__95__identification__58___read__58___Connection_reset_by_peer/#comment-1776b4d3b175c77cdb44fc8656e3352c) which boils down to
> It would be a good idea to `git config annex.sshcaching` false in repositories stored on encfs.
As AFAIK encfs is generally working ok (seems to test ok with bleeding edge [ref](https://github.com/datalad/git-annex/issues/216)), and overall if there is something wrong, it better be fixed and not "avoided", I think that git-annex should either inform user about necessity to not do internal caching within the same repository (e.g. to have it configured to be shared somewhere under ~/.cache/` instead or a like) if file system does not support the ssh caching, or even somehow automagically handle the case, but it should not mislead in its reporting stating that remote does not support git-annex whenever it does.
With this minimalistic reproducer where origin on a regular filesystem and a clone is under `encfs`, we receive a misleading error:
<details>
<summary>reproducer</summary>
```
#!/bin/bash
export PS4='> '
set -x
set -eu
cd "$(mktemp -d ${TMPDIR:-/tmp}/dl-XXXXXXX)"
topd=$PWD
mkdir -p denc{r,}; encfs --standard --extpass="echo 123" $PWD/denc{r,}
cleanup () {
# to possible kill/troubleshoot
if ! fusermount -u $topd; then
lsof $topd/denc
echo "ERROR: failed to unmount"
exit 1
fi
}
trap cleanup SIGINT SIGHUP SIGABRT EXIT
mkdir d
(cd d; git init; git annex init; echo 1 > 1.dat; git annex add 1.dat; git commit -m 1 1.dat)
echo "cloning locally and ssh works ok, testing"
git clone localhost:$topd/d d-clone
git -C d-clone annex wanted origin
echo "but would not be happy under encfs with misleading error:"
cd denc
git clone localhost:$topd/d d-clone
git -C d-clone annex wanted origin
```
</details>
```
> git -C d-clone annex wanted origin
Unable to parse git config from origin
Remote origin does not have git-annex installed; setting annex-ignore
This could be a problem with the git-annex installation on the remote. Please make sure that git-annex-shell is available in PATH when you ssh into the remote. Once you have fixed the git-annex installation, run: git annex enableremote origin
git-annex: cannot determine uuid for origin (perhaps you need to run "git annex sync"?)
```
which is simply due to the fact that git-annex does not only unable to parse, it is unable to connect. But if so, IMHO ideally it should avoid claiming anything about git annex installation there.

View file

@ -0,0 +1,60 @@
reposting since initial post seems has failed with odd https://www.oneukrainian.com/tmp/annex-blew-1.png .
After spending some time in troubleshooting the situation that `git-annex` for some reason claims that `origin` remote does not support git-annex, I have ran into [11 year old discussion](https://git-annex.branchable.com/forum/ssh__95__exchange__95__identification__58___read__58___Connection_reset_by_peer/#comment-1776b4d3b175c77cdb44fc8656e3352c) which boils down to
> It would be a good idea to `git config annex.sshcaching` false in repositories stored on encfs.
As AFAIK encfs is generally working ok (seems to test ok with bleeding edge [ref](https://github.com/datalad/git-annex/issues/216)), and overall if there is something wrong, it better be fixed and not "avoided", I think that git-annex should either inform user about necessity to not do internal caching within the same repository (e.g. to have it configured to be shared somewhere under ~/.cache/` instead or a like) if file system does not support the ssh caching, or even somehow automagically handle the case, but it should not mislead in its reporting stating that remote does not support git-annex whenever it does.
With this minimalistic reproducer where origin on a regular filesystem and a clone is under `encfs`, we receive a misleading error:
<details>
<summary>reproducer</summary>
```
#!/bin/bash
export PS4='> '
set -x
set -eu
cd "$(mktemp -d ${TMPDIR:-/tmp}/dl-XXXXXXX)"
topd=$PWD
mkdir -p denc{r,}; encfs --standard --extpass="echo 123" $PWD/denc{r,}
cleanup () {
# to possible kill/troubleshoot
if ! fusermount -u $topd; then
lsof $topd/denc
echo "ERROR: failed to unmount"
exit 1
fi
}
trap cleanup SIGINT SIGHUP SIGABRT EXIT
mkdir d
(cd d; git init; git annex init; echo 1 > 1.dat; git annex add 1.dat; git commit -m 1 1.dat)
echo "cloning locally and ssh works ok, testing"
git clone localhost:$topd/d d-clone
git -C d-clone annex wanted origin
echo "but would not be happy under encfs with misleading error:"
cd denc
git clone localhost:$topd/d d-clone
git -C d-clone annex wanted origin
```
</details>
```
> git -C d-clone annex wanted origin
Unable to parse git config from origin
Remote origin does not have git-annex installed; setting annex-ignore
This could be a problem with the git-annex installation on the remote. Please make sure that git-annex-shell is available in PATH when you ssh into the remote. Once you have fixed the git-annex installation, run: git annex enableremote origin
git-annex: cannot determine uuid for origin (perhaps you need to run "git annex sync"?)
```
which is simply due to the fact that git-annex does not only unable to parse, it is unable to connect. But if so, IMHO ideally it should avoid claiming anything about git annex installation there.