Merge branch 'master' of ssh://git-annex.branchable.com

This commit is contained in:
Joey Hess 2021-05-12 11:13:00 -04:00
commit 7949cfe318
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
9 changed files with 208 additions and 0 deletions

View file

@ -0,0 +1,13 @@
[[!comment format=mdwn
username="Lukey"
avatar="http://cdn.libravatar.org/avatar/c7c08e2efd29c692cc017c4a4ca3406b"
subject="comment 2"
date="2021-05-11T07:37:07Z"
content="""
This also looks suspicious:
Auto packing the repository in background for optimum performance.
See \"git help gc\" for manual housekeeping.
And after that something (git-annex?) is complaining about missing git objects. So maybe you can try to set `gc.auto` to `0` and see if that works better.
"""]]

View file

@ -0,0 +1,73 @@
[[!comment format=mdwn
username="Atemu"
avatar="http://cdn.libravatar.org/avatar/d1f0f4275931c552403f4c6707bead7a"
subject="comment 3"
date="2021-05-12T14:17:05Z"
content="""
The weird thing is, this only ever happens with my Documents git-annex repo, the only one where I use the assistant. It has never happened in 4 years of actively using regular git nor my manually committed and synced git-annex repos.
Another corruption just occured but this time on the MBP:
```
[2021-05-11 08:04:43.038047] main: starting assistant version 8.20210330
[2021-05-11 08:04:43.169107] Cronner: Consistency check in progress
[2021-05-11 08:04:43.175482] TransferScanner: Syncing with SOTERIA
(scanning...) [2021-05-11 08:04:43.340744] Watcher: Performing startup scan
(recording state in git...)
(recording state in git...)
(recovering from race...)
To ssh://192.168.101.24/~/Annex/Documents.git/
7b587c48b..4c7b284c5 git-annex -> synced/git-annex
[2021-05-11 08:04:47.087899] Cronner: Attempting to repair MB-<REDACTED> [here]
Watcher crashed: PauseWatcher
[2021-05-11 08:04:47.138205] Watcher: warning Watcher crashed: PauseWatcher
Unpacking all pack files.
fatal: early EOF
[2021-05-11 08:05:27.149344] main: starting assistant version 8.20210330
[2021-05-11 08:05:27.325057] Cronner: Consistency check in progress
(recording state in git...)
error: invalid object 100644 f30c0963ac4560878ae6803608310c289de30e26 for '319/854/SHA256E-s1400--dd4744948961710e8369365e439bb37013c5664ed6d8e58ef21a855a08ff4108.o.log'
fatal: git-write-tree: error building trees
.git-annex-wrapped: failed to read sha from git write-tree
CallStack (from HasCallStack):
error, called at ./Git/Sha.hs:23:15 in main:Git.Sha
[2021-05-11 08:05:28.719256] TransferScanner: Syncing with SOTERIA
(recording state in git...)
error: invalid object 100644 f30c0963ac4560878ae6803608310c289de30e26 for '319/854/SHA256E-s1400--dd4744948961710e8369365e439bb37013c5664ed6d8e58ef21a855a08ff4108.o.log'
fatal: git-write-tree: error building trees
Merger crashed: failed to read sha from git write-tree
CallStack (from HasCallStack):
error, called at ./Git/Sha.hs:23:15 in main:Git.Sha
[2021-05-11 08:05:28.905248] Merger: warning Merger crashed: failed to read sha from git write-tree
CallStack (from HasCallStack):
error, called at ./Git/Sha.hs:23:15 in main:Git.Sha
Auto packing the repository in background for optimum performance.
See \"git help gc\" for manual housekeeping.
error: Could not read d3720891bb000797d5fba08403540f8db208fd7a
error: Could not read 0fa98c99e87fd34202e68016c39f7ba00b7e3e1a
error: Could not read 0fa98c99e87fd34202e68016c39f7ba00b7e3e1a
error: Could not read 0fa98c99e87fd34202e68016c39f7ba00b7e3e1a
error: Could not read d3720891bb000797d5fba08403540f8db208fd7a
error: Could not read 0fa98c99e87fd34202e68016c39f7ba00b7e3e1a
error: Could not read d3720891bb000797d5fba08403540f8db208fd7a
error: Could not read 0fa98c99e87fd34202e68016c39f7ba00b7e3e1a
error: Could not read d3720891bb000797d5fba08403540f8db208fd7a
(recording state in git...)
error: invalid object 100644 f30c0963ac4560878ae6803608310c289de30e26 for '319/854/SHA256E-s1400--dd4744948961710e8369365e439bb37013c5664ed6d8e58ef21a855a08ff4108.o.log'
fatal: git-write-tree: error building trees
TransferScanner crashed: failed to read sha from git write-tree
CallStack (from HasCallStack):
error, called at ./Git/Sha.hs:23:15 in main:Git.Sha
[2021-05-11 08:05:29.430437] TransferScanner: warning TransferScanner crashed: failed to read sha from git write-tree
CallStack (from HasCallStack):
error, called at ./Git/Sha.hs:23:15 in main:Git.Sha
[2021-05-11 08:05:52.410361] main: warning git-annex has been shut down
```
This happened just after I started the assistant/webapp and you can see that I shut it down in the end.
My filesystems are ZFS on the Linux machine and APFS on the MBP. Being an Apple product, I'd expect APFS to be built to high standards (though I don't fully trust it) but ZFS corrupting anything or losing data is almost unheard of.
I'll try setting `core.fsyncObjectFiles` on this repo and update this issue accordingly when I see another corruption with it on. Shouldn't take too long as this occurs worringly frequently.
Btw, repair always fails in my setup because git seems to be unable to find the `SOTERIA` remote; is it trying to use the remote name literally as an address perhaps?
"""]]

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="Lukey"
avatar="http://cdn.libravatar.org/avatar/c7c08e2efd29c692cc017c4a4ca3406b"
subject="comment 1"
date="2021-05-11T08:41:41Z"
content="""
Do you have `receive.denyCurrentBranch` set to `warn` or `ignore` anywhere? I could imagine this may cause such problems.
"""]]

View file

@ -0,0 +1,10 @@
[[!comment format=mdwn
username="Lukey"
avatar="http://cdn.libravatar.org/avatar/c7c08e2efd29c692cc017c4a4ca3406b"
subject="comment 2"
date="2021-05-11T10:28:12Z"
content="""
Like [[bugs/git_annex_sync_remove_all_recently_added_files/]], I can also reproduce this with `receive.denyCurrentBranch` set to `warn` or `ignore` and I get similar commit patters sometimes. I guess it's very racy.
So yeah, don't set that config.
"""]]

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="Lukey"
avatar="http://cdn.libravatar.org/avatar/c7c08e2efd29c692cc017c4a4ca3406b"
subject="comment 2"
date="2021-05-11T08:50:36Z"
content="""
Similar to [[bugs/OSX\_\_58\_\_\_Pushed_changes_are_autocommited/]], I guess the `receive.denyCurrentBranch` git config could be at play here too.
"""]]

View file

@ -0,0 +1,35 @@
[[!comment format=mdwn
username="Lukey"
avatar="http://cdn.libravatar.org/avatar/c7c08e2efd29c692cc017c4a4ca3406b"
subject="comment 3"
date="2021-05-11T09:21:37Z"
content="""
And indeed, I can reproduce it like this:
mkdir test
cd test
git init a
cd a
git checkout -b master
git annex init a
touch file
git annex add .
git annex sync
git config receive.denyCurrentBranch warn
cd ..
git clone a b
cd b
git annex init b
git annex sync
touch new-file
git annex add .
git annex sync
cd ../a
git annex sync
cd ../b
git annex sync
ls
# new-file was deleted
So this is definitely a user error. But on the other hand I think that `git annex sync` maybe should only push to the `synced/` branches.
"""]]

View file

@ -0,0 +1,41 @@
### Please describe the problem.
This can be easily reproduced using the latest [[standalone|https://downloads.kitenet.net/git-annex/linux/current/git-annex-standalone-amd64.tar.gz]] build.
### What steps will reproduce the problem?
git init test
cd test
git annex init test
git annex assistant
cat .git/annex/daemon.log
### Please provide any additional information below.
[[!format sh """
$ cat .git/annex/daemon.log
[2021-05-11 12:58:04.142576914] main: starting assistant version 8.20210128-g4fc0c58e2
[2021-05-11 12:58:04.192323786] Cronner: You should enable consistency checking to protect your data.
git-annex: internal error: evacuate: strange closure type 63367248
(GHC version 8.8.4 for x86_64_unknown_linux)
Please report this as a GHC bug: https://www.haskell.org/ghc/reportabug
$ sha256sum git-annex-standalone-amd64.tar.gz
18866ab90823695a65952041ca174fe2df2d82d8dd11d972eceef192358953c3 git-annex-standalone-amd64.tar.gz
$ git annex version
git-annex version: 8.20210331-g1fb59a63a
build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV
dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.26 DAV-1.3.4 feed-1.3.0.1 ghc-8.8.4 http-client-0.6.4.1 persistent-sqlite-2.10.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.1.0
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X*
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external
operating system: linux x86_64
supported repository versions: 8
upgrade supported from repository versions: 0 1 2 3 4 5 6 7
local repository version: 8
"""]]
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)

View file

@ -0,0 +1,12 @@
[[!comment format=mdwn
username="erics"
avatar="http://cdn.libravatar.org/avatar/6e5f74c742128e5d98fd672ed6ea4865"
subject="D'oh!"
date="2021-05-10T21:26:25Z"
content="""
\"Is it in use in git's index?\"
That was it; after *git reset*, the file is now again reported as unused.
Thanks, and sorry for the noise.
"""]]

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="erics"
avatar="http://cdn.libravatar.org/avatar/6e5f74c742128e5d98fd672ed6ea4865"
subject="comment 4"
date="2021-05-10T22:06:02Z"
content="""
May I suggest a *what references this key* command? With that, I'd have been able to figure out for myself what I'd done wrong.
"""]]