mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

Merge branch 'master' of ssh://git-annex.branchable.com

Browse source
This commit is contained in:
Joey Hess 2022-05-31 12:53:39 -04:00
commit a343cdc266
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
7 changed files with 315 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

23
doc/bugs/FreeBSD_patches.mdwn Normal file
View file

@ -0,0 +1,23 @@
diff --git a/git-annex.cabal b/git-annex.cabal
index 0a50878eb..63d79bde9 100644
--- a/git-annex.cabal
+++ b/git-annex.cabal
@@ -396,8 +396,7 @@ Executable git-annex
GHC-Options: -O0
-- Avoid linking with unused dynamic libaries.
- -- (Only tested on Linux).
- if os(Linux)
+ if os(Linux) || os(freebsd)
GHC-Options: -optl-Wl,--as-needed
if (os(windows))
@@ -517,7 +516,7 @@ Executable git-annex
Other-Modules: Utility.DirWatcher.Kqueue
if flag(Dbus)
- if (os(linux))
+ if os(linux) || os(freebsd)
Build-Depends: dbus (>= 0.10.7), fdo-notify (>= 0.3)
CPP-Options: -DWITH_DBUS -DWITH_DESKTOP_NOTIFY -DWITH_DBUS_NOTIFICATIONS
Other-Modules: Utility.DBus

33
doc/bugs/FreeBSD_patches/comment_1_3473c3d4207a8387d962366dd1aa88b1._comment Normal file
View file

@ -0,0 +1,33 @@
[[!comment format=mdwn
username="7thyearold@339ab97e59cee5c85e14df94a5059f251a143e65"
nickname="7thyearold"
avatar="http://cdn.libravatar.org/avatar/bec2e77ce4d1f264936115aa20c6b194"
subject="comment 1"
date="2022-05-30T09:52:11Z"
content="""
```diff
diff --git a/git-annex.cabal b/git-annex.cabal
index 0a50878eb..63d79bde9 100644
--- a/git-annex.cabal
+++ b/git-annex.cabal
@@ -396,8 +396,7 @@ Executable git-annex
GHC-Options: -O0
-- Avoid linking with unused dynamic libaries.
- -- (Only tested on Linux).
- if os(Linux)
+ if os(Linux) || os(freebsd)
GHC-Options: -optl-Wl,--as-needed
if (os(windows))
@@ -517,7 +516,7 @@ Executable git-annex
Other-Modules: Utility.DirWatcher.Kqueue
if flag(Dbus)
- if (os(linux))
+ if os(linux) || os(freebsd)
Build-Depends: dbus (>= 0.10.7), fdo-notify (>= 0.3)
CPP-Options: -DWITH_DBUS -DWITH_DESKTOP_NOTIFY -DWITH_DBUS_NOTIFICATIONS
Other-Modules: Utility.DBus
```
"""]]

99
doc/bugs/error__58___git-annex_filter-process_died_of_signal_15.mdwn Normal file
View file

@ -0,0 +1,99 @@
### Please describe the problem.
When doing
git checkout 3b18836
on a very large repository, where tens of thousands of files will be changed, I get many
error: external filter 'git-annex filter-process' failed
error: git-annex filter-process died of signal 15
and finally
git-annex: git: createProcess: runInteractiveProcess: pipe: resource exhausted (Too many open files)
fatal: the remote end hung up unexpectedly
### What steps will reproduce the problem?
git checkout
on a really big commit.
### What version of git-annex are you using? On what operating system?
The system and all the SW should be up to date.
% git --version
git version 2.34.1
% git annex version
git-annex version: 10.20220322-1~ndall+1
build flags: Assistant Webapp Pairing Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV
dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.26 DAV-1.3.4 feed-1.3.0.1 ghc-8.8.4 http-client-0.6.4.1 persistent-sqlite-2.10.6.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.1.0
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X*
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external
operating system: linux x86_64
supported repository versions: 8 9 10
upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10
local repository version: 9
% uname -a
Linux 5.15.0-33-generic #34-Ubuntu SMP Wed May 18 13:34:26 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
% cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy
The computer has 128GB of main memory, and so is not running out of that.
This is a large repository:
du -sh . p
43G .
(62)% du -sh .git
42G .git
(63)% du -sh .git/annex
19G .git/annex
% git gc
Enumerating objects: 375565, done.
Counting objects: 100% (375565/375565), done.
Delta compression using up to 12 threads
Compressing objects: 100% (153675/153675), done.
Writing objects: 100% (375565/375565), done.
Total 375565 (delta 215701), reused 364628 (delta 207259), pack-reused 0
73.38s real 97.69s user 4.93s system 139% 0,0 socket 13753 mem git gc
I'm trying to recover from a mess caused by creating a new remote repository, making it shallow to save space, and updating it by mounting it on a local system with sshfs, etc. That caused an unintended rebase messing things up, e.g., git log shows most of the commits vanished (merged?). So I was trying to go back to an earlier, good, commit. However that requires changing, basically, everything. And that seems to be too big a change for git annex to process.
FWIW, I'm not using lfs.
### Please provide any additional information below.
[[!format sh """
# If you can, paste a complete transcript of the problem occurring here.
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
# End of transcript or log.
"""]]
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
Otherwise, it's great.

15
doc/bugs/get_is_busy_doing_nothing/comment_5_28da709b2152b1ce1d0c926b099febad._comment Normal file
View file

@ -0,0 +1,15 @@
[[!comment format=mdwn
username="yarikoptic"
avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
subject="What filesystem is this happening on?"
date="2022-05-25T21:09:38Z"
content="""
BTRFS
```
/dev/mapper/vg_backup-lv_backup on /mnt/backup type btrfs (rw,noatime,compress=lzo,space_cache=v2,subvolid=5,subvol=/,x-systemd.mount-timeout=30m)
```
over LVM over mdadm raid6 arrays ;)
"""]]

97
doc/bugs/get_is_busy_doing_nothing/comment_6_c9594a39c44f7fd6649c418dbb5d2c59._comment Normal file
View file

@ -0,0 +1,97 @@
[[!comment format=mdwn
username="yarikoptic"
avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
subject="another/different? stall"
date="2022-05-26T14:31:33Z"
content="""
next one stalled at the end at
```
get sub-dt15/sub-dt15_ses-dt15_6705um_200213_091923_ecephys.nwb (from web...) ok
get sub-bruce/sub-bruce_ses-bruce_0um_210115_155615_ecephys.nwb (from web...) ok
get sub-bruce/sub-bruce_ses-bruce_0um_210110_171041_ecephys.nwb (from web...) ok
get sub-ros/sub-ros_ses-ros_210320_091833_ecephys.nwb (from web...) ok
(recording state in git...)
```
with recent `gets` not complaining about being unable to lock, only some longer before did, e.g.
```
get sub-CGM49/sub-CGM49_ses-CGM49_512um_210227_093748_ecephys.nwb (from web...)
failed to commit changes to sqlite database: Just SQLite3 returned ErrorBusy while attempting to perform step: database is locked
CallStack (from HasCallStack):
error, called at ./Database/Handle.hs:91:26 in main:Database.Handle
(Delaying 1s before retrying....)
44% 9.58 GiB 57 MiB/s 3m36s
get sub-CGM52/sub-CGM52_ses-CGM52_0um_210216_144320_ecephys.nwb (from web...)
```
```
dandi 2013903 0.0 0.0 5492 0 pts/1 S+ May25 0:00 flock -E 0 -e -n /home/dandi/.run/backup2datalad-cron.lock bash -c /mnt/backup/dandi/dandisets/tools/backups2datalad-cron
dandi 2013904 0.0 0.0 7024 1892 pts/1 S+ May25 0:00 /bin/bash /mnt/backup/dandi/dandisets/tools/backups2datalad-cron
dandi 2013933 0.0 0.0 7024 1520 pts/1 S+ May25 0:00 /bin/bash /mnt/backup/dandi/dandisets/tools/backups2datalad-cron
dandi 2113421 0.0 0.0 321448 3140 pts/1 S+ May25 0:04 python -m tools.backups2datalad -l DEBUG -J 5 --target /mnt/backup/dandi/dandisets populate dandi-dandisets-dropbox
dandi 2151114 0.8 0.0 1075306800 32416 pts/1 Sl+ May25 9:39 git-annex get -c annex.retry=3 --jobs 5 --from=web --not --in dandi-dandisets-dropbox --and --not --in here
dandi 2151142 11.6 0.0 1074200808 37352 pts/1 Sl May25 138:50 /home/dandi/miniconda3/envs/dandisets/bin/git-annex transferrer -c annex.retry=3
dandi 2151170 0.0 0.0 10640 1260 pts/1 S May25 0:00 git --git-dir=.git --work-tree=. --literal-pathspecs -c annex.retry=3 cat-file --batch
dandi 2151143 12.9 0.0 1074200808 36384 pts/1 Sl May25 153:23 /home/dandi/miniconda3/envs/dandisets/bin/git-annex transferrer -c annex.retry=3
dandi 2151158 0.0 0.0 10640 1256 pts/1 S May25 0:00 git --git-dir=.git --work-tree=. --literal-pathspecs -c annex.retry=3 cat-file --batch
dandi 2151147 12.7 0.0 1074127044 35308 pts/1 Sl May25 150:49 /home/dandi/miniconda3/envs/dandisets/bin/git-annex transferrer -c annex.retry=3
dandi 2151160 0.0 0.0 10640 1232 pts/1 S May25 0:00 git --git-dir=.git --work-tree=. --literal-pathspecs -c annex.retry=3 cat-file --batch
dandi 2151148 16.3 0.0 1074200776 35596 pts/1 Sl May25 193:42 /home/dandi/miniconda3/envs/dandisets/bin/git-annex transferrer -c annex.retry=3
dandi 2151168 0.0 0.0 10640 1336 pts/1 S May25 0:00 git --git-dir=.git --work-tree=. --literal-pathspecs -c annex.retry=3 cat-file --batch
dandi 2151161 13.0 0.0 1074200808 32060 pts/1 Sl May25 155:15 /home/dandi/miniconda3/envs/dandisets/bin/git-annex transferrer -c annex.retry=3
dandi 2151177 0.0 0.0 10640 1096 pts/1 S May25 0:00 git --git-dir=.git --work-tree=. --literal-pathspecs -c annex.retry=3 cat-file --batch
```
strace end is
```
[pid 2151125] sched_yield( <unfinished ...>
[pid 2151129] <... sched_yield resumed>) = 0
[pid 2151125] <... sched_yield resumed>) = 0
[pid 2151129] sched_yield() = 0
[pid 2151125] sched_yield( <unfinished ...>
[pid 2151129] sched_yield( <unfinished ...>
[pid 2151125] <... sched_yield resumed>) = 0
[pid 2151129] <... sched_yield resumed>) = 0
[pid 2151137] rt_sigprocmask(SIG_SETMASK, [], <unfinished ...>
[pid 2151126] sched_yield( <unfinished ...>
[pid 2151137] <... rt_sigprocmask resumed>NULL, 8) = 0
[pid 2151129] sched_yield( <unfinished ...>
[pid 2151125] sched_yield( <unfinished ...>
[pid 2151137] futex(0x7fe75c000be8, FUTEX_WAIT_PRIVATE, 0, NULL <unfinished ...>
[pid 2151129] <... sched_yield resumed>) = 0
[pid 2151125] <... sched_yield resumed>) = 0
[pid 2151129] futex(0x7fe740000be8, FUTEX_WAIT_PRIVATE, 0, NULL <unfinished ...>
[pid 2151125] futex(0x52eae6c, FUTEX_WAIT_PRIVATE, 0, NULL <unfinished ...>
[pid 2151126] <... sched_yield resumed>) = 0
[pid 2151126] futex(0x52eb068, FUTEX_WAIT_PRIVATE, 0, NULL <unfinished ...>
[pid 2151115] <... read resumed>\"\1\0\0\0\0\0\0\0\", 8) = 8
[pid 2151115] futex(0x4a4844c, FUTEX_WAIT_PRIVATE, 0, NULL^Cstrace: Process 2151114 detached
strace: Process 2151115 detached
<detached ...>
strace: Process 2151116 detached
strace: Process 2151117 detached
strace: Process 2151119 detached
strace: Process 2151124 detached
strace: Process 2151125 detached
strace: Process 2151126 detached
strace: Process 2151127 detached
strace: Process 2151128 detached
strace: Process 2151129 detached
strace: Process 2151132 detached
strace: Process 2151133 detached
strace: Process 2151136 detached
strace: Process 2151137 detached
strace: Process 2151138 detached
strace: Process 2151139 detached
strace: Process 2151140 detached
strace: Process 2677004 detached
strace: Process 2925046 detached
strace: Process 3026979 detached
strace: Process 3027102 detached
```
I have now killed that `get` process and restarted our backup beastie (should now get to `copy` those `get`ed files into backup)
"""]]

5
doc/forum/exporttree__61__yes_always_untrusted__63__.mdwn Normal file
View file

@ -0,0 +1,5 @@
The update for version 10.20220525 says:
>Special remotes with importtree=yes or exporttree=yes are once again treated as untrusted, since files stored in them can be deleted or modified at any time.
Even in an S3 special remote with versioning enabled?

43
doc/tips/android_sync_with_adb/comment_8_036ff21f5eee0c165a41e472ffdfa341._comment Normal file
View file

@ -0,0 +1,43 @@
[[!comment format=mdwn
username="g@aaed65f19d6c3a2a18c33da828e66c7bb915e65a"
nickname="g"
avatar="http://cdn.libravatar.org/avatar/10470d6f8a18833e04dee17126d53372"
subject="adb sync problem (after some upgrades) "
date="2022-05-29T09:59:45Z"
content="""
Hello,
some time ago (it was Jan 2022) I set up an adb remote like explained here and I was able to sync multiple times with my phone.
After some time with no syncs, today I'm trying to sync again but I get this error:
```
[...]
list android_samsung_S4_01 find: bad arg '-printf'
stat: Needs 1 argument (see \"stat --help\")
git-annex: Unable to list contents of android_samsung_S4_01: adb find failed
failed
Merge made by the 'ort' strategy.
(recording state in git...)
sync: 1 failed
```
My doubt is that something changed when I upgraded git-annex, this is my currently installed version:
```
git-annex version: 10.20220504
build flags: Assistant Pairing Inotify TorrentParser MagicMime Feeds Testsuite S3 WebDAV
dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.29 DAV-1.3.4 feed-1.3.2.0 ghc-8.10.7 http-client-0.7.11 persistent-sqlite-2.13.0.3 torrent-10000.1.1 uuid-1.3.15
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X*
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external
operating system: linux x86_64
supported repository versions: 8 9 10
upgrade supported from repository versions: 0 1 2 3 4 5 6 7 8 9 10
local repository version: 8
```
Please do you have any hint on what I should do to make this running again?
Thanks! Giovanni
"""]]