Merge branch 'master' of ssh://git-annex.branchable.com
This commit is contained in:
Joey Hess
commit
d99fead752
4 changed files with 123 additions and 0 deletions
60
doc/bugs/add_config_var_preventing_adjusted_branch_mode.mdwn
Normal file
60
doc/bugs/add_config_var_preventing_adjusted_branch_mode.mdwn
Normal file
|
|
@ -0,0 +1,60 @@
|
|||
### Please describe the problem.
|
||||
|
||||
Somewhat too late for our current usecase since older git-annex would not know about it, but I think could be generalized into adding a configuration variable right away for **any** automated migration. E.g. there is no variable to prevent autoupgrades of the repos (e.g. from v5 to the next one etc), but AFAIK there is none for automated conversion" into `adjusted/master(unlocked)` mode.
|
||||
|
||||
Rationale: With thaw/freeze commands we now can use git-annex in indirect (default) mode on our HPC. But that requires a recent version of git-annex. User might have some other (older) version of git-annex available system-wide by default, and if the user forgets to switch to new version of git-annex before using it, it might trigger git-annex to realize that it operates on crippled FS, and since not knowing about thaw/freeze -- it just migrates repository to adjusted, which is very undesired.
|
||||
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
|
||||
here is a demo of older git-annex going back to adjusted branch mode... yet to discover how else we could have migrated without directly invoking `git annex init`:
|
||||
|
||||
```
|
||||
[d31548v@discovery7 d31548v]$ mkdir repo
|
||||
[d31548v@discovery7 d31548v]$ cd repo
|
||||
[d31548v@discovery7 repo]$ git init
|
||||
Initialized empty Git repository in /dartfs/rc/lab/D/DBIC/DBIC/d31548v/repo/.git/
|
||||
[d31548v@discovery7 repo]$ git config --add annex.thawcontent-command "$HOME/bin-annex/thaw-content %path"
|
||||
[d31548v@discovery7 repo]$ git config --add annex.freezecontent-command "$HOME/bin-annex/freeze-content %path"
|
||||
[d31548v@discovery7 repo]$ git annex init
|
||||
init ok
|
||||
(recording state in git...)
|
||||
[d31548v@discovery7 repo]$ echo 123 > 123
|
||||
[d31548v@discovery7 repo]$ git annex add 123
|
||||
add 123
|
||||
ok
|
||||
(recording state in git...)
|
||||
git comm[d31548v@discovery7 repo]$ git commit -m 'added 123 in indirect mode' 123
|
||||
[master (root-commit) 3ceb200] added 123 in indirect mode
|
||||
1 file changed, 1 insertion(+)
|
||||
create mode 120000 123
|
||||
[d31548v@discovery7 repo]$ ls -ld 123
|
||||
lrwxr-x--- 1 d31548v rc-DBIC 178 May 6 11:19 123 -> .git/annex/objects/G6/qW/SHA256E-s4--181210f8f9c779c26da1d9b2075bde0127302ee0e3fca38c9a83f5b1dd8e5d3b/SHA256E-s4--181210f8f9c779c26da1d9b2075bde0127302ee0e3fca38c9a83f5b1dd8e5d3b
|
||||
[d31548v@discovery7 repo]$ ls -l .git/annex/objects
|
||||
total 3
|
||||
drwxr-x--- 3 d31548v rc-DBIC 20 May 6 11:19 G6
|
||||
[d31548v@discovery7 repo]$ export PATH=/opt/bin:$PATH
|
||||
[d31548v@discovery7 repo]$ git annex version | head -n 1
|
||||
git-annex version: 8.20200502-g55acb2e52
|
||||
[d31548v@discovery7 repo]$ git annex drop 123
|
||||
drop 123
|
||||
git-annex: failed to lock content: .git/annex/objects/G6/qW/SHA256E-s4--181210f8f9c779c26da1d9b2075bde0127302ee0e3fca38c9a83f5b1dd8e5d3b/SHA256E-s4--181210f8f9c779c26da1d9b2075bde0127302ee0e3fca38c9a83f5b1dd8e5d3b: openFd: permission denied (Permission denied)
|
||||
failed
|
||||
git-annex: drop: 1 failed
|
||||
[d31548v@discovery7 repo]$ git annex init
|
||||
init
|
||||
Filesystem allows writing to files whose write bit is not set.
|
||||
|
||||
Detected a crippled filesystem.
|
||||
|
||||
Disabling core.symlinks.
|
||||
(scanning for unlocked files...)
|
||||
|
||||
Entering an adjusted branch where files are unlocked as this filesystem does not support locked files.
|
||||
|
||||
Switched to branch 'adjusted/master(unlocked)'
|
||||
ok
|
||||
```
|
||||
|
||||
[[!meta author=yoh]]
|
||||
[[!tag projects/datalad]]
|
||||
|
|
@ -0,0 +1,20 @@
|
|||
[[!comment format=mdwn
|
||||
username="yarikoptic"
|
||||
avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
|
||||
subject="it is still there"
|
||||
date="2022-05-05T20:22:13Z"
|
||||
content="""
|
||||
each month is a submodule, so the most recent one seems to be from 2 weeks ago
|
||||
|
||||
```
|
||||
(git)smaug:/mnt/datasets/datalad/ci/git-annex[master]builds/2022
|
||||
$> git -C 05 grep 'export_import_subdir:.*FAIL'
|
||||
|
||||
$> git -C 04 grep 'export_import_subdir:.*FAIL'
|
||||
cron-20220424/build-ubuntu.yaml-671-0346631d-failed/1_test-annex (normal, ubuntu-latest).txt:2022-04-24T03:08:45.8548538Z export_import_subdir: FAIL (1.16s)
|
||||
cron-20220424/build-ubuntu.yaml-671-0346631d-failed/test-annex (normal, ubuntu-latest)/7_Run tests.txt:2022-04-24T03:08:45.8548535Z export_import_subdir: FAIL (1.16s)
|
||||
git -C 04 grep 'export_import_subdir:.*FAIL' 5.08s user 2.04s system 369% cpu 1.927 total
|
||||
```
|
||||
|
||||
and there were a few in march (do with `-C 03`)
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
[[!comment format=mdwn
|
||||
username="mnaoumov"
|
||||
avatar="http://cdn.libravatar.org/avatar/6e35380ce54f8cdce1e5466df77f2208"
|
||||
subject="comment 2"
|
||||
date="2022-05-05T22:44:01Z"
|
||||
content="""
|
||||
Thanks for your response.
|
||||
|
||||
Your test for matchexpression doesn't show anything because you put `--mimeencoding=binary` part in it. So obviously expression will match.
|
||||
|
||||
```
|
||||
$ git annex version
|
||||
git-annex version: 10.20220223-g8f6b52b77
|
||||
build flags: Assistant Webapp Pairing TorrentParser MagicMime Feeds Testsuite S3 WebDAV
|
||||
dependency versions: aws-0.22 bloomfilter-2.0.1.0 cryptonite-0.29 DAV-1.3.4 feed-1.3.2.0 ghc-8.10.7 http-client-0.7.9 persistent-sqlite-2.13.0.3 torrent-10000.1.1 uuid-1.3.15 yesod-1.6.1.2
|
||||
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2BP512E BLAKE2BP512 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL X*
|
||||
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar git-lfs httpalso borg hook external
|
||||
operating system: mingw32 x86_64
|
||||
supported repository versions: 8 9 10
|
||||
upgrade supported from repository versions: 2 3 4 5 6 7 8 9 10
|
||||
local repository version: 10
|
||||
```
|
||||
|
||||
I prepared a simple script to reproduce the bug
|
||||
|
||||
```
|
||||
mkdir bug
|
||||
cd bug
|
||||
git init
|
||||
git annex init --version=10
|
||||
git annex config --set annex.largefiles 'mimeencoding=binary and largerthan=0'
|
||||
wget https://picsum.photos/id/237/200/300.jpg -O hello.jpg
|
||||
wget https://picsum.photos/id/237/200/300.jpg -O привет.jpg
|
||||
git annex add hello.jpg --verbose
|
||||
git annex add привет.jpg --verbose
|
||||
```
|
||||
|
||||
It behaves as I described before. The file with Cyrillic letters is added as non-binary.
|
||||
|
||||
I think it proves that something is not right with MagicMime library dealing with Cyrillic filenames
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,2 @@
|
|||
A somewhat follow up to https://git-annex.branchable.com/bugs/add_config_var_preventing_adjusted_branch_mode/ where we ended up in adjusted branch mode and want to get back to original indirect mode using the thaw/freeze commands.
|
||||
Checking out `master` branch is not sufficient since `.git/annex/objects` uses different layout I guess to ensure that symlinks do not jeopardize actual annex storage on systems without read-only protection. But we need some command to migrate .git/annex/objects layout. May be it is already there and I just failed to find
|
||||
Loading…
Add table
Reference in a new issue