0f18636c8a
Instead -J will behave as if it was built without concurrent-output support in this situation. Ie, it will be mostly quiet, except when there's an error. Note that it's not a problem for a filename to contain invalid utf-8 when in a utf-8 locale. That is handled ok by concurrent-output. It's only displaying unicode characters in a non-unicode locale that doesn't work.
88 lines
4.3 KiB
Markdown
88 lines
4.3 KiB
Markdown
### Please describe the problem.
|
|
|
|
If I run git annex fsck with the parallel jobs switch (e.g. -J2) it fails when it encounters files with non-ASCII filenames. It works fine without
|
|
this switch.
|
|
|
|
|
|
### What steps will reproduce the problem?
|
|
|
|
Make a git annex repository with non-ascii filenames, add the files, and then run fsck with the parallel switch. A script to do this
|
|
is included below.
|
|
|
|
|
|
### What version of git-annex are you using? On what operating system?
|
|
|
|
Arch Linux (64bit)
|
|
|
|
git-annex version: 6.20160114-g297a744
|
|
build flags: Assistant Webapp Webapp-secure Pairing Testsuite S3(multipartupload) WebDAV Inotify DBus DesktopNotify XMPP ConcurrentOutput DNS Feeds Quvi TDFA TorrentParser
|
|
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
|
|
remote types: git gcrypt S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external
|
|
local repository version: 5
|
|
supported repository versions: 5 6
|
|
upgrade supported from repository versions: 0 1 2 4 5
|
|
|
|
### Please provide any additional information below.
|
|
|
|
[[!format sh """
|
|
#!/bin/bash
|
|
set -o errexit -o nounset
|
|
|
|
mkdir testing-repo
|
|
cd testing-repo
|
|
git init
|
|
git annex init testing-repo
|
|
|
|
make_fake_file() {
|
|
local filename="$1"
|
|
mkdir -p "$(dirname "$filename")"
|
|
echo "hello world" > "$filename"
|
|
git annex add "$filename"
|
|
}
|
|
|
|
export -f make_fake_file
|
|
parallel -j1 'make_fake_file {}' <<EOF
|
|
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/folder.jpg
|
|
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/01 - 遥か彼方 - ASIAN KUNG-FU GENERATION.flac
|
|
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/02 - 未来の破片 - ASIAN KUNG-FU GENERATION.flac
|
|
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/03 - アンダースタンド - ASIAN KUNG-FU GENERATION.flac
|
|
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/04 - 君という花 - ASIAN KUNG-FU GENERATION.flac
|
|
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/05 - リライト - ASIAN KUNG-FU GENERATION.flac
|
|
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/06 - 君の街まで - ASIAN KUNG-FU GENERATION.flac
|
|
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/07 - ループ & ループ - ASIAN KUNG-FU GENERATION.flac
|
|
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/08 - ブラックアウト - ASIAN KUNG-FU GENERATION.flac
|
|
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/09 - ブルートレイン - ASIAN KUNG-FU GENERATION.flac
|
|
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/10 - 或る街の群青 - ASIAN KUNG-FU GENERATION.flac
|
|
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/11 - アフターダーク - ASIAN KUNG-FU GENERATION.flac
|
|
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/12 - 転がる岩、君に朝が降る - ASIAN KUNG-FU GENERATION.flac
|
|
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/13 - ムスタング - ASIAN KUNG-FU GENERATION.flac
|
|
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/14 - 藤沢ルーザー - ASIAN KUNG-FU GENERATION.flac
|
|
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/15 - 新世紀のラブソング - ASIAN KUNG-FU GENERATION.flac
|
|
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/16 - ソラニン - ASIAN KUNG-FU GENERATION.flac
|
|
ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/17 - マーチングバンド - ASIAN KUNG-FU GENERATION.flac
|
|
EOF
|
|
|
|
git commit -m 'Add testing files.'
|
|
git annex fsck -J2
|
|
"""]]
|
|
|
|
On my system this produces:
|
|
|
|
[[!format sh """
|
|
fsck ASIAN KUNG-FU GENERATION/2012 - BEST HIT AKG/flac/01 - git-annex: <stdout>: commitAndReleaseBuffer: invalid argument (invalid character)
|
|
FAIL 1
|
|
"""]]
|
|
|
|
(The FAIL 1 output is just my terminal printing that the exit code was 1)
|
|
|
|
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
|
|
|
|
Plenty. In fact I've been using it for a long time - I just only recently tried to use -J2 to speed up the fsck'ing :)
|
|
|
|
|
|
[[!meta title="-J can crash on displaying filenames not supported by current locale"]]
|
|
|
|
> I've worked around this by detecting the non-unicode locale and avoiding
|
|
> the fancy concurrent output which needs it. So -J will work, just not
|
|
> with concurrent progress. I think this is the best that can be done
|
|
> reasonably, so [[done]]. --[[Joey]]
|