Merge branch 'master' of ssh://git-annex.branchable.com
This commit is contained in:
Joey Hess
commit
3e85707ccf
3 changed files with 120 additions and 0 deletions
|
|
@ -0,0 +1,49 @@
|
|||
This is a satellite issue the one on [strange mimetype driven addition](http://git-annex.branchable.com/bugs/manages_to_incorrectly_add_to_annex_instead_of_git_based_on___34__mimetype__34___-_we_cannot_figure_it_out_why/?updated)
|
||||
|
||||
We do get report on addition of the file twice (when it switches from annex to git?)
|
||||
|
||||
[[!format sh """
|
||||
(git)smaug:/mnt/btrfs/scrap/tmp/SIMON[master]data_BIDS
|
||||
$> rm -f TEST.txt; cat BADFILE.txt >| TEST.txt; file --mime TEST.txt; git annex add TEST.txt
|
||||
TEST.txt: text/plain; charset=utf-8
|
||||
add TEST.txt ok
|
||||
(recording state in git...)
|
||||
|
||||
$> rm -f TEST.txt; sed -e 's,[{}],,g' BADFILE.txt >| TEST.txt; file --mime TEST.txt; git annex add TEST.txt
|
||||
TEST.txt: text/plain; charset=utf-8
|
||||
add TEST.txt (non-large file; adding content to git repository) ok
|
||||
add TEST.txt (non-large file; adding content to git repository) ok
|
||||
(recording state in git...)
|
||||
|
||||
"""]]
|
||||
|
||||
and here is that run with `--debug`:
|
||||
|
||||
[[!format sh """
|
||||
$> rm -f TEST.txt; sed -e 's,[{}],,g' BADFILE.txt >| TEST.txt; file --mime TEST.txt; git annex add --debug TEST.txt
|
||||
TEST.txt: text/plain; charset=utf-8
|
||||
[2019-04-26 09:57:45.451632679] read: git ["--git-dir=../.git","--work-tree=..","--literal-pathspecs","symbolic-ref","-q","HEAD"]
|
||||
[2019-04-26 09:57:45.455061047] process done ExitSuccess
|
||||
[2019-04-26 09:57:45.455134123] read: git ["--git-dir=../.git","--work-tree=..","--literal-pathspecs","show-ref","refs/heads/master"]
|
||||
[2019-04-26 09:57:45.466478693] process done ExitSuccess
|
||||
[2019-04-26 09:57:45.466612578] read: git ["--git-dir=../.git","--work-tree=..","--literal-pathspecs","ls-files","--others","--exclude-standard","-z","--","TEST.txt"]
|
||||
[2019-04-26 09:57:45.471421295] read: git ["--git-dir=../.git","--work-tree=..","--literal-pathspecs","ls-files","--modified","-z","--","TEST.txt"]
|
||||
[2019-04-26 09:57:45.479659995] chat: git ["--git-dir=../.git","--work-tree=..","--literal-pathspecs","check-attr","-z","--stdin","annex.backend","annex.numcopies","annex.largefiles","--"]
|
||||
[2019-04-26 09:57:45.479997471] read: git ["--version"]
|
||||
[2019-04-26 09:57:45.482476867] process done ExitSuccess
|
||||
add TEST.txt (non-large file; adding content to git repository) ok
|
||||
[2019-04-26 09:57:45.494729569] read: git ["--git-dir=../.git","--work-tree=..","--literal-pathspecs","diff","--name-only","--diff-filter=T","-z","--","TEST.txt"]
|
||||
[2019-04-26 09:57:45.498615035] chat: git ["--git-dir=../.git","--work-tree=..","--literal-pathspecs","cat-file","--batch"]
|
||||
[2019-04-26 09:57:45.499023359] chat: git ["--git-dir=../.git","--work-tree=..","--literal-pathspecs","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"]
|
||||
add TEST.txt (non-large file; adding content to git repository) ok
|
||||
(recording state in git...)
|
||||
[2019-04-26 09:57:45.509449016] feed: xargs ["-0","git","--git-dir=../.git","--work-tree=..","--literal-pathspecs","add","--"]
|
||||
[2019-04-26 09:57:45.54937333] process done ExitSuccess
|
||||
[2019-04-26 09:57:45.550142214] process done ExitSuccess
|
||||
[2019-04-26 09:57:45.550690643] process done ExitSuccess
|
||||
[2019-04-26 09:57:45.551282227] process done ExitSuccess
|
||||
|
||||
|
||||
"""]]
|
||||
|
||||
annex 7.20190219+git191-g2d6a364d4-1~ndall+1
|
||||
|
|
@ -0,0 +1,66 @@
|
|||
We have found a strange file which for some reason gets added to annex instead of git, although `file --mime` reports it to be a text file. Somehow the possible culprit (we also achieved changed in behavior via different means) is the `{}`
|
||||
|
||||
Here is the sample of a BADFILE: http://www.onerussian.com/tmp/BADFILE.txt which gets added to annex instead of git:
|
||||
|
||||
[[!format sh """
|
||||
$> wget http://www.onerussian.com/tmp/BADFILE.txt ; cat .gitattributes; file --mime BAD
|
||||
...
|
||||
BADFILE.txt 100%[=======================================================>] 289 --.-KB/s in 0s
|
||||
|
||||
|
||||
* annex.backend=MD5E
|
||||
* annex.largefiles=(not(mimetype=text/*))
|
||||
**/.git* annex.largefiles=nothingBADFILE.txt: text/plain; charset=utf-8
|
||||
add BADFILE.txt ok
|
||||
(recording state in git...)
|
||||
|
||||
$> ls -l BADFILE.txt
|
||||
lrwxrwxrwx 1 yoh yoh 120 Apr 26 09:43 BADFILE.txt -> .git/annex/objects/xw/3W/MD5E-s289--2aae5dfcc232055ba6c06270b6c6daf0.txt/MD5E-s289--2aae5dfcc232055ba6c06270b6c6daf0.txt
|
||||
|
||||
"""]]
|
||||
|
||||
so we tried to troubleshoot a bit and here is attempt with removing `{}` chars vs without removing which shows differing behavior:
|
||||
|
||||
[[!format sh """
|
||||
(git)smaug:/mnt/btrfs/scrap/tmp/SIMON[master]data_BIDS
|
||||
$> cat ../.gitattributes
|
||||
|
||||
* annex.backend=MD5E
|
||||
* annex.largefiles=(not(mimetype=text/*))
|
||||
**/.git* annex.largefiles=nothing%
|
||||
|
||||
$> git reset --hard; rm -f TEST.txt; sed -e 's,[{}],,g' BADFILE.txt >| TEST.txt; file --mime TEST.txt; git annex add TEST.txt
|
||||
HEAD is now at f97185f badfile into git
|
||||
TEST.txt: text/plain; charset=utf-8
|
||||
add TEST.txt (non-large file; adding content to git repository) ok
|
||||
(recording state in git...)
|
||||
|
||||
$> git reset --hard; rm -f TEST.txt; cat BADFILE.txt >| TEST.txt; file --mime TEST.txt; git annex add TEST.txt
|
||||
HEAD is now at f97185f badfile into git
|
||||
TEST.txt: text/plain; charset=utf-8
|
||||
add TEST.txt ok
|
||||
(recording state in git...)
|
||||
|
||||
$> git annex version
|
||||
git-annex version: 7.20190219+git191-g2d6a364d4-1~ndall+1
|
||||
build flags: Assistant Webapp Pairing S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite
|
||||
dependency versions: aws-0.20 bloomfilter-2.0.1.0 cryptonite-0.25 DAV-1.3.3 feed-1.0.0.0 ghc-8.4.4 http-client-0.5.13.1 persistent-sqlite-2.8.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.0
|
||||
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL
|
||||
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar hook external
|
||||
operating system: linux x86_64
|
||||
supported repository versions: 5 7
|
||||
upgrade supported from repository versions: 0 1 2 3 4 5 6
|
||||
local repository version: 5
|
||||
|
||||
$> apt-cache policy git-annex-standalone
|
||||
git-annex-standalone:
|
||||
Installed: 7.20190219+git191-g2d6a364d4-1~ndall+1
|
||||
Candidate: 7.20190219+git191-g2d6a364d4-1~ndall+1
|
||||
Version table:
|
||||
*** 7.20190219+git191-g2d6a364d4-1~ndall+1 500
|
||||
500 http://neuro.debian.net/debian stretch/main amd64 Packages
|
||||
500 http://neurodebian.ovgu.de/debian stretch/main amd64 Packages
|
||||
100 /var/lib/dpkg/status
|
||||
|
||||
"""]]
|
||||
|
||||
|
|
@ -0,0 +1,5 @@
|
|||
ATM there is no `--json-progress` in `git annex add` (only `--json`), so no feedback to the user could be provided on ETA etc. Would be nice to have `--json-progress` there to echo similar one for `get` and `copy`.
|
||||
|
||||
Cheers!
|
||||
|
||||
[[!meta author=yoh]]
|
||||
Loading…
Add table
Reference in a new issue