mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

Merge branch 'master' of ssh://git-annex.branchable.com

Browse source
This commit is contained in:
Joey Hess 2019-12-20 15:58:15 -04:00
commit 0141c5436e
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
2 changed files with 30 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

22
doc/bugs/be_robust_to_additions_of_more_specific_mime_types_to_libmagic___40__e.g._application__47__json__41__.mdwn Normal file
View file

@ -0,0 +1,22 @@
### Please describe the problem.
[Original report](https://git-annex.branchable.com/bugs/manages_to_incorrectly_add_to_annex_instead_of_git_based_on___34__mimetype__34___-_we_cannot_figure_it_out_why/) was about the change in behavior of git-annex (as a whole installed bundle) that .json files started to be added to git-annex instead of git, whenever .json files remained text files and .gitattributes was configured to add all files with mimetype of text to go into git.
It happened due to the fact that libmagic added handling for detecting .json files and reporting that they are `application/json` instead of `text/plain` as before.
Note that [initial demand/idea behind adding treatment of mime types](https://git-annex.branchable.com/devblog/day_360__annex.largefiles_mimetype/) was actually to provide automation for the most reasonable decision making on what goes into git and what annex, based on either a file a text file or binary.
The bug report referenced above was just marked "done" with a comment that "the magic database changing behavior is not a bug in git-annex" without actually addressing the underlying issue. I even somehow got a wrong impression that "we had it fixed" and was surprised to stumble into it again. I think that the issue should be properly addressed, ideally without requiring users to adjust their `.gitattribute` files (and introducing newer git annex version dependency), so that the desired behavior of having text files going into git, not git-annex, was maintained *even across changes in libmagic DB*.
One, IMHO the easiest way, now that `-k` (keep going) [issue was fixed in libmagic](https://bugs.astron.com/view.php?id=77), would be for git-annex to treat "mimetype" specification as "if any mimetype matches" and ask libmagic about all mimetypes of a file, e.g.:
[[!format sh """
$> file --mime-type -Lk 1.json
1.json: application/json\012- text/plain
"""]]
so that if any structured text file would soon acquire additional, more specific mimetype (e.g., `.md` could be reported as `application/markdown`, just not yet), previous specifications in .gitattributes would still work -- after all those files remain `text/plain` files!
If strict matching (not sure yet about a use case where it would really be needed) by the most specialized mime type is needed, additional "mimetypefirstguess" or alike could be added.
[[!meta author=yoh]]
[[!tag projects/datalad]]

8
doc/bugs/be_robust_to_additions_of_more_specific_mime_types_to_libmagic___40__e.g._application__47__json__41__/comment_1_a18cfae149effed2375b3b0be021ac3a._comment Normal file
View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="yarikoptic"
avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4"
subject="oh hoh, there is mimeencoding now"
date="2019-12-20T19:54:04Z"
content="""
OH!! \"Much ado about nothing\". As [Joey reported in datalad issues](https://github.com/datalad/datalad/issues/3361#issuecomment-488018126) there is now handling of `mimeencoding=binary` as the ultimate decision maker. So we are probably doomed (unless Joey sees reason in the reasoning above and implements that as well) to do go through all datasets and autoadjust them to use `mimeencoding` instead of `mimetype`.
"""]]