Fix encoding of data written to git-annex branch. Avoid truncating unicode characters to 8 bits.
Allow any encoding to be used, as with filenames (but utf8 is the sane choice). Affects metadata and repository descriptions, and preferred content expressions. The question of what's the right encoding for the git-annex branch is a vexing one. utf-8 would be a nice choice, but this leaves the possibility of bad data getting into a git-annex branch somehow, and this resulting in git-annex crashing with encoding errors, which is a failure mode I want to avoid. (Also, preferred content expressions can refer to filenames, and filenames can have any encoding, so limiting to utf-8 would not be ideal.) The union merge code already took care to not assume any encoding for a file. Except it assumes that any \n is a literal newline, and not part of some encoding of a character that happens to contain a newline. (At least utf-8 avoids using newline for anything except liternal newlines.) Adapted the git-annex branch code to use this same approach. Note that there is a potential interop problem with Windows, since FileSystemEncoding doesn't work there, and instead things are always decoded as utf-8. If someone uses non-utf8 encoding for data on the git-annex branch, this can lead to an encoding error on windows. However, this commit doesn't actually make that any worse, because the union merge code would similarly fail with an encoding error on windows in that situation. This commit was sponsored by Kyle Meyer.
This commit is contained in:
parent
2dc0c434e0
commit
95ca3bb022
5 changed files with 19 additions and 8 deletions
4
debian/changelog
vendored
4
debian/changelog
vendored
|
@ -13,6 +13,10 @@ git-annex (5.20140518) UNRELEASED; urgency=medium
|
|||
so that it can be easily enabled elsewhere.
|
||||
* android: Run busybox install with -s, since some versions of Android
|
||||
prohibit making hard links.
|
||||
* Fix encoding of data written to git-annex branch. Avoid truncating
|
||||
unicode characters to 8 bits. Allow any encoding to be used, as with
|
||||
filenames (but utf8 is the sane choice). Affects metadata and repository
|
||||
descriptions, and preferred content expressions.
|
||||
|
||||
-- Joey Hess <joeyh@debian.org> Mon, 19 May 2014 15:59:25 -0400
|
||||
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue