mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
git-annex/doc/bugs/bittorrent_special_url_double-encoding.mdwn
Joey Hess a787cead35 bittorrent: Fix mojibake introduced in parsing arai2c progress output. 
hGetSomeString reads one byte at a time, so unicode bytes are not composed.
The problem comes when outputting that to the console with hPut; that
tried to apply the handle's encoding, and so we get mojibake.

Instead, use ByteStrings, and only convert it to a string for parsing, not
for display.

Note that there are a couple of other things that use hGetSomeString,
which I've left as-is for now.
2015-02-10 12:34:34 -04:00

177 lines
8.3 KiB
Markdown
Raw Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

[[!meta title="mojibake in aria2c output"]]
### Please describe the problem.
French messages in the `aria2c` client called from `git-annex` are garbled, probably double utf-8 encoded.
### What steps will reproduce the problem?
download a bittorrent url with git-torrent using a UTF8 locale.
### What version of git-annex are you using? On what operating system?
`5.20150205-gbf9058a` on current debian jessie.
### Please provide any additional information below.
[[!format sh """
# If you can, paste a complete transcript of the problem occurring here.
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
anarcat@marcos:iso(master)$ /srv/downloads.kitenet.net/git-annex/linux/current/git-annex.linux/git-annex addurl http://images.kali.org/kali-linux-1.1.0-amd64-mini.torrent
(downloading torrent file...)
--2015-02-09 22:08:58-- http://images.kali.org/kali-linux-1.1.0-amd64-mini.torrent
Résolution de images.kali.org (images.kali.org)… 50.7.37.130
Connexion à images.kali.org (images.kali.org)|50.7.37.130|:80… connecté.
requête HTTP transmise, en attente de la réponse… 200 OK
Taille : 2668 (2,6K) [application/octet-stream]
Sauvegarde en : « ../.git/annex/misctmp/torrent31107 »
100%[=====================================================================================================================================================================================================>] 2 668 --.-K/s ds 0,002s
2015-02-09 22:08:59 (1,56 MB/s) — « ../.git/annex/misctmp/torrent31107 » sauvegardé [2668/2668]
addurl images.kali.org_kali_linux_1.1.0_amd64_mini.torrent/kali_linux_1.1.0_amd64_mini.iso (from bittorrent)
02/09 22:08:59 [ERROR] Exception caught while loading DHT routing table from /home/anarcat/.aria2/dht.dat
Exception: [DHTRoutingTableDeserializer.cc:83] errorCode=1 Failed to load DHT routing table from /home/anarcat/.aria2/dht.dat
02/09 22:08:59 [NOTICE] IPv4 DHT: listening on UDP port 6912
02/09 22:08:59 [ERROR] Erreur d'intégrité détectée. fichier=../.git/annex/misctmp/URL--http&c%%images.kali.org%kali-linux-1.1.0-amd64-mini.torrent/kali-linux-1.1.0-amd64-mini
02/09 22:08:59 [NOTICE] IPv4 BitTorrent: listening on TCP port 6910
02/09 22:08:59 [NOTICE] IPv6 BitTorrent: listening on TCP port 6910
[#9acd0a 0B/27MiB(0%) CN:0 SD:0 DL:0B]
[#9acd0a 0B/27MiB(0%) CN:29 SD:0 DL:0B]
[#9acd0a 0B/27MiB(0%) CN:29 SD:1 DL:0B]
[#9acd0a 0B/27MiB(0%) CN:27 SD:5 DL:0B]
[#9acd0a 464KiB/27MiB(1%) CN:27 SD:6 DL:612KiB ETA:44s]
[#9acd0a 1.0MiB/27MiB(3%) CN:27 SD:6 DL:604KiB ETA:44s]
[#9acd0a 1.5MiB/27MiB(5%) CN:27 SD:7 DL:583KiB ETA:44s]
[#9acd0a 2.2MiB/27MiB(8%) CN:26 SD:8 DL:600KiB ETA:42s]
[#9acd0a 2.7MiB/27MiB(10%) CN:26 SD:10 DL:596KiB ETA:41s]
[#9acd0a 3.2MiB/27MiB(12%) CN:26 SD:11 DL:579KiB ETA:41s]
[#9acd0a 3.8MiB/27MiB(14%) CN:27 SD:11 DL:584KiB ETA:40s]
[#9acd0a 4.4MiB/27MiB(16%) CN:27 SD:11 DL:584KiB ETA:39s]
[#9acd0a 5.0MiB/27MiB(18%) CN:26 SD:11 DL:589KiB ETA:38s]
[#9acd0a 5.5MiB/27MiB(20%) CN:26 SD:11 DL:590KiB ETA:37s]
[#9acd0a 6.2MiB/27MiB(23%) CN:26 SD:11 DL:595KiB ETA:35s]
[#9acd0a 6.8MiB/27MiB(25%) CN:26 SD:11 DL:596KiB ETA:34s]
[#9acd0a 7.3MiB/27MiB(27%) CN:24 SD:9 DL:594KiB ETA:33s]
[#9acd0a 7.9MiB/27MiB(29%) CN:24 SD:9 DL:594KiB ETA:32s]
[#9acd0a 8.4MiB/27MiB(31%) CN:24 SD:10 DL:591KiB ETA:32s]
[#9acd0a 9.1MiB/27MiB(33%) CN:24 SD:10 DL:595KiB ETA:30s]
[#9acd0a 9.7MiB/27MiB(35%) CN:24 SD:10 DL:596KiB ETA:29s]
[#9acd0a 10MiB/27MiB(37%) CN:22 SD:9 DL:589KiB ETA:29s]
[#9acd0a 10MiB/27MiB(39%) CN:10 SD:8 DL:588KiB ETA:28s]
[#9acd0a 11MiB/27MiB(41%) CN:9 SD:8 DL:583KiB ETA:27s]
[#9acd0a 11MiB/27MiB(43%) CN:9 SD:8 DL:587KiB ETA:26s]
[#9acd0a 12MiB/27MiB(45%) CN:8 SD:7 DL:586KiB ETA:25s]
[#9acd0a 12MiB/27MiB(48%) CN:8 SD:7 DL:584KiB ETA:24s]
[#9acd0a 13MiB/27MiB(50%) CN:8 SD:8 DL:585KiB ETA:23s]
[#9acd0a 14MiB/27MiB(52%) CN:8 SD:8 DL:574KiB ETA:23s]
[#9acd0a 14MiB/27MiB(53%) CN:8 SD:8 DL:567KiB ETA:22s]
[#9acd0a 14MiB/27MiB(55%) CN:8 SD:8 DL:556KiB ETA:22s]
[#9acd0a 15MiB/27MiB(57%) CN:8 SD:8 DL:555KiB ETA:21s]
[#9acd0a 15MiB/27MiB(59%) CN:8 SD:8 DL:550KiB ETA:20s]
[#9acd0a 16MiB/27MiB(60%) CN:8 SD:8 DL:543KiB ETA:19s]
[#9acd0a 16MiB/27MiB(62%) CN:8 SD:8 DL:540KiB ETA:18s]
[#9acd0a 17MiB/27MiB(64%) CN:8 SD:8 DL:537KiB ETA:18s]
[#9acd0a 18MiB/27MiB(66%) CN:8 SD:8 DL:535KiB ETA:17s]
[#9acd0a 18MiB/27MiB(68%) CN:8 SD:8 DL:525KiB ETA:16s]
[#9acd0a 18MiB/27MiB(70%) CN:8 SD:8 DL:522KiB ETA:15s]
[#9acd0a 19MiB/27MiB(72%) CN:8 SD:8 DL:522KiB ETA:14s]
[#9acd0a 19MiB/27MiB(74%) CN:8 SD:8 DL:523KiB ETA:13s]
[#9acd0a 20MiB/27MiB(76%) CN:8 SD:8 DL:517KiB ETA:12s]
[#9acd0a 21MiB/27MiB(78%) CN:8 SD:8 DL:522KiB ETA:11s]
[#9acd0a 21MiB/27MiB(80%) CN:13 SD:8 DL:532KiB ETA:10s]
[#9acd0a 22MiB/27MiB(82%) CN:13 SD:8 DL:541KiB ETA:8s]
[#9acd0a 22MiB/27MiB(84%) CN:13 SD:8 DL:544KiB ETA:7s]
[#9acd0a 23MiB/27MiB(86%) CN:13 SD:8 DL:552KiB ETA:6s]
[#9acd0a 23MiB/27MiB(88%) CN:13 SD:8 DL:549KiB ETA:5s]
[#9acd0a 24MiB/27MiB(91%) CN:13 SD:8 DL:558KiB ETA:4s]
[#9acd0a 25MiB/27MiB(93%) CN:13 SD:8 DL:570KiB ETA:3s]
[#9acd0a 25MiB/27MiB(95%) CN:13 SD:8 DL:576KiB ETA:2s]
[#9acd0a 26MiB/27MiB(97%) CN:13 SD:8 DL:587KiB ETA:1s]
[#9acd0a 26MiB/27MiB(99%) CN:13 SD:8 DL:592KiB]
02/09 22:09:53 [NOTICE] Le téléchargement des fichiers sélectionnés est terminé.
02/09 22:09:53 [NOTICE] Le partage (seeding) est terminé
[#9acd0a SEED(0.0) CN:9 SD:4]
[#9acd0a SEED(0.0) CN:0 SD:0]
02/09 22:09:55 [NOTICE] Téléchargement terminé: ../.git/annex/misctmp/URL--http&c%%images.kali.org%kali-linux-1.1.0-amd64-mini.torrent/kali-linux-1.1.0-amd64-mini
02/09 22:09:55 [NOTICE] Votre ratio de partage était de 0.0, envoyé/téléchargé=0B/27MiB
Résultats du téléchargement:
gid |stat|avg speed |path/URI
======+====+===========+=======================================================
9acd0a|OK | 518KiB/s|../.git/annex/misctmp/URL--http&c%%images.kali.org%kali-linux-1.1.0-amd64-mini.torrent/kali-linux-1.1.0-amd64-mini/kali-linux-1.1.0-amd64-mini.iso
Légende du statut:
(OK): téléchargement terminé.
ok
addurl images.kali.org_kali_linux_1.1.0_amd64_mini.torrent/kali_linux_1.1.0_amd64_mini.txt.sha1sum (from bittorrent)
02/09 22:09:56 [NOTICE] IPv4 DHT: listening on UDP port 6990
02/09 22:09:56 [NOTICE] Le fichier de contrôle défectueux ../.git/annex/misctmp/URL--http&c%%images.kali.org%kali-linux-1.1.0-amd64-mini.torrent/kali-linux-1.1.0-amd64-mini.aria2 a été supprimé car le fichier téléchargé ../.git/annex/misctmp/URL--http&c%%images.kali.org%kali-linux-1.1.0-amd64-mini.torrent/kali-linux-1.1.0-amd64-mini n'existe pas.
02/09 22:09:56 [ERROR] Erreur d'intégrité détectée. fichier=../.git/annex/misctmp/URL--http&c%%images.kali.org%kali-linux-1.1.0-amd64-mini.torrent/kali-linux-1.1.0-amd64-mini
02/09 22:09:56 [NOTICE] IPv4 BitTorrent: listening on TCP port 6899
02/09 22:09:56 [NOTICE] IPv6 BitTorrent: listening on TCP port 6899
[#5f49f7 0B/75B(0%) CN:0 SD:0 DL:0B]
[#5f49f7 0B/75B(0%) CN:28 SD:0 DL:0B]
[#5f49f7 0B/75B(0%) CN:28 SD:1 DL:0B]
02/09 22:10:00 [NOTICE] Le téléchargement des fichiers sélectionnés est terminé.
02/09 22:10:00 [NOTICE] Le partage (seeding) est terminé
[#5f49f7 SEED(0.0) CN:22 SD:3]
[#5f49f7 SEED(0.0) CN:0 SD:0]
02/09 22:10:02 [NOTICE] Téléchargement terminé: ../.git/annex/misctmp/URL--http&c%%images.kali.org%kali-linux-1.1.0-amd64-mini.torrent/kali-linux-1.1.0-amd64-mini
02/09 22:10:02 [NOTICE] Votre ratio de partage était de 0.0, envoyé/téléchargé=0B/75B
Résultats du téléchargement:
gid |stat|avg speed |path/URI
======+====+===========+=======================================================
5f49f7|OK | 19B/s|../.git/annex/misctmp/URL--http&c%%images.kali.org%kali-linux-1.1.0-amd64-mini.torrent/kali-linux-1.1.0-amd64-mini/kali-linux-1.1.0-amd64-mini.txt.sha1sum
Légende du statut:
(OK): téléchargement terminé.
ok
(recording state in git...)
anarcat@marcos:iso(master +)$ locale
LANG=fr_CA.UTF-8
LANGUAGE=
LC_CTYPE="fr_CA.UTF-8"
LC_NUMERIC="fr_CA.UTF-8"
LC_TIME="fr_CA.UTF-8"
LC_COLLATE="fr_CA.UTF-8"
LC_MONETARY="fr_CA.UTF-8"
LC_MESSAGES="fr_CA.UTF-8"
LC_PAPER="fr_CA.UTF-8"
LC_NAME="fr_CA.UTF-8"
LC_ADDRESS="fr_CA.UTF-8"
LC_TELEPHONE="fr_CA.UTF-8"
LC_MEASUREMENT="fr_CA.UTF-8"
LC_IDENTIFICATION="fr_CA.UTF-8"
LC_ALL=
# End of transcript or log.
"""]]
Previous similar UTF-8 bug: [[forget_corrupts_non-ascii_chars]]. Looks similar. --[[anarcat]]
> [[fixed|done]] --[[Joey]]
Reference in a new issue View git blame Copy permalink