From 6ae6242a42c36cf90eb892c99a41659192a17b0d Mon Sep 17 00:00:00 2001 From: "https://christian.amsuess.com/chrysn" Date: Mon, 25 May 2020 10:03:16 +0000 Subject: [PATCH 1/4] Tried --preserve-filename with a torrent, didn't work --- ...ve-filename_still_modifies_file_names.mdwn | 41 +++++++++++++++++++ 1 file changed, 41 insertions(+) create mode 100644 doc/bugs/addurl_--preserve-filename_still_modifies_file_names.mdwn diff --git a/doc/bugs/addurl_--preserve-filename_still_modifies_file_names.mdwn b/doc/bugs/addurl_--preserve-filename_still_modifies_file_names.mdwn new file mode 100644 index 0000000000..80445ab6a6 --- /dev/null +++ b/doc/bugs/addurl_--preserve-filename_still_modifies_file_names.mdwn @@ -0,0 +1,41 @@ +### Please describe the problem. + +The new `--preserve-filename` option does not have its described effect on torrent files. + +### What steps will reproduce the problem? + +[[!format sh """ +$ git annex addurl --preserve-filename \ + http://downloads.endor.at/chaos-math_multi-language_1080p_mkv.ea15601881aa1be1.torrent +"""]] + +(cancelling when at least a file has arrived) + +[[!format sh """ +$ tree +└── downloads.endor.at_chaos-math_multi-language_1080p_mkv.ea15601881aa1be1.torrent/ + └── 01._Motion_and_determinism_-_Panta_Rhei__1080p_.mkv +$ btcheck -l <(curl 'https://downloads.endor.at/chaos-math_multi-language_1080p_mkv.ea15601881aa1be1.torrent') +Base directory : chaos-math_multi-language_1080p_mkv +01. Motion and determinism - Panta Rhei [1080p].mkv (409315188) +"""]] + +Based on the description of `--preserve-filename`, given that nothing in the names is particularly malicious, I'd have expected the tree output to look like this: + +[[!format sh """ +$ tree +└── chaos-math_multi-language_1080p_mkv/ (as per base directory) + └── 01. Motion and determinism - Panta Rhei [1080p].mkv +"""]] + +### What version of git-annex are you using? On what operating system? + +8.20200522 (built using `gbp buildpackage` from current git master, 87dc9cd0) + +### Please provide any additional information below. + +This option, when working with torrents, would be a building block to [[todo/bittorrent: support offline operation and verification]], and resolve most of the suggestions from there. + +### Have you had any luck using git-annex before? + +Yes: It saved me from potential data loss when my backup cron jobs stopped working and the mails got lost -- `git annex drop` failing on the laptops was both the right thing to do given numcopies=2, and alerted me to the problem in due time. From 91a8b271f7520b06b43a5097aff024504f1f7f1c Mon Sep 17 00:00:00 2001 From: "braun.markus89@51b521a42cc994db864df308627bd6454f9c309d" Date: Mon, 25 May 2020 11:06:05 +0000 Subject: [PATCH 2/4] Added a comment --- ..._8a43d4e00c6b48999fe84d2f7ad55877._comment | 25 +++++++++++++++++++ 1 file changed, 25 insertions(+) create mode 100644 doc/bugs/__34__Cannot_handle_files_this_big__34___error_for_unlocked_files/comment_4_8a43d4e00c6b48999fe84d2f7ad55877._comment diff --git a/doc/bugs/__34__Cannot_handle_files_this_big__34___error_for_unlocked_files/comment_4_8a43d4e00c6b48999fe84d2f7ad55877._comment b/doc/bugs/__34__Cannot_handle_files_this_big__34___error_for_unlocked_files/comment_4_8a43d4e00c6b48999fe84d2f7ad55877._comment new file mode 100644 index 0000000000..b405fe3858 --- /dev/null +++ b/doc/bugs/__34__Cannot_handle_files_this_big__34___error_for_unlocked_files/comment_4_8a43d4e00c6b48999fe84d2f7ad55877._comment @@ -0,0 +1,25 @@ +[[!comment format=mdwn + username="braun.markus89@51b521a42cc994db864df308627bd6454f9c309d" + nickname="braun.markus89" + avatar="http://cdn.libravatar.org/avatar/c11d06a0d9db6a9472b05ee01c342ca4" + subject="comment 4" + date="2020-05-25T11:06:05Z" + content=""" +git version is 2.26.1, so this should be fine. + +I guess, git-annex couldn't do it any better, still git tries to overallocate memory. +My Synology NAS only got 1gb of memory (at least 600mb are used all the time), so I wonder why \"hanging up\" the pipe works when unlocking 1gb file but not for 2gb. But the Synology linux is a little bit weird, so I have to give up on debugging there. The 1gb memory spec of my NAS is ridiculously low (maybe too low for git anyways) and cannot be upgraded.... + +For the sake of documentation for other synology users with a low-spec NAS + +Workaround that did work + +* create a user with uid/gid matching the NAS user and mount via NFSv3. On the client system the memory is sufficient to run every git-annex command. + +Workarounds that did not work out + +* NFSv4 with idmapping (configuring the Kerberos authentication would have taken a lot of time and not even sure if it would have worked out in the end) +* SSHFS (the sshfs server provided by Synology seems to be broken, resulting in broken symlinks on my linux client system -> which is obviously a no go for git annex ;-) ) + + +"""]] From 1ac2b8cc076e3e1098b0abcecc17920358b7a840 Mon Sep 17 00:00:00 2001 From: "braun.markus89@51b521a42cc994db864df308627bd6454f9c309d" Date: Mon, 25 May 2020 11:07:08 +0000 Subject: [PATCH 3/4] Added a comment --- .../comment_5_7829d524f435dc908149f29a53e3a1ab._comment | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 doc/bugs/__34__Cannot_handle_files_this_big__34___error_for_unlocked_files/comment_5_7829d524f435dc908149f29a53e3a1ab._comment diff --git a/doc/bugs/__34__Cannot_handle_files_this_big__34___error_for_unlocked_files/comment_5_7829d524f435dc908149f29a53e3a1ab._comment b/doc/bugs/__34__Cannot_handle_files_this_big__34___error_for_unlocked_files/comment_5_7829d524f435dc908149f29a53e3a1ab._comment new file mode 100644 index 0000000000..89ada2b9ed --- /dev/null +++ b/doc/bugs/__34__Cannot_handle_files_this_big__34___error_for_unlocked_files/comment_5_7829d524f435dc908149f29a53e3a1ab._comment @@ -0,0 +1,9 @@ +[[!comment format=mdwn + username="braun.markus89@51b521a42cc994db864df308627bd6454f9c309d" + nickname="braun.markus89" + avatar="http://cdn.libravatar.org/avatar/c11d06a0d9db6a9472b05ee01c342ca4" + subject="comment 5" + date="2020-05-25T11:07:07Z" + content=""" +so, thank you once again, Joey, in particular for this great tool ;-) +"""]] From d30dd6daa4f94a85a5d34ecce1d10d92c7232120 Mon Sep 17 00:00:00 2001 From: yarikoptic Date: Tue, 26 May 2020 04:14:57 +0000 Subject: [PATCH 4/4] TODO for export --json* options --- .../export_--json_--json-progress_--json-error-messages.mdwn | 4 ++++ 1 file changed, 4 insertions(+) create mode 100644 doc/todo/export_--json_--json-progress_--json-error-messages.mdwn diff --git a/doc/todo/export_--json_--json-progress_--json-error-messages.mdwn b/doc/todo/export_--json_--json-progress_--json-error-messages.mdwn new file mode 100644 index 0000000000..c44d1be529 --- /dev/null +++ b/doc/todo/export_--json_--json-progress_--json-error-messages.mdwn @@ -0,0 +1,4 @@ +Pretty much to mimic `copy` in terms of its interfaceability with external commands. + +[[!meta author=yoh]] +[[!tag projects/datalad]]