From 679b9a09193ab071e3c72913104a7088ccc3aecd Mon Sep 17 00:00:00 2001 From: Ilya_Shlyakhter Date: Mon, 8 Oct 2018 21:43:17 +0000 Subject: [PATCH 1/8] Added a comment --- .../comment_4_df3018b9b3491963311063e8ff202df4._comment | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 doc/git-annex-setpresentkey/comment_4_df3018b9b3491963311063e8ff202df4._comment diff --git a/doc/git-annex-setpresentkey/comment_4_df3018b9b3491963311063e8ff202df4._comment b/doc/git-annex-setpresentkey/comment_4_df3018b9b3491963311063e8ff202df4._comment new file mode 100644 index 0000000000..a7f423a48a --- /dev/null +++ b/doc/git-annex-setpresentkey/comment_4_df3018b9b3491963311063e8ff202df4._comment @@ -0,0 +1,9 @@ +[[!comment format=mdwn + username="Ilya_Shlyakhter" + avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0" + subject="comment 4" + date="2018-10-08T21:43:17Z" + content=""" +@joey thanks. But, besides export.log, the S3 remote also keeps some (undocumented?) internal state, and there's not way to update that state to record the fact that git-annex can GET a given key by downloading s3://mybucket/myobject ? Also, I feel uneasy directly manipulating git-annex internal files. Can you think of any plumbing commands, that could be added to support this use case? +The use case is, I submit a batch job that takes as input some s3:// objects, writes outputs to other s3:// objects, and returns pointers to these new s3:// objects. I want to register these new objects in git-annex, initially without downloading them, but be able to git-annex-get these objects, drop them from the S3 remote, but later be able to put them back under their original s3:// URIs. The latter ability is needed because (1) many workflows expect filenames to be in a particular form, e.g. mysamplename.pN.bam to represent mysample processed with parameter p=N; and (2) some workflow engines can reuse past results if a step is re-run with the same inputs, but they need the results to be at the same s3:// URI as when the step was first run. +"""]] From 3c94f83cbf0ae47bd4e6dec5f318d36b35021eb7 Mon Sep 17 00:00:00 2001 From: Ilya_Shlyakhter Date: Mon, 8 Oct 2018 21:48:27 +0000 Subject: [PATCH 2/8] Added a comment --- .../comment_5_62a0ee0c5b09e6af4a71a8ba55332d00._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/todo/config_option_to_use_all_processors_by_default/comment_5_62a0ee0c5b09e6af4a71a8ba55332d00._comment diff --git a/doc/todo/config_option_to_use_all_processors_by_default/comment_5_62a0ee0c5b09e6af4a71a8ba55332d00._comment b/doc/todo/config_option_to_use_all_processors_by_default/comment_5_62a0ee0c5b09e6af4a71a8ba55332d00._comment new file mode 100644 index 0000000000..61b373f520 --- /dev/null +++ b/doc/todo/config_option_to_use_all_processors_by_default/comment_5_62a0ee0c5b09e6af4a71a8ba55332d00._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="Ilya_Shlyakhter" + avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0" + subject="comment 5" + date="2018-10-08T21:48:27Z" + content=""" +\"why run 64 processes on a 64 CPU system if it has the same speed bus / same network speed as a 4 CPU system\" -- it doesn't always; higher-end AWS instances have 25Gbps networks, and you can buy more throughput as needed. +"""]] From 4a66a3c4474cee195f6d9dbe6a075a7330c786b9 Mon Sep 17 00:00:00 2001 From: ericm Date: Tue, 9 Oct 2018 01:16:47 +0000 Subject: [PATCH 3/8] --- ...dows_only_watches_one_repo_on_startup.mdwn | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) create mode 100644 doc/forum/Assistant_in_windows_only_watches_one_repo_on_startup.mdwn diff --git a/doc/forum/Assistant_in_windows_only_watches_one_repo_on_startup.mdwn b/doc/forum/Assistant_in_windows_only_watches_one_repo_on_startup.mdwn new file mode 100644 index 0000000000..ed0af0f68b --- /dev/null +++ b/doc/forum/Assistant_in_windows_only_watches_one_repo_on_startup.mdwn @@ -0,0 +1,19 @@ +Hi there, + +I've got a problem similar to here: [bugs/Assistant_only_watches_one_repo_on_startup/](https://git-annex.branchable.com/bugs/Assistant_only_watches_one_repo_on_startup/) but in **windows 7**. + +I'm running the latest version and my autostart config file (~/.config/git-annex/autostart) contains the following repos: + +    C:\temp\testrepo1
+    C:\temp\testrepo2 + +From a git bash shell, I launch the assistant using the following command: `git annex assistant --autostart`. +Upon startup, it outputs the following message: + +    git-annex autostart in C:\temp\testrepo1 + +I then copy a test file to both repos. + +In testrepo1, the test file is automatically added to the repo. While in testrepo2, running 'git annex status` shows that file has not been processed. + +Which means that the second repo is not being watched by the assistant. From a6e8c4f157b95bfe86cbcd0f1d7f790be7341dc1 Mon Sep 17 00:00:00 2001 From: ericm Date: Tue, 9 Oct 2018 01:18:02 +0000 Subject: [PATCH 4/8] --- .../Assistant_in_windows_only_watches_one_repo_on_startup.mdwn | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/forum/Assistant_in_windows_only_watches_one_repo_on_startup.mdwn b/doc/forum/Assistant_in_windows_only_watches_one_repo_on_startup.mdwn index ed0af0f68b..9c2e981e46 100644 --- a/doc/forum/Assistant_in_windows_only_watches_one_repo_on_startup.mdwn +++ b/doc/forum/Assistant_in_windows_only_watches_one_repo_on_startup.mdwn @@ -14,6 +14,6 @@ Upon startup, it outputs the following message: I then copy a test file to both repos. -In testrepo1, the test file is automatically added to the repo. While in testrepo2, running 'git annex status` shows that file has not been processed. +In testrepo1, the test file is automatically added to the repo. While in testrepo2, running `git annex status` shows that file has not been processed. Which means that the second repo is not being watched by the assistant. From 87206a985a80dca0970540b3f1358ee07311da0b Mon Sep 17 00:00:00 2001 From: "branchable@bafd175a4b99afd6ed72501042e364ebd3e0c45e" Date: Tue, 9 Oct 2018 09:17:11 +0000 Subject: [PATCH 5/8] Added a comment: aarch64 still broken, please help --- .../comment_20_9651ab0e1e43d193cca16963f5fde858._comment | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 doc/tips/install_on_Android_in_Termux/comment_20_9651ab0e1e43d193cca16963f5fde858._comment diff --git a/doc/tips/install_on_Android_in_Termux/comment_20_9651ab0e1e43d193cca16963f5fde858._comment b/doc/tips/install_on_Android_in_Termux/comment_20_9651ab0e1e43d193cca16963f5fde858._comment new file mode 100644 index 0000000000..815b3b469c --- /dev/null +++ b/doc/tips/install_on_Android_in_Termux/comment_20_9651ab0e1e43d193cca16963f5fde858._comment @@ -0,0 +1,9 @@ +[[!comment format=mdwn + username="branchable@bafd175a4b99afd6ed72501042e364ebd3e0c45e" + nickname="branchable" + avatar="http://cdn.libravatar.org/avatar/ae41dba34ee6000056f00793c695be75" + subject="aarch64 still broken, please help" + date="2018-10-09T09:17:11Z" + content=""" +@joey, any update on this signal 11 issue? Currently we aarch64 users are completely stuck, AFAICS. Thanks! +"""]] From e86cf0898172f36bad9d687074e0a2b5257f8139 Mon Sep 17 00:00:00 2001 From: Ilya_Shlyakhter Date: Tue, 9 Oct 2018 15:34:00 +0000 Subject: [PATCH 6/8] Added a comment --- .../comment_9_6d1a630fbf036144c9559c9c01fca845._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/todo/Natively_support_s3__58____47____47___urls___40__for_addurl__44___get__44___etc__41__/comment_9_6d1a630fbf036144c9559c9c01fca845._comment diff --git a/doc/todo/Natively_support_s3__58____47____47___urls___40__for_addurl__44___get__44___etc__41__/comment_9_6d1a630fbf036144c9559c9c01fca845._comment b/doc/todo/Natively_support_s3__58____47____47___urls___40__for_addurl__44___get__44___etc__41__/comment_9_6d1a630fbf036144c9559c9c01fca845._comment new file mode 100644 index 0000000000..b01e8b8fc0 --- /dev/null +++ b/doc/todo/Natively_support_s3__58____47____47___urls___40__for_addurl__44___get__44___etc__41__/comment_9_6d1a630fbf036144c9559c9c01fca845._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="Ilya_Shlyakhter" + avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0" + subject="comment 9" + date="2018-10-09T15:33:59Z" + content=""" +@joey If I understand correctly, addurl requires a standard URL downloadable by curl? Would it be possible to add 'adduri' and 'registeruri' counterparts, that would be exactly like addurl/registerurl, except they would be for custom URIs not expected to be fetchable by curl? There seems to be an odd asymmetry, where the external special remote protocol has SETURIPRESENT/SETURLPRESENT, but the command-line only has the URL versions. +"""]] From 44a57ba29398715deb5b21a4187727d2fcb90e5b Mon Sep 17 00:00:00 2001 From: Ilya_Shlyakhter Date: Tue, 9 Oct 2018 18:41:33 +0000 Subject: [PATCH 7/8] Added a comment --- .../comment_4_bd5e45a7e668063df9159033a0857075._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/git-annex-sync/comment_4_bd5e45a7e668063df9159033a0857075._comment diff --git a/doc/git-annex-sync/comment_4_bd5e45a7e668063df9159033a0857075._comment b/doc/git-annex-sync/comment_4_bd5e45a7e668063df9159033a0857075._comment new file mode 100644 index 0000000000..099a403423 --- /dev/null +++ b/doc/git-annex-sync/comment_4_bd5e45a7e668063df9159033a0857075._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="Ilya_Shlyakhter" + avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0" + subject="comment 4" + date="2018-10-09T18:41:33Z" + content=""" +When concurrent git-annex commands on the same repo update that repo's git-annex branch, how is that handled? +"""]] From b8148fefa0fc4de9aa4240750597a6b2e3764237 Mon Sep 17 00:00:00 2001 From: Ilya_Shlyakhter Date: Tue, 9 Oct 2018 18:53:30 +0000 Subject: [PATCH 8/8] Added a comment --- .../comment_10_ed3a47b10ce2303041d9b9271bb9a2c8._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/todo/Natively_support_s3__58____47____47___urls___40__for_addurl__44___get__44___etc__41__/comment_10_ed3a47b10ce2303041d9b9271bb9a2c8._comment diff --git a/doc/todo/Natively_support_s3__58____47____47___urls___40__for_addurl__44___get__44___etc__41__/comment_10_ed3a47b10ce2303041d9b9271bb9a2c8._comment b/doc/todo/Natively_support_s3__58____47____47___urls___40__for_addurl__44___get__44___etc__41__/comment_10_ed3a47b10ce2303041d9b9271bb9a2c8._comment new file mode 100644 index 0000000000..0bbb455cea --- /dev/null +++ b/doc/todo/Natively_support_s3__58____47____47___urls___40__for_addurl__44___get__44___etc__41__/comment_10_ed3a47b10ce2303041d9b9271bb9a2c8._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="Ilya_Shlyakhter" + avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0" + subject="comment 10" + date="2018-10-09T18:53:30Z" + content=""" +\"Remote url claiming iterates through remotes ordered by cost\" -- my web remote has lower cost than my dnanexus external special remote; the latter claims dx:// URLs. But git-annex never seems to ask dnanexus to process dx:// URLs, even when I've manually set the URL to be present there. Does the web remote always win, and if it can't handle a URL does git-annex then not try external special remotes? +"""]]