diff --git a/doc/bugs/--debug_is_not_in_effect_if_precedes_-c.mdwn b/doc/bugs/--debug_is_not_in_effect_if_precedes_-c.mdwn new file mode 100644 index 0000000000..ef551b7630 --- /dev/null +++ b/doc/bugs/--debug_is_not_in_effect_if_precedes_-c.mdwn @@ -0,0 +1,67 @@ +### Please describe the problem. + +Identified while troubleshooting another [issue](https://git-annex.branchable.com/bugs/enableremote_stuck_with_a_recentish_git-annex/#comment-2116c5e109aaf39ffd62f3bdeeb14602) + +[[!format sh """ +$> 'git-annex' 'enableremote' --debug -cremote.target1.blah=1 'target1' +enableremote target1 ok + +$> 'git-annex' 'enableremote' -cremote.target1.blah=1 --debug 'target1' +[2020-02-26 14:46:47.789794028] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","remote.target1.blah=1","show-ref","git-annex"] +[2020-02-26 14:46:47.797917978] process done ExitSuccess +[2020-02-26 14:46:47.798350533] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","remote.target1.blah=1","show-ref","--hash","refs/heads/git-annex"] +[2020-02-26 14:46:47.802576899] process done ExitSuccess +[2020-02-26 14:46:47.802884873] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","remote.target1.blah=1","log","refs/heads/git-annex..b1ab0b11fbbc94ffd3d52adb7a0e93c3d45d8b52","--pretty=%H","-n1"] +[2020-02-26 14:46:47.813289406] process done ExitSuccess +[2020-02-26 14:46:47.815873454] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","remote.target1.blah=1","cat-file","--batch"] +[2020-02-26 14:46:47.818598891] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","remote.target1.blah=1","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"] +[2020-02-26 14:46:47.824657055] read: git ["config","--null","--list"] +[2020-02-26 14:46:47.835897478] process done ExitSuccess +enableremote target1 [2020-02-26 14:46:47.83652184] call: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","-c","remote.target1.blah=1","config","remote.target1.annex-ignore","false"] +[2020-02-26 14:46:47.842277017] process done ExitSuccess +[2020-02-26 14:46:47.842703576] read: git ["config","--null","--list"] +[2020-02-26 14:46:47.853478328] process done ExitSuccess +ok +[2020-02-26 14:46:47.855317715] process done ExitSuccess +[2020-02-26 14:46:47.856835556] process done ExitSuccess + +"""]] + +I consider it a bug since options shouldn't be order dependent, and even if they were -- `--debug` is listed before `-c` in `git annex enableremote --help`: + +[[!format sh """ +$> git annex enableremote --help +git-annex enableremote - enables git-annex to use a remote + +Usage: git-annex enableremote [NAME K=V ...] + +Available options: + --force allow actions that may lose annexed data + -F,--fast avoid slow operations + -q,--quiet avoid verbose output + -v,--verbose allow verbose output (default) + -d,--debug show debug messages + --no-debug don't show debug messages + -b,--backend NAME specify key-value backend to use + -N,--numcopies NUMBER override default number of copies + --trust REMOTE override trust setting + --semitrust REMOTE override trust setting back to default + --untrust REMOTE override trust setting to untrusted + -c,--config NAME=VALUE override git configuration setting + --user-agent NAME override default User-Agent + --trust-glacier Trust Amazon Glacier inventory + --notify-finish show desktop notification after transfer finishes + --notify-start show desktop notification after transfer starts + -h,--help Show this help text + +For details, run: git-annex help enableremote + +$> git annex version +git-annex version: 7.20190819+git2-g908476a9b-1~ndall+1 + +"""]] + +[[!meta author=yoh]] +[[!tag projects/datalad]] + + diff --git a/doc/bugs/S3-remote_fail_to_get_data_on_some_clones__47__machines/comment_8_dd40cb01999b4d05db0df84c07b3d67b._comment b/doc/bugs/S3-remote_fail_to_get_data_on_some_clones__47__machines/comment_8_dd40cb01999b4d05db0df84c07b3d67b._comment new file mode 100644 index 0000000000..79fffe6da2 --- /dev/null +++ b/doc/bugs/S3-remote_fail_to_get_data_on_some_clones__47__machines/comment_8_dd40cb01999b4d05db0df84c07b3d67b._comment @@ -0,0 +1,9 @@ +[[!comment format=mdwn + username="basile.pinsard@f1a7fae9f3bd9d5282fca11f62ad53b45a8eb317" + nickname="basile.pinsard" + avatar="http://cdn.libravatar.org/avatar/87e1f73acf277ad0337b90fc0253c62e" + subject="comment 8" + date="2020-02-26T19:11:53Z" + content=""" +The only relevant traffic when running the `git-annex get` is the DNS call for the A record, the answer from the DNS is right (and AAAA record but no IPV6 on our network so 0 results). +"""]] diff --git a/doc/bugs/S3-remote_fail_to_get_data_on_some_clones__47__machines/comment_9_ce0d2cd9b4a1ca205144d294c1e021da._comment b/doc/bugs/S3-remote_fail_to_get_data_on_some_clones__47__machines/comment_9_ce0d2cd9b4a1ca205144d294c1e021da._comment new file mode 100644 index 0000000000..0760801d57 --- /dev/null +++ b/doc/bugs/S3-remote_fail_to_get_data_on_some_clones__47__machines/comment_9_ce0d2cd9b4a1ca205144d294c1e021da._comment @@ -0,0 +1,21 @@ +[[!comment format=mdwn + username="basile.pinsard@f1a7fae9f3bd9d5282fca11f62ad53b45a8eb317" + nickname="basile.pinsard" + avatar="http://cdn.libravatar.org/avatar/87e1f73acf277ad0337b90fc0253c62e" + subject="comment 9" + date="2020-02-26T19:15:11Z" + content=""" +The only difference in the `dig` answer when running from the outside is additional answer sections: +``` +;; AUTHORITY SECTION: +. 3600 IN NS ns22.domaincontrol.com. +. 3600 IN NS ns21.domaincontrol.com. + +;; ADDITIONAL SECTION: +ns22.domaincontrol.com. 21 IN A 173.201.68.11 +ns22.domaincontrol.com. 21 IN AAAA 2603:5:2241::b +ns21.domaincontrol.com. 21 IN A 97.74.100.11 +ns21.domaincontrol.com. 21 IN AAAA 2603:5:2141::b + +``` +"""]] diff --git a/doc/bugs/enableremote_stuck_with_a_recentish_git-annex/comment_2_de71be6bdc853567ad52a05c591b3bb5._comment b/doc/bugs/enableremote_stuck_with_a_recentish_git-annex/comment_2_de71be6bdc853567ad52a05c591b3bb5._comment new file mode 100644 index 0000000000..0e5d4fbdf5 --- /dev/null +++ b/doc/bugs/enableremote_stuck_with_a_recentish_git-annex/comment_2_de71be6bdc853567ad52a05c591b3bb5._comment @@ -0,0 +1,36 @@ +[[!comment format=mdwn + username="yarikoptic" + avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4" + subject="comment 2" + date="2020-02-26T20:14:47Z" + content=""" +thank you for the quick follow up joey. I got confused about + +> --debug output is often useful. ... and maybe a --debug output + +since all `--debug` output from running `enableremote` is present in the log extract I have shown AFAIK... started digging, if `--debug` preceds `-c` -- it is not in effect, filed a [dedicated issue](https://git-annex.branchable.com/bugs/--debug_is_not_in_effect_if_precedes_-c/?updated). + + +I repositioned `--debug` after `-c`. Results will appear later on [travis log](https://travis-ci.org/datalad/datalad/jobs/655516059?utm_medium=notification&utm_source=github_status) and I will update here. + +Re type of remote: it is actually a regular git remote. Before running `enableremote` on my laptop it looks like: + +``` +[remote \"target2\"] + url = ssh://datalad-test/home/yoh/.tmp/datalad_temp_test_publish_dependsfgrfi8tn + fetch = +refs/heads/*:refs/remotes/target2/* + datalad-publish-depends = target1 +``` + +and then after `enableremote`: + +``` +[remote \"target2\"] + url = ssh://datalad-test/home/yoh/.tmp/datalad_temp_test_publish_dependsfgrfi8tn + fetch = +refs/heads/*:refs/remotes/target2/* + datalad-publish-depends = target1 + annex-uuid = 31b92c58-edd1-4675-8b06-7d40f3bcc40d + annex-ignore = false +``` + +"""]] diff --git a/doc/bugs/enableremote_stuck_with_a_recentish_git-annex/comment_3_36b91cb600c0a14dedd58e91e97132d5._comment b/doc/bugs/enableremote_stuck_with_a_recentish_git-annex/comment_3_36b91cb600c0a14dedd58e91e97132d5._comment new file mode 100644 index 0000000000..5e74e198d1 --- /dev/null +++ b/doc/bugs/enableremote_stuck_with_a_recentish_git-annex/comment_3_36b91cb600c0a14dedd58e91e97132d5._comment @@ -0,0 +1,100 @@ +[[!comment format=mdwn + username="yarikoptic" + avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4" + subject="--debug output" + date="2020-02-26T20:36:28Z" + content=""" +
+here is the --debug output (well, logged by datalad, so there is some decoration) for the stuck `enableremote` + +```shell +[DEBUG ] Async run ['git-annex', 'enableremote', '-c', 'remote.target1.annex-ssh-options=-o ControlMaster=auto -S /home/travis/.cache/datalad/sockets/44786c35', 'target2', '--debug'] +[DEBUG ] Launching process ['git-annex', 'enableremote', '-c', 'remote.target1.annex-ssh-options=-o ControlMaster=auto -S /home/travis/.cache/datalad/sockets/44786c35', 'target2', '--debug'] +[DEBUG ] Process 16886 started +[DEBUG ] Waiting for process 16886 to complete +[Level 5] Read 232 bytes from 16886[stderr]: +[Level 9] stderr| [2020-02-26 20:21:00.411270506] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"-c\",\"remote.target1.annex-ssh-options=-o ControlMaster=auto -S /home/travis/.cache/datalad/sockets/44786c35\",\"show-ref\",\"git-annex\"] +| +[Level 5] Read 309 bytes from 16886[stderr]: +[Level 9] stderr| [2020-02-26 20:21:00.414030967] process done ExitSuccess +| [2020-02-26 20:21:00.414120854] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"-c\",\"remote.target1.annex-ssh-options=-o ControlMaster=auto -S /home/travis/.cache/datalad/sockets/44786c35\",\"show-ref\",\"--hash\",\"refs/heads/git-annex\"] +| +[Level 5] Read 57 bytes from 16886[stderr]: +[Level 9] stderr| [2020-02-26 20:21:00.416528464] process done ExitSuccess +| +[Level 5] Read 300 bytes from 16886[stderr]: +[Level 9] stderr| [2020-02-26 20:21:00.416774225] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"-c\",\"remote.target1.annex-ssh-options=-o ControlMaster=auto -S /home/travis/.cache/datalad/sockets/44786c35\",\"log\",\"refs/heads/git-annex..4205e0fe09bdb15c518d8475778452b26a0b931a\",\"--pretty=%H\",\"-n1\"] +| +[Level 5] Read 57 bytes from 16886[stderr]: +[Level 9] stderr| [2020-02-26 20:21:00.420055075] process done ExitSuccess +| +[Level 5] Read 300 bytes from 16886[stderr]: +[Level 9] stderr| [2020-02-26 20:21:00.420146676] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"-c\",\"remote.target1.annex-ssh-options=-o ControlMaster=auto -S /home/travis/.cache/datalad/sockets/44786c35\",\"log\",\"refs/heads/git-annex..e7c5d7333e2ba07d8eb6a4ddadd9affae13c40eb\",\"--pretty=%H\",\"-n1\"] +| +[Level 5] Read 56 bytes from 16886[stderr]: +[Level 9] stderr| [2020-02-26 20:21:00.42378637] process done ExitSuccess +| +[Level 5] Read 230 bytes from 16886[stderr]: +[Level 9] stderr| [2020-02-26 20:21:00.430135272] chat: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"-c\",\"remote.target1.annex-ssh-options=-o ControlMaster=auto -S /home/travis/.cache/datalad/sockets/44786c35\",\"cat-file\",\"--batch\"] +| +[Level 5] Read 278 bytes from 16886[stderr]: +[Level 9] stderr| [2020-02-26 20:21:00.432035636] chat: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"-c\",\"remote.target1.annex-ssh-options=-o ControlMaster=auto -S /home/travis/.cache/datalad/sockets/44786c35\",\"cat-file\",\"--batch-check=%(objectname) %(objecttype) %(objectsize)\"] +| +[Level 5] Read 46 bytes from 16886[stdout]: +[Level 9] stdout| (merging target2/git-annex into git-annex...) +| +[Level 5] Read 259 bytes from 16886[stderr]: +[Level 9] stderr| [2020-02-26 20:21:00.435904786] chat: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"-c\",\"remote.target1.annex-ssh-options=-o ControlMaster=auto -S /home/travis/.cache/datalad/sockets/44786c35\",\"hash-object\",\"-w\",\"--stdin-paths\",\"--no-filters\"] +| +[Level 5] Read 244 bytes from 16886[stderr]: +[Level 9] stderr| [2020-02-26 20:21:00.438156653] feed: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"-c\",\"remote.target1.annex-ssh-options=-o ControlMaster=auto -S /home/travis/.cache/datalad/sockets/44786c35\",\"update-index\",\"-z\",\"--index-info\"] +| +[Level 5] Read 320 bytes from 16886[stderr]: +[Level 9] stderr| [2020-02-26 20:21:00.440579446] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"-c\",\"remote.target1.annex-ssh-options=-o ControlMaster=auto -S /home/travis/.cache/datalad/sockets/44786c35\",\"diff-index\",\"--raw\",\"-z\",\"-r\",\"--no-renames\",\"-l0\",\"--cached\",\"e7c5d7333e2ba07d8eb6a4ddadd9affae13c40eb\",\"--\"] +| +[Level 5] Read 57 bytes from 16886[stderr]: +[Level 9] stderr| [2020-02-26 20:21:00.444964246] process done ExitSuccess +| +[Level 5] Read 56 bytes from 16886[stderr]: +[Level 9] stderr| [2020-02-26 20:21:00.44534127] process done ExitSuccess +| +[Level 5] Read 300 bytes from 16886[stderr]: +[Level 9] stderr| [2020-02-26 20:21:00.447359547] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"-c\",\"remote.target1.annex-ssh-options=-o ControlMaster=auto -S /home/travis/.cache/datalad/sockets/44786c35\",\"log\",\"e7c5d7333e2ba07d8eb6a4ddadd9affae13c40eb..refs/heads/git-annex\",\"--pretty=%H\",\"-n1\"] +| +[Level 5] Read 57 bytes from 16886[stderr]: +[Level 9] stderr| [2020-02-26 20:21:00.450117916] process done ExitSuccess +| +[Level 5] Read 28 bytes from 16886[stdout]: +[Level 9] stdout| (recording state in git...) +| +[Level 5] Read 221 bytes from 16886[stderr]: +[Level 9] stderr| [2020-02-26 20:21:00.45026688] read: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"-c\",\"remote.target1.annex-ssh-options=-o ControlMaster=auto -S /home/travis/.cache/datalad/sockets/44786c35\",\"write-tree\"] +| +[Level 5] Read 415 bytes from 16886[stderr]: +[Level 9] stderr| [2020-02-26 20:21:00.453577825] process done ExitSuccess +| [2020-02-26 20:21:00.453656296] chat: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"-c\",\"remote.target1.annex-ssh-options=-o ControlMaster=auto -S /home/travis/.cache/datalad/sockets/44786c35\",\"commit-tree\",\"bc6e2bf931ad9e846895d60189f9af41d9705724\",\"--no-gpg-sign\",\"-p\",\"refs/heads/git-annex\",\"-p\",\"e7c5d7333e2ba07d8eb6a4ddadd9affae13c40eb\"] +| +[Level 5] Read 57 bytes from 16886[stderr]: +[Level 9] stderr| [2020-02-26 20:21:00.456300928] process done ExitSuccess +| +[Level 5] Read 287 bytes from 16886[stderr]: +[Level 9] stderr| [2020-02-26 20:21:00.45636449] call: git [\"--git-dir=.git\",\"--work-tree=.\",\"--literal-pathspecs\",\"-c\",\"remote.target1.annex-ssh-options=-o ControlMaster=auto -S /home/travis/.cache/datalad/sockets/44786c35\",\"update-ref\",\"refs/heads/git-annex\",\"c06d34b848f1a4095d0e2d59bf36b96469e168ea\"] +| +[Level 5] Read 57 bytes from 16886[stderr]: +[Level 9] stderr| [2020-02-26 20:21:00.459600142] process done ExitSuccess +| +[Level 5] Read 71 bytes from 16886[stderr]: +[Level 9] stderr| [2020-02-26 20:21:00.460816011] read: git [\"config\",\"--null\",\"--list\"] +| +[Level 5] Read 57 bytes from 16886[stderr]: +[Level 9] stderr| [2020-02-26 20:21:00.462948244] process done ExitSuccess +| +[Level 5] Read 246 bytes from 16886[stderr]: +[Level 9] stderr| [2020-02-26 20:21:00.464013355] read: ssh [\"datalad-test\",\"-S\",\".git/annex/ssh/datalad-test\",\"-o\",\"ControlMaster=auto\",\"-o\",\"ControlPersist=yes\",\"-n\",\"-T\",\"git-annex-shell 'configlist' '/tmp/datalad_temp_test_publish_depends_q8i1lpv' '--debug'\"] +| +No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself. +Check the details on how to adjust your build configuration on: https://docs.travis-ci.com/user/common-build-problems/#build-times-out-because-no-output-was-received +``` +
+ +"""]] diff --git a/doc/bugs/fails_to___96__get__96___in_parallel_for_a_freshly_clone_from_public_s3_bucket_where_versioning_info_was_forgotten/comment_5_6c1e74df4c7e566c4c1ba43fbbbd3beb._comment b/doc/bugs/fails_to___96__get__96___in_parallel_for_a_freshly_clone_from_public_s3_bucket_where_versioning_info_was_forgotten/comment_5_6c1e74df4c7e566c4c1ba43fbbbd3beb._comment new file mode 100644 index 0000000000..11704aa75f --- /dev/null +++ b/doc/bugs/fails_to___96__get__96___in_parallel_for_a_freshly_clone_from_public_s3_bucket_where_versioning_info_was_forgotten/comment_5_6c1e74df4c7e566c4c1ba43fbbbd3beb._comment @@ -0,0 +1,14 @@ +[[!comment format=mdwn + username="yarikoptic" + avatar="http://cdn.libravatar.org/avatar/f11e9c84cb18d26a1748c33b48c924b4" + subject="comment 5" + date="2020-02-26T20:58:16Z" + content=""" +> How do I produce such a repo, I thought that git-annex has fixed the problem that made it not include the S3 versioning information? I don't want to see a lot of repos being created with that information missing. + +Sorry if I was not clear. Those repositories were produced in 2018 when available/used version of git annex was not yet \"fixed\". Newer datasets, have .rmet files, e.g. just checked https://github.com/OpenNeuroDatasets/ds002596 (full of .rmet files in git-annex branch ;)) + +> Easily fixed. + +Thanks. FTR in 7.20200219-129-ge52034150 +"""]]