From 66cb09b47ed1c9664461c23b133539d8a843b3f3 Mon Sep 17 00:00:00 2001 From: yarikoptic Date: Mon, 19 Mar 2018 20:50:40 +0000 Subject: [PATCH] initial whining about 2 ssh prompts --- ...on___34__transient__34___errors__63__.mdwn | 27 +++++++++++++++++++ 1 file changed, 27 insertions(+) create mode 100644 doc/bugs/Add_explicit_retries_for_curl__47__wget_on___34__transient__34___errors__63__.mdwn diff --git a/doc/bugs/Add_explicit_retries_for_curl__47__wget_on___34__transient__34___errors__63__.mdwn b/doc/bugs/Add_explicit_retries_for_curl__47__wget_on___34__transient__34___errors__63__.mdwn new file mode 100644 index 0000000000..4b9c21a75a --- /dev/null +++ b/doc/bugs/Add_explicit_retries_for_curl__47__wget_on___34__transient__34___errors__63__.mdwn @@ -0,0 +1,27 @@ +### Please describe the problem. + +While download content from dropbox (via regular urls associated with the files, so "web" remote), some fail some time with error code returned 500 (internal server failure without explicit reason), which causes git-annex get also to fail if that was the only source for the file to try. + +### What steps will reproduce the problem? + +This is all happening with http://datasets.datalad.org/workshops/nih-2017/ds000114/derivatives/freesurfer/ repository (on eg https://dl.dropboxusercontent.com/s/sn4et1e3d2run9g/rh.aparc.dktatlas.annot?dl=0 url ), but for testing wget/curl I just created http://www.onerussian.com/tmp/errors/500 which would always return 500. + +### What version of git-annex are you using? On what operating system? + +6.20180316+gitg308f3ecf6 + +### Please provide any additional information below. + +here are options for wget and curl which could help us out here: + +[[!format sh """ + +# to make wget retry +> wget --retry-on-http-error=500 http://www.onerussian.com/tmp/errors/500 + +# to make curl retry, just needs --retry 10 which then considers 5xx errors intermittent +> curl --retry 10 http://www.onerussian.com/tmp/errors/500 + +"""]] + +Could git-annex add those options to curl/wget invocations for more robust access to the web remote?