Merge branch 'master' of ssh://git-annex.branchable.com

This commit is contained in:
Joey Hess 2018-10-09 15:58:45 -04:00
commit bd24814d42
No known key found for this signature in database
GPG key ID: DB12DB0FF05F8F38
7 changed files with 69 additions and 0 deletions

View file

@ -0,0 +1,19 @@
Hi there,
I've got a problem similar to here: [bugs/Assistant_only_watches_one_repo_on_startup/](https://git-annex.branchable.com/bugs/Assistant_only_watches_one_repo_on_startup/) but in **windows 7**.
I'm running the latest version and my autostart config file (~/.config/git-annex/autostart) contains the following repos:
&nbsp;&nbsp;&nbsp;&nbsp;C:\temp\testrepo1<br>
&nbsp;&nbsp;&nbsp;&nbsp;C:\temp\testrepo2
From a git bash shell, I launch the assistant using the following command: `git annex assistant --autostart`.
Upon startup, it outputs the following message:
&nbsp;&nbsp;&nbsp;&nbsp;git-annex autostart in C:\temp\testrepo1
I then copy a test file to both repos.
In testrepo1, the test file is automatically added to the repo. While in testrepo2, running `git annex status` shows that file has not been processed.
Which means that the second repo is not being watched by the assistant.

View file

@ -0,0 +1,9 @@
[[!comment format=mdwn
username="Ilya_Shlyakhter"
avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
subject="comment 4"
date="2018-10-08T21:43:17Z"
content="""
@joey thanks. But, besides export.log, the S3 remote also keeps some (undocumented?) internal state, and there's not way to update that state to record the fact that git-annex can GET a given key by downloading s3://mybucket/myobject ? Also, I feel uneasy directly manipulating git-annex internal files. Can you think of any plumbing commands, that could be added to support this use case?
The use case is, I submit a batch job that takes as input some s3:// objects, writes outputs to other s3:// objects, and returns pointers to these new s3:// objects. I want to register these new objects in git-annex, initially without downloading them, but be able to git-annex-get these objects, drop them from the S3 remote, but later be able to put them back under their original s3:// URIs. The latter ability is needed because (1) many workflows expect filenames to be in a particular form, e.g. mysamplename.pN.bam to represent mysample processed with parameter p=N; and (2) some workflow engines can reuse past results if a step is re-run with the same inputs, but they need the results to be at the same s3:// URI as when the step was first run.
"""]]

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="Ilya_Shlyakhter"
avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
subject="comment 4"
date="2018-10-09T18:41:33Z"
content="""
When concurrent git-annex commands on the same repo update that repo's git-annex branch, how is that handled?
"""]]

View file

@ -0,0 +1,9 @@
[[!comment format=mdwn
username="branchable@bafd175a4b99afd6ed72501042e364ebd3e0c45e"
nickname="branchable"
avatar="http://cdn.libravatar.org/avatar/ae41dba34ee6000056f00793c695be75"
subject="aarch64 still broken, please help"
date="2018-10-09T09:17:11Z"
content="""
@joey, any update on this signal 11 issue? Currently we aarch64 users are completely stuck, AFAICS. Thanks!
"""]]

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="Ilya_Shlyakhter"
avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
subject="comment 10"
date="2018-10-09T18:53:30Z"
content="""
\"Remote url claiming iterates through remotes ordered by cost\" -- my web remote has lower cost than my dnanexus external special remote; the latter claims dx:// URLs. But git-annex never seems to ask dnanexus to process dx:// URLs, even when I've manually set the URL to be present there. Does the web remote always win, and if it can't handle a URL does git-annex then not try external special remotes?
"""]]

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="Ilya_Shlyakhter"
avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
subject="comment 9"
date="2018-10-09T15:33:59Z"
content="""
@joey If I understand correctly, addurl requires a standard URL downloadable by curl? Would it be possible to add 'adduri' and 'registeruri' counterparts, that would be exactly like addurl/registerurl, except they would be for custom URIs not expected to be fetchable by curl? There seems to be an odd asymmetry, where the external special remote protocol has SETURIPRESENT/SETURLPRESENT, but the command-line only has the URL versions.
"""]]

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="Ilya_Shlyakhter"
avatar="http://cdn.libravatar.org/avatar/1647044369aa7747829c38b9dcc84df0"
subject="comment 5"
date="2018-10-08T21:48:27Z"
content="""
\"why run 64 processes on a 64 CPU system if it has the same speed bus / same network speed as a 4 CPU system\" -- it doesn't always; higher-end AWS instances have 25Gbps networks, and you can buy more throughput as needed.
"""]]