Merge branch 'master' of ssh://git-annex.branchable.com

This commit is contained in:
Joey Hess 2015-01-26 14:43:53 -04:00
commit e66cc74031
15 changed files with 199 additions and 2 deletions

View file

@ -0,0 +1 @@
Currently, in a git annex view, filenames take the form basename%path%.ext. I understand that this is a carefully drafted mapping to allow changes to be merged back into metadata. However, maybe it would be possible to make the separator ('%') and the order (e.g. path%basename.ext instead) configurable?

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="https://www.google.com/accounts/o8/id?id=AItOawkXzY87FdO942LR-qNyJFeCth0LjWORNuw"
nickname="Rob"
subject="Proposed resolution"
date="2015-01-24T17:18:14Z"
content="""
see [my other bug](https://git-annex.branchable.com/bugs/How_to_use_a_DRA_bucket_in_Google_cloud_storage__63__/) for a proposed resolution: a GCS-optimized external special remote
"""]]

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="https://www.google.com/accounts/o8/id?id=AItOawkXzY87FdO942LR-qNyJFeCth0LjWORNuw"
nickname="Rob"
subject="proposed resolution"
date="2015-01-24T17:10:20Z"
content="""
I believe that I could write an external special remote to enable a full GCS feature set to resolve this. I'll do this. If someone has already done so, please add to the comments.
"""]]

View file

@ -0,0 +1,7 @@
[[!comment format=mdwn
username="anarcat"
subject="still an issue"
date="2015-01-26T17:27:25Z"
content="""
this is still a major annoyance here. so much so that i disabled the assistant completely here. i ended up blacklisting myself from my own server... repeatedly.
"""]]

View file

@ -0,0 +1,30 @@
### Please describe the problem.
git-annex dies of signal 11 when syncing content with Box.
I noticed that the assistant stopped syncing after a few files, so I tried to sync manually through the terminal and saw the error.
### What steps will reproduce the problem?
$ git annex sync box.com --content
commit ok
copy myfile1 (checking box.com...) (to box.com...)
ok
copy myfile2 (checking box.com...) (to box.com...)
ok
copy myfile3 (checking box.com...) (to box.com...)
ok
copy myfile4 (checking box.com...) (to box.com...)
ok
copy myfile5 (checking box.com...) error: git-annex died of signal 11
The number of files that are copied correctly is quite random, some times it syncs 200 files and other times only 2 or 3.
### What version of git-annex are you using? On what operating system?
5.20141125 on Debian Sid (everything install from official packages)
### Please provide any additional information below.
I don't know how to get more details of the problem, I haven't manage to get a more precise error activating logs or verbose mode, if there's anything I can do to debug the problem I'll be more than willing.

13
doc/forum/.mdwn Normal file
View file

@ -0,0 +1,13 @@
Hi!
I just discovered *git-annex* as a great way to move away from Dropbox, but there are some open questions I couldn't find answers for and hope someone will be able to answer them here:
I didn't completely understand how remotes are treated. As far as I understand there are two different parts to sync, the metadata (which is stored in a regular git repository) and the regular file data (or only chunks of it?). When syncing, I can choose whether I want to only sync the git (-metadata) part, or if I want to include the file contents, right?
1. Is it possible to sync only a *part* of the file contents to certain remotes? For example I would want all my ebooks and photos to be synced to my remote server, but wouldn't want my private accounting and tax stuff to sync over the internet.
2. I have several (regular) git repositories used for e.g. development. Can I store a regular (bare) git repository within a git-annex repository? Does this make sense, or should I (for whatever reason) avoid it?
3. A great feature of the Dropbox app is the automatic upload of photos taken with my iPhone. I would like to adopt this kind of backup. So considering a remotely accessible server. Is there some way to make use of *special remotes* like WebDAV to allow uploads over a *non-git interface*?
4. Is there a way to access files (read-only) over a non-git interface from some git-annex remote? So could I access e.g. some pdf document from a git-annex repository, when there is a WebDAV special remote? Or do I just misunderstand the concept of *special remotes*? Is there another (useful) way? Maybe some kind of (remotely accessible) web interface?
5. Why is git-annex [not a backup system](http://git-annex.branchable.com/not/)?
Thank you in advance!

View file

@ -0,0 +1,28 @@
I just created a gcrypt special remote and got an error from (I think) `git-annex-shell`, though my local git-annex didn't complain and seemed to think the creation went okay.
annex $ ga initremote ma type=gcrypt encryption=hybrid gitrepo=ssh://ma/meta/s/spw/local/annex keyid=3B6D411B
(merging synced/git-annex into git-annex...)
(Recording state in git...)
initremote ma (encryption setup) (hybrid cipher with gpg key 0F56D0553B6D411B) gcrypt: Development version -- Repository format MAY CHANGE
gcrypt: Repository not found: ssh://ma/meta/s/spw/local/annex
gcrypt: Development version -- Repository format MAY CHANGE
gcrypt: Repository not found: ssh://ma/meta/s/spw/local/annex
gcrypt: Setting up new repository
gcrypt: Remote ID is :id:icUy+qU392R5L5H7g8Yk
Counting objects: 111456, done.
Compressing objects: 100% (74045/74045), done.
Total 111456 (delta 59312), reused 85857 (delta 33717)
gcrypt: Encrypting to: -r 0F56D0553B6D411B
gcrypt: Requesting manifest signature
To gcrypt::ssh://ma/meta/s/spw/local/annex
* [new branch] git-annex -> git-annex
fatal: What do you think I am? A shell?
git-annex-shell: git-shell failed
ok
(Recording state in git...)
Further `git annex sync ma` runs go off without a hitch and the master branch gets pushed a synced/master, though it didn't get pushed on this initial creation as can be seen from the above output.
Is it a bug that my local git-annex said "ok" when there was an error, or is this just a case of my local git-annex falling back to rsync instead of git-annex-shell because I didn't set up my $PATH properly on the remote? Should I be worried that my gcrypt repo wasn't set up correctly and thus can't be trusted to hold my annexed data and metadata? Thanks.
Local git-annex: 5.20141125. Remote git-annex (standalone build): 5.20150113-gcf247cf.

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="https://www.google.com/accounts/o8/id?id=AItOawntodrSgODU27WUCyN2PV7TC14YMkyaoxQ"
nickname="Dennis"
subject="Ubuntu Trusty 14.04"
date="2015-01-24T03:19:41Z"
content="""
I am running into the same issue with Ubuntu Trusty 14.04. The version shipped is 5.20140412ubuntu1, but I would like to try a more recent version to see if an issue I experience still persists. Is there any up-to-date PPA available for Trusty?
"""]]

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="https://www.google.com/accounts/o8/id?id=AItOawntodrSgODU27WUCyN2PV7TC14YMkyaoxQ"
nickname="Dennis"
subject="Ubuntu Trusty 14.04"
date="2015-01-24T03:20:48Z"
content="""
I am searching for current packages for Ubuntu Trusty 14.04. The version shipped is 5.20140412ubuntu1, but I would like to try a more recent version to see if an issue I experience still persists. Is there any up-to-date PPA available for Trusty?
"""]]

View file

@ -0,0 +1,14 @@
[[!comment format=mdwn
username="https://www.google.com/accounts/o8/id?id=AItOawm_YXzEdPHzbSGVwtmTR7g1BqDtTnIBB5s"
nickname="Matthias"
subject="comment 17"
date="2015-01-22T22:04:09Z"
content="""
git sync … >> fetches from each remote
Well, I have two git annex-ed repositories where \"git remote -v\" properly lists the other repo, and \"git annex sync foo\" manages to pull from foo, but \"git annex sync\" without a remote name simply does a local sync. Also, neither command pushes anything anywhere.
So, where does \"git annex\" get its list of remotes from? What could prevent it from accessing them?
"""]]

View file

@ -0,0 +1,26 @@
[[!comment format=mdwn
username="https://www.google.com/accounts/o8/id?id=AItOawnjDzZkA6SzZHgT5_Jy3Rw3s7m_W3-oNLY"
nickname="Antoni"
subject="Process seems to hang on Synology"
date="2015-01-22T20:18:12Z"
content="""
I think I've completed the setup correctly, as I
* am able to execute git-annex-shell on the server
* locally running ``git-annex sync`` triggers the git-annex process on the server
However, once I provide the password to the key on my local machine, nothing seemingly happens - there is no output after the password prompt.
The repository I'm trying to sync with the remote on the server is ~200 mb and is in my lan.
When monitoring the remote server, I see the following in process being triggered:
````
31433 31431 amk D 656m270.7 7.1 /volume1/homes/amk/bin/git-annex.linux//lib/ld-linux.so.3 --library-path /volume1/homes/amk/bin/git-annex.linux//etc/ld.so.conf.d:/volume1/homes/amk/bin/git-annex.linux//usr/lib/arm-linux-gnueabi/audit:/volume1/homes/amk/bin/git-annex.linux//usr/lib/arm-linux-gnueabi/gconv:/volume1/homes/amk/bin/git-annex.linux//usr/lib:/volume1/homes/amk/bin/git-annex.linux//usr/lib/arm-linux-gnueabi:/volume1/homes/amk/bin/git-annex.linux//lib:/volume1/homes/amk/bin/git-annex.linux//lib/arm-linux-gnueabi: /volume1/homes/amk/bin/git-annex.linux/shimmed/git-annex-shell/git-annex-shell -c git-annex-shell 'configlist' '/volume1/homes/amk/archive/annex'
````
The memory usage goes up graudally, reaching well over 200mb, hogging the cpu significantly. I have never used git-annex before, so I'm not sure what else I should be looking for. This is what the server runs on:
``armv7l GNU/Linux synology_armada370_214se``
Is this an issue caused by the host architecture mismatch?
"""]]

View file

@ -0,0 +1,10 @@
[[!comment format=mdwn
username="https://www.google.com/accounts/o8/id?id=AItOawn8d6WBPNtmyeANT-gjlN19QnCTQ0psRPs"
nickname="James"
subject="S3 special remote?"
date="2015-01-22T21:52:13Z"
content="""
Is it possible to use special remotes (like S3) with this centralized method?
Public web content works fine, but I've had no luck with S3.
"""]]

View file

@ -0,0 +1,24 @@
[[!comment format=mdwn
username="https://www.google.com/accounts/o8/id?id=AItOawn8d6WBPNtmyeANT-gjlN19QnCTQ0psRPs"
nickname="James"
subject="comment 11"
date="2015-01-22T22:16:46Z"
content="""
Even after enableremote I can't get from s3.
get Docs.zip (from s3...)
Unable to access these remotes: s3
Try making some of these repositories available:
02b4e373-02b1-42b9-8285-c353c047895c -- my laptop
f287b4f7-4e5a-4734-bd6b-62b1f4498838 -- [s3]
(Note that these git remotes have annex-ignore set: origin)
failed
git-annex: get: 1 failed
This is after all branches are pushed from my original repo. Any suggestions?
"""]]

View file

@ -0,0 +1,12 @@
[[!comment format=mdwn
username="https://www.google.com/accounts/o8/id?id=AItOawn8d6WBPNtmyeANT-gjlN19QnCTQ0psRPs"
nickname="James"
subject="RE: last comment"
date="2015-01-23T01:02:18Z"
content="""
RE: my last comment
The reason I couldn't get it to work is because I didn't have proper read access to the bucket. My bad for not checking first but it would be great it there was a clearer error message from git-annex and/or a way to get more detailed information on the s3 extension (-d doesn't do much).
Regardless git-annex is pretty cool, thanks to all the maintainers for their hard work.
"""]]

View file

@ -19,7 +19,7 @@ even if the world has forgotten about git-annex and git.
<small>[[more about future-proofing|future_proofing]]</small>
Run in a cron job, git-annex adds new files to archival drives at night. It
also helps Bob keep track of intentional, and unintentional copies of
files, and logs information he can use to decide when it's time to duplicate
also helps Bob keep track of intentional and unintentional copies of
files and logs information he can use to decide when it's time to duplicate
the content of old drives.
<small>[[more about backup copies|copies]]</small>