Merge branch 'master' of ssh://git-annex.branchable.com

This commit is contained in:
Joey Hess 2015-12-04 18:05:18 -04:00
commit 1b597cbee6
Failed to extract signature
6 changed files with 84 additions and 0 deletions

View file

@ -0,0 +1,11 @@
[[!comment format=mdwn
username="OlivierBerger"
subject="Difficult to tell"
date="2015-12-04T10:21:26Z"
content="""
Sorry, Joey, but I don't have the details of the version which crahed on us, and we updated afterwards : no time to try and reproduce fully : really busy producing videos, and anxious to find a solution on backing up videos reliably (which by the way I solved like this : https://www-public.tem-tsp.eu/~berger_o/weblog/2015/11/26/handling-video-files-produced-for-a-mooc-on-windows-with-git-and-git-annex/).
I don't see however how we could have touched the contents of the .git/ : we were only doing Windows explorer actions : dragging files here and there, renaming dirs, saving stuff in Camtasia, but all inside subdirs of the git clone.
Sorry for the poor quality of the report, but we were in a real hurry and had to find solutions without distracting us from the main topic : video editing :-/
"""]]

View file

@ -0,0 +1,7 @@
[[!comment format=mdwn
username="OlivierBerger"
subject="Again, difficult to tell"
date="2015-12-04T12:24:06Z"
content="""
Well, I really don't understand either what may have caused the crashes... but the path issue might be a hint. I think our repo was cloned in a path something like : \"C:\Users\Majdi\git\git-annex.tem-tsp.eu\testscpo\" which already takes around 48 chars.
"""]]

View file

@ -0,0 +1,36 @@
### Please describe the problem.
git-annex's WebDAV support does not like (aka it does not work) the WebDAV server of the freenet cloud.
### What steps will reproduce the problem?
My first attempt was:
WEBDAV_USERNAME='XXX' WEBDAV_PASSWORD='XXX' git annex initremote webdav type=webdav url='https://webmail.freenet.de/webdav' encryption=none
initremote webdav (testing WebDAV server...)
git-annex: WebDAV test failed: StatusCodeException (Status {statusCode = 401, statusMessage = "Unauthorized"}) [("Date","Fri, 04 Dec 2015 12:20:57 GMT"),("Server","Apache/2.2.16 (Debian)"),("WWW-Authenticate","Basic realm=\"MD-Cloud\""),
("Vary","Accept-Encoding"),("Content-Encoding","gzip"),("Content-Length","20"),("Connection","close"),("Content-Type","text/html; charset=iso-8859-15"),("X-Response-Body-Start",""),("X-Request-URL","MKCOL https://webmail.freenet.de:443/webdav/tmp")]
(CJ {expose = []}): user error failed
git-annex: initremote: 1 failed
Ok this fails (what is the error?). However, it does create a folder "tmp" in the "cloud". A second attempt yields another error:
WEBDAV_USERNAME='XXX' WEBDAV_PASSWORD='XXX' git annex initremote webdav type=webdav url='https://webmail.freenet.de/webdav' encryption=none
initremote webdav (testing WebDAV server...)
git-annex: WebDAV test failed: StatusCodeException (Status {statusCode = 501, statusMessage = "Not Implemented"}) [("Date","Fri, 04 Dec 2015 12:21:22 GMT"),("Server","Apache/2.2.16 (Debian)"),("Content-Length","349"),("Connection","close"),
("Content-Type","application/xml; charset=utf-8"),("X-Response-Body-Start","<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<d:error xmlns:d=\"DAV:\" xmlns:s=\"http://sabredav.org/ns\">\n <s:exception>Sabre\\DAV\\Exception\\NotImplemented</s:exception>\n
<s:message>This server is not compatible with OS/X finder. Consider using a different WebDAV client or webserver.</s:message>\n <s:sabredav-version>1.8.6</s:sabredav-version>\n</d:error>\n"),("X-Request-URL","PUT https://webmail.freenet.de:443/webdav
/tmp/git-annex-test")] (CJ {expose = []}): user error failed
git-annex: initremote: 1 failed
which is I guess the same. The WebDAV server does support writing and locking files. I tried writing using davfs2 and locking-unlocking using cadaver. I guess in the end that it's the server's fault, but it would be great to know what exactly fails at this point :).
### What version of git-annex are you using? On what operating system?
git-annex version: 5.20151116-gbe86081
Gentoo Linux
### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders)
First time trying WebDAV support.

View file

@ -0,0 +1,9 @@
I am interested in using `git annex` to manage encrypted backups to Amazon S3/Glacier. So `git annex` will be used with the main file directory in direct mode and an encrypted S3 or Glacier remote set up in archive mode and then `git annex add .` and `git annex sync` will be run periodically. The intent is for this set up to be a backup for catastrophic failure, so I want to make sure I take care of future-proofing and disaster recovery properly. So my basic question is what would I need to have backed up and what would I have to do if the computer with the main repository died. I try to break that out into more specific questions below.
0. Do the S3/Glacier remotes just store the contents of `.git/annex/objects` in encrypted form and nothing else? So if I was left with nothing but the AWS bucket and couldn't get `git annex` to work for whatever reason, I could recover my files by hand if I had the encryption key (though I wouldn't know the file names or directory structure)?
1. For `shared` encryption, I see the cipher text in `remote.log` in the `git-annex` branch. Assuming I didn't have access to `git annex`, what would I need to do to convert that cipher text into a form that I could use with `gpg` to decrypt files?
2. Same question but for `hybrid` encryption rather than `shared`. I assume the answer is similar but I need to decrypt the cipher first with my gpg key? How do I do that?
3. Assuming I did have access to `git annex`, what would I need to create a new repo on a new computer with access to all of the files in the S3/Glacier bucket? I think I would need my Amazon credentials, my gpg key if using hybrid or public key encryption, and the `.git` folder as it was the last time files were pushed to the S3/Glacier remote (which would have the necessary decryption information for shared encryption). Is that right? I guess mainly I am checking that the remote does not store any metadata about the repo, so for `git annex` to be able to pull files back out I would need a backup of the `.git` directory and that back up would need to be up to date (can't just copy remote.log and have `git annex` work out the rest from the remote's contents). So for a full backup, my script would need to `tar` the `.git` directory, encrypt it, and push it to S3/Glacier separately after `git annex` does a sync. Then I could recover everything as long as I had a secure backup of my Amazon credentials and my encryption key(s).

View file

@ -0,0 +1,13 @@
I have a setup with a windows desktop and a linux laptop with both an annex repository, interconnected via a backup repository in the middle (connected via ssh).
Both computers are setup as clients and all 3 machines are running the assistant.
Today i tried creating an archive folder and a putting some files in it. To my understanding those files should have been dropped in the 2 machines but kept on the server.. instead I have them copied on each machine. Listing where the files are on both computers shows that both know that themselves and the server have a copy, but they don't know anything about each other.
AFAIK it looks like the bug reported in https://git-annex.branchable.com/forum/question_about_assistant_and___47__archive__47__/
laptop: git-annex version: 5.20151116+gitg5416a1a-1~ndall+1
server: git-annex version: 5.20151109-g08bb3b1
windows: the latest build on the website

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="encryptio@2557a3f5b4ef0cddf8f011d7dbb3e76d46ed9c79"
nickname="encryptio"
subject="git-annex-remote-b2"
date="2015-12-04T03:19:29Z"
content="""
I wrote an external remote for [Backblaze's B2](https://www.backblaze.com/b2/cloud-storage.html) object storage service: [git-annex-remote-b2](https://github.com/encryptio/git-annex-remote-b2). I can't be arsed to write a full page on it, but the README covers everything an experienced git-annex user needs to know to use it.
"""]]