Merge branch 'master' of ssh://git-annex.branchable.com

This commit is contained in:
Joey Hess 2016-01-18 13:03:48 -04:00
commit daf852ede1
Failed to extract signature
13 changed files with 136 additions and 1 deletions

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="wsha.code+ga@b38779424f41c5701bbe5937340be43ff1474b2d"
nickname="wsha.code+ga"
subject="comment 6"
date="2016-01-16T12:20:05Z"
content="""
I just upgraded to 6.20160114-g297a744. `git annex version` shows me `S3(multipartupload)`, and trying to set `storageclass=STANDARD_IA` still leaves the storageclass as `Standard`. Would it show `S3(multipartupload,storageclasses)` if it had `STANDARD_IA` support from `aws-0.13`? I see that `aws-0.13.0` was added to Debian unstable on 2016/01/11 -- was that too close to make the 2016/01/14 release?
"""]]

View file

@ -0,0 +1,7 @@
[[!comment format=mdwn
username="https://me.yahoo.com/a/EbvxpTI_xP9Aod7Mg4cwGhgjrCrdM5s-#7c0f4"
subject="comment 3"
date="2016-01-15T21:11:24Z"
content="""
For clarity -- I was devnulling (in 2nd example) just to show what stdout (which is what to be used for getting status) spits out. Thanks for fixing it -- I will check it out asap
"""]]

View file

@ -0,0 +1,15 @@
[[!comment format=mdwn
username="https://me.yahoo.com/a/EbvxpTI_xP9Aod7Mg4cwGhgjrCrdM5s-#7c0f4"
subject="comment 2"
date="2016-01-15T21:27:55Z"
content="""
would it be possible to get something like
[[!format sh \"\"\"
git annex evaluate-expression [--size X] [--filename Y] [--mime Z] [--file F] [--repository A and so on] ... EXPRESSION
\"\"\"]]
which, given the EXPRESSION (as the one for largefiles) would use provided information and if necessary (e.g. size is not provided but needed) consult --file value, and return 0 if condition satisfied, 1 if unsatisfied, 2 if no sufficient information provided (e.g. --file was needed since --size was not provided but used by the condition)? ;-)
I understand that you are aiming for the most flexible and generic solution. in my case I am aiming for a less generic use-case, e.g. when I still use largefiles for a repository which in general links to the content using --fast (or --relaxed) but where I need to add some files (e.g. *.txt) straight into git. I thought to resort to largefiles handling within git-annex for that instead of creating my own specifications (which files to add to git or to annex based on name and/or size)
"""]]

View file

@ -0,0 +1,7 @@
[[!comment format=mdwn
username="spwhitton"
subject="comment 1"
date="2016-01-16T05:14:40Z"
content="""
Congratulations!
"""]]

View file

@ -0,0 +1,10 @@
Hi,
I have been attempting to use `git annex` for archiving about 300gb of data (a usb disk that is a copy of a laptop drive). I have an S3 special remote setup as a backup remote right now (the plan is once I feel like the data is safe on s3 I will change the remote type to archive and get rid of the local copies). I am using the WORM backend. I think something has gone wrong:
1. As of right now I have about 200mb of data on S3, but 'git annex copy --to S3 .' runs for a while, then exits, without any indication that it has uploaded any additional files.
2. .git/ is around 200gb in size
3. 'git annex add .' adds files for a while, then exits, and does the same thing when run again.
4. At this point I decided to try and restart from scratch so I ran 'git annex direct' to get the files replaced in the working directory, this runs for a while then hangs.
This has been my first outing with `git annex`, so I am sort of at a loss as to what there is I can poke at to try and determine what is going on. It would be nice to be able to get the data back to the original state, but more than that, I would like to understand what I did wrong, and I would appreciate any pointers or advice.

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="https://launchpad.net/~felixonmars"
nickname="felixonmars"
subject="comment 2"
date="2016-01-16T11:35:22Z"
content="""
Thanks. I have updated the page :)
"""]]

View file

@ -0,0 +1,28 @@
I just set up one local repo and one remote server repo (accessible via SSH) that are synced with each other. However, when I add a file to the local repo it never appears on the remote repo. From the log:
***************************************************************************
(scanning...) [2016-01-18 14:55:26.566983] Watcher: Performing startup scan
[2016-01-18 14:55:26.584387] Committer: Committing changes to git
(recording state in git...)
(started...)
Permission denied, please try again.
Permission denied, please try again.
Permission denied (publickey,password,keyboard-interactive).
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: unexplained error (code 255) at /SourceCache/rsync/rsync-45/rsync/io.c(453) [sender=2.6.9]
rsync failed -- run git annex again to resume file transfer
[2016-01-18 14:55:30.429824] main: Syncing with lacieannex
Permission denied, please try again.
Permission denied, please try again.
Permission denied (publickey,password,keyboard-interactive).
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: error in rsync protocol data stream
rsync failed -- run git annex again to resume file transfer
(code 12) at /SourceCache/rsync/rsync-45/rsync/io.c(453) [sender=2.6.9]
***************************************************************************
I suspect there is something with the ssh authentication. I tried to remove all keys from my known_hosts that was related to the server (a handful since I had accessed it at server.local, server.lan, 192.168.1.3, fqdn:2222 etc). My plan was to re-run the setup screen where you enter the details about the remote server, including your username and password, but that step seems unavailable after the initial setup.
How do I fix the issues above?

View file

@ -10,7 +10,7 @@ detailed instructions | quick install
  [[Ubuntu]] | `apt-get install git-annex`   [[Ubuntu]] | `apt-get install git-annex`
  [[Fedora]] | `yum install git-annex`   [[Fedora]] | `yum install git-annex`
  [[FreeBSD]] | `pkg install hs-git-annex`   [[FreeBSD]] | `pkg install hs-git-annex`
  [[ArchLinux]] | `yaourt -Sy git-annex-bin`   [[ArchLinux]] | `pacman -S git-annex`
  [[NixOS]] | `nix-env -i git-annex`   [[NixOS]] | `nix-env -i git-annex`
  [[Gentoo]] | `emerge git-annex`   [[Gentoo]] | `emerge git-annex`
  [[Void]] | `xbps-install git-annex`   [[Void]] | `xbps-install git-annex`

View file

@ -0,0 +1,10 @@
[[!comment format=mdwn
username="wsha.code+ga@b38779424f41c5701bbe5937340be43ff1474b2d"
nickname="wsha.code+ga"
subject="comment 3"
date="2016-01-16T13:03:26Z"
content="""
1. If you want to save a committed version of a file, is there a way to do that, other than `sync`ing to a remote that does not have `annex.thin` set?
2. If you `add` and `commit` a file multiple times in a repo without `sync`ing to a remote, what does the commit history look like on a remote when you do `sync` it? It just has several commits for which the file contents are not available?
3. If you want to preserve history with `annex.thin` set, do you just have to `sync` manually after each commit? I guess you might want to set up a git commit hook to do that in that case.
"""]]

View file

@ -0,0 +1,19 @@
Please consider making the `*E` backends lower-case the file extensions. This improves deduplication especially with image files, where cameras often use `*.JPG`, while other software such as the Dropbox Android app rename the images to `*.jpg`.
We talked about that very shortly a few months ago:
(Log starts 2015-08-06 21:11:40 CET)
tribut is there a backend that lowercases the extension? or do the *E backends do that anyway?
joeyh it does not
tribut but would it make sense? or am i missing something?
joeyh I don't know.. the extension is only there for stupid programs that follow symlinks and
check extensions. If such a program cares about .GIF vss .gif, you might have a problem
joeyh I think that you can git-annex migrate from hashE to hash, then migrate back, and it'll
update to the new file extension.
tribut i was thinking about content-identical images with .JPG or .jpg extension
tribut and because even the most retarded of programs wont care, i thought the backend could
lowercase the extension
joeyh ah, sure, using the E backend reduces the ability to de-duplicate
joeyh I'd not want to add a e backend set just for this. There's no requirement that the
extension extraction code be stable, so it could be considered changing it to lower-case
joeyh otoh, I have no idea if some programs are dumb enough to care about .git vs .GIF

View file

@ -0,0 +1,13 @@
[[!comment format=mdwn
username="CandyAngel"
subject="comment 1"
date="2016-01-18T13:16:59Z"
content="""
Joey implemented the '--force' option in *git annex migrate* at my request to deal with this issue.
* Rename symlinks to lowercase extension
* git annex migrate --force (causes the keys to be recreated using the new, lowercase extension)
* git annex unused (for removing the now unused upper/mixed case files in the store).
It's not the smoothest method, but it works well enough :)
"""]]

View file

@ -0,0 +1,9 @@
[[!comment format=mdwn
username="tribut"
subject="comment 2"
date="2016-01-18T16:22:54Z"
content="""
Thanks for the hint, CandyAngel.
This is useful, however it has to be done periodically. I would still prefer if those keys were never created in the first place.
"""]]

View file

@ -0,0 +1 @@
Would it be possible to create a package for Lacie's NacOS?