Merge branch 'master' of ssh://git-annex.branchable.com

This commit is contained in:
Joey Hess 2013-02-25 22:44:26 -04:00
commit 50fa0b2cba
15 changed files with 279 additions and 2 deletions

View file

@ -0,0 +1,81 @@
Creating additional branches in history seems to slow down the 'git annex unused' command quadratically, even if the location of the branches should be irrelevant as far as unused data goes.
This was tested on:
$ git annex version
git-annex version: 3.20130216
local repository version: 3
default repository version: 3
supported repository versions: 3
upgrade supported from repository versions: 0 1 2
What steps will reproduce the problem?
$ mkdir a
$ cd a
$ git init
$ git annex init
$ i=0 ; while test $i -lt 1000; do dd if=/dev/urandom of=$i.img bs=1M count=1; i=$(($i+1)); done
$ git annex add .
$ git commit -m"foo"
$ git rm 1*
$ git commit -m"bar"
$ git log --oneline --decorate
ffcca3a (HEAD, master) bar
3e7793d foo
$ time -p git annex unused
unused . (checking for unused data...) (checking master...)
(...)
real 0.76
user 0.40
sys 0.06
git commit --allow-empty -m"baz"
$ git log --oneline --decorate
4390c32 (HEAD, master) baz
ffcca3a bar
3e7793d foo
$ time -p git annex unused
unused . (checking for unused data...) (checking master...)
(...)
real 0.75
user 0.38
sys 0.07
$ git branch boo HEAD^
$ time -p git annex unused
unused . (checking for unused data...) (checking boo...) (checking master...)
(...)
real 1.29
user 0.62
sys 0.08
arand@mas:~/tmp/more/a(master)$ git branch beeboo HEAD^
4390c32 (HEAD, master) baz
ffcca3a (boo, beeboo) bar
3e7793d foo
arand@mas:~/tmp/more/a(master)$ time -p git annex unused
unused . (checking for unused data...) (checking beeboo...) (checking master...)
(...)
real 2.50
user 1.12
sys 0.14
$ git branch -d boo beeboo
$ git log --oneline --decorate
4390c32 (HEAD, master) baz
ffcca3a bar
3e7793d foo
$ time -p git annex unused
unused . (checking for unused data...) (checking master...)
(...)
real 0.77
user 0.42
sys 0.04
What is the expected output? What do you see instead?
I would expect the time to be the same in all the above cases.
What version of git-annex are you using? On what operating system?
$ git annex version
git-annex version: 3.20130216
On current Debian sid/experimental

View file

@ -0,0 +1,57 @@
One of my remotes, on a USB drive, is behaving exceedingly strangely. Files sometimes refuse to copy to it - whether I copy to it from my home annex, or whether I "cd" to that USB drive and try to "get" files to it.
Note that the external HD is a FAT32 filesystem. This has never caused problems in the past, but I am wondering if some of the recent work on "crippled" filesystems might have caused breakage on existing repositories which had been working well on FAT32 filesystems?
What steps will reproduce the problem?
On my annex, something like this:
```
Talislanta Books$ git annex whereis talislanta_fantasy_roleplaying.pdf
whereis talislanta_fantasy_roleplaying.pdf (2 copies)
d16d0d1a-3cdd-11e2-9161-67c83599f720 -- homeworld
fa2bd02e-3ce2-11e2-a675-47389975a32e -- here (macbook)
ok
Talislanta Books$ git annex copy --to=toshiba talislanta_fantasy_roleplaying.pdf
copy talislanta_fantasy_roleplaying.pdf ok
Talislanta Books$ git annex whereis talislanta_fantasy_roleplaying.pdf
whereis talislanta_fantasy_roleplaying.pdf (2 copies)
d16d0d1a-3cdd-11e2-9161-67c83599f720 -- homeworld
fa2bd02e-3ce2-11e2-a675-47389975a32e -- here (macbook)
ok
Talislanta Books$ cd /Volumes/TOSHIBAEXT/annex/Books/archive/Talislanta\ Books/
Talislanta Books$ git annex whereis talislanta_fantasy_roleplaying.pdf
whereis talislanta_fantasy_roleplaying.pdf (2 copies)
d16d0d1a-3cdd-11e2-9161-67c83599f720 -- homeworld
fa2bd02e-3ce2-11e2-a675-47389975a32e -- macbook
ok
Talislanta Books$ git annex get talislanta_fantasy_roleplaying.pdf
Talislanta Books$ git annex whereis talislanta_fantasy_roleplaying.pdf
whereis talislanta_fantasy_roleplaying.pdf (2 copies)
d16d0d1a-3cdd-11e2-9161-67c83599f720 -- homeworld
fa2bd02e-3ce2-11e2-a675-47389975a32e -- macbook
ok
Talislanta Books$
```
What is the expected output? What do you see instead?
I should be able to copy files to my external hard drive, /Volumes/TOSHIBAEXT/annex
What version of git-annex are you using? On what operating system?
Talislanta Books$ git annex version
git-annex version: 3.20130216
local repository version: 3
default repository version: 3
supported repository versions: 3
upgrade supported from repository versions: 0 1 2
OS X 10.6 (lion)
Please provide any additional information below.
Most files are affected by this, a few are not. I don't see any pattern to which is which.

View file

@ -0,0 +1,10 @@
[[!comment format=mdwn
username="http://edheil.wordpress.com/"
ip="99.54.57.201"
subject="comment 1"
date="2013-02-25T06:18:58Z"
content="""
Just tried downgrading to 3.20130207 and the behavior is the same. :(
"""]]

View file

@ -0,0 +1,9 @@
[[!comment format=mdwn
username="http://edheil.wordpress.com/"
ip="99.54.57.201"
subject="comment 2"
date="2013-02-25T12:44:18Z"
content="""
Update: git annex fsck --fast --from=toshiba fixed this. Guess I was up too late to think about the obvious. Sorry bout that.
"""]]

View file

@ -0,0 +1,20 @@
[[!comment format=mdwn
username="http://edheil.wordpress.com/"
ip="99.54.57.201"
subject="comment 3"
date="2013-02-25T14:24:29Z"
content="""
This is still weird though.
Setup: I have my home drive, an ssh remote, and a usb remote.
Home drive is a \"client\" and the other two are \"backups.\" numcopies = 2.
WHen I start the assistant and watch it in the webapp, it starts furiously copying content in \"archive\" subdirectories from the USB remote to the home drive, and then dropping it from the USB remote!
I then kill the assistant because I do not want content dropped from my backup, that's why it's a backup.
Is there any way to tell *why* the assistant is doing this crazy thing?
"""]]

View file

@ -0,0 +1,9 @@
[[!comment format=mdwn
username="http://edheil.wordpress.com/"
ip="99.54.57.201"
subject="comment 4"
date="2013-02-25T14:57:20Z"
content="""
Yeah, the assistant wants to drop *everything* on my USB drive. When I vicfg I see that the usb drive repo is \"untrusted/backup/standard\". Why in the world would the assistant want to drop everything on it?
"""]]

View file

@ -0,0 +1,10 @@
[[!comment format=mdwn
username="http://edheil.wordpress.com/"
ip="99.54.57.201"
subject="comment 5"
date="2013-02-25T16:02:44Z"
content="""
Downgrading to 3.20130207 and testing.... this seems to have been a regression since 3.20130207. 3.20130207 seems right now to be behaving normally, not trying to drop things it shouldn't. Is it possible something went terribly wrong with preferred content settings since 3.20130207?
Still testing, but things look good so far.
"""]]

View file

@ -4,7 +4,7 @@ Only report of 100% failure so far is on Cyanogenmod 7.2 (Android 2.3.7).
Worked today on some of the obvious bugs.
* Turns out that getEnvironment is broken on Android, returning no
environment, which explains the weird git beheavior where it complains
environment, which explains the weird git behavior where it complains
that it cannot determine the username and email (because it sees no USER
or HOST), and suggests setting them in the global git config (which it
ignores, because it sees no HOME). Put in a work around for this
@ -17,7 +17,7 @@ Worked today on some of the obvious bugs.
* Enabled a few useful things in busybox. Including vi.
* Replaced the service notification icon with one with the git-annex logo.
* Made the terminal not close immediately when the shell exits, which
should aid in debugging of certian types of crashes.
should aid in debugging of certain types of crashes.
I want to set up an autobuilder for Android, but to do that I need to
install all the haskell libraries on my server. Since getting them built

View file

@ -0,0 +1,19 @@
[[!comment format=mdwn
username="http://meep.pl/"
ip="193.23.174.18"
subject="Android 2.3 again"
date="2013-02-24T23:07:28Z"
content="""
I got _a lot_ further with the latest upgrade.
* installation worked
* `git init` worked
* first `git annex init` asked to set user.{email,name}
* `git config --global ...` worked
* second `git annex init` segfaulted, but not immediately:
init
Detected a crippled filesystem.
error: git-annex died of signal 11
"""]]

View file

@ -0,0 +1,12 @@
[[!comment format=mdwn
username="https://www.google.com/accounts/o8/id?id=AItOawm5iosFbL2By7UFeViqkc6v-hoAtqILeDA"
nickname="Laszlo"
subject="comment 2"
date="2013-02-25T11:22:53Z"
content="""
Is the autiobuilder for linux available already?
I can't find it, and I'm eager to try the thumbdrive related fixes (where I can specify a simple folder).
Best,
Laszlo
"""]]

View file

@ -0,0 +1,10 @@
[[!comment format=mdwn
username="http://joeyh.name/"
ip="4.152.108.210"
subject="comment 3"
date="2013-02-25T15:53:30Z"
content="""
@meep try running it with --debug for some more hint of where it's crashing
@Laszlo see [[install/Linux_standalone]]
"""]]

View file

@ -0,0 +1,12 @@
[[!comment format=mdwn
username="https://www.google.com/accounts/o8/id?id=AItOawm5iosFbL2By7UFeViqkc6v-hoAtqILeDA"
nickname="Laszlo"
subject="comment 4"
date="2013-02-25T18:28:09Z"
content="""
joeyh: Is the 20130216 is the newest? Because you just wrote about two days ago of direct mode fixes, I'm interested in.
I tried 20130216 already.
Laszlo
"""]]

View file

@ -0,0 +1,10 @@
[[!comment format=mdwn
username="http://joeyh.name/"
ip="4.152.108.210"
subject="comment 1"
date="2013-02-24T21:51:19Z"
content="""
You're basically right. Additionally, if you use encryption, the keys used on the remote are themselves encrypted and so even the same content will result in different keys.
With this said, it's easy to use a subdirectory of a rsync or box.com remote, and I'd rather recommend doing that, rather than jumbling multiple repositories data into a single directory.
"""]]

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="http://joeyh.name/"
ip="4.152.108.210"
subject="comment 1"
date="2013-02-24T21:52:41Z"
content="""
I welcome people doing this kind of thing with git-annex. Since you can have multiple files linking to the same content easily, it should be a reasonable platform with which to build something like this.
"""]]

View file

@ -0,0 +1,10 @@
[[!comment format=mdwn
username="http://joeyh.name/"
ip="4.152.108.210"
subject="Ever figure this out?"
date="2013-02-24T22:00:40Z"
content="""
I find it's useful, when stuck on this kind of thing, to run `git annex find` at the command line passing it parts of the expression and making sure it works, and gradually building up to the complete expression.
You have to do some conversion due to differences between command line and preferred content expression format. For example `git annex find -\( --in here --and --include \"music/*\" -\)`
"""]]