Merge branch 'master' of ssh://git-annex.branchable.com
This commit is contained in:
Joey Hess
commit
cc25ec0ad9
10 changed files with 192 additions and 1 deletions
|
|
@ -0,0 +1,25 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://www.google.com/accounts/o8/id?id=AItOawm5iosFbL2By7UFeViqkc6v-hoAtqILeDA"
|
||||
nickname="Laszlo"
|
||||
subject="comment 6"
|
||||
date="2013-03-02T10:05:38Z"
|
||||
content="""
|
||||
$ ls -lah daemon.log
|
||||
-rw-r--r-- 1 user user 3,3G 2013-03-02 10:54 daemon.log
|
||||
|
||||
$ cat daemon.log |wc -l
|
||||
30746274
|
||||
|
||||
$ grep \"Auto packing the repository for optimum performance.\" daemon.log |wc -l
|
||||
568
|
||||
|
||||
$ grep \"typechange: \" daemon.log |wc -l
|
||||
30713158
|
||||
|
||||
I'm not sure where should I issue the git command. Inside the annex/.git directory?
|
||||
The useful data is also in git repository locally.
|
||||
I try to RTFM over the internet, thanks for the pointer!
|
||||
|
||||
|
||||
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://www.google.com/accounts/o8/id?id=AItOawm5iosFbL2By7UFeViqkc6v-hoAtqILeDA"
|
||||
nickname="Laszlo"
|
||||
subject="comment 7"
|
||||
date="2013-03-02T11:03:46Z"
|
||||
content="""
|
||||
Ok, quick question: Should I issue git config gc.auto 0, in both repositories?
|
||||
(it puts variable inside .git/config)
|
||||
|
||||
With this gc.auto 0, should the \"typechange: \" logging change? (because it still in the log)
|
||||
|
||||
I'm testing the whole experience, and report back,
|
||||
also when new build is available i will switch to it, to report about the logging size.
|
||||
|
||||
Thank you for your fast response!
|
||||
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://www.google.com/accounts/o8/id?id=AItOawm5iosFbL2By7UFeViqkc6v-hoAtqILeDA"
|
||||
nickname="Laszlo"
|
||||
subject="comment 8"
|
||||
date="2013-03-02T12:54:14Z"
|
||||
content="""
|
||||
The startup time is somewhere between 4-5 hours
|
||||
(dunno exactly, its only running about 3 hours) with this gc.auto 0.
|
||||
|
||||
I will try to relaunch git-annex-assistant once it finishes startup,
|
||||
but I can report back only tomorrow because it may take 5 hours/try.
|
||||
|
||||
Laszlo
|
||||
|
||||
"""]]
|
||||
12
doc/bugs/host_with_rysnc_installed__44___not_recognized.txt
Normal file
12
doc/bugs/host_with_rysnc_installed__44___not_recognized.txt
Normal file
|
|
@ -0,0 +1,12 @@
|
|||
What steps will reproduce the problem?
|
||||
Set up a remote server using ssh to a FreeNAS box
|
||||
|
||||
What is the expected output? What do you see instead?
|
||||
Neither rsync nor git-annex are installed
|
||||
rsync is in the path, user with permissions is able to run it
|
||||
|
||||
What version of git-annex are you using? On what operating system?
|
||||
4.20130227 OSX
|
||||
|
||||
Please provide any additional information below.
|
||||
ssh keys were installed to allow login, when ssh-askpass was not found on osx version
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://www.google.com/accounts/o8/id?id=AItOawmmLfBDEDFfEOba8Ra46nsnTmoNHFsLObo"
|
||||
nickname="Brian"
|
||||
subject="comment 13"
|
||||
date="2013-03-01T23:52:53Z"
|
||||
content="""
|
||||
I got the latest autobuild version installed and the fallback works for me--thanks. FYI, the fallback error message says: \"cannot find expected files in /data/app-lib\"
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,78 @@
|
|||
Before I start on what's gone wrong, many thanks for a great program: finally a way of finding out where I've put all those files, and I enjoyed your talk in Australia. Not quite to New Zealand, but a good start :-)
|
||||
|
||||
To play with git-annex, I decided to convert my ~/Downloads directory to a git-annex repository, as there were a wide variety of files in it, mostly easily replaceable, and also handy to have on multiple machines. In hindsight, probably wasn't a great idea, as I'd regularly forget it was a git-annex repo and move files and directories out manually, which caused all sorts of fun trying to sort out the dead symlinks. Then I after one update, the repository format changed, from WORM to xxx256 format which meant there were now both sorts in the GA object store.
|
||||
|
||||
More recently I'd tried converting the repo to direct mode, you get the idea: lots of playing with git-annex commands, and now that I think about it, possibly some git commands too trying to repair missing files.
|
||||
|
||||
Anyway I've ended up with a 27GB git-annex repo that now manages to kill git-annex whenever I try to check it using "git-annex fsck".
|
||||
|
||||
Not only does the fsck subcommand cause it to die, but also "find", "whereis" and "status". It dies on the same file (for find/whereis/fsck).
|
||||
|
||||
e.g.
|
||||
|
||||
... lots of stuff deleted ...
|
||||
whereis 1wolf14.zip (2 copies)
|
||||
051f0b00-e265-11e1-894e-3b0b3f3844f2 -- Laptop
|
||||
2c4e11e0-a1b4-11e1-9a02-73e17b04c00f -- here (myPC - Downloads)
|
||||
ok
|
||||
git-annex: out of memory (requested 2097152 bytes)
|
||||
|
||||
Now "git status" for some repo data:
|
||||
|
||||
myPC:~/Downloads$ git annex status
|
||||
supported backends: SHA256E SHA1E SHA512E SHA224E SHA384E SHA256 SHA1 SHA512 SHA224 SHA384 WORM URL
|
||||
supported remote types: git S3 bup directory rsync web webdav glacier hook
|
||||
repository mode: direct
|
||||
trusted repositories: 0
|
||||
semitrusted repositories: 4
|
||||
00000000-0000-0000-0000-000000000001 -- web
|
||||
051f0b00-e265-11e1-894e-3b0b3f3844f2 -- Laptop
|
||||
2c4e11e0-a1b4-11e1-9a02-73e17b04c00f -- here (myPC - Downloads)
|
||||
48fbe52a-a1b3-11e1-bb80-ebc15118871d -- netbk
|
||||
untrusted repositories: 0
|
||||
dead repositories: 0
|
||||
transfers in progress: none
|
||||
available local disk space: 100 gigabytes (+1 megabyte reserved)
|
||||
local annex keys: 1719
|
||||
local annex size: 27 gigabytes
|
||||
known annex keys: git-annex: out of memory (requested 2097152 bytes)
|
||||
|
||||
It always seems to die at about 3.5GB memory usage. This is running on Ubuntu 12.04, using the latest GA release built using cabal:
|
||||
|
||||
git-annex version: 4.20130227
|
||||
local repository version: 3
|
||||
default repository version: 3
|
||||
supported repository versions: 3 4
|
||||
upgrade supported from repository versions: 0 1 2
|
||||
|
||||
There are also dead symlinks that point to directories that have meta-data but not the symlink target (manually line-wrapped):
|
||||
|
||||
myPC:~/Downloads$ ls -l precise*
|
||||
lrwxrwxrwx 1 nino nino 194 Oct 21 12:48 precise-dvd-i386.iso ->
|
||||
.git/annex/objects/0x/Xz/SHA256-s3590631424--
|
||||
b08ecdd4846948ec076b23afae7f87be9cfba5218fb9ba4160f26c0b8d4b5dd0/
|
||||
SHA256-s3590631424--b08ecdd4846948ec076b23afae7f87be9cfba5218fb9ba4160f26c0b8d4b5dd0
|
||||
|
||||
But looking in the symlink destination directory, there's no corresponding object, only metadata:
|
||||
|
||||
myPC:~/Downloads/.git/annex/objects/0x/Xz/SHA256-s3590631424--b08ecdd4846948ec076b23afae7f87be9cfba5218fb9ba4160f26c0b8d4b5dd0$ ls -l
|
||||
total 8
|
||||
-rw-rw-r-- 1 nino nino 30 Jan 8 23:15 SHA256-s3590631424--b08ecdd4846948ec076b23afae7f87be9cfba5218fb9ba4160f26c0b8d4b5dd0.cache
|
||||
-rw-rw-r-- 1 nino nino 49 Jan 8 23:15 SHA256-s3590631424--b08ecdd4846948ec076b23afae7f87be9cfba5218fb9ba4160f26c0b8d4b5dd0.map
|
||||
|
||||
But there is another version somewhere else.
|
||||
|
||||
-r--r--r-- 1 nino nino 3590631424 Mar 18 2012 ./.git/annex/objects/Kj/wM/WORM-s3590631424-m1331991509
|
||||
--precise-dvd-i386.iso/WORM-s3590631424-m1331991509--precise-dvd-i386.iso
|
||||
|
||||
This actual file does exist in the "Used" directory:
|
||||
|
||||
-rw-r--r-- 1 nino nino 3.4G May 27 2012 precise-dvd-i386.iso
|
||||
|
||||
I'm not so worried about the mangled repo - it's quite possibly because of clueless git/git-annex command usage - but the inability to use the fsck command is concerning
|
||||
|
||||
I could just uninit everything, but as it dies prematurely, I'm not certain that all the contents would be restored.
|
||||
Any thoughts on how I can get git-annex (esp. fsck) to complete would be appreciated.
|
||||
|
||||
Thanks
|
||||
Giovanni
|
||||
|
|
@ -0,0 +1,14 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://www.google.com/accounts/o8/id?id=AItOawmdLmVxigDvxiGhnQ34xhyAaxl0CRRe9gs"
|
||||
nickname="Bryan"
|
||||
subject="Alternate version - smarter integration with other remotes/web special remote?"
|
||||
date="2013-03-03T14:19:12Z"
|
||||
content="""
|
||||
I love the idea of pushing files to \"visible\" using annex, but I'm wondering how difficult to would be to make this even fancier.
|
||||
|
||||
For instance, if there are files that were originally imported from the web special remote (and have http:// prefixes, since that's not guaranteed), and they're not actually in the pushed annex, generate a redirect for all of them and write a special remote-local .htaccess file.
|
||||
|
||||
Or, similarly, allow each remote to have some metadata which specifies what, if any, the URL prefix for a directory in annex is. Then, whenever you update a web remote, the auto-generated redirects list could contain not only files not present but with web-remote locations, but also other remotes which have the file under a web-visible path prefix. Bonus for allowing URL prefixes for S3-style remotes that are not encrypted.
|
||||
|
||||
My use case for this sort of fancy is that I have a large server with gobs of space but on a personal (slower) internet connection, and a much smaller hosted instance with much faster bandwidth. With this smarts, I could push and pull large/expected high traffic files to the higher bandwidth/smaller machine ahead of, say, announcing a new feature or sharing a link to a video via e-mail.. or even reactively if the lower-bandwidth server is getting overwhelmed.
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,12 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://www.google.com/accounts/o8/id?id=AItOawnwz07HJdp3nzLwCAl7OGKn_3m0tTO4Mxw"
|
||||
nickname="Teemu"
|
||||
subject="use existing ssh keys?"
|
||||
date="2013-03-02T12:05:30Z"
|
||||
content="""
|
||||
I also found the key setup confusing. I have many identity files for several server / service combinations, so the webapp key setup for an ssh+rsync cloud server always failed with too many identification attempts before I had the sense to move my keys temporarily away from ~/.ssh
|
||||
|
||||
It would be really nice if the webapp would allow one either to pick an existing identity, or to create a new one with the aid of a couple of password logins.
|
||||
|
||||
Also, having the created key to be restricted to only running rsync in the server would improve the chances of 'Share with a friend' use case being used. I don't think many people will want to give out full shell access to their servers in order to share some files.
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
[[!comment format=mdwn
|
||||
username="http://joeyh.name/"
|
||||
nickname="joey"
|
||||
subject="comment 3"
|
||||
date="2013-03-03T16:57:22Z"
|
||||
content="""
|
||||
When pairing with a local computer, both systems set up locked down ssh keys that only allow the pair to run git-annex on the repository being paired with. This does *not* involve full shell access.
|
||||
|
||||
The \"remote server\" necessarily involves you already having shell access to the remote server. However, the ssh key that the assistant generates is again locked down to only being able to run git-annex on a single repository on the remote server.
|
||||
"""]]
|
||||
|
|
@ -19,7 +19,7 @@ more.
|
|||
|
||||
A daily build is also available.
|
||||
|
||||
* [download tarball](http://downloads.kitenet.net/git-annex/autobuild/i386/git-annex-standalone-i386.tar.gz) ([build logs](http://downloads.kitenet.net/git-annex/autobuild/android/))
|
||||
* [download apk](http://downloads.kitenet.net/git-annex/autobuild/android/git-annex.apk) ([build logs](http://downloads.kitenet.net/git-annex/autobuild/android/))
|
||||
|
||||
## building it yourself
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue