Merge branch 'master' of ssh://git-annex.branchable.com

This commit is contained in:
Joey Hess 2013-07-10 15:35:26 -04:00
commit 8697976975
15 changed files with 333 additions and 0 deletions

View file

@ -0,0 +1,18 @@
[[!comment format=mdwn
username="https://www.google.com/accounts/o8/id?id=AItOawkeyfC88gIU4fsEqqkvhzVlSKDUxbtZaTE"
nickname="Selem"
subject="comment 2"
date="2013-07-10T01:41:24Z"
content="""
I got the same problem (Mac OS X 10.8.4, whatever the latest download package, local computers same network repos)
To get rid of the noname repo, I just removed git remote in the annex dir.
# List remote
$ git remote -v
# Remove remote grabbed from running the previous command
$ git remote rm your_remote
After that, I added the second computer repo successfully.
"""]]

View file

@ -0,0 +1,63 @@
### Please describe the problem.
I created somelargefile on host1. A file with that name appears in the right place on host2 but I can't use it:
host2:~/annex% cat somelargefile
cat: somelargefile: No such file or directory
### What steps will reproduce the problem?
I am running http://downloads.kitenet.net/git-annex/linux/current/git-annex-standalone-amd64.tar.gz and I connected two hosts with the 'Local computer' workflow. I run assistant on both sides.
I make a file on either side, wait a minute for it to sync, and then observe this on the remote side:
lrwxrwxrwx 1 drewp drewp 178 Jul 9 23:58 somelargefile -> .git/annex/objects/Pw/vw/SHA256E-s8--bf87165d313027621936809a01da1994f9bd20ff9580c1380e7636e2443fe4ed/SHA256E-s8--bf87165d313027621936809a01da1994f9bd20ff9580c1380e7636e2443fe4ed
host2:~/annex% ls -l .git/annex/objects/Pw/vw/SHA256E-s8--bf87165d313027621936809a01da1994f9bd20ff9580c1380e7636e2443fe4ed/
total 4
-rw------- 1 drewp drewp 14 Jul 9 23:58 SHA256E-s8--bf87165d313027621936809a01da1994f9bd20ff9580c1380e7636e2443fe4ed.map
My "large file" is in fact 8 bytes long. The .map file contains the string "somelargefile\n"
### What version of git-annex are you using? On what operating system?
git-annex version: 4.20130709-g339d1e0
build flags: Assistant Webapp Pairing Testsuite S3 WebDAV Inotify DBus XMPP
local repository version: 3
default repository version: 3
supported repository versions: 3 4
upgrade supported from repository versions: 0 1 2
ubuntu 12.10 64-bit
### Please provide any additional information below.
[[!format sh """
# If you can, paste a complete transcript of the problem occurring here.
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
New lines in .git/annex/daemon.log when I add another new file:
(merging synced/git-annex into git-annex...)
Updating bac8b65..0422d27
Fast-forward
newfile2 | 1 +
1 file changed, 1 insertion(+)
create mode 120000 newfile2
[2013-07-10 00:08:38 PDT] Committer: Committing changes to git
(Recording state in git...)
[2013-07-10 00:08:38 PDT] Pusher: Syncing with host1.local_annex
To ssh://drewp@git-annex-host1.local-drewp_annex/~/annex/
a80b6f5..72c0865 git-annex -> synced/git-annex
Already up-to-date.
# End of transcript or log.
"""]]

View file

@ -0,0 +1,10 @@
[[!comment format=mdwn
username="http://joeyh.name/"
ip="4.154.7.235"
subject="comment 1"
date="2013-07-10T16:31:40Z"
content="""
Everything you've described is expected behavior, when the content of the file has not yet been transferred to the repository.
There should be something in daemon.log about the transfer of the file. If there's an error message for the transfer, it will certainly show up there. You can enable debugging to get more information in daemon.log about file transfers.
"""]]

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="http://joeyh.name/"
ip="4.154.7.235"
subject="comment 10"
date="2013-07-10T17:05:49Z"
content="""
It has its own ssh server written in java too, which hardcodes the set of commands it supports. AFAICS it would require some java coding to add a command to this, it does not seem to be designed for easy extensability.
"""]]

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="http://joeyh.name/"
ip="4.154.7.235"
subject="comment 8"
date="2013-07-10T16:42:18Z"
content="""
If gerrit is handling incoming ssh connections, it's probable running its own restricted shell. That shell would need to be configured to spawn git-annex-shell when asked to do so in order for git-annex to transfer files to that repository.
"""]]

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="https://www.google.com/accounts/o8/id?id=AItOawln3ckqKx0x_xDZMYwa9Q1bn4I06oWjkog"
nickname="Michael"
subject="comment 9"
date="2013-07-10T16:57:29Z"
content="""
AFAIK Gerrit relies on jgit implementation of git in java; allowing it to execute helper programs like git-annex-shell may be tricky.
"""]]

View file

@ -0,0 +1,13 @@
[[!comment format=mdwn
username="https://www.google.com/accounts/o8/id?id=AItOawln3ckqKx0x_xDZMYwa9Q1bn4I06oWjkog"
nickname="Michael"
subject="multiple ways to access a special remote?"
date="2013-07-10T05:52:28Z"
content="""
The use case is like this: an external USB drive could be accessed both locally (as a special directory remote) or it could be mounted remotely (then it would need to be an rsync special remote).
Is there a way to handle this?
Failing that, can one switch the same remote from being a directory to an rsync without having to --move all of the content?
Finally, what's the best way to check directory remote's directory= parameter?
"""]]

View file

@ -0,0 +1,14 @@
[[!comment format=mdwn
username="http://joeyh.name/"
ip="4.154.7.235"
subject="comment 3"
date="2013-07-10T16:38:04Z"
content="""
@Michael, like I said above, this use case is completely supported. That's why git-annex uses UUIDs to uniquely identify repositories, no matter where or how many urls are used for them.
Just set up the remotes you need, and if they end up pointing to the same repository by different routes, git-annex will automatically notice.
---
The directory= parameter used when initializing a directory remote is only used to set up the remote in the .git/config file. It is not stored anywhere else, since the directory could be mounted at different locations on different computers, eg when a drive is moved between computers.
"""]]

View file

@ -0,0 +1,40 @@
[[!comment format=mdwn
username="https://www.google.com/accounts/o8/id?id=AItOawkQvRLq7nMMEGoEKuYx9oaf67IC0nZfmVI"
nickname="chung yan"
subject="owncloud hook exited nonzero"
date="2013-07-10T05:03:47Z"
content="""
hi
I got the above message, and cannot sync to my owncloud. Here is the log file i had:
>[2013-07-10 12:30:31 HKT] main: starting assistant version 4.20130627
(scanning...) [2013-07-10 12:30:31 HKT] Watcher: Performing startup scan
(started...) git-annex: Daemon is already running.
git-annex: Daemon is already running.
[2013-07-10 12:44:04 HKT] Committer: Adding sync2Servers.sh
>add syncByRsync/sync2Servers.sh (checksum...) [2013-07-10 12:44:04 HKT] Committer: Committing changes to git
(gpg)
>owncloud hook exited nonzero!
>git-annex: Daemon is already running.
[2013-07-10 12:51:07 HKT] main: Syncing with owncloud
>owncloud hook exited nonzero!
>[2013-07-10 12:53:10 HKT] Committer: Adding 2 files
ok
(Recording state in git...)
(Recording state in git...)
add syncByRsync/DSCN1810.JPG (checksum...) ok
add syncByRsync/DSCN1810.JPG (checksum...) [2013-07-10 12:53:10 HKT] Committer: Committing changes to git
>owncloud hook exited nonzero!
[2013-07-10 12:53:50 HKT] main: Syncing with owncloud
owncloud hook exited nonzero!
owncloud hook exited nonzero!
thanks
yan
"""]]

View file

@ -0,0 +1,15 @@
[[!comment format=mdwn
username="https://www.google.com/accounts/o8/id?id=AItOawmLB39PC89rfGaA8SwrsnB6tbumezj-aC0"
nickname="Tobias"
subject="comment 2"
date="2013-07-10T08:21:39Z"
content="""
Personally i've only seen that when the server ran out of space. But lets see what is going on.
Please run this command
git config annex.owncloud-hook '/usr/bin/python2 ~/owncloudannex/owncloudannex.py --dbglevel 1 --stderr'
And then replicate the error. It should give me some debug information to work with.
"""]]

View file

@ -0,0 +1,87 @@
[[!comment format=mdwn
username="https://www.google.com/accounts/o8/id?id=AItOawkQvRLq7nMMEGoEKuYx9oaf67IC0nZfmVI"
nickname="chung yan"
subject="comment 3"
date="2013-07-10T08:46:27Z"
content="""
hi Tobias
Thanks your suggestion for list of my error log, here it is:
[2013-07-10 16:27:58 HKT] main: starting assistant version 4.20130627
(scanning...) [2013-07-10 16:27:58 HKT] Watcher: Performing startup scan
(started...) [2013-07-10 16:28:14 HKT] Committer: Adding 2 files
add syncByRsync/sync2Servers.sh (checksum...) ok
add syncByRsync/sync2Servers.sh (checksum...) [2013-07-10 16:28:14 HKT] Committer: Committing changes to git
[2013-07-10 16:31:12 HKT] Watcher: add direct DSCN1810.JPG
[2013-07-10 16:31:12 HKT] read: lsof [\"-F0can\",\"+d\",\"/home/yan/annex_testing/.git/annex/tmp/\"]
[2013-07-10 16:31:12 HKT] Committer: Adding DSCN1810.JPG
ok
(Recording state in git...)
(Recording state in git...)
add DSCN1810.JPG [2013-07-10 16:31:12 HKT](checksum...) Watcher: add direct DSCN1810.JPG
[2013-07-10 16:31:12 HKT] read: sha256sum [\"/home/yan/annex_testing/.git/annex/tmp/DSCN181012023.JPG\"]
DSCN1810.JPG changed while it was being added
[2013-07-10 16:31:12 HKT] Committer: delaying commit of 1 changes
[2013-07-10 16:31:13 HKT] read: lsof [\"-F0can\",\"+d\",\"/home/yan/annex_testing/.git/annex/tmp/\"]
[2013-07-10 16:31:13 HKT] Committer: Adding 2 files
failed
add DSCN1810.JPG (checksum...) [2013-07-10 16:31:13 HKT] read: sha256sum [\"/home/yan/annex_testing/.git/annex/tmp/DSCN181012023.JPG\"]
[2013-07-10 16:31:13 HKT] chat: git [\"--git-dir=/home/yan/annex_testing/.git\",\"--work-tree=/home/yan/annex_testing\",\"hash-object\",\"-t\",\"blob\",\"-w\",\"--stdin\",\"--no-filters\"]
ok
add DSCN1810.JPG (checksum...) [2013-07-10 16:31:13 HKT] read: sha256sum [\"/home/yan/annex_testing/.git/annex/tmp/DSCN181012024.JPG\"]
[2013-07-10 16:31:13 HKT] chat: git [\"--git-dir=/home/yan/annex_testing/.git\",\"--work-tree=/home/yan/annex_testing\",\"hash-object\",\"-t\",\"blob\",\"-w\",\"--stdin\",\"--no-filters\"]
[2013-07-10 16:31:13 HKT] Committer: committing 2 changes
[2013-07-10 16:31:13 HKT] Committer: Committing changes to git
[2013-07-10 16:31:13 HKT] feed: git [\"--git-dir=/home/yan/annex_testing/.git\",\"--work-tree=/home/yan/annex_testing\",\"update-index\",\"-z\",\"--index-info\"]
[2013-07-10 16:31:13 HKT] read: git [\"--git-dir=/home/yan/annex_testing/.git\",\"--work-tree=/home/yan/annex_testing\",\"commit\",\"--allow-empty-message\",\"--no-edit\",\"-m\",\"\",\"--quiet\",\"--no-verify\"]
[2013-07-10 16:31:13 HKT] Committer: queued Upload UUID \"df02d32a-7e3a-4e12-a417-7f1d1a1cf1a6\" DSCN1810.JPG Nothing : new file created
[2013-07-10 16:31:13 HKT] chat: git [\"--git-dir=/home/yan/annex_testing/.git\",\"--work-tree=/home/yan/annex_testing\",\"hash-object\",\"-w\",\"--stdin-paths\",\"--no-filters\"]
[2013-07-10 16:31:13 HKT] feed: git [\"--git-dir=/home/yan/annex_testing/.git\",\"--work-tree=/home/yan/annex_testing\",\"update-index\",\"-z\",\"--index-info\"]
[2013-07-10 16:31:13 HKT] read: git [\"--git-dir=/home/yan/annex_testing/.git\",\"--work-tree=/home/yan/annex_testing\",\"show-ref\",\"--hash\",\"refs/heads/git-annex\"]
[2013-07-10 16:31:13 HKT] read: git [\"--git-dir=/home/yan/annex_testing/.git\",\"--work-tree=/home/yan/annex_testing\",\"write-tree\"]
[2013-07-10 16:31:13 HKT] chat: git [\"--git-dir=/home/yan/annex_testing/.git\",\"--work-tree=/home/yan/annex_testing\",\"commit-tree\",\"a8337a989b58b29eee5fd2fa7a4c0b8ec45d5e59\",\"-p\",\"refs/heads/git-annex\"]
[2013-07-10 16:31:13 HKT] call: git [\"--git-dir=/home/yan/annex_testing/.git\",\"--work-tree=/home/yan/annex_testing\",\"update-ref\",\"refs/heads/git-annex\",\"267bf35da4d9abe5ed7fe82ea5df8a8df2ddf940\"]
[2013-07-10 16:31:13 HKT] read: git [\"--git-dir=/home/yan/annex_testing/.git\",\"--work-tree=/home/yan/annex_testing\",\"symbolic-ref\",\"HEAD\"]
[2013-07-10 16:31:13 HKT] read: git [\"--git-dir=/home/yan/annex_testing/.git\",\"--work-tree=/home/yan/annex_testing\",\"show-ref\",\"refs/heads/master\"]
[2013-07-10 16:31:13 HKT] Transferrer: Transferring: Upload UUID \"df02d32a-7e3a-4e12-a417-7f1d1a1cf1a6\" DSCN1810.JPG Nothing
[2013-07-10 16:31:13 HKT] read: git [\"--git-dir=/home/yan/annex_testing/.git\",\"--work-tree=/home/yan/annex_testing\",\"show-ref\",\"git-annex\"]
[2013-07-10 16:31:13 HKT] call: git-annex [\"transferkeys\",\"--readfd\",\"46\",\"--writefd\",\"36\"]
[2013-07-10 16:31:13 HKT] read: git [\"--git-dir=/home/yan/annex_testing/.git\",\"--work-tree=/home/yan/annex_testing\",\"show-ref\",\"--hash\",\"refs/heads/git-annex\"]
[2013-07-10 16:31:13 HKT] read: git [\"--git-dir=/home/yan/annex_testing/.git\",\"--work-tree=/home/yan/annex_testing\",\"log\",\"refs/heads/git-annex..267bf35da4d9abe5ed7fe82ea5df8a8df2ddf940\",\"--oneline\",\"-n1\"]
[2013-07-10 16:31:13 HKT] read: git [\"--git-dir=/home/yan/annex_testing/.git\",\"--work-tree=/home/yan/annex_testing\",\"show-ref\",\"git-annex\"]
[2013-07-10 16:31:13 HKT] read:[2013-07-10 16:31:13 HKT] read: git [\"--git-dir=/home/yan/annex_testing/.git\",\"--work-tree=/home/yan/annex_testing\",\"ls-tree\",\"-z\",\"--\",\"refs/heads/git-annex\",\"uuid.log\",\"remote.log\",\"trust.log\",\"group.log\",\"preferred-content.log\"]
git [\"--git-dir=/home/yan/annex_testing/.git\",\"--work-tree=/home/yan/annex_testing\",\"show-ref\",\"--hash\",\"refs/heads/git-annex\"]
[2013-07-10 16:31:13 HKT] read: git [\"--git-dir=/home/yan/annex_testing/.git\",\"--work-tree=/home/yan/annex_testing\",\"log\",\"refs/heads/git-annex..267bf35da4d9abe5ed7fe82ea5df8a8df2ddf940\",\"--oneline\",\"-n1\"]
[2013-07-10 16:31:13 HKT] chat: git [\"--git-dir=/home/yan/annex_testing/.git\",\"--work-tree=/home/yan/annex_testing\",\"cat-file\",\"--batch\"]
(gpg) [2013-07-10 16:31:13 HKT] TransferWatcher: transfer starting: Upload UUID \"df02d32a-7e3a-4e12-a417-7f1d1a1cf1a6\" DSCN1810.JPG Nothing
[2013-07-10 16:31:13 HKT] chat: gpg [\"--batch\",\"--no-tty\",\"--use-agent\",\"--quiet\",\"--trust-model\",\"always\",\"--batch\",\"--passphrase-fd\",\"48\",\"--symmetric\",\"--force-mdc\"]
[2013-07-10 16:31:14 HKT] call: sh [\"-c\",\"/usr/bin/python2 /home/yan/annex/SocialBusiness/Resources/Infrastructure/Computer_Tools/Records/AsusU24/owncloudannex/owncloudannex.py --dbglevel 1 --stderr\"]
16:31:16 [owncloudannex-0.1.1] <module> : 'START'
16:31:16 [owncloudannex-0.1.1] main : 'ARGS: 'ANNEX_ACTION=store ANNEX_KEY=GPGHMACSHA1--0179fbbf559d5c25cf69b4e92025ff1f007d4f1f ANNEX_HASH_1=fp ANNEX_HASH_2=23 ANNEX_FILE=/home/yan/annex_testing/.git/annex/tmp/GPGHMACSHA1--0179fbbf559d5c25cf69b4e92025ff1f007d4f1f /home/yan/annex/SocialBusiness/Resources/Infrastructure/Computer_Tools/Records/AsusU24/owncloudannex/owncloudannex.py --dbglevel 1 --stderr''
16:31:16 [owncloudannex-0.1.1] readFile : ''/home/yan/annex/SocialBusiness/Resources/Infrastructure/Computer_Tools/Records/AsusU24/owncloudannex/owncloudannex.conf' - 'r''
16:31:16 [owncloudannex-0.1.1] readFile : 'Done'
16:31:16 [owncloudannex-0.1.1] login : ''
16:31:16 [owncloudannex-0.1.1] login : 'Using base: wingyan.no-ip.org - {'Authorization': 'Basic Y2h1bmd5YW41QGdtYWlsLmNvbTp5YW5fd2lraQ=='}'
16:31:16 [owncloudannex-0.1.1] login : 'res: <davlib.DAV instance at 0x12915f0>'
16:31:16 [owncloudannex-0.1.1] findInFolder : 'u'gitannex'(<type 'unicode'>) - '/'(<type 'str'>)'
16:31:17 [owncloudannex-0.1.1] findInFolder : 'propfind: /owncloud/remote.php/webdav - '<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<d:error xmlns:d=\"DAV:\" xmlns:s=\"http://sabredav.org/ns\">\n <s:exception>Sabre_DAV_Exception_NotAuthenticated</s:exception>\n <s:message>Username or password does not match</s:message>\n <s:sabredav-version>1.7.6</s:sabredav-version>\n</d:error>\n''
16:31:17 [owncloudannex-0.1.1] findInFolder : 'Failure'
16:31:17 [owncloudannex-0.1.1] createFolder : '/gitannex'
16:31:18 [owncloudannex-0.1.1] createFolder : 'Failure: 401 - '<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<d:error xmlns:d=\"DAV:\" xmlns:s=\"http://sabredav.org/ns\">\n <s:exception>Sabre_DAV_Exception_NotAuthenticated</s:exception>\n <s:message>Username or password does not match</s:message>\n <s:sabredav-version>1.7.6</s:sabredav-version>\n</d:error>\n''
owncloud hook exited nonzero!
[2013-07-10 16:31:18 HKT] TransferWatcher: transfer finishing: Transfer {transferDirection = Upload, transferUUID = UUID \"df02d32a-7e3a-4e12-a417-7f1d1a1cf1a6\", transferKey = Key {keyName = \"911ba6148fbcbe4afe53772f1216b8204f403ed4ee06cb90c3c3ac25e56d9402.JPG\", keyBackendName = \"SHA256E\", keySize = Just 1893431, keyMtime = Nothing}}
I figured out it is a *Username or password does not match*, and i saw the content of owncloudannex.conf as \"uname\": \"myGmail@gmail.com\", but i quickly saw my WebDAV login to owncloud as my user name, not gmail address, so i changed this \"uname\": \"my_normal_user_name_not_gmail_acc\" inside owncloudannex.conf. Finally, i got it work. So, i think, user name should not be a gmail address, should be owncloud login user name.
Another issue, i had a look into /WebDAV.../gitannex/, it is git repos. file, for my user opinion, it is better that it is a real file content that we can see the file(such as photos) by owncloud web client directly, rather that owncloud is a file server to keep git repos only.
Thanks
yan
"""]]

View file

@ -0,0 +1,17 @@
[[!comment format=mdwn
username="https://www.google.com/accounts/o8/id?id=AItOawmLB39PC89rfGaA8SwrsnB6tbumezj-aC0"
nickname="Tobias"
subject="comment 4"
date="2013-07-10T08:54:13Z"
content="""
Hm, an error like that it really should print without the need of debug information. I'll look into it.
And if you want the real file content, not encrypted, just change \"shared\" to \"none\" in the line:
git annex initremote owncloud type=hook hooktype=owncloud encryption=shared
I'm not sure if you can change this after you initialized the remote though. And also, the folder structure and filenames will be as you see them now on the owncloud. I'd need some more data from gitannex to make it show the real directory structure. But that doesn't seem feasible.
Why would you even want the data unencrypted on owncloud anyways? i mean on flickr, or googledocs, i kinda get it. but for owncloud?
"""]]

View file

@ -0,0 +1,16 @@
[[!comment format=mdwn
username="https://www.google.com/accounts/o8/id?id=AItOawkQvRLq7nMMEGoEKuYx9oaf67IC0nZfmVI"
nickname="chung yan"
subject="comment 5"
date="2013-07-10T09:43:55Z"
content="""
hi Tobias
thanks your sharing, i am still in have a trial of git-annex, so i do not yet have real data on the server, so i just del. all and re-create both client and server repos in owncloud, i can got what you said, and see my photo.
Actually, i am studying git-annex. 1) It can syncs the data from my client computer to server. 2) i would like that other and anywhere computers through browser can view my server data which do not need to download all data into a new client. So my original goal is i can have a web to display the content of my files, and see owncloud have a nice web.
Regarding to GDoc, flickr, etc. they have space limitation, so i install owncloud as my own computer. I also see the <http://git-annex.branchable.com/design/assistant/partial_content/>, but it is not yet at roadmap.
thanks
"""]]

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="https://www.google.com/accounts/o8/id?id=AItOawmLB39PC89rfGaA8SwrsnB6tbumezj-aC0"
nickname="Tobias"
subject="comment 6"
date="2013-07-10T09:50:14Z"
content="""
A 1 terabyte limit(for flickr) should be enough for most people. No?
"""]]

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="https://www.google.com/accounts/o8/id?id=AItOawkQvRLq7nMMEGoEKuYx9oaf67IC0nZfmVI"
nickname="chung yan"
subject="comment 7"
date="2013-07-10T09:56:54Z"
content="""
i have not got this 1 terabyte limit from flick before. I will have a look. I still consider some my own data, it is better to keep at my own server.
"""]]