mirrors/git-annex
2
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

Merge branch 'master' of ssh://git-annex.branchable.com

Browse source
This commit is contained in:
Joey Hess 2013-08-26 16:09:45 -04:00
commit 8f131c8298
16 changed files with 356 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

10
doc/bugs/Resource_exhausted/comment_10_bccf9528ffe963154c92ce49762e7ea6._comment Normal file
View file

@ -0,0 +1,10 @@
[[!comment format=mdwn
username="https://www.google.com/accounts/o8/id?id=AItOawln3ckqKx0x_xDZMYwa9Q1bn4I06oWjkog"
nickname="Michael"
subject="comment 10"
date="2013-08-26T18:33:40Z"
content="""
@Joey: it was a \"pretty large\" transfer, several hundred gigabytes in perhaps ~100000 files. The copying was going to a GPG-encrypted directory remote.
The error only happened once or twice so far. Point taken about find in /proc; I'll do that if it happens next time.
"""]]

25
doc/bugs/android:_high_CPU_usage__44___unclear_how_to_quit.mdwn Normal file
View file

@ -0,0 +1,25 @@
### Please describe the problem.
I installed git-annex on my android device (Nook HD+, with Cyanogenmod 10.1 installed) for the first time today and was excited to get it working. However, I noticed the device warming alarmingly, and, after installing a CPU usage monitor, it became clear that git annex was the problem, as it was hovering around 30-40% even when idle.
I tried quitting git-annex using the webapp's "Shutdown Daemon" menu option, and it seemed to shut down successfully, but the CPU monitor still showed that process present and taking up high amounts of CPU (sometimes well over 50%). I used the android app switcher and noticed that the terminal emulator for git annex was still running; I tried to quit this by using the X button and it seemed to close, but the CPU monitor still showed the git-annex process consuming large amounts of CPU. Finally I had to quit the process forcefully from the monitor.
### What steps will reproduce the problem?
Install & run; observe CPU. I used a dedicated CPU monitor to stop it the first time; another time, I tried stopping it by going to Preferences, Apps, Running Applications, where it told me it had one process and one service running. I stopped the service without issue; it said the process could not be safely stopped but I stopped it anyway and that successfully stopped the app.
### What version of git-annex are you using? On what operating system?
the current (4.20130826-g46f422) version on Android.
### Please provide any additional information below.
[[!format sh """
# If you can, paste a complete transcript of the problem occurring here.
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
(I'm not sure how to get a log out of the web app to paste here unfortunately.
# End of transcript or log.
"""]]

9
doc/bugs/android:_high_CPU_usage__44___unclear_how_to_quit/comment_1_8e7bc6965ea967a8d43240791a30c5bc._comment Normal file
View file

@ -0,0 +1,9 @@
[[!comment format=mdwn
username="http://edheil.wordpress.com/"
ip="173.162.44.162"
subject="comment 1"
date="2013-08-26T18:34:44Z"
content="""
Just noticed I was using the autobuild instead of the last release version; I'll try the release version and see if that makes any difference.
"""]]

8
doc/bugs/android:_high_CPU_usage__44___unclear_how_to_quit/comment_2_891c1073f908b204651899d41599f944._comment Normal file
View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="http://joeyh.name/"
ip="4.154.0.63"
subject="comment 2"
date="2013-08-26T18:42:02Z"
content="""
If you shut down the daemon, it seems to me that the process that was still running would probably be `git annex transferkey` which runs in its own process to upload/download file contents. I normally see 0% to 1% cpu use from git-annex when it is running on my android tablet. It's possible that the 1% use is due to it waking up every second, which got fixed in 9dc2373977d583b4c4aa6cf0555dc97309f89991.
"""]]

11
doc/bugs/android:_high_CPU_usage__44___unclear_how_to_quit/comment_3_de02b8f1b5928fa1a7078c4aa2124bea._comment Normal file
View file

@ -0,0 +1,11 @@
[[!comment format=mdwn
username="http://edheil.wordpress.com/"
ip="173.162.44.162"
subject="comment 3"
date="2013-08-26T19:03:45Z"
content="""
OK, so I might have a rogue transferkey process going on, but one which starts over again when I restart the app?
I've set this all up purely as a \"can I do this\" experiment right now; there is literally zero important data there to lose, either on the tablet or on the laptop, so maybe I could just write this off as a bad result of my first few shaky attempts to get the device paired with my laptop, wipe out both annexes, and start completely from scratch.
"""]]

99
doc/bugs/android:_high_CPU_usage__44___unclear_how_to_quit/comment_4_0f1aeaaf1bb3d79393bb77ea65f81089._comment Normal file
View file

@ -0,0 +1,99 @@
[[!comment format=mdwn
username="http://edheil.wordpress.com/"
ip="173.162.44.162"
subject="comment 4"
date="2013-08-26T19:32:35Z"
content="""
Wiped out the repository I was using. Cleared my data on the android app completely so I was starting from scratch. Created a new repository on the laptop with a linked XMPP account and a ssh repo. Fired up the android app and gave it the same XMPP credentials and the same ssh repo.
I did not actually add any files to the repo.
The application consumed 35% CPU on the Android tablet, idling, and has been ever since.
This is the log; maybe something in it will explain the problem:
\"\"\"
× Synced with ejayaitch
× Synced with edheil.noip.org_androidannex
× Synced with ejayaitch
× Pairing with ejayaitch complete
× Performed startup scan
Here's everything that has been logged by the git-annex assistant, as well as by programs it has run.
[2013-08-26 15:10:58 EDT] main: starting assistant version 4.20130802-g60ecabd
(scanning...) [2013-08-26 15:10:58 EDT] Watcher: Performing startup scan
(started...) [2013-08-26 15:11:31 EDT] XMPPClient: Pairing with ejayaitch in progress
[2013-08-26 15:11:32 EDT] XMPPSendPack: Syncing with ejayaitch
Already up-to-date.
[2013-08-26 15:11:33 EDT] XMPPReceivePack: Syncing with ejayaitch
[2013-08-26 15:11:33 EDT] XMPPSendPack: Unable to download files from your other devices.
[2013-08-26 15:11:33 EDT] XMPPSendPack: Syncing with ejayaitch
[2013-08-26 15:11:33 EDT] XMPPSendPack: Unable to download files from your other devices.
[2013-08-26 15:11:58 EDT] XMPPSendPack: Syncing with ejayaitch
[2013-08-26 15:11:59 EDT] XMPPSendPack: Unable to download files from your other devices.
[2013-08-26 15:11:59 EDT] XMPPSendPack: Syncing with ejayaitch
[2013-08-26 15:11:59 EDT] XMPPSendPack: Unable to download files from your other devices.
Generating public/private rsa key pair.
Your identification has been saved in /data/data/ga.androidterm/tmp/git-annex-keygen.0/key.
Your public key has been saved in /data/data/ga.androidterm/tmp/git-annex-keygen.0/key.pub.
The key fingerprint is:
bb:85:f2:e7:05:ad:82:62:de:27:32:5b:7a:90:c7:3d u0_a91@localhost
The key's randomart image is:
+--[ RSA 2048]----+
| |
| |
| |
| . |
| o .S . . |
| o o.Eo o |
| oo+ +.o . |
| oo=oo.+.. |
| +=.ooo. |
+-----------------+
[2013-08-26 15:13:07 EDT] main: Syncing with edheil.noip.org_androidannex
warning: no common commits
From ssh://git-annex-edheil.no-ip.org-ed_androidannex/~/androidannex
* [new branch] git-annex -> edheil.noip.org_androidannex/git-annex
* [new branch] master -> edheil.noip.org_androidannex/master
* [new branch] synced/git-annex -> edheil.noip.org_androidannex/synced/git-annex
* [new branch] synced/master -> edheil.noip.org_androidannex/synced/master
Already up-to-date!
Merge made by the 'recursive' strategy.
Already up-to-date.
[2013-08-26 15:13:10 EDT] Pusher: Syncing with edheil.noip.org_androidannex
Already up-to-date.
Already up-to-date.
warning: no threads support, ignoring --threads
warning: no threads support, ignoring --threads
To ssh://ed@git-annex-edheil.no-ip.org-ed_androidannex/~/androidannex/
54a6ede..b502e21 git-annex -> synced/git-annex
67690e7..0bdd057 master -> synced/master
error: Ref refs/heads/synced/git-annex is at b502e2176d04a713d49b019e10b13dc9d30835bd but expected 54a6eded4adf9c926a15675351e3feaa0083cf46
remote: error: failed to lock refs/heads/synced/git-annex
error: Ref refs/heads/synced/master is at 0bdd0575caf5c1992015175363e767ab526b5db7 but expected 67690e7d9c6da81792d14e7389e43a75abfd5a36
remote: error: failed to lock refs/heads/synced/master
To ssh://ed@git-annex-edheil.no-ip.org-ed_androidannex/~/androidannex/
! [remote rejected] git-annex -> synced/git-annex (failed to lock)
! [remote rejected] master -> synced/master (failed to lock)
error: failed to push some refs to 'ssh://ed@git-annex-edheil.no-ip.org-ed_androidannex/~/androidannex/'
Already up-to-date.
Everything up-to-date
fatal: The remote end hung up unexpectedly
[2013-08-26 15:13:33 EDT] XMPPReceivePack: Syncing with ejayaitch
Already up-to-date.
From ssh://git-annex-edheil.no-ip.org-ed_androidannex/~/androidannex
b502e21..5c2452d synced/git-annex -> edheil.noip.org_androidannex/synced/git-annex
[2013-08-26 15:15:10 EDT] read: git [\"--git-dir=/storage/emulated/legacy/annex/.git\",\"--work-tree=/storage/emulated/legacy/annex\",\"ls-tree\",\"-z\",\"--\",\"refs/heads/git-annex\",\"uuid.log\",\"remote.log\",\"trust.log\",\"group.log\",\"preferred-content.log\"]
[2013-08-26 15:15:12 EDT] XMPPClient: received: [\"Unknown message\"]
[2013-08-26 15:15:33 EDT] XMPPClient: received: [\"Pushing \\"e28\\" (ReceivePackDone (ExitFailure 128))\"]
[2013-08-26 15:15:33 EDT] XMPPClient: NetMessager stored Pushing \"e28\" (ReceivePackDone (ExitFailure 128)) in SendPack inbox \"e28\"
[2013-08-26 15:17:12 EDT] XMPPClient: received: [\"Unknown message\"]
[2013-08-26 15:17:33 EDT] XMPPClient: received: [\"Presence from h10/messaging-TalkGadgetwADC317A1 Nothing\"]
[2013-08-26 15:17:42 EDT] XMPPClient: received: [\"Presence from 326/MessagingA7afbdd8d Nothing\"]
[2013-08-26 15:17:42 EDT] XMPPClient: received: [\"Presence from 326/MessagingA7afbdd8d Nothing\"]
[2013-08-26 15:18:25 EDT] XMPPClient: received: [\"Presence from h10/messaging-TalkGadgetwADC317A1 Nothing\"]
[2013-08-26 15:19:12 EDT] XMPPClient: received: [\"Unknown message\"]
\"\"\"
"""]]

90
doc/bugs/android:_high_CPU_usage__44___unclear_how_to_quit/comment_5_30120dee80aeab74d3f480b83b08440b._comment Normal file
View file

@ -0,0 +1,90 @@
[[!comment format=mdwn
username="http://edheil.wordpress.com/"
ip="173.162.44.162"
subject="comment 5"
date="2013-08-26T19:34:23Z"
content="""
hopefully better formatted debug info:
× Synced with ejayaitch
× Synced with edheil.noip.org_androidannex
× Synced with ejayaitch
× Pairing with ejayaitch complete
× Performed startup scan
Here's everything that has been logged by the git-annex assistant, as well as by programs it has run.
[2013-08-26 15:10:58 EDT] main: starting assistant version 4.20130802-g60ecabd
(scanning...) [2013-08-26 15:10:58 EDT] Watcher: Performing startup scan
(started...) [2013-08-26 15:11:31 EDT] XMPPClient: Pairing with ejayaitch in progress
[2013-08-26 15:11:32 EDT] XMPPSendPack: Syncing with ejayaitch
Already up-to-date.
[2013-08-26 15:11:33 EDT] XMPPReceivePack: Syncing with ejayaitch
[2013-08-26 15:11:33 EDT] XMPPSendPack: Unable to download files from your other devices.
[2013-08-26 15:11:33 EDT] XMPPSendPack: Syncing with ejayaitch
[2013-08-26 15:11:33 EDT] XMPPSendPack: Unable to download files from your other devices.
[2013-08-26 15:11:58 EDT] XMPPSendPack: Syncing with ejayaitch
[2013-08-26 15:11:59 EDT] XMPPSendPack: Unable to download files from your other devices.
[2013-08-26 15:11:59 EDT] XMPPSendPack: Syncing with ejayaitch
[2013-08-26 15:11:59 EDT] XMPPSendPack: Unable to download files from your other devices.
Generating public/private rsa key pair.
Your identification has been saved in /data/data/ga.androidterm/tmp/git-annex-keygen.0/key.
Your public key has been saved in /data/data/ga.androidterm/tmp/git-annex-keygen.0/key.pub.
The key fingerprint is:
bb:85:f2:e7:05:ad:82:62:de:27:32:5b:7a:90:c7:3d u0_a91@localhost
The key's randomart image is:
+--[ RSA 2048]----+
| |
| |
| |
| . |
| o .S . . |
| o o.Eo o |
| oo+ +.o . |
| oo=oo.+.. |
| +=.ooo. |
+-----------------+
[2013-08-26 15:13:07 EDT] main: Syncing with edheil.noip.org_androidannex
warning: no common commits
From ssh://git-annex-edheil.no-ip.org-ed_androidannex/~/androidannex
* [new branch] git-annex -> edheil.noip.org_androidannex/git-annex
* [new branch] master -> edheil.noip.org_androidannex/master
* [new branch] synced/git-annex -> edheil.noip.org_androidannex/synced/git-annex
* [new branch] synced/master -> edheil.noip.org_androidannex/synced/master
Already up-to-date!
Merge made by the 'recursive' strategy.
Already up-to-date.
[2013-08-26 15:13:10 EDT] Pusher: Syncing with edheil.noip.org_androidannex
Already up-to-date.
Already up-to-date.
warning: no threads support, ignoring --threads
warning: no threads support, ignoring --threads
To ssh://ed@git-annex-edheil.no-ip.org-ed_androidannex/~/androidannex/
54a6ede..b502e21 git-annex -> synced/git-annex
67690e7..0bdd057 master -> synced/master
error: Ref refs/heads/synced/git-annex is at b502e2176d04a713d49b019e10b13dc9d30835bd but expected 54a6eded4adf9c926a15675351e3feaa0083cf46
remote: error: failed to lock refs/heads/synced/git-annex
error: Ref refs/heads/synced/master is at 0bdd0575caf5c1992015175363e767ab526b5db7 but expected 67690e7d9c6da81792d14e7389e43a75abfd5a36
remote: error: failed to lock refs/heads/synced/master
To ssh://ed@git-annex-edheil.no-ip.org-ed_androidannex/~/androidannex/
! [remote rejected] git-annex -> synced/git-annex (failed to lock)
! [remote rejected] master -> synced/master (failed to lock)
error: failed to push some refs to 'ssh://ed@git-annex-edheil.no-ip.org-ed_androidannex/~/androidannex/'
Already up-to-date.
Everything up-to-date
fatal: The remote end hung up unexpectedly
[2013-08-26 15:13:33 EDT] XMPPReceivePack: Syncing with ejayaitch
Already up-to-date.
From ssh://git-annex-edheil.no-ip.org-ed_androidannex/~/androidannex
b502e21..5c2452d synced/git-annex -> edheil.noip.org_androidannex/synced/git-annex
[2013-08-26 15:15:10 EDT] read: git [\"--git-dir=/storage/emulated/legacy/annex/.git\",\"--work-tree=/storage/emulated/legacy/annex\",\"ls-tree\",\"-z\",\"--\",\"refs/heads/git-annex\",\"uuid.log\",\"remote.log\",\"trust.log\",\"group.log\",\"preferred-content.log\"]
[2013-08-26 15:15:12 EDT] XMPPClient: received: [\"Unknown message\"]
[2013-08-26 15:15:33 EDT] XMPPClient: received: [\"Pushing \\"e28\\" (ReceivePackDone (ExitFailure 128))\"]
[2013-08-26 15:15:33 EDT] XMPPClient: NetMessager stored Pushing \"e28\" (ReceivePackDone (ExitFailure 128)) in SendPack inbox \"e28\"
[2013-08-26 15:17:12 EDT] XMPPClient: received: [\"Unknown message\"]
[2013-08-26 15:17:33 EDT] XMPPClient: received: [\"Presence from h10/messaging-TalkGadgetwADC317A1 Nothing\"]
[2013-08-26 15:17:42 EDT] XMPPClient: received: [\"Presence from 326/MessagingA7afbdd8d Nothing\"]
[2013-08-26 15:17:42 EDT] XMPPClient: received: [\"Presence from 326/MessagingA7afbdd8d Nothing\"]
[2013-08-26 15:18:25 EDT] XMPPClient: received: [\"Presence from h10/messaging-TalkGadgetwADC317A1 Nothing\"]
[2013-08-26 15:19:12 EDT] XMPPClient: received: [\"Unknown message\"]
"""]]

9
doc/bugs/android:_high_CPU_usage__44___unclear_how_to_quit/comment_6_506acc4275a81ed9e9b08e8a40fcf96a._comment Normal file
View file

@ -0,0 +1,9 @@
[[!comment format=mdwn
username="http://edheil.wordpress.com/"
ip="173.162.44.162"
subject="comment 6"
date="2013-08-26T19:38:12Z"
content="""
a final note -- according to the process monitor, the process eating CPU is \"ga.androidterm\". There are also a small swarm of about 6 git processes none of which are consuming much CPU time at all.
"""]]

10
doc/bugs/android:_high_CPU_usage__44___unclear_how_to_quit/comment_7_d38d6f40db4c9437764c7b2ddf36b5a9._comment Normal file
View file

@ -0,0 +1,10 @@
[[!comment format=mdwn
username="http://joeyh.name/"
ip="4.154.0.63"
subject="comment 7"
date="2013-08-26T20:00:39Z"
content="""
It's certianly possible that the terminal app eats cpu for some reason even when sitting idle. It's hard for me to tell since I've been measuring cpu use by running top inside that terminal, which necessarily seems to use a lot of the CPU just to draw the screen.
If it's the terminal at fault, it would continue after you shutdown the git-annex daemon, since that doesn't close the terminal.
"""]]

23
doc/bugs/git-annex_quit_unexpectedly___40__macosx__41__/comment_1_97abb8442329d19c9687002f43afac74._comment Normal file
View file

@ -0,0 +1,23 @@
[[!comment format=mdwn
username="http://joeyh.name/"
ip="4.154.0.63"
subject="comment 1"
date="2013-08-26T19:51:48Z"
content="""
So this is a crash in the gnutls library used for XMPP. Someone else using OSX reported a similar crash to me by email, reproducible reliably by setting up xmpp with google. This is great debugging info:
<pre>
Thread 3 Crashed:
0 libsystem_c.dylib 0x00007fff86ae0bf9 pthread_mutex_lock + 20
1 H 0x000000010e9fd29f gnutls_system_mutex_lock + 12
2 H 0x000000010ea7fa29 wrap_nettle_rnd_refresh + 20
3 H 0x000000010e9fee89 gnutls_deinit + 42
4 git-annex 0x000000010caf0a3a 0x10b5e6000 + 22063674
</pre>
Looks like `gnutls_deinit` was called and it crashed there, inside pthread code. So I think git-annex has already managed to connect to the XMPP server (to test it) and the cleanup is where it's crashing.
This is the second time I have seen a gnutls-related crash using XMPP. The other one was tracked down by John Millikin to a resource allocation bug in haskell-gnutls and fixed.
I have written a test case that reproduces the problem -- just forking a dozen threads that each try to connect to the google xmpp server and then close the connection. After a dozen or so succeed, one will reliably cause a segfault. Forwarded this test case to John.
"""]]

12
doc/forum/Manual_Setup_of_a_Central_Repo/comment_1_3a163fd5629dc40423f1290a78ae1c07._comment Normal file
View file

@ -0,0 +1,12 @@
[[!comment format=mdwn
username="http://joeyh.name/"
ip="4.154.0.63"
subject="comment 1"
date="2013-08-26T18:46:06Z"
content="""
You could certianly do that. I don't think it's the easiest way.
Note that this is essentially a git question. It really has nothing to do with git-annex, unless you want to use the git-annex assistant, which can sync a repository over XMPP without needing a central git repository at all.
If I had this problem with git in general, I would make a new empty repository on the server, and push the local repository I have on the one machine to it. Then on the other machine, I would clone from the server. Problem solved, I think?
"""]]

8
doc/forum/howto_update_feed/comment_1_bec356619f370a618f19a187d09d2e35._comment Normal file
View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="http://joeyh.name/"
ip="4.154.0.63"
subject="comment 1"
date="2013-08-26T18:11:07Z"
content="""
I have purposefully kept a list of feeds out of git-annex. I handle this by having a `feeds` file, which I check into git. Then a cron job runs: `xargs git-annex importfeed < feeds`
"""]]

8
doc/forum/howto_update_feed/comment_2_84dfb80ba3db8d41164eb97022becae3._comment Normal file
View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="ringprince"
ip="134.76.140.110"
subject="comment 2"
date="2013-08-26T19:04:21Z"
content="""
Thanks for that. I'll do the same ;-)
"""]]

10
doc/forum/webapp___47___assistant_without_watch/comment_1_1bcd99aa81f937ded683e19a69d33dd9._comment Normal file
View file

@ -0,0 +1,10 @@
[[!comment format=mdwn
username="http://joeyh.name/"
ip="4.154.0.63"
subject="comment 1"
date="2013-08-26T18:14:17Z"
content="""
The assistant is currently only able to transfer files that it has added itself. So if you disable syncing, you have to manually upload any files you add.
I doubt that I will change this in the assistant, because supporting this use case would complicate it unnecessarily for a use case that is not what it's designed to do. It's more likely that `git annex sync` will get an option to also transfer file contents.
"""]]

16
doc/forum/webapp___47___assistant_without_watch/comment_2_9f5b3f5bf7fedcd5baec519d97d3aa8c._comment Normal file
View file

@ -0,0 +1,16 @@
[[!comment format=mdwn
username="ringprince"
ip="134.76.140.110"
subject="comment 2"
date="2013-08-26T18:54:40Z"
content="""
Thanks for the clarification.
Although
* not what I hoped for ;-)
* I don't see that it would get complicated
* I do not consider my use case special
Anyway, thanks for this great piece of software.
"""]]

8
doc/special_remotes/rsync/comment_11_8cafc1a8b37e6fb056185ec058c0c3e8._comment Normal file
View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="http://joeyh.name/"
ip="4.154.0.63"
subject="comment 11"
date="2013-08-26T18:36:03Z"
content="""
@Chris yes in that case you still need, a central git repository (which need not be on a host that supports git-annex), or the assistant can use xmpp to sync the git data.
"""]]