From b52c54ac36433756a458b0a5d4a7876bf76615f0 Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Thu, 6 Mar 2014 18:12:57 +0000 Subject: [PATCH 01/12] Added a comment --- ...comment_5_6b9b87bfb8b94171b3dba51919fd1ceb._comment | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 doc/bugs/Assistant_lost_dbus_connection_spamming_log/comment_5_6b9b87bfb8b94171b3dba51919fd1ceb._comment diff --git a/doc/bugs/Assistant_lost_dbus_connection_spamming_log/comment_5_6b9b87bfb8b94171b3dba51919fd1ceb._comment b/doc/bugs/Assistant_lost_dbus_connection_spamming_log/comment_5_6b9b87bfb8b94171b3dba51919fd1ceb._comment new file mode 100644 index 0000000000..d612cf35f6 --- /dev/null +++ b/doc/bugs/Assistant_lost_dbus_connection_spamming_log/comment_5_6b9b87bfb8b94171b3dba51919fd1ceb._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="209.250.56.146" + subject="comment 5" + date="2014-03-06T18:12:57Z" + content=""" +Again the accept message does not seem to be related to dbus. A dbus client has no reason to do that; a web server does. The use of `O_NONBLOCK` with accept4 seems likely to be the culprit to me. + +How frequently is dbus mentioned in the log? +"""]] From 9efc14a164500062a1df35aea5e38c6fdb24a24a Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Thu, 6 Mar 2014 18:14:37 +0000 Subject: [PATCH 02/12] Added a comment --- .../comment_5_9fe529034ad0115792b58d7da99c167e._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/bugs/Auto-repair_greatly_slows_down_the_machine/comment_5_9fe529034ad0115792b58d7da99c167e._comment diff --git a/doc/bugs/Auto-repair_greatly_slows_down_the_machine/comment_5_9fe529034ad0115792b58d7da99c167e._comment b/doc/bugs/Auto-repair_greatly_slows_down_the_machine/comment_5_9fe529034ad0115792b58d7da99c167e._comment new file mode 100644 index 0000000000..4683165c37 --- /dev/null +++ b/doc/bugs/Auto-repair_greatly_slows_down_the_machine/comment_5_9fe529034ad0115792b58d7da99c167e._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="209.250.56.146" + subject="comment 5" + date="2014-03-06T18:14:37Z" + content=""" +auto-repair is only done if git fsck detects a problem. You can run git fsck yourself to see. +"""]] From c8fe50f8f4a86b5d004e42ac0b0a714d5bc29a07 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawmUJBh1lYmvfCCiGr3yrdx-QhuLCSRnU5c" Date: Thu, 6 Mar 2014 18:21:53 +0000 Subject: [PATCH 03/12] Added a comment --- ...3_46305aa2d43da000c1a7cb003c822572._comment | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) create mode 100644 doc/forum/copy_fails_for_some_fails_without_explanation/comment_3_46305aa2d43da000c1a7cb003c822572._comment diff --git a/doc/forum/copy_fails_for_some_fails_without_explanation/comment_3_46305aa2d43da000c1a7cb003c822572._comment b/doc/forum/copy_fails_for_some_fails_without_explanation/comment_3_46305aa2d43da000c1a7cb003c822572._comment new file mode 100644 index 0000000000..297c28a40e --- /dev/null +++ b/doc/forum/copy_fails_for_some_fails_without_explanation/comment_3_46305aa2d43da000c1a7cb003c822572._comment @@ -0,0 +1,18 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawmUJBh1lYmvfCCiGr3yrdx-QhuLCSRnU5c" + nickname="Justin" + subject="comment 3" + date="2014-03-06T18:21:53Z" + content=""" +> How many files copied are we talking about before it begins to fail? + +Tens of thousands of files processed, but many of them were already on the other remote so didn't invoke cp (or anything else). ~3300 invocations of cp. + +I saved a log of ps aux, and, while the memory used by git annex remains relatively constant, I do observe /tons/ of zombie processes. 3300, actually. + +I didn't check all of them, but all of the zombie pids I checked appear to have corresponded to this command: + + /home/pi/git-annex.linux/shimmed/git/git --git-dir=/home/pi/hdd/annex/.git --work-tree=/home/pi/hdd/annex cat-file --batch + +Perhaps git annex is forgetting to reap this processes? +"""]] From 8d4eb114c6469c0b29d53270d82a23ac82b2aad7 Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Thu, 6 Mar 2014 18:22:31 +0000 Subject: [PATCH 04/12] Added a comment --- ...mment_5_50526283b35997cece2f087507cdd4ee._comment | 12 ++++++++++++ 1 file changed, 12 insertions(+) create mode 100644 doc/forum/workspace.xml_file_disappeared__44___broken_symlink_showed_up/comment_5_50526283b35997cece2f087507cdd4ee._comment diff --git a/doc/forum/workspace.xml_file_disappeared__44___broken_symlink_showed_up/comment_5_50526283b35997cece2f087507cdd4ee._comment b/doc/forum/workspace.xml_file_disappeared__44___broken_symlink_showed_up/comment_5_50526283b35997cece2f087507cdd4ee._comment new file mode 100644 index 0000000000..ebf175cf22 --- /dev/null +++ b/doc/forum/workspace.xml_file_disappeared__44___broken_symlink_showed_up/comment_5_50526283b35997cece2f087507cdd4ee._comment @@ -0,0 +1,12 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="209.250.56.146" + subject="comment 5" + date="2014-03-06T18:22:31Z" + content=""" +Do you edit this file in multiple places? This could be an occurance of this bug: [[bugs/direct_mode_merge_can_overwrite_local,_non-annexed_files]] which is fixed in the latest release. + +If the program that writes the workspace.xml file did so by first deleting it, and then writing the new version, this could result in the assistant committing the deletion, which makes the new version a local, non-annexed file, and then if a pull is received that modified the file, I think the above bug could happen. + +You could tell if this was the case by looking at the git log of the directory containing the file, and see if it has been repeatedly deleted and added back to the repository. The git log snippet you pasted unfortunately does not let me tell this information. +"""]] From ae7b77a62ad27361135b2e87fcf7065954530d93 Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Thu, 6 Mar 2014 18:32:33 +0000 Subject: [PATCH 05/12] Added a comment --- .../comment_4_1dbdeded7f587e8fc2d1ac5170ecb928._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/forum/copy_fails_for_some_fails_without_explanation/comment_4_1dbdeded7f587e8fc2d1ac5170ecb928._comment diff --git a/doc/forum/copy_fails_for_some_fails_without_explanation/comment_4_1dbdeded7f587e8fc2d1ac5170ecb928._comment b/doc/forum/copy_fails_for_some_fails_without_explanation/comment_4_1dbdeded7f587e8fc2d1ac5170ecb928._comment new file mode 100644 index 0000000000..537736a729 --- /dev/null +++ b/doc/forum/copy_fails_for_some_fails_without_explanation/comment_4_1dbdeded7f587e8fc2d1ac5170ecb928._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="209.250.56.146" + subject="comment 4" + date="2014-03-06T18:32:33Z" + content=""" +Old versions of git-annex have known bugs involving zombies. What version? +"""]] From b442d0fa40e06cf065465385999d61b391ee90e5 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawmUJBh1lYmvfCCiGr3yrdx-QhuLCSRnU5c" Date: Thu, 6 Mar 2014 18:35:00 +0000 Subject: [PATCH 06/12] Added a comment --- .../comment_5_1e0c06a07345d85b3712339e6f0d9a9f._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/forum/copy_fails_for_some_fails_without_explanation/comment_5_1e0c06a07345d85b3712339e6f0d9a9f._comment diff --git a/doc/forum/copy_fails_for_some_fails_without_explanation/comment_5_1e0c06a07345d85b3712339e6f0d9a9f._comment b/doc/forum/copy_fails_for_some_fails_without_explanation/comment_5_1e0c06a07345d85b3712339e6f0d9a9f._comment new file mode 100644 index 0000000000..ad2b80d666 --- /dev/null +++ b/doc/forum/copy_fails_for_some_fails_without_explanation/comment_5_1e0c06a07345d85b3712339e6f0d9a9f._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawmUJBh1lYmvfCCiGr3yrdx-QhuLCSRnU5c" + nickname="Justin" + subject="comment 5" + date="2014-03-06T18:35:00Z" + content=""" +5.20140221-g1a47f5f -- I just downloaded it a week or two ago. +"""]] From f3452dbe7f0e31901d0096f3df7a47eb6883fe4c Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Thu, 6 Mar 2014 18:38:43 +0000 Subject: [PATCH 07/12] Added a comment --- ...comment_6_41798e92068eb227c5e75cae2edef68a._comment | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 doc/forum/copy_fails_for_some_fails_without_explanation/comment_6_41798e92068eb227c5e75cae2edef68a._comment diff --git a/doc/forum/copy_fails_for_some_fails_without_explanation/comment_6_41798e92068eb227c5e75cae2edef68a._comment b/doc/forum/copy_fails_for_some_fails_without_explanation/comment_6_41798e92068eb227c5e75cae2edef68a._comment new file mode 100644 index 0000000000..a0554cd8ab --- /dev/null +++ b/doc/forum/copy_fails_for_some_fails_without_explanation/comment_6_41798e92068eb227c5e75cae2edef68a._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="209.250.56.146" + subject="comment 6" + date="2014-03-06T18:38:43Z" + content=""" +Hmm, that version should only start git cat-file --batch a maximum of 10 times (if it is crashing for some reason), and appears to wait on the process if it does crash. And if not, should only start one. + +I think you need to post some git-annex --debug output , to show when it's running this command. +"""]] From 0a920a0cd8b58b17cd28c134f6fcc8f5684d4233 Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Thu, 6 Mar 2014 18:40:26 +0000 Subject: [PATCH 08/12] Added a comment --- .../comment_7_1f33d694a08d8dcbf04595e3442b8cd5._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/forum/copy_fails_for_some_fails_without_explanation/comment_7_1f33d694a08d8dcbf04595e3442b8cd5._comment diff --git a/doc/forum/copy_fails_for_some_fails_without_explanation/comment_7_1f33d694a08d8dcbf04595e3442b8cd5._comment b/doc/forum/copy_fails_for_some_fails_without_explanation/comment_7_1f33d694a08d8dcbf04595e3442b8cd5._comment new file mode 100644 index 0000000000..f846e290d0 --- /dev/null +++ b/doc/forum/copy_fails_for_some_fails_without_explanation/comment_7_1f33d694a08d8dcbf04595e3442b8cd5._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="209.250.56.146" + subject="comment 7" + date="2014-03-06T18:40:26Z" + content=""" +Actually, NM, I have reproduced the bug. +"""]] From 208ff86fe6ec296ecd59b7094fd051fe12409e8f Mon Sep 17 00:00:00 2001 From: "http://grossmeier.net/" Date: Thu, 6 Mar 2014 18:54:08 +0000 Subject: [PATCH 09/12] Added a comment --- ...nt_8_1aaeb808e20c67f89eaac5e45d9309f0._comment | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 doc/bugs/copy_unused_and_unused_not_agreeing/comment_8_1aaeb808e20c67f89eaac5e45d9309f0._comment diff --git a/doc/bugs/copy_unused_and_unused_not_agreeing/comment_8_1aaeb808e20c67f89eaac5e45d9309f0._comment b/doc/bugs/copy_unused_and_unused_not_agreeing/comment_8_1aaeb808e20c67f89eaac5e45d9309f0._comment new file mode 100644 index 0000000000..3480ffa7e5 --- /dev/null +++ b/doc/bugs/copy_unused_and_unused_not_agreeing/comment_8_1aaeb808e20c67f89eaac5e45d9309f0._comment @@ -0,0 +1,15 @@ +[[!comment format=mdwn + username="http://grossmeier.net/" + nickname="greg" + subject="comment 8" + date="2014-03-06T18:54:08Z" + content=""" +The last key listed by unused (111 of 111): + +[[!format sh \"\"\" +greg@x200s:~/Documents/.git/annex/objects/q4/22/SHA256E-s12289--68a93144e03274664d50754882bdaf196134e06ec2b912157bdccae436d577d6.ods$ ls +SHA256E-s12289--68a93144e03274664d50754882bdaf196134e06ec2b912157bdccae436d577d6.ods.cache +SHA256E-s12289--68a93144e03274664d50754882bdaf196134e06ec2b912157bdccae436d577d6.ods.map +greg@x200s:~/Documents/.git/annex/objects/q4/22/SHA256E-s12289--68a93144e03274664d50754882bdaf196134e06ec2b912157bdccae436d577d6.ods$ +\"\"\"]] +"""]] From d8f65aa9573c69cc77ee73573da214ba3d62373e Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Thu, 6 Mar 2014 20:21:28 +0000 Subject: [PATCH 10/12] Added a comment --- ...comment_8_884f31ce917c8e5ce9a32a55da9b42d6._comment | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 doc/forum/copy_fails_for_some_fails_without_explanation/comment_8_884f31ce917c8e5ce9a32a55da9b42d6._comment diff --git a/doc/forum/copy_fails_for_some_fails_without_explanation/comment_8_884f31ce917c8e5ce9a32a55da9b42d6._comment b/doc/forum/copy_fails_for_some_fails_without_explanation/comment_8_884f31ce917c8e5ce9a32a55da9b42d6._comment new file mode 100644 index 0000000000..5ba7442dc6 --- /dev/null +++ b/doc/forum/copy_fails_for_some_fails_without_explanation/comment_8_884f31ce917c8e5ce9a32a55da9b42d6._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="209.250.56.146" + subject="comment 8" + date="2014-03-06T20:21:16Z" + content=""" +Analysis: Remote.Git's onLocal calls Annex.new to make a new AnnexState for the local remote. This state is not cached, and is regenerated for each file. Since it runs a Annex.Branch check of the location log on the remote, it needs to start catFile, and since the state is not reused, a new CatFileHandle is allocated each time. I'm not sure, but there may have been a recent-ish change that caused the location log to get checked and so catfile to be run; the general inneficiency of making a new AnnexState each time is not new. + +Fixing this by caching the AnnexState will not only fix the resource leak, but should speed up local to local copies significantly! +"""]] From 504538c4a8428bae122086d5ae7cb6fa3b6f4f34 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawnR6E5iUghMWdUGlbA9CCs8DKaoigMjJXw" Date: Thu, 6 Mar 2014 20:37:37 +0000 Subject: [PATCH 11/12] Added a comment --- .../comment_1_0d5c90eb0e8fe61b82a19c5fea343613._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/forum/Feature_Request:_Sync_Now_Button_in_Webapp/comment_1_0d5c90eb0e8fe61b82a19c5fea343613._comment diff --git a/doc/forum/Feature_Request:_Sync_Now_Button_in_Webapp/comment_1_0d5c90eb0e8fe61b82a19c5fea343613._comment b/doc/forum/Feature_Request:_Sync_Now_Button_in_Webapp/comment_1_0d5c90eb0e8fe61b82a19c5fea343613._comment new file mode 100644 index 0000000000..a5f631d50f --- /dev/null +++ b/doc/forum/Feature_Request:_Sync_Now_Button_in_Webapp/comment_1_0d5c90eb0e8fe61b82a19c5fea343613._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawnR6E5iUghMWdUGlbA9CCs8DKaoigMjJXw" + nickname="Efraim" + subject="comment 1" + date="2014-03-06T20:37:36Z" + content=""" +not quite a sync button, but when I want to force sync now I turn off and turn on sync for one of the repos from the webapp and then it syncs. +"""]] From fb09e4c4a96890ac43e823e0a1bf6edbb6981639 Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Thu, 6 Mar 2014 21:17:15 +0000 Subject: [PATCH 12/12] Added a comment --- ...comment_9_ab770dafee3bd9212f553db222adbfe6._comment | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 doc/forum/copy_fails_for_some_fails_without_explanation/comment_9_ab770dafee3bd9212f553db222adbfe6._comment diff --git a/doc/forum/copy_fails_for_some_fails_without_explanation/comment_9_ab770dafee3bd9212f553db222adbfe6._comment b/doc/forum/copy_fails_for_some_fails_without_explanation/comment_9_ab770dafee3bd9212f553db222adbfe6._comment new file mode 100644 index 0000000000..e6a23eed5a --- /dev/null +++ b/doc/forum/copy_fails_for_some_fails_without_explanation/comment_9_ab770dafee3bd9212f553db222adbfe6._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="209.250.56.146" + subject="comment 9" + date="2014-03-06T21:17:14Z" + content=""" +Fixed in git. Also reduced the non-data-transfer work done by `git-annex copy` by around 8%. + +I'm going to move this thread to [[bugs]] so I can close it. ;) +"""]]