From dce2de9216d59ebcbcb04500fa84ece5cf96d509 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawkbuIwySiJbYqXEP3w9vfHGAnE3SDv3C9s" Date: Mon, 6 Jan 2014 13:40:15 +0000 Subject: [PATCH 01/15] --- doc/forum/How_do_I_do_with_.gitrefs__47_____63__.mdwn | 1 + 1 file changed, 1 insertion(+) create mode 100644 doc/forum/How_do_I_do_with_.gitrefs__47_____63__.mdwn diff --git a/doc/forum/How_do_I_do_with_.gitrefs__47_____63__.mdwn b/doc/forum/How_do_I_do_with_.gitrefs__47_____63__.mdwn new file mode 100644 index 0000000000..d4a2a71b14 --- /dev/null +++ b/doc/forum/How_do_I_do_with_.gitrefs__47_____63__.mdwn @@ -0,0 +1 @@ +There is .gitrefs directory in my annexed repo while I am unaware. Should I do `git add .gitrefs/` or `git annex add .gitrefs/` or just ignore? From 33b8f9b474c48af75f48974df6661440ee0448aa Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Mon, 6 Jan 2014 15:04:34 +0000 Subject: [PATCH 02/15] Added a comment --- .../comment_3_44a1a6d2db9097de9ae68ea1ff1b08a2._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/design/assistant/transfer_control/comment_3_44a1a6d2db9097de9ae68ea1ff1b08a2._comment diff --git a/doc/design/assistant/transfer_control/comment_3_44a1a6d2db9097de9ae68ea1ff1b08a2._comment b/doc/design/assistant/transfer_control/comment_3_44a1a6d2db9097de9ae68ea1ff1b08a2._comment new file mode 100644 index 0000000000..33ab5565bd --- /dev/null +++ b/doc/design/assistant/transfer_control/comment_3_44a1a6d2db9097de9ae68ea1ff1b08a2._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="209.250.56.35" + subject="comment 3" + date="2014-01-06T15:04:33Z" + content=""" +Just put the box.com remote in the transfer group. There is no need for special subdirectories, the transfer group makes the remote only want files that have not yet reached all the known clients. +"""]] From fd5c9cd6b7d699aacda8339426e96c5cd54063ea Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawkWQXo7MLYIDa-ep04UTbdz8KuvfjwLDCQ" Date: Mon, 6 Jan 2014 15:05:28 +0000 Subject: [PATCH 03/15] --- ...nce_with_Dropbox_results_in_data_loss.mdwn | 46 +++++++++++++++++++ 1 file changed, 46 insertions(+) create mode 100644 doc/bugs/interference_with_Dropbox_results_in_data_loss.mdwn diff --git a/doc/bugs/interference_with_Dropbox_results_in_data_loss.mdwn b/doc/bugs/interference_with_Dropbox_results_in_data_loss.mdwn new file mode 100644 index 0000000000..bf8641dcc1 --- /dev/null +++ b/doc/bugs/interference_with_Dropbox_results_in_data_loss.mdwn @@ -0,0 +1,46 @@ +### Please describe the problem. + +When working with git-annex in a folder managed by Dropbox, the folders names in .git/annex/objects/ are mangled by Dropbox. This leads to lost files. + +This is not a git-annex bug, but the problem is severe and it would be helpful when git-annex can deal with it somehow. Possibilities are: + +* include a warning in documentation (this will be probably found after someone encounters the problem actually) +* issue a warning when running in a Dropbox managed folder +* avoid using filenames that differs in case only (this would make git-annex also more robust against corruption i.e. when a repository is copied over filesystems/protocols without filename case support) + +### What steps will reproduce the problem? + cd ~/Dropbox + mkdir test_annex + cd test_annex + git init + git annex init "test_annex" + cp ~/somefiles/* . + git annex add * + # wait for Dropbox to upload and mangle folder names in .git/annex/objects/ + find -L . -type l | wc -l # print number of broken symlinks + git annex fsck # prints all the missing files + + +### What version of git-annex are you using? On what operating system? + +3.20130207 +Fedora 19 + +### Please provide any additional information below. + +[[!format sh """ +# filder names mangled by Dropbox + +$ ls .git/annex/objects/ +Ff/ M9 (Case Conflict (1))/ x8 (Case Conflict (1))/ +Ff (Case Conflict)/ pF/ x9/ +Ff (Case Conflict (1))/ pF (Case Conflict)/ x9 (Case Conflict)/ +Ff (Case Conflict (2))/ pF (Case Conflict (1))/ Zf/ +Ff (Case Conflict (3))/ pF (Case Conflict (2))/ Zf (Case Conflict)/ +Ff (Case Conflict (4))/ pF (Case Conflict (3))/ Zf (Case Conflict (1))/ +Ff (Case Conflict (5))/ pF (Case Conflict (4))/ Zf (Case Conflict (2))/ +Ff (Case Conflict (6))/ PG/ Zf (Case Conflict (3))/ +... + +# End of transcript or log. +"""]] From 06077b402c2cacd296d597042b2662e691a7cf32 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawkWQXo7MLYIDa-ep04UTbdz8KuvfjwLDCQ" Date: Mon, 6 Jan 2014 15:10:52 +0000 Subject: [PATCH 04/15] added link to Dropbox file naming limitations --- doc/bugs/interference_with_Dropbox_results_in_data_loss.mdwn | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/bugs/interference_with_Dropbox_results_in_data_loss.mdwn b/doc/bugs/interference_with_Dropbox_results_in_data_loss.mdwn index bf8641dcc1..c68588c833 100644 --- a/doc/bugs/interference_with_Dropbox_results_in_data_loss.mdwn +++ b/doc/bugs/interference_with_Dropbox_results_in_data_loss.mdwn @@ -8,6 +8,8 @@ This is not a git-annex bug, but the problem is severe and it would be helpful w * issue a warning when running in a Dropbox managed folder * avoid using filenames that differs in case only (this would make git-annex also more robust against corruption i.e. when a repository is copied over filesystems/protocols without filename case support) +Dropbox file name limitations: [[https://www.dropbox.com/help/145/en]] + ### What steps will reproduce the problem? cd ~/Dropbox mkdir test_annex From 1aea48aa7dce6b78c9f1392ac283b887a8de5236 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawkWQXo7MLYIDa-ep04UTbdz8KuvfjwLDCQ" Date: Mon, 6 Jan 2014 15:12:23 +0000 Subject: [PATCH 05/15] typos --- .../interference_with_Dropbox_results_in_data_loss.mdwn | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/doc/bugs/interference_with_Dropbox_results_in_data_loss.mdwn b/doc/bugs/interference_with_Dropbox_results_in_data_loss.mdwn index c68588c833..a60fd765ea 100644 --- a/doc/bugs/interference_with_Dropbox_results_in_data_loss.mdwn +++ b/doc/bugs/interference_with_Dropbox_results_in_data_loss.mdwn @@ -25,13 +25,13 @@ Dropbox file name limitations: [[https://www.dropbox.com/help/145/en]] ### What version of git-annex are you using? On what operating system? -3.20130207 -Fedora 19 +* git-annex 3.20130207 +* Fedora 19 ### Please provide any additional information below. [[!format sh """ -# filder names mangled by Dropbox +# folder names mangled by Dropbox $ ls .git/annex/objects/ Ff/ M9 (Case Conflict (1))/ x8 (Case Conflict (1))/ From 94149c88f40f213bee2e346db744313d5a912980 Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Mon, 6 Jan 2014 15:44:15 +0000 Subject: [PATCH 06/15] Added a comment --- .../comment_1_5e235af2ea13fd4f6a226c842f69965e._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/forum/How_do_I_do_with_.gitrefs__47_____63__/comment_1_5e235af2ea13fd4f6a226c842f69965e._comment diff --git a/doc/forum/How_do_I_do_with_.gitrefs__47_____63__/comment_1_5e235af2ea13fd4f6a226c842f69965e._comment b/doc/forum/How_do_I_do_with_.gitrefs__47_____63__/comment_1_5e235af2ea13fd4f6a226c842f69965e._comment new file mode 100644 index 0000000000..9e8d044a1e --- /dev/null +++ b/doc/forum/How_do_I_do_with_.gitrefs__47_____63__/comment_1_5e235af2ea13fd4f6a226c842f69965e._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="209.250.56.35" + subject="comment 1" + date="2014-01-06T15:44:15Z" + content=""" +AFAIK neither git nor git-annex creates a .gitrefs directory. +"""]] From 2eb326cc8c3567b04ba6c2049bb54754e8648b90 Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Mon, 6 Jan 2014 15:48:36 +0000 Subject: [PATCH 07/15] Added a comment --- ...ment_4_99286f17a87049c303f2aa34c0a90286._comment | 13 +++++++++++++ 1 file changed, 13 insertions(+) create mode 100644 doc/forum/git_annex_on_osx_only_creating_symlinks__63____63__/comment_4_99286f17a87049c303f2aa34c0a90286._comment diff --git a/doc/forum/git_annex_on_osx_only_creating_symlinks__63____63__/comment_4_99286f17a87049c303f2aa34c0a90286._comment b/doc/forum/git_annex_on_osx_only_creating_symlinks__63____63__/comment_4_99286f17a87049c303f2aa34c0a90286._comment new file mode 100644 index 0000000000..d21d871458 --- /dev/null +++ b/doc/forum/git_annex_on_osx_only_creating_symlinks__63____63__/comment_4_99286f17a87049c303f2aa34c0a90286._comment @@ -0,0 +1,13 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="209.250.56.35" + subject="comment 4" + date="2014-01-06T15:48:36Z" + content=""" +@Tim, you do not seem to be having the problem that the OP described. + +It is completely normal for git-annex to represent a file whose content is not present in the repository as a broken symlink. +Typically, git-annex preserves at least one copy of the file in one of your repositories. But if you `git annex add $file; git annex drop --force $file`, you will be left in exactly the situation you describe. + +You can use `git annex log $file` to see a log of which repositories contained a copy of the file in the past, and see what times the file was removed from each repository. This might give clues to what operation caused the last copy of the file to be lose. +"""]] From 953ac5eb56ff53a907648c55864dbc3acffda8c9 Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Mon, 6 Jan 2014 16:02:45 +0000 Subject: [PATCH 08/15] Added a comment --- ..._837c7ab2d31531ac8a61509225926814._comment | 20 +++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100644 doc/bugs/interference_with_Dropbox_results_in_data_loss/comment_1_837c7ab2d31531ac8a61509225926814._comment diff --git a/doc/bugs/interference_with_Dropbox_results_in_data_loss/comment_1_837c7ab2d31531ac8a61509225926814._comment b/doc/bugs/interference_with_Dropbox_results_in_data_loss/comment_1_837c7ab2d31531ac8a61509225926814._comment new file mode 100644 index 0000000000..256ae83b49 --- /dev/null +++ b/doc/bugs/interference_with_Dropbox_results_in_data_loss/comment_1_837c7ab2d31531ac8a61509225926814._comment @@ -0,0 +1,20 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="209.250.56.35" + subject="comment 1" + date="2014-01-06T16:02:45Z" + content=""" +I'm not seeing data loss. It looks like the contents of your files are still there, you just need to rename them back to the right directory names. If you're using indirect mode, you should be able to do this: + + mv .git/annex/objects . + chmod -R u+w objects + git annex add objects + +That will re-inject all the file contents with the right names. You can then delete the objects directory, which should only contain symlinks at that point. + +---- + +Moving git-annex to using lower case hash directories is on the very long term todo list, due to various crappy filesystems (problems mostly worked around by direct mode). Since it requires changing every symlink in every existing git-annex repository, which will be extremely disruptive, I'd need a much better reason than dropbox to do it now. + +I have no interest, in general, in making git-annex use filenames that meet whatever limitations dropbox wants to impose on its users. +"""]] From 2d3ccd4e4dbb9d24668395a7801490ed2fe3cb87 Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Mon, 6 Jan 2014 16:14:30 +0000 Subject: [PATCH 09/15] Added a comment --- .../comment_2_77295c0b749e984a6fb200d3b73b5765._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/bugs/fatal:_git-write-tree:_error_building_trees/comment_2_77295c0b749e984a6fb200d3b73b5765._comment diff --git a/doc/bugs/fatal:_git-write-tree:_error_building_trees/comment_2_77295c0b749e984a6fb200d3b73b5765._comment b/doc/bugs/fatal:_git-write-tree:_error_building_trees/comment_2_77295c0b749e984a6fb200d3b73b5765._comment new file mode 100644 index 0000000000..dabc1d4f35 --- /dev/null +++ b/doc/bugs/fatal:_git-write-tree:_error_building_trees/comment_2_77295c0b749e984a6fb200d3b73b5765._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="209.250.56.35" + subject="comment 2" + date="2014-01-06T16:14:29Z" + content=""" +As far as I can tell from git's code, this involves a \"cache-tree\" data structure stored in git's index file. git-annex has nothing to do with writing index files, beyond running git plumbing. So I don't see how this could be anything other than a corrupt index file (of a sort that git fsck doesn't detect), or a git bug. The best thing to do would be to send an example of a repository having this problem to the git developers for analysis. +"""]] From a8d1513a887a0e9ba7f9045f81da0dd8f76b409e Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Mon, 6 Jan 2014 16:16:08 +0000 Subject: [PATCH 10/15] Added a comment --- .../comment_4_70e62e6fe75774a65b049c8f000dc6a3._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/forum/Git_annex___39__corrupting__39___itself/comment_4_70e62e6fe75774a65b049c8f000dc6a3._comment diff --git a/doc/forum/Git_annex___39__corrupting__39___itself/comment_4_70e62e6fe75774a65b049c8f000dc6a3._comment b/doc/forum/Git_annex___39__corrupting__39___itself/comment_4_70e62e6fe75774a65b049c8f000dc6a3._comment new file mode 100644 index 0000000000..728b4d855e --- /dev/null +++ b/doc/forum/Git_annex___39__corrupting__39___itself/comment_4_70e62e6fe75774a65b049c8f000dc6a3._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="209.250.56.35" + subject="comment 4" + date="2014-01-06T16:16:08Z" + content=""" +This appears to be some form of corruption of git's index file. Someone will need to publish a git repository that has this problem in order for it to be investigated. +"""]] From 2283d5191fa5f549817cfe8d6edb8a8dbeb79693 Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Mon, 6 Jan 2014 16:22:55 +0000 Subject: [PATCH 11/15] Added a comment --- ...omment_3_10fa5082909f5e568623cf6d901d5161._comment | 11 +++++++++++ 1 file changed, 11 insertions(+) create mode 100644 doc/bugs/Assistant_lost_dbus_connection_spamming_log/comment_3_10fa5082909f5e568623cf6d901d5161._comment diff --git a/doc/bugs/Assistant_lost_dbus_connection_spamming_log/comment_3_10fa5082909f5e568623cf6d901d5161._comment b/doc/bugs/Assistant_lost_dbus_connection_spamming_log/comment_3_10fa5082909f5e568623cf6d901d5161._comment new file mode 100644 index 0000000000..249eb9c79e --- /dev/null +++ b/doc/bugs/Assistant_lost_dbus_connection_spamming_log/comment_3_10fa5082909f5e568623cf6d901d5161._comment @@ -0,0 +1,11 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="209.250.56.35" + subject="comment 3" + date="2014-01-06T16:22:49Z" + content=""" +Is dbus running at all? + +What is the frequency of the messages? + +"""]] From fcc566c7348be33f4e637b73d14cca75128f62a7 Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Mon, 6 Jan 2014 16:26:17 +0000 Subject: [PATCH 12/15] Added a comment --- ...comment_4_1c46b4ad0c981c6105ffb8531223f0b1._comment | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 doc/bugs/gsha256sum_crashes/comment_4_1c46b4ad0c981c6105ffb8531223f0b1._comment diff --git a/doc/bugs/gsha256sum_crashes/comment_4_1c46b4ad0c981c6105ffb8531223f0b1._comment b/doc/bugs/gsha256sum_crashes/comment_4_1c46b4ad0c981c6105ffb8531223f0b1._comment new file mode 100644 index 0000000000..a188bf6f64 --- /dev/null +++ b/doc/bugs/gsha256sum_crashes/comment_4_1c46b4ad0c981c6105ffb8531223f0b1._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="209.250.56.35" + subject="comment 4" + date="2014-01-06T16:26:16Z" + content=""" +So far, only Jan has seen this problem, and it has affected both wget and gsha256sum. + +This makes me wonder if it's somehow a problem with running 64 bit code on his system. +"""]] From cb3351b67deda8ca593e038bd20c5ba7d481642f Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Mon, 6 Jan 2014 16:47:49 +0000 Subject: [PATCH 13/15] Added a comment --- ...t_5_3fa106ed7fb30226ee7c48b66edb963c._comment | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) create mode 100644 doc/bugs/gsha256sum_crashes/comment_5_3fa106ed7fb30226ee7c48b66edb963c._comment diff --git a/doc/bugs/gsha256sum_crashes/comment_5_3fa106ed7fb30226ee7c48b66edb963c._comment b/doc/bugs/gsha256sum_crashes/comment_5_3fa106ed7fb30226ee7c48b66edb963c._comment new file mode 100644 index 0000000000..43ca8d31c5 --- /dev/null +++ b/doc/bugs/gsha256sum_crashes/comment_5_3fa106ed7fb30226ee7c48b66edb963c._comment @@ -0,0 +1,16 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="209.250.56.35" + subject="comment 5" + date="2014-01-06T16:47:43Z" + content=""" +Searching for \"OSX SIGILL\" finds rather a lot of reports of problems like this. + + + + + +> My libraries were built using homebrew. By default homebrew optimizes for native architecture, resulting in machine instructions that don't work on older machines. + +Suggests to me that the OSX autobuilder probably needs to run the build with PATH set to a directory with un-optimised builds (or fat binary builds?) of all the utilities included in the app. Seems we're lucky and git-annex itself is not built with whatever sort of optimisations (SSE?) is causing this. +"""]] From 9480022dcef3f588ef9da7a76828d053cceb815b Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Mon, 6 Jan 2014 18:01:44 +0000 Subject: [PATCH 14/15] Added a comment --- ...mment_6_276b181b2aeb1512e0468b88598e0a84._comment | 12 ++++++++++++ 1 file changed, 12 insertions(+) create mode 100644 doc/bugs/gsha256sum_crashes/comment_6_276b181b2aeb1512e0468b88598e0a84._comment diff --git a/doc/bugs/gsha256sum_crashes/comment_6_276b181b2aeb1512e0468b88598e0a84._comment b/doc/bugs/gsha256sum_crashes/comment_6_276b181b2aeb1512e0468b88598e0a84._comment new file mode 100644 index 0000000000..9c0c40d671 --- /dev/null +++ b/doc/bugs/gsha256sum_crashes/comment_6_276b181b2aeb1512e0468b88598e0a84._comment @@ -0,0 +1,12 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="209.250.56.35" + subject="comment 6" + date="2014-01-06T18:01:43Z" + content=""" +I have updated the daily autobuild to use wget and coreutils (for gsha256sum) built using `brew install --build-bottle`. This should mean they are built portably without optimisations for the local CPU. + +That seems to be all the software on the build host that was installed using homebrew. So I hope the included rsync, curl xargs, etc are ok without being rebuilt. + +This all needs to be tested on a system affected by this problem.. +"""]] From ae4d8a66b1a239213d8d55f3d478c7f8b2787999 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawkipQLNyt8RHREHpg2k5wdYeRSCCvSNSBg" Date: Mon, 6 Jan 2014 18:17:44 +0000 Subject: [PATCH 15/15] Added a comment: I'll start another thread --- .../comment_5_39bad7441dcea4da4b389700301233de._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/forum/git_annex_on_osx_only_creating_symlinks__63____63__/comment_5_39bad7441dcea4da4b389700301233de._comment diff --git a/doc/forum/git_annex_on_osx_only_creating_symlinks__63____63__/comment_5_39bad7441dcea4da4b389700301233de._comment b/doc/forum/git_annex_on_osx_only_creating_symlinks__63____63__/comment_5_39bad7441dcea4da4b389700301233de._comment new file mode 100644 index 0000000000..f7975fc0ee --- /dev/null +++ b/doc/forum/git_annex_on_osx_only_creating_symlinks__63____63__/comment_5_39bad7441dcea4da4b389700301233de._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawkipQLNyt8RHREHpg2k5wdYeRSCCvSNSBg" + nickname="Tim" + subject="I'll start another thread" + date="2014-01-06T18:17:44Z" + content=""" +Let me start another thread then and not pollute this one. Thanks! +"""]]