From 45a5103e50b45042656b2920fab1fcefbccd7ead Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Sat, 20 Jul 2013 20:33:37 +0000 Subject: [PATCH 01/12] Added a comment --- .../comment_3_0ef8c37350fc192d9b784fbab1d9f318._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/forum/wishlist:_make_copy_stop_on_exhausted_disk_space/comment_3_0ef8c37350fc192d9b784fbab1d9f318._comment diff --git a/doc/forum/wishlist:_make_copy_stop_on_exhausted_disk_space/comment_3_0ef8c37350fc192d9b784fbab1d9f318._comment b/doc/forum/wishlist:_make_copy_stop_on_exhausted_disk_space/comment_3_0ef8c37350fc192d9b784fbab1d9f318._comment new file mode 100644 index 0000000000..a33cbcd27c --- /dev/null +++ b/doc/forum/wishlist:_make_copy_stop_on_exhausted_disk_space/comment_3_0ef8c37350fc192d9b784fbab1d9f318._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.154.0.140" + subject="comment 3" + date="2013-07-20T20:33:37Z" + content=""" +Indeed, that was a bug with encrypted directory special remotes. Fixed it. +"""]] From a505e8d47787bf6d0090554176a5e4b3859f4dad Mon Sep 17 00:00:00 2001 From: andy Date: Sat, 20 Jul 2013 21:34:08 +0000 Subject: [PATCH 02/12] Added a comment --- ...mment_5_7ede5ee312f3abdf78979c0d52a7871a._comment | 12 ++++++++++++ 1 file changed, 12 insertions(+) create mode 100644 doc/bugs/git_annex_add_removes_file_with_no_data_left/comment_5_7ede5ee312f3abdf78979c0d52a7871a._comment diff --git a/doc/bugs/git_annex_add_removes_file_with_no_data_left/comment_5_7ede5ee312f3abdf78979c0d52a7871a._comment b/doc/bugs/git_annex_add_removes_file_with_no_data_left/comment_5_7ede5ee312f3abdf78979c0d52a7871a._comment new file mode 100644 index 0000000000..ccf5e574f1 --- /dev/null +++ b/doc/bugs/git_annex_add_removes_file_with_no_data_left/comment_5_7ede5ee312f3abdf78979c0d52a7871a._comment @@ -0,0 +1,12 @@ +[[!comment format=mdwn + username="andy" + ip="108.202.17.204" + subject="comment 5" + date="2013-07-20T21:34:08Z" + content=""" +Yes, the symlink was still broken. Or rather, `tar -tf filename.tar` didn't like the file. + +The file's about 193 M bzipped. I'll be e-mailing you a link to an S3 copy with DojOsEf2 in the subject line of the email--once the upload finishes. :) + +I'm not sure if this is relevant, but the file is a tar of an annex and another git repo. Perhaps that is influencing something (although I presume it shouldn't)? +"""]] From fb15dc3e7cd9c89a1ddedeacac52cd37955c9faf Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Sat, 20 Jul 2013 22:31:02 +0000 Subject: [PATCH 03/12] Added a comment --- ...ment_6_02f0beef1188bfa336bf4220eb5c6286._comment | 13 +++++++++++++ 1 file changed, 13 insertions(+) create mode 100644 doc/bugs/Watcher_crashed:_addWatch:_does_not_exist/comment_6_02f0beef1188bfa336bf4220eb5c6286._comment diff --git a/doc/bugs/Watcher_crashed:_addWatch:_does_not_exist/comment_6_02f0beef1188bfa336bf4220eb5c6286._comment b/doc/bugs/Watcher_crashed:_addWatch:_does_not_exist/comment_6_02f0beef1188bfa336bf4220eb5c6286._comment new file mode 100644 index 0000000000..a21352693e --- /dev/null +++ b/doc/bugs/Watcher_crashed:_addWatch:_does_not_exist/comment_6_02f0beef1188bfa336bf4220eb5c6286._comment @@ -0,0 +1,13 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.154.0.140" + subject="comment 6" + date="2013-07-20T22:31:02Z" + content=""" +Isolated the bug to a problem with the upstream inotify library. + I've sent in a patch to that library that fixes the problem. + +Unfortunately, I cannot work around it in git-annex more than I already have. It'll no longer crash, but will skip over files or directories that contain characters not valid in the current locale. + +I have applied my patch to the haskell-hinotify package in Debian unstable, and have deployed fixed versions to all my linux autobuilds, including Android. (An Android user had mentioned also seeing this bug.) +"""]] From 2e6641281ea118ffbda483f8274864eb86b3191b Mon Sep 17 00:00:00 2001 From: GLITTAH Date: Sat, 20 Jul 2013 22:38:19 +0000 Subject: [PATCH 04/12] --- doc/todo/Bittorrent-like_features.mdwn | 31 ++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) create mode 100644 doc/todo/Bittorrent-like_features.mdwn diff --git a/doc/todo/Bittorrent-like_features.mdwn b/doc/todo/Bittorrent-like_features.mdwn new file mode 100644 index 0000000000..4ec991a65b --- /dev/null +++ b/doc/todo/Bittorrent-like_features.mdwn @@ -0,0 +1,31 @@ +I made an oops and created a wishlist thread in the forum regarding bittorrent-like behaviour. Sorry, my bad. + +Here's the original thread: +http://git-annex.branchable.com/forum/Wishlist:_Bittorrent-like_transfers/ + +I think I summed up pretty well what bittorrent-like features could be added to git-annex in one of the posts, so I'll copy and paste some of it here (with slight clarifications added in). + +>Disclaimer: I'm thinking out loud of what could make git-annex even more awesome. I don't expect this to be implemented any time soon. Please pardon any dumbassery. + +>Having your remotes (optionally!) act like a swarm would be an awesome feature to have because you bring in a lot of new features that optimize storage, bandwidth, and overall traffic usage. This would be made a lot easier if parts of it were implemented in small steps that added a nifty feature. The best part is, each of these could be implemented by themselves, and they're all features that would be really useful. +> +>Step 1. Concurrent downloads of a file from remotes. +> +>This would make sense to have, it saves upload traffic on your remotes, and you also get faster DL speeds on the receiving end. +> +>Step 2. Implementing part of the super-seeding capabilities. +> +>You upload pieces of a file to different remotes from your laptop, and on your desktop you can download all those pieces and put them together again to get a complete file. If you really wanted to get fancy, you could build in redundancy (ala RAID) so if a remote or two gets lost, you don't lose the entire file. This would be a very efficient use of storage if you have a bunch of free cloud storage accounts (~1GB each) and some big files you want to back up. +> +>Step 3. Setting it up so that those remotes could talk to one another and share those pieces. +> +>This is where it gets more like bittorrent. Useful because you upload 1 copy and in a few hours, have say, 5 complete copies on 5 different remotes. You could add or remove remotes from a swarm locally, and push those changes to those remotes, which then adapt themselves to suit the new rules and share those with other remotes in the swarm (rules should be GPG-signed as a safety precaution). Also, if/when deltas get implemented, you could push that delta to the swarm and have all the remotes adopt it. This is cooler than regular bittorrent because the shared file can be updated. As a safety precaution, the delta could be GPG signed so a corrupt file doesn't contaminate the entire swarm. Each remote could have bandwidth/storage limits set in a dotfile. +> +>This is a high-level idea of how it might work, and it's also a HUGE set of features to add, but if implemented, you'd be saving a ton of resources, adding new use cases, and making git-annex more flexible. + +And this: + +>Obviously, Step 3 would only work on remotes that you have control of processes on, but if given login credentials to cloud storage remotes (potentially dangerous!) they could read/write to something like dropbox or rsync. +> +>Another thing, this would be completely trackerless. You just use remote groups (or create swarm definitions) and share those with your remotes. **It's completely decentralized!** + From 97f3579bdceb240bb7743e5f3d5b5f34799e7b8b Mon Sep 17 00:00:00 2001 From: GLITTAH Date: Sat, 20 Jul 2013 22:40:16 +0000 Subject: [PATCH 05/12] --- doc/forum/Wishlist:_Bittorrent-like_transfers.mdwn | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/forum/Wishlist:_Bittorrent-like_transfers.mdwn b/doc/forum/Wishlist:_Bittorrent-like_transfers.mdwn index effd62bdac..8352a4edad 100644 --- a/doc/forum/Wishlist:_Bittorrent-like_transfers.mdwn +++ b/doc/forum/Wishlist:_Bittorrent-like_transfers.mdwn @@ -1,3 +1,5 @@ +**EDIT: Mistakenly posted this thread in the forum. I created a new post in todo. Link: http://git-annex.branchable.com/todo/Bittorrent-like_features/?updated** + Do you think it would be possible to have bittorrent-like transfers between remotes, so that no one remote gets pegged too hard with transfers? It would be great if you distribute your files between multiple bandwidth-capped remotes, and want fast down speed. Obviously, this isn't a simple task, but the protocol is already there, it just needs to be adapted for the purpose (and re-written in Haskell...). Maybe some day in the future after the more important stuff gets taken care of? It could be an enticing stretch goal. PS: still working on getting BTC, will be donating soon! From 8f8d1c4d4e0223522e21512bbb4aece7335aeed0 Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Sat, 20 Jul 2013 22:43:27 +0000 Subject: [PATCH 06/12] Added a comment --- ...ment_6_e37cf18708f09619442c3a9532d12ed9._comment | 13 +++++++++++++ 1 file changed, 13 insertions(+) create mode 100644 doc/bugs/git_annex_add_removes_file_with_no_data_left/comment_6_e37cf18708f09619442c3a9532d12ed9._comment diff --git a/doc/bugs/git_annex_add_removes_file_with_no_data_left/comment_6_e37cf18708f09619442c3a9532d12ed9._comment b/doc/bugs/git_annex_add_removes_file_with_no_data_left/comment_6_e37cf18708f09619442c3a9532d12ed9._comment new file mode 100644 index 0000000000..c1056de9e0 --- /dev/null +++ b/doc/bugs/git_annex_add_removes_file_with_no_data_left/comment_6_e37cf18708f09619442c3a9532d12ed9._comment @@ -0,0 +1,13 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.154.0.140" + subject="comment 6" + date="2013-07-20T22:43:27Z" + content=""" +Got the file. Verified checksum. + +Reproduced bug! + +Wow, it really seems to be a bug specific to this one particular +file content. That's crazy. +"""]] From 3e061cac2eca7955764e1df4c4415e2a2c4f3e45 Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Sat, 20 Jul 2013 23:00:59 +0000 Subject: [PATCH 07/12] Added a comment --- .../comment_7_a744ef7dd3a224a911ebb24858bc2fd6._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/bugs/git_annex_add_removes_file_with_no_data_left/comment_7_a744ef7dd3a224a911ebb24858bc2fd6._comment diff --git a/doc/bugs/git_annex_add_removes_file_with_no_data_left/comment_7_a744ef7dd3a224a911ebb24858bc2fd6._comment b/doc/bugs/git_annex_add_removes_file_with_no_data_left/comment_7_a744ef7dd3a224a911ebb24858bc2fd6._comment new file mode 100644 index 0000000000..aa60ef2fc8 --- /dev/null +++ b/doc/bugs/git_annex_add_removes_file_with_no_data_left/comment_7_a744ef7dd3a224a911ebb24858bc2fd6._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.154.0.140" + subject="comment 7" + date="2013-07-20T23:00:59Z" + content=""" +Can be reproduced with first 500kb of file. I have deleted all the rest of the file, without looking at it. (Scout's honor!) +"""]] From 50b35c3e29ed6a0626cc240d2019467c7b89c9d3 Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Sat, 20 Jul 2013 23:07:17 +0000 Subject: [PATCH 08/12] Added a comment --- ...comment_8_f97141b255073b90120895148220c2d7._comment | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 doc/bugs/git_annex_add_removes_file_with_no_data_left/comment_8_f97141b255073b90120895148220c2d7._comment diff --git a/doc/bugs/git_annex_add_removes_file_with_no_data_left/comment_8_f97141b255073b90120895148220c2d7._comment b/doc/bugs/git_annex_add_removes_file_with_no_data_left/comment_8_f97141b255073b90120895148220c2d7._comment new file mode 100644 index 0000000000..8bde554708 --- /dev/null +++ b/doc/bugs/git_annex_add_removes_file_with_no_data_left/comment_8_f97141b255073b90120895148220c2d7._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.154.0.140" + subject="comment 8" + date="2013-07-20T23:07:17Z" + content=""" +Ok, it is in fact relevant that the file is a tarball of a git-annex repository, because git-annex add turns out to be looking at the beginning of the file, and seeing that it contains a git-annex link. + +I thought that code was only supposed to fire in repos on FAT filesystems that don't have symlinks. So, several issues to fix here, it seems.. +"""]] From 09245d06ae9e97837b8a2630e73797214c46538a Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawm_cen0223TLcWCTPwCPecCQC5JxGnPO04" Date: Sat, 20 Jul 2013 23:12:50 +0000 Subject: [PATCH 09/12] --- ...d_indirect_don__39__t_work_on_android.mdwn | 21 +++++++++++++++++++ 1 file changed, 21 insertions(+) create mode 100644 doc/bugs/uninit_and_indirect_don__39__t_work_on_android.mdwn diff --git a/doc/bugs/uninit_and_indirect_don__39__t_work_on_android.mdwn b/doc/bugs/uninit_and_indirect_don__39__t_work_on_android.mdwn new file mode 100644 index 0000000000..39f2028e0b --- /dev/null +++ b/doc/bugs/uninit_and_indirect_don__39__t_work_on_android.mdwn @@ -0,0 +1,21 @@ +### Please describe the problem. +I am unable to restore a git-annex dir to its pre init state. + +### What steps will reproduce the problem? +init a git-annex dir on android with a file system with out symlinks. +use for a while. +Run: "git-annex uninit" -> You cannot run this command in a direct mode repository. +Run: "git-annex indirect" -> Git is configured to not use symlinks, so you must use direct mode. + +### What version of git-annex are you using? On what operating system? +git-annex version: 4.20130601-g7483ca4 + +### Please provide any additional information below. + +[[!format sh """ +# If you can, paste a complete transcript of the problem occurring here. +# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log + + +# End of transcript or log. +"""]] From 9424461cbe3564a8b79fe64b776f86e5585f2d57 Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Sat, 20 Jul 2013 23:20:48 +0000 Subject: [PATCH 10/12] Added a comment --- .../comment_1_fec69c4c41987b9469eaa8f745c0a124._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/bugs/uninit_and_indirect_don__39__t_work_on_android/comment_1_fec69c4c41987b9469eaa8f745c0a124._comment diff --git a/doc/bugs/uninit_and_indirect_don__39__t_work_on_android/comment_1_fec69c4c41987b9469eaa8f745c0a124._comment b/doc/bugs/uninit_and_indirect_don__39__t_work_on_android/comment_1_fec69c4c41987b9469eaa8f745c0a124._comment new file mode 100644 index 0000000000..e310c1afd8 --- /dev/null +++ b/doc/bugs/uninit_and_indirect_don__39__t_work_on_android/comment_1_fec69c4c41987b9469eaa8f745c0a124._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.154.0.140" + subject="comment 1" + date="2013-07-20T23:20:48Z" + content=""" +There's no way to make indirect mode work on a filesystem w/o symlinks, but it should be possible to make unannex (required for uninit) work in direct mode. Just has not been done yet. +"""]] From 875d4f0489208f648da7c4dfd102101013757920 Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Sat, 20 Jul 2013 23:26:09 +0000 Subject: [PATCH 11/12] Added a comment --- ...comment_9_dd2be11dfd190129d491f5f891e7cd1a._comment | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 doc/bugs/git_annex_add_removes_file_with_no_data_left/comment_9_dd2be11dfd190129d491f5f891e7cd1a._comment diff --git a/doc/bugs/git_annex_add_removes_file_with_no_data_left/comment_9_dd2be11dfd190129d491f5f891e7cd1a._comment b/doc/bugs/git_annex_add_removes_file_with_no_data_left/comment_9_dd2be11dfd190129d491f5f891e7cd1a._comment new file mode 100644 index 0000000000..f7ae63cf8b --- /dev/null +++ b/doc/bugs/git_annex_add_removes_file_with_no_data_left/comment_9_dd2be11dfd190129d491f5f891e7cd1a._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.154.0.140" + subject="comment 9" + date="2013-07-20T23:26:09Z" + content=""" +Ok, I've put in 2 separate bug fixes, any one of which would have been sufficient to prevent this data loss. Am working on a third fix to detect this kind of problem at a higher level and avoid losing content even if it gets all confused. + +This bug may be a candidate to be backported to Debian stable, since it causes data loss. +"""]] From 9d95ea70f03d021c3dbe6ea3eb91cd73763f855d Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawm_cen0223TLcWCTPwCPecCQC5JxGnPO04" Date: Sat, 20 Jul 2013 23:26:23 +0000 Subject: [PATCH 12/12] Added a comment --- .../comment_2_54c3fa77a069b36d03c41aad08fee9af._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/bugs/uninit_and_indirect_don__39__t_work_on_android/comment_2_54c3fa77a069b36d03c41aad08fee9af._comment diff --git a/doc/bugs/uninit_and_indirect_don__39__t_work_on_android/comment_2_54c3fa77a069b36d03c41aad08fee9af._comment b/doc/bugs/uninit_and_indirect_don__39__t_work_on_android/comment_2_54c3fa77a069b36d03c41aad08fee9af._comment new file mode 100644 index 0000000000..ad6d651917 --- /dev/null +++ b/doc/bugs/uninit_and_indirect_don__39__t_work_on_android/comment_2_54c3fa77a069b36d03c41aad08fee9af._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawm_cen0223TLcWCTPwCPecCQC5JxGnPO04" + nickname="Eric" + subject="comment 2" + date="2013-07-20T23:26:23Z" + content=""" +unannex an uninit are the main thing, so that people have an exit strategy for git-annex on android. +"""]]