From d3faa74233e5a23441bfd94357f35b726d254935 Mon Sep 17 00:00:00 2001 From: "https://rmunn.myopenid.com/" Date: Sat, 24 Aug 2013 15:31:36 +0000 Subject: [PATCH 01/14] Added a comment --- .../comment_4_a57947ed257b28bbe995a68bfeb5eeaa._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/todo/wishlist:_special-case_handling_of_Youtube_URLs_in_Web_special_remote/comment_4_a57947ed257b28bbe995a68bfeb5eeaa._comment diff --git a/doc/todo/wishlist:_special-case_handling_of_Youtube_URLs_in_Web_special_remote/comment_4_a57947ed257b28bbe995a68bfeb5eeaa._comment b/doc/todo/wishlist:_special-case_handling_of_Youtube_URLs_in_Web_special_remote/comment_4_a57947ed257b28bbe995a68bfeb5eeaa._comment new file mode 100644 index 0000000000..ee0ab45e71 --- /dev/null +++ b/doc/todo/wishlist:_special-case_handling_of_Youtube_URLs_in_Web_special_remote/comment_4_a57947ed257b28bbe995a68bfeb5eeaa._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="https://rmunn.myopenid.com/" + nickname="rmunn" + subject="comment 4" + date="2013-08-24T15:31:36Z" + content=""" +Either quvi or youtube-dl might be a good possibility: youtube-dl has the --get-url option (or -g for short) that outputs just the download URL (and nothing else) to stdout. So if for some reason quvi turned out not to be suitable, it wouldn't necessarily mean the idea would have to be abandoned. +"""]] From e58b237a955f86c5607270606c95fd8ec1b1faf0 Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Sat, 24 Aug 2013 15:54:03 +0000 Subject: [PATCH 02/14] Added a comment --- ...comment_2_ecb411a2c4d67917b734a90bd460d44b._comment | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 doc/forum/Use_reflinks_on_BTRFS_instead_of_symlinks___63__/comment_2_ecb411a2c4d67917b734a90bd460d44b._comment diff --git a/doc/forum/Use_reflinks_on_BTRFS_instead_of_symlinks___63__/comment_2_ecb411a2c4d67917b734a90bd460d44b._comment b/doc/forum/Use_reflinks_on_BTRFS_instead_of_symlinks___63__/comment_2_ecb411a2c4d67917b734a90bd460d44b._comment new file mode 100644 index 0000000000..a4ade57926 --- /dev/null +++ b/doc/forum/Use_reflinks_on_BTRFS_instead_of_symlinks___63__/comment_2_ecb411a2c4d67917b734a90bd460d44b._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.154.0.63" + subject="comment 2" + date="2013-08-24T15:54:02Z" + content=""" +git-annex has always used cp --reflink=auto, including when it cp's the file in `git annex unlock`. + +You can verify this with --debug. +"""]] From 38086457f4a805120fa419e04400dec538f347ed Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Sat, 24 Aug 2013 15:56:47 +0000 Subject: [PATCH 03/14] Added a comment --- .../comment_15_1cd32456630b25d5aaa6d2763e6eb384._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/direct_mode/comment_15_1cd32456630b25d5aaa6d2763e6eb384._comment diff --git a/doc/direct_mode/comment_15_1cd32456630b25d5aaa6d2763e6eb384._comment b/doc/direct_mode/comment_15_1cd32456630b25d5aaa6d2763e6eb384._comment new file mode 100644 index 0000000000..d265118a04 --- /dev/null +++ b/doc/direct_mode/comment_15_1cd32456630b25d5aaa6d2763e6eb384._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.154.0.63" + subject="comment 15" + date="2013-08-24T15:56:47Z" + content=""" +No, you can sync before you copy, get, or whatever. git-annex will replace the symlinks with the actual files when they arrive at the repository. +"""]] From a7767effdd1ae2f1c55819a2eb358135211cd13a Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Sat, 24 Aug 2013 15:58:29 +0000 Subject: [PATCH 04/14] Added a comment --- .../comment_6_ae323b16ddb9342e91be955408eca3b1._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/forum/Using_git_annex_with_a_SMB__47__FTP_TV_NAS_with_preconfigured_dirs/comment_6_ae323b16ddb9342e91be955408eca3b1._comment diff --git a/doc/forum/Using_git_annex_with_a_SMB__47__FTP_TV_NAS_with_preconfigured_dirs/comment_6_ae323b16ddb9342e91be955408eca3b1._comment b/doc/forum/Using_git_annex_with_a_SMB__47__FTP_TV_NAS_with_preconfigured_dirs/comment_6_ae323b16ddb9342e91be955408eca3b1._comment new file mode 100644 index 0000000000..44d3f5a85c --- /dev/null +++ b/doc/forum/Using_git_annex_with_a_SMB__47__FTP_TV_NAS_with_preconfigured_dirs/comment_6_ae323b16ddb9342e91be955408eca3b1._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.154.0.63" + subject="comment 6" + date="2013-08-24T15:58:28Z" + content=""" +You can run any of these operations in any order and it will work. Ie, this is far more complicated and detailed than it needs to be. Just run git-annex commands in any order to do what you want to do. If they don't, file a (detailed) bug report. +"""]] From 6f4de7c9cbbb32d27cd1a3acd109fcf80b4c8248 Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Sat, 24 Aug 2013 16:04:21 +0000 Subject: [PATCH 05/14] Added a comment --- ...mment_3_d106a87101db52f957da84d90dafcdbb._comment | 12 ++++++++++++ 1 file changed, 12 insertions(+) create mode 100644 doc/bugs/Problem_when_dropping_unused_files/comment_3_d106a87101db52f957da84d90dafcdbb._comment diff --git a/doc/bugs/Problem_when_dropping_unused_files/comment_3_d106a87101db52f957da84d90dafcdbb._comment b/doc/bugs/Problem_when_dropping_unused_files/comment_3_d106a87101db52f957da84d90dafcdbb._comment new file mode 100644 index 0000000000..8c739552da --- /dev/null +++ b/doc/bugs/Problem_when_dropping_unused_files/comment_3_d106a87101db52f957da84d90dafcdbb._comment @@ -0,0 +1,12 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.154.0.63" + subject="comment 3" + date="2013-08-24T16:04:21Z" + content=""" +Unfortunately, the git error message you pasted suggests very strongly that your git repository has gotten corrupted. You can probably verify that by running `git annex fsck`. Assuming it is corrupted, the best thing to do is to make a new clone and move .git/annex and .git/config over from the corrupted repository to it, and finally run `git annex fsck` + +You then seem to have some reason decided to go delete git-annex's .git/annex/journal/*.log files, which are just files that have not yet been committed to the git-annex branch. The only reason this \"restored some sanity\" is that git-annex was trying to commit that data to git, and failing because your git repository is corrupted. + +(This has nothing at all to do with direct mode.) +"""]] From 4dc4dc4a3ee32b16df32060d1267a7cd3e49002e Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Sat, 24 Aug 2013 16:20:30 +0000 Subject: [PATCH 06/14] Added a comment --- ...comment_1_4a0468b6ca2ffff8ef8f19800597567d._comment | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 doc/forum/Manual_mode_option_in_assistant_auto-syncs/comment_1_4a0468b6ca2ffff8ef8f19800597567d._comment diff --git a/doc/forum/Manual_mode_option_in_assistant_auto-syncs/comment_1_4a0468b6ca2ffff8ef8f19800597567d._comment b/doc/forum/Manual_mode_option_in_assistant_auto-syncs/comment_1_4a0468b6ca2ffff8ef8f19800597567d._comment new file mode 100644 index 0000000000..56e68f779b --- /dev/null +++ b/doc/forum/Manual_mode_option_in_assistant_auto-syncs/comment_1_4a0468b6ca2ffff8ef8f19800597567d._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.154.0.63" + subject="comment 1" + date="2013-08-24T16:20:29Z" + content=""" +In the assistant, putting a repository in manual mode prevents the assistant from automatically downloading every file that is added to other clones of the repository. As you've noticed, it does not prevent the assistant from automatically adding new files that are put into the local repository, or from uploading those files to other clients that want them (ie, clients not themselves set to use manual mode). + +You can prevent the assistant from noticing when you add new files to a repository by clicking on \"syncing enabled\" in the first repository in the list. (The repository labeled as \"here\".) You can then `git annex add` the files you want to add by hand, and manually `copy annex copy` them to other repositories, and manually `git annex sync`. +"""]] From 578700d88a39ef2a44b230c0119637efa32eaa6c Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Sat, 24 Aug 2013 16:35:34 +0000 Subject: [PATCH 07/14] Added a comment --- ...ent_3_b9f73375e2c732e798495f8ee6970c7c._comment | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 doc/todo/sync_my_local_git-annex_from_a_dump_remote/comment_3_b9f73375e2c732e798495f8ee6970c7c._comment diff --git a/doc/todo/sync_my_local_git-annex_from_a_dump_remote/comment_3_b9f73375e2c732e798495f8ee6970c7c._comment b/doc/todo/sync_my_local_git-annex_from_a_dump_remote/comment_3_b9f73375e2c732e798495f8ee6970c7c._comment new file mode 100644 index 0000000000..df4be033bb --- /dev/null +++ b/doc/todo/sync_my_local_git-annex_from_a_dump_remote/comment_3_b9f73375e2c732e798495f8ee6970c7c._comment @@ -0,0 +1,14 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.154.0.63" + subject="comment 3" + date="2013-08-24T16:35:33Z" + content=""" +Seems to me that this could easily be dealt with by installing git-annex on the webserver, making the directory there a git repository, and using either a cron job or `git annex watch` to commit files as they were changed there. + +Then you can make a direct mode, indirect mode, or even a bare clone on your local machine and use git-annex to get the files. + +Maybe you have good reasons for not wanting to go that route. And rsync on a direct mode repository should work, provided to tell it to not delete `.git`. :P I don't see any way to make rsync work in an indirect mode repository. As for trying to make git-annex handle this import over rsync itself in a way that would work in an indirect mode repository, let alone a bare repository -- I don't see a good way to do it and it seems quite special case and likely to get quite complicated to implement. + +In the meantime, I did implement `git annex mirror`, which I think is a much more interesting and generally useful tool to have. And could even be used in my recommended solution above. +"""]] From f992f73610a0382e44bfcb221fae7ce770d3e5b2 Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Sat, 24 Aug 2013 16:41:04 +0000 Subject: [PATCH 08/14] Added a comment --- ...comment_1_9ef6e694ef8a8eee7a42f88554475db7._comment | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 doc/bugs/immediately_drops_files/comment_1_9ef6e694ef8a8eee7a42f88554475db7._comment diff --git a/doc/bugs/immediately_drops_files/comment_1_9ef6e694ef8a8eee7a42f88554475db7._comment b/doc/bugs/immediately_drops_files/comment_1_9ef6e694ef8a8eee7a42f88554475db7._comment new file mode 100644 index 0000000000..9d630e385e --- /dev/null +++ b/doc/bugs/immediately_drops_files/comment_1_9ef6e694ef8a8eee7a42f88554475db7._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.154.0.63" + subject="comment 1" + date="2013-08-24T16:41:04Z" + content=""" +You are running the git annex assistant, which takes care of getting and dropping files as it's configured to do, and you are then going in and manually running `git annex get`. If the assistant sees a file's content has appeared, and that file is in a directory that it has been configured to not want the content of file in (the `archive` directory by default), it will immediately try to drop it. + +The only thing I don't understand is why the number of `..` in the symlink would change. +"""]] From 1dc27524757337bfccc2d8869723b09f48df945b Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Sat, 24 Aug 2013 16:41:47 +0000 Subject: [PATCH 09/14] Added a comment --- .../comment_2_76e4f8b73ab60b2540dd2a3e5379791d._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/bugs/immediately_drops_files/comment_2_76e4f8b73ab60b2540dd2a3e5379791d._comment diff --git a/doc/bugs/immediately_drops_files/comment_2_76e4f8b73ab60b2540dd2a3e5379791d._comment b/doc/bugs/immediately_drops_files/comment_2_76e4f8b73ab60b2540dd2a3e5379791d._comment new file mode 100644 index 0000000000..274090c63e --- /dev/null +++ b/doc/bugs/immediately_drops_files/comment_2_76e4f8b73ab60b2540dd2a3e5379791d._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.154.0.63" + subject="comment 2" + date="2013-08-24T16:41:47Z" + content=""" +BTW if you don't want the assistant to automatically get and drop files in your repository, you can edit the repository in the webapp and select \"manual mode\". +"""]] From 623960454397e4cc0ab3c36a1091a81c45402f41 Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Sat, 24 Aug 2013 17:09:43 +0000 Subject: [PATCH 10/14] Added a comment --- ...5_33a2e783e5355e981497b9861997570b._comment | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) create mode 100644 doc/bugs/git-annex_opens_too_many_files/comment_5_33a2e783e5355e981497b9861997570b._comment diff --git a/doc/bugs/git-annex_opens_too_many_files/comment_5_33a2e783e5355e981497b9861997570b._comment b/doc/bugs/git-annex_opens_too_many_files/comment_5_33a2e783e5355e981497b9861997570b._comment new file mode 100644 index 0000000000..786672eb20 --- /dev/null +++ b/doc/bugs/git-annex_opens_too_many_files/comment_5_33a2e783e5355e981497b9861997570b._comment @@ -0,0 +1,18 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.154.0.63" + subject="comment 5" + date="2013-08-24T17:09:43Z" + content=""" +Thanks for an excellent amount of debug information. + +I can see what's leaking is UDP connections to 224.0.0.251. That address is used for the \"local pairing\" option in the webapp. + +I was able to reproduce the problem by disabling all network interfaces except `lo`. The PairListener then failed to open a multicast listening socket. When it fails that way, it retried every second, as you noticed. And there is a socket leak in that failure mode. + +I wonder if you're seeing this even when on the network? If so, perhaps your Ubuntu system has something going on that prevents opening a multicast listening socket on even `eth0` or `wlan0` or whatever. + +Unfortunately, the actual socket leak bug is in the [network-multicast](http://hackage.haskell.org/package/network-multicast) library, and not in git-annex. I have filed an upstream bug report: + +Hopefully that will be dealt with soon. There is a workaround I could do in git-annex: If it fails (leaking one socket), it could wait until the NetworkListener indicated a new network interface was opened, before trying again (possibly leaking one socket again). This would change it from a 1 per second leak to a 1 per change of network leak at worst, which is probably much less likely to cause problems. +"""]] From 9fde4aad23bd6a2eb807c0c43593768aa34b76a3 Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Sat, 24 Aug 2013 18:38:56 +0000 Subject: [PATCH 11/14] Added a comment --- ...nt_6_b3a5a4e4ca29c5cd2840bfeb4c63ea68._comment | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 doc/bugs/git-annex_opens_too_many_files/comment_6_b3a5a4e4ca29c5cd2840bfeb4c63ea68._comment diff --git a/doc/bugs/git-annex_opens_too_many_files/comment_6_b3a5a4e4ca29c5cd2840bfeb4c63ea68._comment b/doc/bugs/git-annex_opens_too_many_files/comment_6_b3a5a4e4ca29c5cd2840bfeb4c63ea68._comment new file mode 100644 index 0000000000..60c268fc03 --- /dev/null +++ b/doc/bugs/git-annex_opens_too_many_files/comment_6_b3a5a4e4ca29c5cd2840bfeb4c63ea68._comment @@ -0,0 +1,15 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.154.0.63" + subject="comment 6" + date="2013-08-24T18:38:56Z" + content=""" +My NetListener workaround turned out to not be portable enough. + +However, I have sent a patch to fix the FD leak: +Hopefully it gets applied soon. + +I have also made the PairListener only retry every 60 seconds. Which makes the leak 1/60th as bad, for whatever that's worth. + +Once a fix for this gets into Debian, I need to remember to backport it to stable, and update the autobuilders to use it. Also need to remember to update the Android autobuilder. Leaving this bug report open until that happens. +"""]] From 11cca2b183dd890e5b41d2aa5ed2f02f7fd0b774 Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Sat, 24 Aug 2013 18:43:08 +0000 Subject: [PATCH 12/14] Added a comment --- .../comment_1_e14757c2c106770c2d7069ace4987b3b._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/forum/How_do_I_cleanly_remove_an_Android_git-annex_installation__63__/comment_1_e14757c2c106770c2d7069ace4987b3b._comment diff --git a/doc/forum/How_do_I_cleanly_remove_an_Android_git-annex_installation__63__/comment_1_e14757c2c106770c2d7069ace4987b3b._comment b/doc/forum/How_do_I_cleanly_remove_an_Android_git-annex_installation__63__/comment_1_e14757c2c106770c2d7069ace4987b3b._comment new file mode 100644 index 0000000000..b051aa704a --- /dev/null +++ b/doc/forum/How_do_I_cleanly_remove_an_Android_git-annex_installation__63__/comment_1_e14757c2c106770c2d7069ace4987b3b._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.154.0.63" + subject="comment 1" + date="2013-08-24T18:43:08Z" + content=""" +It should be sufficient to delete `/sdcard/git-annex.home` and delete the .git directory inside whatever directory you set up as the repository. +"""]] From 4f3256c218f0610aacf61e883b06518545d06622 Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Sat, 24 Aug 2013 18:49:18 +0000 Subject: [PATCH 13/14] Added a comment --- ...mment_1_b3197993dbdfaf2db5e4651ac54a896e._comment | 12 ++++++++++++ 1 file changed, 12 insertions(+) create mode 100644 doc/bugs/Git_annexed_files_symlink_are_wrong_when_submodule_is_not_in_the_same_path/comment_1_b3197993dbdfaf2db5e4651ac54a896e._comment diff --git a/doc/bugs/Git_annexed_files_symlink_are_wrong_when_submodule_is_not_in_the_same_path/comment_1_b3197993dbdfaf2db5e4651ac54a896e._comment b/doc/bugs/Git_annexed_files_symlink_are_wrong_when_submodule_is_not_in_the_same_path/comment_1_b3197993dbdfaf2db5e4651ac54a896e._comment new file mode 100644 index 0000000000..8ff2321591 --- /dev/null +++ b/doc/bugs/Git_annexed_files_symlink_are_wrong_when_submodule_is_not_in_the_same_path/comment_1_b3197993dbdfaf2db5e4651ac54a896e._comment @@ -0,0 +1,12 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.154.0.63" + subject="comment 1" + date="2013-08-24T18:49:18Z" + content=""" +git-annex assumes that it can make a stable symlink from a file in the working tree to a file in the .git directory. There are several ways to break this. One, as noted, is sometimes using a repository as a submodule, and sometimes not. Another would be to play around with `GIT_DIR`. + +I don't see a way git-annex can support those use cases, at least in indirect mode. + +It does seem like, in direct mode, it should just work. git-annex will commit various symlinks to git, but these symlinks will never be followed to get at the content of a file, since direct mode arranges for the content to be directly present in the working tree. +"""]] From 840a10b155cdb85e62de8c7a531aff00434edf18 Mon Sep 17 00:00:00 2001 From: "http://joeyh.name/" Date: Sat, 24 Aug 2013 18:52:12 +0000 Subject: [PATCH 14/14] Added a comment --- .../comment_2_dc128eeddeaaf3f84e71aca0fb7d341f._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/bugs/Hangs_on_creating_repository_when_using_--listen/comment_2_dc128eeddeaaf3f84e71aca0fb7d341f._comment diff --git a/doc/bugs/Hangs_on_creating_repository_when_using_--listen/comment_2_dc128eeddeaaf3f84e71aca0fb7d341f._comment b/doc/bugs/Hangs_on_creating_repository_when_using_--listen/comment_2_dc128eeddeaaf3f84e71aca0fb7d341f._comment new file mode 100644 index 0000000000..55d7efa1ce --- /dev/null +++ b/doc/bugs/Hangs_on_creating_repository_when_using_--listen/comment_2_dc128eeddeaaf3f84e71aca0fb7d341f._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="http://joeyh.name/" + ip="4.154.0.63" + subject="comment 2" + date="2013-08-24T18:52:11Z" + content=""" +This also affects creating a second repository in the webapp, not just the repository creation at first startup. +"""]]