From fd238314679de173cd8d30439d9b06270dc0319f Mon Sep 17 00:00:00 2001 From: xloem Date: Wed, 18 May 2016 01:49:11 +0000 Subject: [PATCH 1/8] Added a comment: dangling objects --- .../comment_1_6da0787fa9749ee2d1f91842f10a5d3c._comment | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100644 doc/devblog/day_392__v6_fixes/comment_1_6da0787fa9749ee2d1f91842f10a5d3c._comment diff --git a/doc/devblog/day_392__v6_fixes/comment_1_6da0787fa9749ee2d1f91842f10a5d3c._comment b/doc/devblog/day_392__v6_fixes/comment_1_6da0787fa9749ee2d1f91842f10a5d3c._comment new file mode 100644 index 0000000000..a123937b53 --- /dev/null +++ b/doc/devblog/day_392__v6_fixes/comment_1_6da0787fa9749ee2d1f91842f10a5d3c._comment @@ -0,0 +1,7 @@ +[[!comment format=mdwn + username="xloem" + subject="dangling objects" + date="2016-05-18T01:49:11Z" + content=""" +My repos have accumulated thousands of dangling objects recently (every one). I'm wondering, would this issue be resolved with the syncing fix? If so, should the objects just be deleted? +"""]] From e86688651cedfa51daa52981dc8c1a2b47f53d98 Mon Sep 17 00:00:00 2001 From: xloem Date: Wed, 18 May 2016 09:43:13 +0000 Subject: [PATCH 2/8] Added a comment --- .../comment_2_4c6ced0d3163735c7f4bd5a4a7053159._comment | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 doc/bugs/broken_repo_when_inodes_exhausted/comment_2_4c6ced0d3163735c7f4bd5a4a7053159._comment diff --git a/doc/bugs/broken_repo_when_inodes_exhausted/comment_2_4c6ced0d3163735c7f4bd5a4a7053159._comment b/doc/bugs/broken_repo_when_inodes_exhausted/comment_2_4c6ced0d3163735c7f4bd5a4a7053159._comment new file mode 100644 index 0000000000..270968bbcc --- /dev/null +++ b/doc/bugs/broken_repo_when_inodes_exhausted/comment_2_4c6ced0d3163735c7f4bd5a4a7053159._comment @@ -0,0 +1,9 @@ +[[!comment format=mdwn + username="xloem" + subject="comment 2" + date="2016-05-18T09:43:13Z" + content=""" +Thanks. It turns out I'd deleted that repo in order to free the inodes up. Next time I will see if I can repair. + +But it would be nice if git-annex tracked inodes the way it tracks free space, so that it could refrain from causing the inode exhaustion. This also would have alerted me to how few inodes I had remaining. +"""]] From 57a7981e1b4b8517f7c44fe2655e48e533261c96 Mon Sep 17 00:00:00 2001 From: xloem Date: Thu, 19 May 2016 03:02:19 +0000 Subject: [PATCH 3/8] Added a comment: Freenet Special Remote --- .../comment_5_0c40bb64126957a25ec1b20dc856529c._comment | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 doc/todo/wishlist__58___spec.remotes_for_other_peer_network_data_stores___40__gnunet__44___freenet__41__/comment_5_0c40bb64126957a25ec1b20dc856529c._comment diff --git a/doc/todo/wishlist__58___spec.remotes_for_other_peer_network_data_stores___40__gnunet__44___freenet__41__/comment_5_0c40bb64126957a25ec1b20dc856529c._comment b/doc/todo/wishlist__58___spec.remotes_for_other_peer_network_data_stores___40__gnunet__44___freenet__41__/comment_5_0c40bb64126957a25ec1b20dc856529c._comment new file mode 100644 index 0000000000..f345043dfe --- /dev/null +++ b/doc/todo/wishlist__58___spec.remotes_for_other_peer_network_data_stores___40__gnunet__44___freenet__41__/comment_5_0c40bb64126957a25ec1b20dc856529c._comment @@ -0,0 +1,9 @@ +[[!comment format=mdwn + username="xloem" + subject="Freenet Special Remote" + date="2016-05-19T03:02:18Z" + content=""" +The freenet external special remote at https://github.com/xloem/gitlakepy is working now. + +No bells and whistles, but you can install it and start git-annex copy --to=freenet and git-annex get --from=freenet . +"""]] From 51ab43ca589874ef9a2a1afd48559c06ddd04e05 Mon Sep 17 00:00:00 2001 From: Gus Date: Thu, 19 May 2016 11:27:16 +0000 Subject: [PATCH 4/8] Added a comment: Thank you --- ..._eeae2f172b4481fcb08745fc28aa4d4e._comment | 42 +++++++++++++++++++ 1 file changed, 42 insertions(+) create mode 100644 doc/forum/What_am_I_doing_wrong_with_move__63__/comment_2_eeae2f172b4481fcb08745fc28aa4d4e._comment diff --git a/doc/forum/What_am_I_doing_wrong_with_move__63__/comment_2_eeae2f172b4481fcb08745fc28aa4d4e._comment b/doc/forum/What_am_I_doing_wrong_with_move__63__/comment_2_eeae2f172b4481fcb08745fc28aa4d4e._comment new file mode 100644 index 0000000000..2ec35684c7 --- /dev/null +++ b/doc/forum/What_am_I_doing_wrong_with_move__63__/comment_2_eeae2f172b4481fcb08745fc28aa4d4e._comment @@ -0,0 +1,42 @@ +[[!comment format=mdwn + username="Gus" + subject="Thank you" + date="2016-05-19T11:27:15Z" + content=""" +The external unit seems to hold a NTFS. Here is the `git-annex info` output: + + repository mode: direct + trusted repositories: 0 + semitrusted repositories: 4 + 00000000-0000-0000-0000-000000000001 -- web + 00000000-0000-0000-0000-000000000002 -- bittorrent + 069de9a2-dc53-4c0a-82e0-a61a1f29e6b3 -- stratus PC [stratus] + 49b5b3a4-56ac-4cf2-aed9-1c23d3181c97 -- Toshiba USB HDD [here] + untrusted repositories: 0 + transfers in progress: none + available local disk space: 4.42 gigabytes (+1 megabyte reserved) + local annex keys: 5556 + local annex size: 1.66 terabytes + annexed files in working tree: 6618 + size of annexed files in working tree: 1.1 terabytes + bloom filter size: 32 mebibytes (1.1% full) + backend usage: + SHA256E: 6618 + +However, `git status` says: + + fatal: This operation must be run in a work tree + +Which is not the same message as \"no git found here\", but is also not what I expected to see. `git log` seems to work but says nothing about the file at hand. + +On the PC side, however, I can see three commits on the file (I wish the commit message contained the command line with arguments, rather than the less descriptive \"git-annex in Toshiba USB HDD\"). +Using `git show` and `git cat-file` I managed to determine the following: + +March 4: the initial version of the file was committed. + +May 17 11:51: the file's content changed to `../../../../../../../../../media/TOSHIBA EXT/annex/.git/annex/objects/kx/3W/SHA256E-s96418--c6164e17d88914b2e6781e2cb8e7b91e9669ddf2d9ee6f5cbb17f3212bccfba4.jpg/SHA256E-s96418--c6164e17d88914b2e6781e2cb8e7b91e9669ddf2d9ee6f5cbb17f3212bccfba4.jpg`. This is blob `../.git/annex/objects/4z/gf/SHA256E-s245--ae647e7ad31089255413a9290ca9344542f3cd15ecef66884613bf776387633d.jpg/SHA256E-s245--ae647e7ad31089255413a9290ca9344542f3cd15ecef66884613bf776387633d.jpg`. + +May 17 12:22 the file's content changed again to `../../../../../../../../../media/TOSHIBA EXT/annex/.git/annex/objects/4z/gf/SHA256E-s245--ae647e7ad31089255413a9290ca9344542f3cd15ecef66884613bf776387633d.jpg/SHA256E-s245--ae647e7ad31089255413a9290ca9344542f3cd15ecef66884613bf776387633d.jpg`, i.e., a reference to the previous object. This is blob `../.git/annex/objects/ZQ/WF/SHA256E-s241--f3d7e5d1f788235b8eec0af58cc0c526b112b9e834a47ba7a475876c49dce343.jpg/SHA256E-s241--f3d7e5d1f788235b8eec0af58cc0c526b112b9e834a47ba7a475876c49dce343.jpg`. + +What else can I do in order to work out what went wrong? Is having concurrent commands manipulating the same repository a bad idea? +"""]] From 58bdc5be88f855bd1ac2ee79e62c6c33456be3ec Mon Sep 17 00:00:00 2001 From: "https://me.yahoo.com/a/EbvxpTI_xP9Aod7Mg4cwGhgjrCrdM5s-#7c0f4" Date: Thu, 19 May 2016 19:11:14 +0000 Subject: [PATCH 5/8] --- doc/todo/make_copy_--fast__faster.mdwn | 3 +++ 1 file changed, 3 insertions(+) create mode 100644 doc/todo/make_copy_--fast__faster.mdwn diff --git a/doc/todo/make_copy_--fast__faster.mdwn b/doc/todo/make_copy_--fast__faster.mdwn new file mode 100644 index 0000000000..2aea946695 --- /dev/null +++ b/doc/todo/make_copy_--fast__faster.mdwn @@ -0,0 +1,3 @@ +I was trying to copy files which failed to copy (3 out of 6,000) to remote host after copy -J4. Succeeded. But with subsequent runs, apparently even with copy --fast it takes annex 10 sec for annex to realize there is nothing to copy. git ls-files which annex calls returns list immediately, so it is really some parsing/access to data under git-annex branch which takes awhile. I think we had similar discussion before but couldn't find. So I wondered to whine again to see if some optimization is possible to make --fast copies faster, especially whenever there is nothing to copy. + +[[!meta author=yoh]] From f317ecb77b58514ef758d4d8a7be06c3563a3fbe Mon Sep 17 00:00:00 2001 From: EskildHustvedt Date: Fri, 20 May 2016 07:37:38 +0000 Subject: [PATCH 6/8] --- ...eps_deleting_all_the_files_in_my_repo.mdwn | 24 +++++++++++++++++++ 1 file changed, 24 insertions(+) create mode 100644 doc/bugs/Assistant_keeps_deleting_all_the_files_in_my_repo.mdwn diff --git a/doc/bugs/Assistant_keeps_deleting_all_the_files_in_my_repo.mdwn b/doc/bugs/Assistant_keeps_deleting_all_the_files_in_my_repo.mdwn new file mode 100644 index 0000000000..c9752f098b --- /dev/null +++ b/doc/bugs/Assistant_keeps_deleting_all_the_files_in_my_repo.mdwn @@ -0,0 +1,24 @@ +### Please describe the problem. +The assistant keeps making merges that deletes all the files in the repository. I "git revert" the merge commit, but the next day it does it again. It also seems to have gone into a merge loop before this happens (will post a screenshot of tig). I can make the repo available privately if that will help. + +### What steps will reproduce the problem? +Unknown. It does it on its own without me even interacting with files in the annex. + +### What version of git-annex are you using? On what operating system? +The host that keeps deleting is running 6.20160428-g1f253e8 on ArchLinux (x86_64). A remote that it keeps syncing with is running 6.20160511-g4633f0b, also on ArchLinux x86_64. + +### Please provide any additional information below. + +[[!format sh """ +# If you can, paste a complete transcript of the problem occurring here. +# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log +Updating 8cb69d3..c589c5e +Fast-forward + .gitignore | 3 --- + Bilete/8mars-2013-1.jpg | 1 - +etc. +# End of transcript or log. +"""]] + +### Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil' positive end note does wonders) +I use git-annex for everything. I've got 10 repositories and around 2.5TB of data in those repos which in turn is synced all over the place. It's excellent. From 41a5155b09ab87f743826ab05fa6e9a6ab0a655d Mon Sep 17 00:00:00 2001 From: EskildHustvedt Date: Fri, 20 May 2016 07:45:52 +0000 Subject: [PATCH 7/8] Added a comment --- ...omment_1_951d03de942c19f760e9ff01a3b61f33._comment | 11 +++++++++++ 1 file changed, 11 insertions(+) create mode 100644 doc/bugs/Assistant_keeps_deleting_all_the_files_in_my_repo/comment_1_951d03de942c19f760e9ff01a3b61f33._comment diff --git a/doc/bugs/Assistant_keeps_deleting_all_the_files_in_my_repo/comment_1_951d03de942c19f760e9ff01a3b61f33._comment b/doc/bugs/Assistant_keeps_deleting_all_the_files_in_my_repo/comment_1_951d03de942c19f760e9ff01a3b61f33._comment new file mode 100644 index 0000000000..a65054ca7a --- /dev/null +++ b/doc/bugs/Assistant_keeps_deleting_all_the_files_in_my_repo/comment_1_951d03de942c19f760e9ff01a3b61f33._comment @@ -0,0 +1,11 @@ +[[!comment format=mdwn + username="EskildHustvedt" + subject="comment 1" + date="2016-05-20T07:45:52Z" + content=""" +Right, so I forgot to mention, perhaps crucially. I'm using v6 for these repos. + +Here's the tig grap for master: [[https://ssl.zerodogg.org/~zerodogg/tmp/annexmerge1-2016-05-20.png]] + +Here's the merge commit itself for master: [[https://ssl.zerodogg.org/~zerodogg/tmp/annexmerge2-2016-05-20.png]] +"""]] From bc87b2d9f9df8d5db5a6858a711fb6aaeefc8033 Mon Sep 17 00:00:00 2001 From: "ilovezfs@25b1c2e6e4c5194d062624bafc4507b8398d6588" Date: Fri, 20 May 2016 10:33:02 +0000 Subject: [PATCH 8/8] S3 support lacking in a stock cabal install build in a sandbox on OS X --- doc/bugs/.mdwn | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 doc/bugs/.mdwn diff --git a/doc/bugs/.mdwn b/doc/bugs/.mdwn new file mode 100644 index 0000000000..58ae5d9700 --- /dev/null +++ b/doc/bugs/.mdwn @@ -0,0 +1,15 @@ +A default cabal install on OS X in a sandbox of git-annex 6.20160511 will result in no S3 support, as reported to Homebrew in the following two issues: +https://github.com/Homebrew/homebrew-core/issues/1268 +https://github.com/Homebrew/legacy-homebrew/issues/47737 + +The underlying cause is that aws-0.13.0 lacks commit https://github.com/aristidb/aws/commit/402bfe5aa9ef4bec84186880faafcbfdae1ad91d, which allows data-default 0.6. + +I attempted to mitigate the issue using --flags="s3", but that does not seem to help (nor does it force the build to fail): still no s3 support. I'd expect that either to constrain data-default to 0.5.3 and produce a build with s3 support, or fail the build, but for some reason it doesn't. Is this not working because we're in a sandbox or some other reason? + +Currently, I'm planning to just patch aws with https://github.com/aristidb/aws/commit/402bfe5aa9ef4bec84186880faafcbfdae1ad91d rather than resorting to a fixed configuration (e.g., lts-5.5 or whatever), as you can see here: +https://github.com/Homebrew/homebrew-core/pull/1307 + +It would be great if git-annex could work around the issue itself, though. + +Meanwhile, I have also pinged aws to request a 0.13.1 release, which would solve the problem "the right way": +https://github.com/aristidb/aws/issues/202