From 062a0a9ca2480861af3a75ff1c8930ea0f4d65e1 Mon Sep 17 00:00:00 2001 From: "https://id.koumbit.net/anarcat" Date: Sat, 7 Mar 2015 02:05:26 +0000 Subject: [PATCH 01/26] Added a comment: similar to a forum question --- .../comment_1_c803bfef1b881c6f64ed44d49dfb4547._comment | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100644 doc/todo/Facilitate_public_pretty_S3_URLs/comment_1_c803bfef1b881c6f64ed44d49dfb4547._comment diff --git a/doc/todo/Facilitate_public_pretty_S3_URLs/comment_1_c803bfef1b881c6f64ed44d49dfb4547._comment b/doc/todo/Facilitate_public_pretty_S3_URLs/comment_1_c803bfef1b881c6f64ed44d49dfb4547._comment new file mode 100644 index 0000000000..3eb005b945 --- /dev/null +++ b/doc/todo/Facilitate_public_pretty_S3_URLs/comment_1_c803bfef1b881c6f64ed44d49dfb4547._comment @@ -0,0 +1,7 @@ +[[!comment format=mdwn + username="https://id.koumbit.net/anarcat" + subject="similar to a forum question" + date="2015-03-07T02:05:26Z" + content=""" +this is similar to a forum question i asked: [[forum/original_filename_on_s3/]]. --[[anarcat]] +"""]] From 01bb69535ffa37fbba682cc56e3afb655e2848c9 Mon Sep 17 00:00:00 2001 From: "https://id.koumbit.net/anarcat" Date: Sat, 7 Mar 2015 02:06:06 +0000 Subject: [PATCH 02/26] Added a comment: dupe? --- .../comment_1_870f3f01f95b5878d483ffedd2ae2698._comment | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100644 doc/forum/original_filename_on_s3/comment_1_870f3f01f95b5878d483ffedd2ae2698._comment diff --git a/doc/forum/original_filename_on_s3/comment_1_870f3f01f95b5878d483ffedd2ae2698._comment b/doc/forum/original_filename_on_s3/comment_1_870f3f01f95b5878d483ffedd2ae2698._comment new file mode 100644 index 0000000000..165b7f8f65 --- /dev/null +++ b/doc/forum/original_filename_on_s3/comment_1_870f3f01f95b5878d483ffedd2ae2698._comment @@ -0,0 +1,7 @@ +[[!comment format=mdwn + username="https://id.koumbit.net/anarcat" + subject="dupe?" + date="2015-03-07T02:06:06Z" + content=""" +see also [[todo/Facilitate_public_pretty_S3_URLs]] +"""]] From fb11179d05d89cf6d192ef04b30d5a8c964e4dc0 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawksOCeakibYmGDt3wLLo4nkY0FkB72I2Uo" Date: Sat, 7 Mar 2015 02:50:19 +0000 Subject: [PATCH 03/26] Added a comment --- ...mment_1_b2d2bf6be441dc2e4443d06ef8db7134._comment | 12 ++++++++++++ 1 file changed, 12 insertions(+) create mode 100644 doc/forum/NTFS_usb_on_linux_unable_to_connect_to_ssh_remote/comment_1_b2d2bf6be441dc2e4443d06ef8db7134._comment diff --git a/doc/forum/NTFS_usb_on_linux_unable_to_connect_to_ssh_remote/comment_1_b2d2bf6be441dc2e4443d06ef8db7134._comment b/doc/forum/NTFS_usb_on_linux_unable_to_connect_to_ssh_remote/comment_1_b2d2bf6be441dc2e4443d06ef8db7134._comment new file mode 100644 index 0000000000..d5a0d6460a --- /dev/null +++ b/doc/forum/NTFS_usb_on_linux_unable_to_connect_to_ssh_remote/comment_1_b2d2bf6be441dc2e4443d06ef8db7134._comment @@ -0,0 +1,12 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawksOCeakibYmGDt3wLLo4nkY0FkB72I2Uo" + nickname="Source" + subject="comment 1" + date="2015-03-07T02:50:19Z" + content=""" +I think I have tracked down part of the issue. It due to the options that are called by `git-annex` when trying to `rsync` between the NTFS local hard drive and the SSH remote. + +I got `rsync` to work by using `rsync -vrc source dest` as options (outside of git-annex). Found [here](http://askubuntu.com/questions/112863/rsync-not-working-between-ntfs-fat-and-ext). + +Is there any way to change the rsync options put out by git-annex when copying/moving to and from local ntfs drives on linux? +"""]] From ef241504627b776c4cd37ba5f1d0b0d084149794 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawkWHj0RxNMfuwvFzo2d-V6vBKOYwW_Fnfk" Date: Sat, 7 Mar 2015 03:00:14 +0000 Subject: [PATCH 04/26] Added a comment: Thanks for the clarification --- ...mment_2_b76dd79aabe87ec717eea164aa965d74._comment | 12 ++++++++++++ 1 file changed, 12 insertions(+) create mode 100644 doc/forum/Using_standard_groups_and_sync_to_preserve_history:_--all_not_recognised/comment_2_b76dd79aabe87ec717eea164aa965d74._comment diff --git a/doc/forum/Using_standard_groups_and_sync_to_preserve_history:_--all_not_recognised/comment_2_b76dd79aabe87ec717eea164aa965d74._comment b/doc/forum/Using_standard_groups_and_sync_to_preserve_history:_--all_not_recognised/comment_2_b76dd79aabe87ec717eea164aa965d74._comment new file mode 100644 index 0000000000..2b5a8672b2 --- /dev/null +++ b/doc/forum/Using_standard_groups_and_sync_to_preserve_history:_--all_not_recognised/comment_2_b76dd79aabe87ec717eea164aa965d74._comment @@ -0,0 +1,12 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawkWHj0RxNMfuwvFzo2d-V6vBKOYwW_Fnfk" + nickname="Andrew" + subject="Thanks for the clarification" + date="2015-03-07T03:00:14Z" + content=""" +I will post the todo, and in the meantime I can script a `git annex copy --unused` followed by a `git annex sync --content` to capture the full history in the archive + +regards + +Andrew +"""]] From 6d3fd2f4f96318d03d8a850196d1148ae3985395 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawkWHj0RxNMfuwvFzo2d-V6vBKOYwW_Fnfk" Date: Sat, 7 Mar 2015 03:31:45 +0000 Subject: [PATCH 05/26] --- doc/todo/wishlist:_--all_option_for_sync.mdwn | 20 +++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100644 doc/todo/wishlist:_--all_option_for_sync.mdwn diff --git a/doc/todo/wishlist:_--all_option_for_sync.mdwn b/doc/todo/wishlist:_--all_option_for_sync.mdwn new file mode 100644 index 0000000000..75e4b27d38 --- /dev/null +++ b/doc/todo/wishlist:_--all_option_for_sync.mdwn @@ -0,0 +1,20 @@ +I wish to preserve all history on the backup drives using standard groups and the `sync` command. Namely, if I do the following + + touch test-of-annex-backup.txt + git annex add test-of-annex-backup.txt + git commit --message='test: Create empty test-of-annex-backup.txt file' + git annex edit test-of-annex-backup.txt + echo "This line creates version 2 of this file" > test-of-annex-backup.txt + git annex add test-of-annex-backup.txt + git commit --message='test: Create version 2 of test-of-annex-backup.txt' + git annex sync --content --all + +I expect to see 2 copies of `test-of-annex-backup.txt` be copied to each accessible annex repository in the `backup` [standard group](http://git-annex.branchable.com/preferred_content/standard_groups/) + +At present, the `backup` standard group prefers unused files, but the `sync` command cannot act on this configuration, since it lacks an `--all` option. This is surprising to me as a user, and appears to contradict the intent of the preferred content, as evinced by the awkward explanation of why it is there in [preferred content](http://git-annex.branchable.com/preferred_content/) + +Please add an `--all` option to the `sync` command + +Thanks + +Andrew From 0e9f9019d192feb0c9ff4fec4e257ec137d34672 Mon Sep 17 00:00:00 2001 From: "https://id.koumbit.net/anarcat" Date: Sat, 7 Mar 2015 13:16:02 +0000 Subject: [PATCH 06/26] Added a comment: about copying to the local store --- .../comment_1_d1b2da148715476015716a2f866558b9._comment | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100644 doc/special_remotes/ipfs/comment_1_d1b2da148715476015716a2f866558b9._comment diff --git a/doc/special_remotes/ipfs/comment_1_d1b2da148715476015716a2f866558b9._comment b/doc/special_remotes/ipfs/comment_1_d1b2da148715476015716a2f866558b9._comment new file mode 100644 index 0000000000..ea56ab66bb --- /dev/null +++ b/doc/special_remotes/ipfs/comment_1_d1b2da148715476015716a2f866558b9._comment @@ -0,0 +1,7 @@ +[[!comment format=mdwn + username="https://id.koumbit.net/anarcat" + subject="about copying to the local store" + date="2015-03-07T13:16:02Z" + content=""" +there's a [discussion](https://github.com/jbenet/go-ipfs/issues/875) happening upstream about how copying to the local datastore could be avoided. +"""]] From b7e9a089092f626a9c27004eac28e817d76a23e7 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawmInOIq3X0IygZm4PEKlh2-aW1AGIxN97A" <@web> Date: Sat, 7 Mar 2015 18:47:54 +0000 Subject: [PATCH 07/26] --- .../Windows:_can__39__t_clone_repository.mdwn | 43 +++++++++++++++++++ 1 file changed, 43 insertions(+) create mode 100644 doc/bugs/Windows:_can__39__t_clone_repository.mdwn diff --git a/doc/bugs/Windows:_can__39__t_clone_repository.mdwn b/doc/bugs/Windows:_can__39__t_clone_repository.mdwn new file mode 100644 index 0000000000..cd10e39aa9 --- /dev/null +++ b/doc/bugs/Windows:_can__39__t_clone_repository.mdwn @@ -0,0 +1,43 @@ +### Please describe the problem. + +Can't clone repository on Windows 7 64bit + +### What steps will reproduce the problem? +git clone git://git-annex.branchable.com/ gitannex + +... + +error: Invalid path 'doc/walkthrough/fsck:_verifying_your_data.mdwn' + +error: Invalid path 'doc/walkthrough/fsck:_when_things_go_wrong.mdwn' + +error: Invalid path 'doc/walkthrough/quiet_please:_When_git-annex_seems_to_skip_files.mdwn' + +error: Invalid path 'doc/walkthrough/removing_files:_When_things_go_wrong.mdwn' + +error: Invalid path 'doc/walkthrough/transferring_files:_When_things_go_wrong.mdwn' + +Checking out files: 100% (7235/7235), done. + +git status shows many deleted +git reset --hard shows same error as clone + +### What version of git-annex are you using? On what operating system? +git annex version + +git-annex version: 5.20150219-g3fc8d83 +build flags: Assistant Webapp Webapp-secure Pairing Testsuite S3 WebDAV DNS Feeds Quvi TDFA TorrentParser +key/value backends: SHA256E SHA1E SHA512E SHA224E SHA384E SKEIN256E SKEIN512E MD5E SHA256 SHA1 SHA512 SHA224 SHA384 SKEIN256 SKEIN512 MD5 WORM + URL +remote types: git gcrypt S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external +local repository version: unknown +supported repository version: 5 +upgrade supported from repository versions: 2 3 4 + +git --version + +git version 1.9.5.msysgit.0 + +Windows 7 64bit + + From a03a3617e58a3888ac98fc10aa72e0e2854a7fdf Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawmInOIq3X0IygZm4PEKlh2-aW1AGIxN97A" <@web> Date: Sat, 7 Mar 2015 18:48:54 +0000 Subject: [PATCH 08/26] --- doc/bugs/Windows:_can__39__t_clone_repository.mdwn | 1 + 1 file changed, 1 insertion(+) diff --git a/doc/bugs/Windows:_can__39__t_clone_repository.mdwn b/doc/bugs/Windows:_can__39__t_clone_repository.mdwn index cd10e39aa9..f6e611251d 100644 --- a/doc/bugs/Windows:_can__39__t_clone_repository.mdwn +++ b/doc/bugs/Windows:_can__39__t_clone_repository.mdwn @@ -20,6 +20,7 @@ error: Invalid path 'doc/walkthrough/transferring_files:_When_things_go_wrong.md Checking out files: 100% (7235/7235), done. git status shows many deleted + git reset --hard shows same error as clone ### What version of git-annex are you using? On what operating system? From f788d7f353e97d5ed8d7e7678e26b706d44f0e4b Mon Sep 17 00:00:00 2001 From: James Date: Sun, 8 Mar 2015 03:25:42 +0000 Subject: [PATCH 09/26] --- doc/forum/noob_question._VPS_web_assistant.mdwn | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100644 doc/forum/noob_question._VPS_web_assistant.mdwn diff --git a/doc/forum/noob_question._VPS_web_assistant.mdwn b/doc/forum/noob_question._VPS_web_assistant.mdwn new file mode 100644 index 0000000000..410cb74d55 --- /dev/null +++ b/doc/forum/noob_question._VPS_web_assistant.mdwn @@ -0,0 +1,7 @@ +Hi + +So I've installed git-annex on my Debian VPS. + +How do I find out the URL to access the web UI / assistant? + +Thanks! From 84b4ba19e07ed43928f74271f86b2f475dda3406 Mon Sep 17 00:00:00 2001 From: ahmaddamra Date: Mon, 9 Mar 2015 03:20:28 +0000 Subject: [PATCH 10/26] --- ...Reputation.1_on_git-annex_windows_installer.mdwn | 13 +++++++++++++ 1 file changed, 13 insertions(+) create mode 100644 doc/forum/Symantec_endpoint_protection_raises_WS.Reputation.1_on_git-annex_windows_installer.mdwn diff --git a/doc/forum/Symantec_endpoint_protection_raises_WS.Reputation.1_on_git-annex_windows_installer.mdwn b/doc/forum/Symantec_endpoint_protection_raises_WS.Reputation.1_on_git-annex_windows_installer.mdwn new file mode 100644 index 0000000000..16be75923e --- /dev/null +++ b/doc/forum/Symantec_endpoint_protection_raises_WS.Reputation.1_on_git-annex_windows_installer.mdwn @@ -0,0 +1,13 @@ +Hello, + +I downloaded git-annex-installer.exe from https://downloads.kitenet.net/git-annex/windows/current/git-annex-installer.exe . +When I try to run it on my work windows 7 64-bit laptop, Symantec Encpoint Protection complains with a WS.Reputation.1 flag that the file has a low reputation and quarantines the file. + +I searched online trying to find why git-annex is flagged as such by Symantec but was unable to find anything. +I prefer not to build from source if possible as this is my work laptop and it would require that I install Haskell, MingW, and Cygwin. + +Any pointers/suggestions would be greatly appreciated. + +Thanks, + +Ahmad From 50d85ecefb004cee8bbd100179db20b985f6d2f8 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawkSq2FDpK2n66QRUxtqqdbyDuwgbQmUWus" Date: Mon, 9 Mar 2015 16:48:18 +0000 Subject: [PATCH 11/26] Added a comment --- .../comment_2_c0a59549409faa355a461e85a1c3f908._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/design/iabackup/comment_2_c0a59549409faa355a461e85a1c3f908._comment diff --git a/doc/design/iabackup/comment_2_c0a59549409faa355a461e85a1c3f908._comment b/doc/design/iabackup/comment_2_c0a59549409faa355a461e85a1c3f908._comment new file mode 100644 index 0000000000..135917b4ee --- /dev/null +++ b/doc/design/iabackup/comment_2_c0a59549409faa355a461e85a1c3f908._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawkSq2FDpK2n66QRUxtqqdbyDuwgbQmUWus" + nickname="Jimmy" + subject="comment 2" + date="2015-03-09T16:48:18Z" + content=""" +I've tried throwing about ~16 million files at git/git-annex in the past where some files were 1-2kb in size (around 30% of them). git/git-annex doesn't work well at that scale. +"""]] From 34cea0834094b325b9efaf233f1b4a743c59ea09 Mon Sep 17 00:00:00 2001 From: effigies Date: Mon, 9 Mar 2015 17:32:19 +0000 Subject: [PATCH 12/26] Added a comment: Strategy for getting up and running --- ..._55fcf44eefe6abac3f61a14e9ce9206c._comment | 34 +++++++++++++++++++ 1 file changed, 34 insertions(+) create mode 100644 doc/bugs/assistant_does_not_allow_adding_an_existing_repo/comment_2_55fcf44eefe6abac3f61a14e9ce9206c._comment diff --git a/doc/bugs/assistant_does_not_allow_adding_an_existing_repo/comment_2_55fcf44eefe6abac3f61a14e9ce9206c._comment b/doc/bugs/assistant_does_not_allow_adding_an_existing_repo/comment_2_55fcf44eefe6abac3f61a14e9ce9206c._comment new file mode 100644 index 0000000000..8ebdb4ff98 --- /dev/null +++ b/doc/bugs/assistant_does_not_allow_adding_an_existing_repo/comment_2_55fcf44eefe6abac3f61a14e9ce9206c._comment @@ -0,0 +1,34 @@ +[[!comment format=mdwn + username="effigies" + subject="Strategy for getting up and running" + date="2015-03-09T17:32:19Z" + content=""" +The following is the set of steps I use when setting assistant up on a new repository: + + git clone user@host:repo.git + pushd repo + git annex init + touch EMPTY + git annex add EMPTY + git commit -m 'Initial commit' + git push --all + git annex copy --to origin + git annex direct + git annex sync + git annex untrust . + popd + +Entering the folder path now lets the assistant take over. + +For an existing repository: + + git clone user@host:repo.git + pushd repo + git annex get . + git annex direct + git annex sync + git annex untrust . + popd + +Not sure if this would be helpful for working out what the assistant behavior should be, but maybe it'll help others get to the point where the assistant works with a gitolite managed repo. +"""]] From 684a267a72eebd39d0333f9ba7f7b61d42b96812 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawlc-3pdibcizrdz4WmZooECL0k6AvM1cWc" Date: Mon, 9 Mar 2015 17:40:59 +0000 Subject: [PATCH 13/26] Added a comment: direct-mode to direct-mode perhaps? --- ...mment_1_0d13792a92dc4cde6db73123eeb88218._comment | 12 ++++++++++++ 1 file changed, 12 insertions(+) create mode 100644 doc/bugs/Windows:_Annex_can_not_get_files/comment_1_0d13792a92dc4cde6db73123eeb88218._comment diff --git a/doc/bugs/Windows:_Annex_can_not_get_files/comment_1_0d13792a92dc4cde6db73123eeb88218._comment b/doc/bugs/Windows:_Annex_can_not_get_files/comment_1_0d13792a92dc4cde6db73123eeb88218._comment new file mode 100644 index 0000000000..baf8bed8ea --- /dev/null +++ b/doc/bugs/Windows:_Annex_can_not_get_files/comment_1_0d13792a92dc4cde6db73123eeb88218._comment @@ -0,0 +1,12 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawlc-3pdibcizrdz4WmZooECL0k6AvM1cWc" + nickname="Joe" + subject="direct-mode to direct-mode perhaps?" + date="2015-03-09T17:40:59Z" + content=""" +It seems like the problem is when using a direct-mode repository on both sides of the copy. + +I can get and copy files to and from the USB stick when I'm dealing with my Linux-based (indirect mode) repository. + +Could it be that git-annex isn't properly noticing that c:\annex is in direct mode, and so it should copy from c:\annex\readme.txt instead of c:\annex\.git\annex\objects\... ? +"""]] From 1f8e5221a689d943344f574fcd076f059212c75d Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawlc-3pdibcizrdz4WmZooECL0k6AvM1cWc" Date: Mon, 9 Mar 2015 17:46:58 +0000 Subject: [PATCH 14/26] Added a comment: Ah, but I can directly overwrite the file --- ...mment_2_66aff09094d593f1c95213b9f8cbf26e._comment | 12 ++++++++++++ 1 file changed, 12 insertions(+) create mode 100644 doc/bugs/Windows:_Annex_can_not_get_files/comment_2_66aff09094d593f1c95213b9f8cbf26e._comment diff --git a/doc/bugs/Windows:_Annex_can_not_get_files/comment_2_66aff09094d593f1c95213b9f8cbf26e._comment b/doc/bugs/Windows:_Annex_can_not_get_files/comment_2_66aff09094d593f1c95213b9f8cbf26e._comment new file mode 100644 index 0000000000..f16e09c0d4 --- /dev/null +++ b/doc/bugs/Windows:_Annex_can_not_get_files/comment_2_66aff09094d593f1c95213b9f8cbf26e._comment @@ -0,0 +1,12 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawlc-3pdibcizrdz4WmZooECL0k6AvM1cWc" + nickname="Joe" + subject="Ah, but I can directly overwrite the file" + date="2015-03-09T17:46:58Z" + content=""" +If I copy f:\annex\bin\s3.exe into c:\annex\bin\s3.exe, (overwriting the .git symlink record) git annex sync will properly record that the file is now in this replica. + +So I guess I have a workaround(ish) + +--Joe +"""]] From ae61c20ee422ee93b379afca8d6c6f6e90a6287d Mon Sep 17 00:00:00 2001 From: "http://hendry.iki.fi/" Date: Tue, 10 Mar 2015 07:09:00 +0000 Subject: [PATCH 15/26] --- doc/forum/Where_are_my_remote_ssh_files__63__.mdwn | 6 ++++++ 1 file changed, 6 insertions(+) create mode 100644 doc/forum/Where_are_my_remote_ssh_files__63__.mdwn diff --git a/doc/forum/Where_are_my_remote_ssh_files__63__.mdwn b/doc/forum/Where_are_my_remote_ssh_files__63__.mdwn new file mode 100644 index 0000000000..342e4df1a5 --- /dev/null +++ b/doc/forum/Where_are_my_remote_ssh_files__63__.mdwn @@ -0,0 +1,6 @@ + + +I managed to sync files to a remote ssh store 'bible' with `git annex sync --content` however, where I ssh to bible, I was surprised not to see any of the JPG files that were copied there. + + +What am I missing? From 25b82d59831fe01a17baa558a399f5eb65eb7612 Mon Sep 17 00:00:00 2001 From: "http://hendry.iki.fi/" Date: Tue, 10 Mar 2015 07:14:53 +0000 Subject: [PATCH 16/26] --- doc/forum/Where_are_my_remote_ssh_files__63__.mdwn | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/doc/forum/Where_are_my_remote_ssh_files__63__.mdwn b/doc/forum/Where_are_my_remote_ssh_files__63__.mdwn index 342e4df1a5..704f95ff09 100644 --- a/doc/forum/Where_are_my_remote_ssh_files__63__.mdwn +++ b/doc/forum/Where_are_my_remote_ssh_files__63__.mdwn @@ -1,6 +1,10 @@ -I managed to sync files to a remote ssh store 'bible' with `git annex sync --content` however, where I ssh to bible, I was surprised not to see any of the JPG files that were copied there. +I managed to sync files to a remote ssh store "bible" with `git annex sync --content` however, where I ssh to bible, I was surprised not to see any of the JPG files that were copied there. What am I missing? + +# Solution + +I need to run `git annex sync` on the host bible too! From bc226d69301028199dc55d5cc6fe9ba863c78927 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawlZ-6dtxJY4cP7shhvV8E6YyuV0Rak8it4" Date: Tue, 10 Mar 2015 22:09:42 +0000 Subject: [PATCH 17/26] Added a comment: problem adding keyid to existing repo --- ..._4c6ffd6dd8d0ea16667866139b181d41._comment | 24 +++++++++++++++++++ 1 file changed, 24 insertions(+) create mode 100644 doc/encryption/comment_1_4c6ffd6dd8d0ea16667866139b181d41._comment diff --git a/doc/encryption/comment_1_4c6ffd6dd8d0ea16667866139b181d41._comment b/doc/encryption/comment_1_4c6ffd6dd8d0ea16667866139b181d41._comment new file mode 100644 index 0000000000..c0be6ca77a --- /dev/null +++ b/doc/encryption/comment_1_4c6ffd6dd8d0ea16667866139b181d41._comment @@ -0,0 +1,24 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawlZ-6dtxJY4cP7shhvV8E6YyuV0Rak8it4" + nickname="Giovanni" + subject="problem adding keyid to existing repo" + date="2015-03-10T22:09:42Z" + content=""" +I have a gcrypt special remote encrypted in hybrid mode, when I try to add a keyid using \"git annex enableremote myremote keyid+=XXXXXXXX\" I get this error: + +enableremote myremote (encryption update) (hybrid cipher with gpg keys XXXXXXXX XXXXXXX) fatal: remote myremote already exists. +git-annex: git [Params \"remote add\",Param \"myremote\",Param \"gcrypt::XXXXXXXXXXX:gcrypt-tests\"] failed + +this is my git-annex version info: +git-annex version: 5.20141125 +build flags: Assistant Webapp Webapp-secure Pairing Testsuite S3 WebDAV Inotify DBus DesktopNotify XMPP DNS Feeds Quvi TDFA CryptoHash +key/value backends: SHA256E SHA1E SHA512E SHA224E SHA384E SKEIN256E SKEIN512E SHA256 SHA1 SHA512 SHA224 SHA384 SKEIN256 SKEIN512 WORM URL +remote types: git gcrypt S3 bup directory rsync web webdav tahoe glacier ddar hook external +local repository version: 5 +supported repository version: 5 +upgrade supported from repository versions: 0 1 2 4 + +am I doing something wrong? +thank you +Giovanni +"""]] From e3c7f93b9c6cd4c91c0a803a3bd8df8a78f49cbd Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawlZ-6dtxJY4cP7shhvV8E6YyuV0Rak8it4" Date: Tue, 10 Mar 2015 22:11:01 +0000 Subject: [PATCH 18/26] removed --- ..._4c6ffd6dd8d0ea16667866139b181d41._comment | 24 ------------------- 1 file changed, 24 deletions(-) delete mode 100644 doc/encryption/comment_1_4c6ffd6dd8d0ea16667866139b181d41._comment diff --git a/doc/encryption/comment_1_4c6ffd6dd8d0ea16667866139b181d41._comment b/doc/encryption/comment_1_4c6ffd6dd8d0ea16667866139b181d41._comment deleted file mode 100644 index c0be6ca77a..0000000000 --- a/doc/encryption/comment_1_4c6ffd6dd8d0ea16667866139b181d41._comment +++ /dev/null @@ -1,24 +0,0 @@ -[[!comment format=mdwn - username="https://www.google.com/accounts/o8/id?id=AItOawlZ-6dtxJY4cP7shhvV8E6YyuV0Rak8it4" - nickname="Giovanni" - subject="problem adding keyid to existing repo" - date="2015-03-10T22:09:42Z" - content=""" -I have a gcrypt special remote encrypted in hybrid mode, when I try to add a keyid using \"git annex enableremote myremote keyid+=XXXXXXXX\" I get this error: - -enableremote myremote (encryption update) (hybrid cipher with gpg keys XXXXXXXX XXXXXXX) fatal: remote myremote already exists. -git-annex: git [Params \"remote add\",Param \"myremote\",Param \"gcrypt::XXXXXXXXXXX:gcrypt-tests\"] failed - -this is my git-annex version info: -git-annex version: 5.20141125 -build flags: Assistant Webapp Webapp-secure Pairing Testsuite S3 WebDAV Inotify DBus DesktopNotify XMPP DNS Feeds Quvi TDFA CryptoHash -key/value backends: SHA256E SHA1E SHA512E SHA224E SHA384E SKEIN256E SKEIN512E SHA256 SHA1 SHA512 SHA224 SHA384 SKEIN256 SKEIN512 WORM URL -remote types: git gcrypt S3 bup directory rsync web webdav tahoe glacier ddar hook external -local repository version: 5 -supported repository version: 5 -upgrade supported from repository versions: 0 1 2 4 - -am I doing something wrong? -thank you -Giovanni -"""]] From 50ddad1a6b758d3a10d01882d427eddac3e892a2 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawlZ-6dtxJY4cP7shhvV8E6YyuV0Rak8it4" Date: Tue, 10 Mar 2015 22:16:09 +0000 Subject: [PATCH 19/26] Added a comment --- ..._4257e3c4ae559f1c0595a903f738fd7e._comment | 28 +++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100644 doc/encryption/comment_1_4257e3c4ae559f1c0595a903f738fd7e._comment diff --git a/doc/encryption/comment_1_4257e3c4ae559f1c0595a903f738fd7e._comment b/doc/encryption/comment_1_4257e3c4ae559f1c0595a903f738fd7e._comment new file mode 100644 index 0000000000..e1decc397c --- /dev/null +++ b/doc/encryption/comment_1_4257e3c4ae559f1c0595a903f738fd7e._comment @@ -0,0 +1,28 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawlZ-6dtxJY4cP7shhvV8E6YyuV0Rak8it4" + nickname="Giovanni" + subject="comment 1" + date="2015-03-10T22:16:09Z" + content=""" +I have a gcrypt special remote encrypted in hybrid mode, when I try to add a keyid using: + + git annex enableremote myremote keyid+=XXXXXXXX + +I get this error: + + enableremote myremote (encryption update) (hybrid cipher with gpg keys XXXXXXXX XXXXXXX) fatal: remote myremote already exists. + git-annex: git [Params \"remote add\",Param \"myremote\",Param \"gcrypt::XXXXXXXXXXX:gcrypt-tests\"] failed + +this is my git-annex version info: + + git-annex version: 5.20141125 + build flags: Assistant Webapp Webapp-secure Pairing Testsuite S3 WebDAV Inotify DBus DesktopNotify XMPP DNS Feeds Quvi TDFA CryptoHash + key/value backends: SHA256E SHA1E SHA512E SHA224E SHA384E SKEIN256E SKEIN512E SHA256 SHA1 SHA512 SHA224 SHA384 SKEIN256 SKEIN512 WORM URL + remote types: git gcrypt S3 bup directory rsync web webdav tahoe glacier ddar hook external + local repository version: 5 + supported repository version: 5 + upgrade supported from repository versions: 0 1 2 4 + +am I doing something wrong? thank you Giovanni + +"""]] From f418ccc6d70398599601254309bd44c897df8f79 Mon Sep 17 00:00:00 2001 From: "http://hendry.iki.fi/" Date: Thu, 12 Mar 2015 04:28:05 +0000 Subject: [PATCH 20/26] --- doc/todo/Nearline_support.mdwn | 4 ++++ 1 file changed, 4 insertions(+) create mode 100644 doc/todo/Nearline_support.mdwn diff --git a/doc/todo/Nearline_support.mdwn b/doc/todo/Nearline_support.mdwn new file mode 100644 index 0000000000..2c99478303 --- /dev/null +++ b/doc/todo/Nearline_support.mdwn @@ -0,0 +1,4 @@ +This has been described as Google's [[special_remotes/glacier]]. + +* [Announcement](http://googlecloudplatform.blogspot.in/2015/03/introducing-Google-Cloud-Storage-Nearline-near-online-data-at-an-offline-price.html) +* From bfe29b6a1c20a17b10807e08768ec6b159adfaf5 Mon Sep 17 00:00:00 2001 From: clement Date: Thu, 12 Mar 2015 11:00:29 +0000 Subject: [PATCH 21/26] --- ...etween_to_regular_repositories_hangs_.mdwn | 28 +++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100644 doc/bugs/assistant:_synchronisation_between_to_regular_repositories_hangs_.mdwn diff --git a/doc/bugs/assistant:_synchronisation_between_to_regular_repositories_hangs_.mdwn b/doc/bugs/assistant:_synchronisation_between_to_regular_repositories_hangs_.mdwn new file mode 100644 index 0000000000..8e6c4f468f --- /dev/null +++ b/doc/bugs/assistant:_synchronisation_between_to_regular_repositories_hangs_.mdwn @@ -0,0 +1,28 @@ +### Please describe the problem. + +Two regular repositories created with the assistant, one on the computer and one on an USB stick cannot synchronise. Log reports problem with the index file: + + /media/usb/annex/.git/index: copyFile: does not exist (No such file or directory) + +### What steps will reproduce the problem? + +Create a repository and then another on an usb stick (with `Add another repository`) and add a file to the first one, it doesn't synchronise. + +### What version of git-annex are you using? On what operating system? + +git-annex 5.20141125 from Debian Jessie + +### Please provide any additional information below. + +I was able to solve the problem by copying the .git/index file from the first repository to the second one. + +Also note that the second repository is on a FAT32 usb stick + + +[[!format sh """ +# If you can, paste a complete transcript of the problem occurring here. +# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log + + +# End of transcript or log. +"""]] From 6231b7a369c6b124743ada7056a90162f5427cfe Mon Sep 17 00:00:00 2001 From: clement Date: Thu, 12 Mar 2015 11:46:00 +0000 Subject: [PATCH 22/26] rename bugs/assistant:_synchronisation_between_to_regular_repositories_hangs_.mdwn to bugs/Assistant:_synchronisation_between_to_regular_repositories_hangs_.mdwn --- ...:_synchronisation_between_to_regular_repositories_hangs_.mdwn} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename doc/bugs/{assistant:_synchronisation_between_to_regular_repositories_hangs_.mdwn => Assistant:_synchronisation_between_to_regular_repositories_hangs_.mdwn} (100%) diff --git a/doc/bugs/assistant:_synchronisation_between_to_regular_repositories_hangs_.mdwn b/doc/bugs/Assistant:_synchronisation_between_to_regular_repositories_hangs_.mdwn similarity index 100% rename from doc/bugs/assistant:_synchronisation_between_to_regular_repositories_hangs_.mdwn rename to doc/bugs/Assistant:_synchronisation_between_to_regular_repositories_hangs_.mdwn From 7c839cb38064ad2cee65da498148ee7395aa9f73 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawnudiRFAyVAwehTACPhuNV_dhhWsHKIACw" Date: Thu, 12 Mar 2015 18:32:18 +0000 Subject: [PATCH 23/26] Added a comment: git annex on chromebook --- ..._5431a57aa7d659faf0aee12e638dd1ba._comment | 56 +++++++++++++++++++ 1 file changed, 56 insertions(+) create mode 100644 doc/forum/Manual_webapp_behaviour_on_ARM/comment_1_5431a57aa7d659faf0aee12e638dd1ba._comment diff --git a/doc/forum/Manual_webapp_behaviour_on_ARM/comment_1_5431a57aa7d659faf0aee12e638dd1ba._comment b/doc/forum/Manual_webapp_behaviour_on_ARM/comment_1_5431a57aa7d659faf0aee12e638dd1ba._comment new file mode 100644 index 0000000000..f045b5c3ec --- /dev/null +++ b/doc/forum/Manual_webapp_behaviour_on_ARM/comment_1_5431a57aa7d659faf0aee12e638dd1ba._comment @@ -0,0 +1,56 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawnudiRFAyVAwehTACPhuNV_dhhWsHKIACw" + nickname="Jose" + subject="git annex on chromebook" + date="2015-03-12T18:32:18Z" + content=""" +I know I'm commenting on something old but I was looking this topic up and felt this may help any future questions. :S + +* its just like any other machine with git annex. You can connect your instance of git annex that runs in the chroot (crouton) with your existing set up like normal. usb, ssh, xmpp+cloud + +* you will need a transfer repository, that vps should do nicely (I use a few USB drives) + + * this depends if the VPS also has git annex installed or not. +if it does, great! If you want it configured for you, you can run git annex webapp to add a remote server and give it the details to connect to your VPS via ssh. Then tell it where your transfer repository will be and that should be it. If you plan on sharing this repo with your friend, or if you want more control over the data, you may want to set up your ssh repo via the command line instead. you may be able to add encryption and share your repository without necessarily letting your friend have access to everything on your account (if you have separate accounts) + + +if it does not, also doable. Again if you want simple configuration git annex webapp will prompt you for credentials; in this case it may as you to encrypt the data (http://git-annex.branchable.com/encryption/). + +* how to add an ssh remote via commandline (please don't just copy paste, a simple google of the commands will help you understand what they are doing) + +$ cd ~/Downloads/Documents + +$ git init + +$ git annex init \"chromebook\" + +$ git remote add my-ssh ssh://user@google.com/home/user/annex/Documents #I recommend you use an 'ssh alias' so you can just do 'ssh://alias1/home/user/annex/Documents' + +$ git annex add + +$ git annex sync + +$ git annex sync --content #I do not remember but I believe it may be necessary to do this for some versions of git annex. + +git annex may or may not complain about git annex being installed on your machine. (it does for me each time I do this... perhaps I'm missing a step?) +If you know git annex is installed in the $PATH of your server (also your machine) then you can just run + +$ git config remote.my-ssh.annex-ignore false + +$ git annex sync + +$ git annex sync --content + + +* how come I can not see my annexed files on chromeos but I can on crouton? +because you are using git annex in 'indirect' mode + +$ cd ~/Downloads/Documents + +$ git annex direct + +You can also 'unlock' the files too but you will need to run git annex unlock every time you need to read/write a file using. + +$ git annex unlock + +"""]] From cbbb60eff8925b3e52d7fa2124dce6a18451019b Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawmsy_GIefGlGGD_XJp_R6EsWIRUC4ev9XU" Date: Fri, 13 Mar 2015 20:48:56 +0000 Subject: [PATCH 24/26] Added a comment: This is a BIG task --- ...ment_3_560d3f65d543c3af9722ed7e9a11e920._comment | 13 +++++++++++++ 1 file changed, 13 insertions(+) create mode 100644 doc/design/iabackup/comment_3_560d3f65d543c3af9722ed7e9a11e920._comment diff --git a/doc/design/iabackup/comment_3_560d3f65d543c3af9722ed7e9a11e920._comment b/doc/design/iabackup/comment_3_560d3f65d543c3af9722ed7e9a11e920._comment new file mode 100644 index 0000000000..dca02f8af8 --- /dev/null +++ b/doc/design/iabackup/comment_3_560d3f65d543c3af9722ed7e9a11e920._comment @@ -0,0 +1,13 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawmsy_GIefGlGGD_XJp_R6EsWIRUC4ev9XU" + nickname="David" + subject="This is a BIG task" + date="2015-03-13T20:48:56Z" + content=""" +If I understand it correctly, 20PB at 2400 shards of 8TB each with 3 copies is 24TB/shard at 1TB/client is 2400*24 = ~60K clients assuming no churn. So it would probably need ~100K clients to cover the churn and have a good chance that each shard had 3 copies at all times. That's 1/3 the size of BOINC's active population. + +It would take time to scale to that population. And it would take time to get three copies out of the Archive. During that time, the Archive is growing. The back of my envelope says that doing this in 2.5yrs roughly doubles the Archive's outbound bandwidth if you average it across the 2.5 years. But the population would grow slowly to start with, then faster, so that the bandwidth impact would be back-loaded. And at the end of the 2.5 years, you would need a lot more than the 100K users. + +A design that used erasure coding or entanglement would reduce the storage and bandwidth demand considerably while providing adequate reliability. + +"""]] From 1d28478a8435bc27a3cfeb836a3ff7fc70bbf8ca Mon Sep 17 00:00:00 2001 From: "https://me.yahoo.com/a/_yfGpKsE29lA9Hb0UEVOeJ1nKYc-#6f39b" Date: Sat, 14 Mar 2015 07:38:02 +0000 Subject: [PATCH 25/26] git-annex sync on new remote is slow --- ...c_on_new_remote___40__USB_drive__41__.mdwn | 82 +++++++++++++++++++ 1 file changed, 82 insertions(+) create mode 100644 doc/forum/4hr+_sync_on_new_remote___40__USB_drive__41__.mdwn diff --git a/doc/forum/4hr+_sync_on_new_remote___40__USB_drive__41__.mdwn b/doc/forum/4hr+_sync_on_new_remote___40__USB_drive__41__.mdwn new file mode 100644 index 0000000000..35bb24fb68 --- /dev/null +++ b/doc/forum/4hr+_sync_on_new_remote___40__USB_drive__41__.mdwn @@ -0,0 +1,82 @@ +I need some help understanding what would cause ``git-annex sync`` to still be running 4hrs+ on a new remote (FAT32 USB drive - BTEST) ? From all my searching, it appears to be part of git's optimization routines. But to be this long and slow seems odd. Also I am not sure what there would be optimize since it's a brand new remote ? The src (WTEST) doesn't seem to show any need to run ``git gc``. + +I followed the walkthrough's setup of a remote and followed that up with ``git-annex sync``. I did not specify the source for the sync as there is only one other (WTEST). The content is mostly ISO files and Win32 executables in 7 separate commits. In between ``git-annex import`` and ``git commit``, the working directory was removed with ``git rm '*'`` but no ``git-annex drop``. There were lots of duplicate files and it truly was satisfying to not see the disk usage increase. My backend is also MD5E for purposes of quicker imports and ease of hash lookup from other sources. + +I would like to have 20-30 USB remotes. The length of time to get one remote up and running at this point is horrendous. Other people seem to have better experiences with it. Only odd thing that I see with my configuration is that the remote FAT32 drive is in ``indirect`` mode. My understanding is that ``git-annex`` would automatically switch to ``direct`` if it detected a filesystem that did not support symbolic links. + +What is wrong with my setup ? How can I fix it ? + +[[!format sh """ +BTEST$ git-annex sync + +commit ok + +pull origin + +Auto packing the repository for optimum performance. You may also + +run "git gc" manually. See "git help gc" for more information. + +Counting objects: 521834, done. + +Delta compression using up to 4 threads. + +Compressing objects: 12% (59687/462488) +"""]] + +[[!format sh """ + +WTEST$ git-annex info + +repository mode: indirect +trusted repositories: 0 +semitrusted repositories: 4 + 00000000-0000-0000-0000-000000000001 -- web + 00000000-0000-0000-0000-000000000002 -- bittorrent + 98dfasdf-ab83-4a0e-8b73-4dfasffdsaff1ae -- WTEST [here] + 9dfdsfdf8-c5d5-4761-ab67-ffsadfsadfsa83 -- BTEST +untrusted repositories: 0 +transfers in progress: none +available local disk space: 488.16 gigabytes (+1 megabyte reserved) +local annex keys: 57327 +local annex size: 58.57 gigabytes +annexed files in working tree: 4322 +size of annexed files in working tree: 6.82 gigabytes +bloom filter size: 16 mebibytes (11.5% full) +backend usage: + MD5E: 61649 +"""]] + +[[!format sh """ +WTEST$git object-count -v + +count: 521829 +size: 66794240 +in-pack: 0 +packs: 0 +size-pack: 0 +prune-packable: 0 +garbage: 0 +size-garbage: 0 +"""]] + + +[[!format sh """ +WTEST$git config -l + +core.repositoryformatversion=0 +core.filemode=false +core.bare=false +core.logallrefupdates=true +core.ignorecase=true +core.precomposeunicode=true +annex.uuid=98dfasdf-ab83-4a0e-8b73-4dfasffdsaff1ae +annex.sshcaching=false +annex.version=5 +annex.backends=MD5E +annex.queuesize=102400 +annex.genmetadata=true +remote.BTEST.url=/Volumes/BTEST +remote.BTEST.fetch=+refs/heads/*:refs/remotes/BTEST/* +remote.BTEST.annex-uuid=9dfdsfdf8-c5d5-4761-ab67-ffsadfsadfsa83 +"""]] From 5cf6a55f56eb9736bac6c3e02436033d60bf2347 Mon Sep 17 00:00:00 2001 From: "https://me.yahoo.com/a/_yfGpKsE29lA9Hb0UEVOeJ1nKYc-#6f39b" Date: Sat, 14 Mar 2015 14:34:40 +0000 Subject: [PATCH 26/26] Update after 7hrs: 70% done writing objects./Formatting changes --- ...c_on_new_remote___40__USB_drive__41__.mdwn | 44 ++++++++++++------- 1 file changed, 27 insertions(+), 17 deletions(-) diff --git a/doc/forum/4hr+_sync_on_new_remote___40__USB_drive__41__.mdwn b/doc/forum/4hr+_sync_on_new_remote___40__USB_drive__41__.mdwn index 35bb24fb68..167c090f18 100644 --- a/doc/forum/4hr+_sync_on_new_remote___40__USB_drive__41__.mdwn +++ b/doc/forum/4hr+_sync_on_new_remote___40__USB_drive__41__.mdwn @@ -6,23 +6,7 @@ I would like to have 20-30 USB remotes. The length of time to get one remote up What is wrong with my setup ? How can I fix it ? -[[!format sh """ -BTEST$ git-annex sync - -commit ok - -pull origin - -Auto packing the repository for optimum performance. You may also - -run "git gc" manually. See "git help gc" for more information. - -Counting objects: 521834, done. - -Delta compression using up to 4 threads. - -Compressing objects: 12% (59687/462488) -"""]] +Here's the basic config of WTEST. [[!format sh """ @@ -80,3 +64,29 @@ remote.BTEST.url=/Volumes/BTEST remote.BTEST.fetch=+refs/heads/*:refs/remotes/BTEST/* remote.BTEST.annex-uuid=9dfdsfdf8-c5d5-4761-ab67-ffsadfsadfsa83 """]] + +**The long running sync.** + +[[!format sh """ +BTEST$ git-annex sync + +commit ok + +pull origin + +Auto packing the repository for optimum performance. You may also + +run "git gc" manually. See "git help gc" for more information. + +Counting objects: 521834, done. + +Delta compression using up to 4 threads. + +Compressing objects: 12% (59687/462488) +"""]] + +**Update 7hrs later:** + +[[!format sh """ +Writing objects: 70% (366184/521834) +"""]]