From f1bd615d378fefa74f74b0dde9341e3de8891bf5 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawnX6DJ6XrW6-omVUgOKuKe0r_c9cLsR7io" Date: Mon, 9 Sep 2013 21:34:54 +0000 Subject: [PATCH 01/62] Added a comment: Well --- .../comment_1_64832ff265661d920993e9e3bcea7971._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/devblog/day_7__release_day/comment_1_64832ff265661d920993e9e3bcea7971._comment diff --git a/doc/devblog/day_7__release_day/comment_1_64832ff265661d920993e9e3bcea7971._comment b/doc/devblog/day_7__release_day/comment_1_64832ff265661d920993e9e3bcea7971._comment new file mode 100644 index 0000000000..7559bc503e --- /dev/null +++ b/doc/devblog/day_7__release_day/comment_1_64832ff265661d920993e9e3bcea7971._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawnX6DJ6XrW6-omVUgOKuKe0r_c9cLsR7io" + nickname="Andrew" + subject="Well" + date="2013-09-09T21:34:53Z" + content=""" +There are some cheap +"""]] From 0df81bb8701a6c7cb6c5118957085069cdcc3e2e Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawnX6DJ6XrW6-omVUgOKuKe0r_c9cLsR7io" Date: Mon, 9 Sep 2013 21:36:23 +0000 Subject: [PATCH 02/62] removed --- .../comment_1_64832ff265661d920993e9e3bcea7971._comment | 8 -------- 1 file changed, 8 deletions(-) delete mode 100644 doc/devblog/day_7__release_day/comment_1_64832ff265661d920993e9e3bcea7971._comment diff --git a/doc/devblog/day_7__release_day/comment_1_64832ff265661d920993e9e3bcea7971._comment b/doc/devblog/day_7__release_day/comment_1_64832ff265661d920993e9e3bcea7971._comment deleted file mode 100644 index 7559bc503e..0000000000 --- a/doc/devblog/day_7__release_day/comment_1_64832ff265661d920993e9e3bcea7971._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="https://www.google.com/accounts/o8/id?id=AItOawnX6DJ6XrW6-omVUgOKuKe0r_c9cLsR7io" - nickname="Andrew" - subject="Well" - date="2013-09-09T21:34:53Z" - content=""" -There are some cheap -"""]] From a0b895fdf981d08530c6a38b2dee4495d7c6f5da Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawnsuhFUIfWNT-Q-C02FDaSQqceFDge5M9w" Date: Mon, 9 Sep 2013 21:45:29 +0000 Subject: [PATCH 03/62] --- doc/forum/Sync_with_one_offline_peer.txt | 11 +++++++++++ 1 file changed, 11 insertions(+) create mode 100644 doc/forum/Sync_with_one_offline_peer.txt diff --git a/doc/forum/Sync_with_one_offline_peer.txt b/doc/forum/Sync_with_one_offline_peer.txt new file mode 100644 index 0000000000..6ee05c6e67 --- /dev/null +++ b/doc/forum/Sync_with_one_offline_peer.txt @@ -0,0 +1,11 @@ +Hello, + +I use the assistant to set up two repositories A and B synced using jabber. A third repository C on my server is used as rsync transfer. Syncing works fine between both repos when both are online. + +But when either A or B is offline the sync does not happen when it comes online again, though the file was synced to C. + +Is this because C is only a rsync repository and can't hold metadata? How can I achieve that the sync happens also when one of the repositories is offline? + +I also tried using the static build of git annex on my server. It seemed to run fine but during the setup the assistant got an error about too many command line arguments. On A und B I use the ArchLinux AUR build (https://aur.archlinux.org/packages/git-annex-standalone/), on C I use the static build. Could it be a version mismatch? + +Thanks! From 3a1d0d0265eeddb6831504c6c814bbe42cd14aee Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawnsuhFUIfWNT-Q-C02FDaSQqceFDge5M9w" Date: Mon, 9 Sep 2013 21:49:17 +0000 Subject: [PATCH 04/62] Added a comment --- ..._3859d842d4f7e2ef44877b05ebe881fb._comment | 20 +++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100644 doc/forum/Sync_with_one_offline_peer/comment_1_3859d842d4f7e2ef44877b05ebe881fb._comment diff --git a/doc/forum/Sync_with_one_offline_peer/comment_1_3859d842d4f7e2ef44877b05ebe881fb._comment b/doc/forum/Sync_with_one_offline_peer/comment_1_3859d842d4f7e2ef44877b05ebe881fb._comment new file mode 100644 index 0000000000..5fe670d35c --- /dev/null +++ b/doc/forum/Sync_with_one_offline_peer/comment_1_3859d842d4f7e2ef44877b05ebe881fb._comment @@ -0,0 +1,20 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawnsuhFUIfWNT-Q-C02FDaSQqceFDge5M9w" + nickname="Florian" + subject="comment 1" + date="2013-09-09T21:49:17Z" + content=""" +Another try on the layout: + +Hello, + +I use the assistant to set up two repositories A and B synced using jabber. A third repository C on my server is used as rsync transfer. Syncing works fine between both repos when both are online. + +But when either A or B is offline the sync does not happen when it comes online again, though the file was synced to C. + +Is this because C is only a rsync repository and can't hold metadata? How can I achieve that the sync happens also when one of the repositories is offline? + +I also tried using the static build of git annex on my server. It seemed to run fine but during the setup the assistant got an error about too many command line arguments. On A und B I use the ArchLinux AUR build (https://aur.archlinux.org/packages/git-annex-standalone/), on C I use the static build. Could it be a version mismatch? + +Thanks! +"""]] From 77723fe047909304870193bce6c8bec734765172 Mon Sep 17 00:00:00 2001 From: "https://me.yahoo.com/a/2grhJvAC049fJnvALDXek.6MRZMTlg--#eec89" Date: Mon, 9 Sep 2013 21:51:47 +0000 Subject: [PATCH 05/62] Added a comment --- .../comment_1_12bb94d903868ecddb3e348c9c4afeaf._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/devblog/day_7__release_day/comment_1_12bb94d903868ecddb3e348c9c4afeaf._comment diff --git a/doc/devblog/day_7__release_day/comment_1_12bb94d903868ecddb3e348c9c4afeaf._comment b/doc/devblog/day_7__release_day/comment_1_12bb94d903868ecddb3e348c9c4afeaf._comment new file mode 100644 index 0000000000..266e843fd5 --- /dev/null +++ b/doc/devblog/day_7__release_day/comment_1_12bb94d903868ecddb3e348c9c4afeaf._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="https://me.yahoo.com/a/2grhJvAC049fJnvALDXek.6MRZMTlg--#eec89" + nickname="John" + subject="comment 1" + date="2013-09-09T21:51:47Z" + content=""" +Joey, I'd be happy to host 64-bit builds on both Mac and Linux using my personal Jenkins server (both machines are local and have 16+ GB of RAM). I might even be able to get you SSH jail access to the build tree for the 64-bit Linux build. Just let me know at johnw@fpcomplete.com. +"""]] From e34c3b087b603623d9632f651571c0909c52f358 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawmkBwMWvNKZZCge_YqobCSILPMeK6xbFw8" Date: Mon, 9 Sep 2013 21:55:45 +0000 Subject: [PATCH 06/62] --- ...sanitychecker_fix_wrong_UUID__47__duplicate_remote.mdwn | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100644 doc/todo/Wishlist:_sanitychecker_fix_wrong_UUID__47__duplicate_remote.mdwn diff --git a/doc/todo/Wishlist:_sanitychecker_fix_wrong_UUID__47__duplicate_remote.mdwn b/doc/todo/Wishlist:_sanitychecker_fix_wrong_UUID__47__duplicate_remote.mdwn new file mode 100644 index 0000000000..ff7773d3e6 --- /dev/null +++ b/doc/todo/Wishlist:_sanitychecker_fix_wrong_UUID__47__duplicate_remote.mdwn @@ -0,0 +1,7 @@ +In certain situations different client annexes might get the same remote repository added, but before being synced. + +Once the two clients sync they will both have two remotes with the same name. But only one UUID will have any content(Assuming only one client pushed). + +It would be nice to have some (automatic?) way to resolve this conflict. + +Not sure if anything sane can be done if both clients have pushed? From d268abf78d638583d49d36190e7ea88b89d3f97a Mon Sep 17 00:00:00 2001 From: "http://grossmeier.net/" Date: Mon, 9 Sep 2013 22:06:31 +0000 Subject: [PATCH 07/62] --- ...portfeed_fails__44___bad_feed_content.mdwn | 34 +++++++++++++++++++ 1 file changed, 34 insertions(+) create mode 100644 doc/bugs/importfeed_fails__44___bad_feed_content.mdwn diff --git a/doc/bugs/importfeed_fails__44___bad_feed_content.mdwn b/doc/bugs/importfeed_fails__44___bad_feed_content.mdwn new file mode 100644 index 0000000000..2435e3304b --- /dev/null +++ b/doc/bugs/importfeed_fails__44___bad_feed_content.mdwn @@ -0,0 +1,34 @@ +### Please describe the problem. + +I just added a new podcast to my feeds list and it is complaining of "bad feed content" when trying to import. + +### What steps will reproduce the problem? + +1. git-annex importfeed --fast http://redecentralize.org/podcast/feed.rss +3. see the sad error "warning: bad feed content" + +### What version of git-annex are you using? On what operating system? + +git-annex version: 4.20130827 + +### Please provide any additional information below. + +[[!format sh """ +greg@x200s:~/annex/Podcasts$ git-annex importfeed --fast http://redecentralize.org/podcast/feed.rss +(checking known urls...) +importfeed http://redecentralize.org/podcast/feed.rss +--2013-09-09 15:05:17-- http://redecentralize.org/podcast/feed.rss +Resolving redecentralize.org (redecentralize.org)... 204.232.175.78 +Connecting to redecentralize.org (redecentralize.org)|204.232.175.78|:80... connected. +HTTP request sent, awaiting response... 200 OK +Length: 4331 (4.2K) [text/xml] +Saving to: ‘/tmp/feed12649’ + +100%[==============================================================>] 4,331 --.-K/s in 0.08s + +2013-09-09 15:05:17 (56.0 KB/s) - ‘/tmp/feed12649’ saved [4331/4331] + + + warning: bad feed content +ok +"""]] From 94970f4d0b3d180b0cbe787db723c9f70fa0742f Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawlQsgfXntaEvZPKgd_K0dfoFcYBJXKcaFE" Date: Mon, 9 Sep 2013 22:54:42 +0000 Subject: [PATCH 08/62] --- ...sync_in_preferred_content_mode_manual.mdwn | 22 +++++++++++++++++++ 1 file changed, 22 insertions(+) create mode 100644 doc/bugs/assistant_fails_to_sync_in_preferred_content_mode_manual.mdwn diff --git a/doc/bugs/assistant_fails_to_sync_in_preferred_content_mode_manual.mdwn b/doc/bugs/assistant_fails_to_sync_in_preferred_content_mode_manual.mdwn new file mode 100644 index 0000000000..f3bff86837 --- /dev/null +++ b/doc/bugs/assistant_fails_to_sync_in_preferred_content_mode_manual.mdwn @@ -0,0 +1,22 @@ +Thanks for git annex! + +### Please describe the problem. + +I've set up two computers A, B to sync to (and each other through) a remote server S. If I add files to A, it syncs with S. But B is not getting synced unless I run 'git annex sync' or restart the daemon in the webapp, i.e. I don't see these files on B at all, not even as broken symlinks. +The same happens if I manually copy files from A to S. B is not updating the copy count in 'git annex whereis' until I manually sync on B. + +It would be great if B could automatically sync when S is changed, either file locations or contents. + +### What steps will reproduce the problem? + +Both computers' local repos were setup with annex assistant, and switched into indirect mode. The server contains git repo and file data. All systems, computers and server are set to manual in the preferred content mode. + +### What version of git-annex are you using? On what operating system? + +A is running Debian testing with git-annex 4.20130827, B and S are running Debian stable with the backports version of git-annex 4.20130815~bpo70+1. A and S are 64 bit, B is 32 bit. All are up to date. + +### Please provide any additional information below. + +I tried this setup before with direct mode and different preferred content settings (A, B as client, S as backup or archive), and syncing of git and file data was working then. + +Despite being able to run 'sync' it is at least inconvenient to not have automatically updated file location information when I run the assistant. (I could then just run without it, but I had it happen to me that I setup a local repo w/o assistant, and when I ran assistant, expecting it would tell me "there are no local repos, would you like to create one?" it somehow automatically found the manually created one and started copying files. But that's another problem.) From 7f079d62934485e1651e19ffac88db3e9dfd1554 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawlQsgfXntaEvZPKgd_K0dfoFcYBJXKcaFE" Date: Mon, 9 Sep 2013 23:36:49 +0000 Subject: [PATCH 09/62] Added a comment: Direct oder indirect mode makes no difference --- .../comment_1_37acb3afafb1b4c4da7c778130cf3035._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/bugs/assistant_fails_to_sync_in_preferred_content_mode_manual/comment_1_37acb3afafb1b4c4da7c778130cf3035._comment diff --git a/doc/bugs/assistant_fails_to_sync_in_preferred_content_mode_manual/comment_1_37acb3afafb1b4c4da7c778130cf3035._comment b/doc/bugs/assistant_fails_to_sync_in_preferred_content_mode_manual/comment_1_37acb3afafb1b4c4da7c778130cf3035._comment new file mode 100644 index 0000000000..94dc51d0a7 --- /dev/null +++ b/doc/bugs/assistant_fails_to_sync_in_preferred_content_mode_manual/comment_1_37acb3afafb1b4c4da7c778130cf3035._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawlQsgfXntaEvZPKgd_K0dfoFcYBJXKcaFE" + nickname="Sten" + subject="Direct oder indirect mode makes no difference" + date="2013-09-09T23:36:49Z" + content=""" +Direct or indirect mode does not make a difference. I created another setup like this, only different is direct mode, otherwise the same (in particular preferred content mode is manual). Result: Still no sync. +"""]] From 0b60f8b810864533c00ae42e0aab6dfb9c506a20 Mon Sep 17 00:00:00 2001 From: "https://id.koumbit.net/anarcat" Date: Tue, 10 Sep 2013 01:26:35 +0000 Subject: [PATCH 10/62] share the results for my search for binary packages --- doc/install/cabal.mdwn | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/doc/install/cabal.mdwn b/doc/install/cabal.mdwn index 6c5c0d4c69..1e4470833d 100644 --- a/doc/install/cabal.mdwn +++ b/doc/install/cabal.mdwn @@ -40,3 +40,13 @@ you want, and use cabal as follows inside its source tree: cabal configure cabal build cabal install --bindir=$HOME/bin + +## installing some binary dependencies + +Some binary dependencies can be installed by hand. This saves compilation time because `cabal` will find those packages and use them as is: + + apt-get install libghc-mtl-dev libghc-missingh-dev libghc-hslogger-dev libghc-pcre-light-dev libghc-sha-dev libghc-regex-tdfa-dev libghc-dataenc-dev libghc-utf8-string-dev libghc-hs3-dev libghc-quickcheck2-dev libghc-monad-control-dev libghc-monadcatchio-transformers-dev libghc-unix-compat-dev libghc-dlist-dev libghc-uuid-dev libghc-json-dev libghc-aeson-dev libghc-ifelse-dev libghc-bloomfilter-dev libghc-edit-distance-dev libghc-extensible-exceptions-dev libghc-stm-dev libghc-dbus-dev libghc-case-insensitive-dev libghc-http-types-dev libghc-blaze-builder-dev libghc-crypto-api-dev libghc-network-multicast-dev libghc-network-info-dev libghc-safesemaphore-dev libghc-network-protocol-xmpp-dev libghc-gnutls-dev libghc-async-dev libghc-http-dev libghc-feed-dev + +Those may also be available at the right version if you are running sid or jessie: + + apt-get install libghc-yesod-dev libghc-yesod-static-dev libghc-yesod-default-dev libghc-warp-dev libghc-wai-dev libghc-wai-logger-dev libghc-hamlet-dev libghc-clientsession-dev libghc-dav-dev libghc-extensible-exceptions-dev libghc-hinotify-dev From 63af14b24811317446ce814379395201367a351c Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawlQsgfXntaEvZPKgd_K0dfoFcYBJXKcaFE" Date: Tue, 10 Sep 2013 02:21:59 +0000 Subject: [PATCH 11/62] Added a comment: computer in manual mode seems to be the reason, not server --- ...comment_2_9d58887ee0184663852bde83b8d497c7._comment | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 doc/bugs/assistant_fails_to_sync_in_preferred_content_mode_manual/comment_2_9d58887ee0184663852bde83b8d497c7._comment diff --git a/doc/bugs/assistant_fails_to_sync_in_preferred_content_mode_manual/comment_2_9d58887ee0184663852bde83b8d497c7._comment b/doc/bugs/assistant_fails_to_sync_in_preferred_content_mode_manual/comment_2_9d58887ee0184663852bde83b8d497c7._comment new file mode 100644 index 0000000000..8d388ceeeb --- /dev/null +++ b/doc/bugs/assistant_fails_to_sync_in_preferred_content_mode_manual/comment_2_9d58887ee0184663852bde83b8d497c7._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawlQsgfXntaEvZPKgd_K0dfoFcYBJXKcaFE" + nickname="Sten" + subject="computer in manual mode seems to be the reason, not server" + date="2013-09-10T02:21:59Z" + content=""" +If I create a setup like originally described with the only difference being S set to full backup mode, I see no change in sync behavior. + +In detail, what happens is: A gets files added. These changes are committed and synced. Because S is in full backup mode, the file content is also sent from A to S (this is different, but expected). However, B still does not sync until manually run. I would expect that B's manual mode only applies to file content (it is the preferred content setting), not to changes to the git repo, thus I'd expect git changes to sync to all involved (and accessible) repos. +"""]] From 0e2879ce4285fe9e8fba5829847ea5266b0add22 Mon Sep 17 00:00:00 2001 From: "https://id.koumbit.net/anarcat" Date: Tue, 10 Sep 2013 02:40:44 +0000 Subject: [PATCH 12/62] nevermind, that failed --- doc/install/cabal.mdwn | 10 ---------- 1 file changed, 10 deletions(-) diff --git a/doc/install/cabal.mdwn b/doc/install/cabal.mdwn index 1e4470833d..6c5c0d4c69 100644 --- a/doc/install/cabal.mdwn +++ b/doc/install/cabal.mdwn @@ -40,13 +40,3 @@ you want, and use cabal as follows inside its source tree: cabal configure cabal build cabal install --bindir=$HOME/bin - -## installing some binary dependencies - -Some binary dependencies can be installed by hand. This saves compilation time because `cabal` will find those packages and use them as is: - - apt-get install libghc-mtl-dev libghc-missingh-dev libghc-hslogger-dev libghc-pcre-light-dev libghc-sha-dev libghc-regex-tdfa-dev libghc-dataenc-dev libghc-utf8-string-dev libghc-hs3-dev libghc-quickcheck2-dev libghc-monad-control-dev libghc-monadcatchio-transformers-dev libghc-unix-compat-dev libghc-dlist-dev libghc-uuid-dev libghc-json-dev libghc-aeson-dev libghc-ifelse-dev libghc-bloomfilter-dev libghc-edit-distance-dev libghc-extensible-exceptions-dev libghc-stm-dev libghc-dbus-dev libghc-case-insensitive-dev libghc-http-types-dev libghc-blaze-builder-dev libghc-crypto-api-dev libghc-network-multicast-dev libghc-network-info-dev libghc-safesemaphore-dev libghc-network-protocol-xmpp-dev libghc-gnutls-dev libghc-async-dev libghc-http-dev libghc-feed-dev - -Those may also be available at the right version if you are running sid or jessie: - - apt-get install libghc-yesod-dev libghc-yesod-static-dev libghc-yesod-default-dev libghc-warp-dev libghc-wai-dev libghc-wai-logger-dev libghc-hamlet-dev libghc-clientsession-dev libghc-dav-dev libghc-extensible-exceptions-dev libghc-hinotify-dev From 0ed4c17851ed2b3cae854aaecaeabf7dc61e1121 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawlQsgfXntaEvZPKgd_K0dfoFcYBJXKcaFE" Date: Tue, 10 Sep 2013 02:55:44 +0000 Subject: [PATCH 13/62] Added a comment: available copies update need manual sync too --- .../comment_3_b70881c8026e30fd3ddc051bd01a888b._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/bugs/assistant_fails_to_sync_in_preferred_content_mode_manual/comment_3_b70881c8026e30fd3ddc051bd01a888b._comment diff --git a/doc/bugs/assistant_fails_to_sync_in_preferred_content_mode_manual/comment_3_b70881c8026e30fd3ddc051bd01a888b._comment b/doc/bugs/assistant_fails_to_sync_in_preferred_content_mode_manual/comment_3_b70881c8026e30fd3ddc051bd01a888b._comment new file mode 100644 index 0000000000..d42cf6cd2e --- /dev/null +++ b/doc/bugs/assistant_fails_to_sync_in_preferred_content_mode_manual/comment_3_b70881c8026e30fd3ddc051bd01a888b._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawlQsgfXntaEvZPKgd_K0dfoFcYBJXKcaFE" + nickname="Sten" + subject="available copies update need manual sync too" + date="2013-09-10T02:55:44Z" + content=""" +After copying files from S to B (as expected manually), I have to (unexpectedly) manually sync again on A to update the #copies on A. +"""]] From 22c7e464116307874f1c7ecba6c62f5c624b9104 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawlQsgfXntaEvZPKgd_K0dfoFcYBJXKcaFE" Date: Tue, 10 Sep 2013 03:19:51 +0000 Subject: [PATCH 14/62] Added a comment: output of git annex sync --- ..._43f756e2e6ff985c8e050da0e369d486._comment | 25 +++++++++++++++++++ 1 file changed, 25 insertions(+) create mode 100644 doc/bugs/assistant_fails_to_sync_in_preferred_content_mode_manual/comment_4_43f756e2e6ff985c8e050da0e369d486._comment diff --git a/doc/bugs/assistant_fails_to_sync_in_preferred_content_mode_manual/comment_4_43f756e2e6ff985c8e050da0e369d486._comment b/doc/bugs/assistant_fails_to_sync_in_preferred_content_mode_manual/comment_4_43f756e2e6ff985c8e050da0e369d486._comment new file mode 100644 index 0000000000..51e9cc135a --- /dev/null +++ b/doc/bugs/assistant_fails_to_sync_in_preferred_content_mode_manual/comment_4_43f756e2e6ff985c8e050da0e369d486._comment @@ -0,0 +1,25 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawlQsgfXntaEvZPKgd_K0dfoFcYBJXKcaFE" + nickname="Sten" + subject="output of git annex sync " + date="2013-09-10T03:19:51Z" + content=""" +When running sync on B after copying file content from A to S (manually or automatically), I saw the following output (copied just the end of it): + ... + file.pdf | 1 + + file.txt | 1 + + 762 files changed, 762 insertions(+) + ... + create mode 120000 file.pdf + create mode 120000 file.txt + +git-annex: /home/sten/Documents/.git/annex/merge/: getDirectoryContents: does not exist (No such file or directory) +failed +push server +Everything up-to-date +ok +git-annex: sync: 1 failed + +A subsequent sync worked without error. + +"""]] From af590215d1b2c1b96db535cf633844166b255a10 Mon Sep 17 00:00:00 2001 From: "https://id.koumbit.net/anarcat" Date: Tue, 10 Sep 2013 05:00:50 +0000 Subject: [PATCH 15/62] Added a comment: manpages? --- .../comment_19_74639fe525042aca12f9557d5d6a4f98._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/install/cabal/comment_19_74639fe525042aca12f9557d5d6a4f98._comment diff --git a/doc/install/cabal/comment_19_74639fe525042aca12f9557d5d6a4f98._comment b/doc/install/cabal/comment_19_74639fe525042aca12f9557d5d6a4f98._comment new file mode 100644 index 0000000000..e9fcb94b8c --- /dev/null +++ b/doc/install/cabal/comment_19_74639fe525042aca12f9557d5d6a4f98._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="https://id.koumbit.net/anarcat" + ip="2001:1928:1:9::1" + subject="manpages?" + date="2013-09-10T05:00:49Z" + content=""" +the manual install process doesn't seem to install manpages - how do we do that? +"""]] From 20975d1b373205dea03cc1e4771826dfd01f6e33 Mon Sep 17 00:00:00 2001 From: eezec Date: Tue, 10 Sep 2013 06:05:32 +0000 Subject: [PATCH 16/62] Added a comment --- ...comment_2_932f6aaa712298a47868002872e16310._comment | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 doc/forum/git-annex:_status:_1_failed/comment_2_932f6aaa712298a47868002872e16310._comment diff --git a/doc/forum/git-annex:_status:_1_failed/comment_2_932f6aaa712298a47868002872e16310._comment b/doc/forum/git-annex:_status:_1_failed/comment_2_932f6aaa712298a47868002872e16310._comment new file mode 100644 index 0000000000..98fb1d8054 --- /dev/null +++ b/doc/forum/git-annex:_status:_1_failed/comment_2_932f6aaa712298a47868002872e16310._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="eezec" + ip="93.21.246.236" + subject="comment 2" + date="2013-09-10T06:05:32Z" + content=""" +i have + git-annex version: 4.20130815 + +"""]] From 2af2817eaaed40759c1e4b5de935bd85e059447f Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawkQrsXK-icnYXg6kJNd-jDjMgOixnhwE34" Date: Tue, 10 Sep 2013 13:46:45 +0000 Subject: [PATCH 17/62] Added a comment: reply to Comment 1 --- ..._c0880ce3ee13d388ab5b46a740170845._comment | 50 +++++++++++++++++++ 1 file changed, 50 insertions(+) create mode 100644 doc/forum/Git_annex_on_Windows/comment_2_c0880ce3ee13d388ab5b46a740170845._comment diff --git a/doc/forum/Git_annex_on_Windows/comment_2_c0880ce3ee13d388ab5b46a740170845._comment b/doc/forum/Git_annex_on_Windows/comment_2_c0880ce3ee13d388ab5b46a740170845._comment new file mode 100644 index 0000000000..bdf30b3b7e --- /dev/null +++ b/doc/forum/Git_annex_on_Windows/comment_2_c0880ce3ee13d388ab5b46a740170845._comment @@ -0,0 +1,50 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawkQrsXK-icnYXg6kJNd-jDjMgOixnhwE34" + nickname="x" + subject="reply to Comment 1" + date="2013-09-10T13:46:45Z" + content=""" +Thanks for your helpful comment.I add the path to windows, and the +problem disappears now. +But I meet another problem when I download the data using git annex. +I use the command \"git annex get foo.nc\" to download data. The foo.nc +is actually symlink. For example: +foo.nc -> ../../.git/annex/objects/jK/v7/WORM-s16009120-m1337754158-- +foo.nc/WORM-s16009120-m1337754158--foo.nc +Could you help me to figure it out? +Thank you. + +The erro information when using git annex in Windows, is as below: + +$ git annex get tau.nc + + Detected a crippled filesystem. + + Enabling direct mode. + + Detected a filesystem without fifo support. + + Disabling ssh connection caching. + +*** Please tell me who you are. + + + +Run + + git config --global user.email \"you@example.com\" + + git config --user.name \"Your Name\" + +to set your account's default identity. + +Omit --global to set the identity only in this repository. + + + +fatal: unable to auto-detect email address (got 'xshao@DELL-PC.(none') + +git-annex :tau.nc not found + +(Recording state in git ...) +"""]] From 72ac459c2c6d0089bfd0f8eb778a9668d5577949 Mon Sep 17 00:00:00 2001 From: "https://me.yahoo.com/a/FHnTlSBo1eCGJRwueeKeB6.RCaPbGMPr5jxx8A--#ce0d8" Date: Tue, 10 Sep 2013 14:22:49 +0000 Subject: [PATCH 18/62] --- doc/forum/Annex_dropping_files.mdwn | 12 ++++++++++++ 1 file changed, 12 insertions(+) create mode 100644 doc/forum/Annex_dropping_files.mdwn diff --git a/doc/forum/Annex_dropping_files.mdwn b/doc/forum/Annex_dropping_files.mdwn new file mode 100644 index 0000000000..7ec19caa15 --- /dev/null +++ b/doc/forum/Annex_dropping_files.mdwn @@ -0,0 +1,12 @@ +It seems like one of my repos dropped files by itself, my setup involved 3 distributed repos (non bare repos, not using assistant), A/B/C then one client died I've created another one in its place called D cloned from B and marked C as dead. I have now A/B/D. I have updated files in A synced it to D (central) I then tried dropping unused, it listed none (I've dropped the files from A which has full copy) at this point I thought B did not sync to it maybe thats why it did not drop them. I went home sync my laptop B (B dropped its share of unused files) but cental D still show no unused. + +A-D has full copies of the repo +B has partial. + +At this point I checked out a old commit from a week ago, checked where the links for the files are pointing to, they point to no where files are not there. Repo sized reported with du -hcs show both repos are the same size (132GB) looks like D managed to drop unused files by itself. I am wondering what caused this? + +My annex version is, + +git-annex version: 4.20130902-g307537a + +on all clients. From 0481a014adaea97c4f0d85a9d450e903615ef87a Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawnlotDRSLW2JVXY3SLSwhrcHteqUHhTtoY" Date: Tue, 10 Sep 2013 14:23:09 +0000 Subject: [PATCH 19/62] --- doc/forum/manual_update_of_.git__47__annex__47__objects.txt | 1 + 1 file changed, 1 insertion(+) create mode 100644 doc/forum/manual_update_of_.git__47__annex__47__objects.txt diff --git a/doc/forum/manual_update_of_.git__47__annex__47__objects.txt b/doc/forum/manual_update_of_.git__47__annex__47__objects.txt new file mode 100644 index 0000000000..5d67516a69 --- /dev/null +++ b/doc/forum/manual_update_of_.git__47__annex__47__objects.txt @@ -0,0 +1 @@ +Let's suppose that I've manually modified files in .git/annex/objects, for example I ran an rsync or some other file synchronization software to copy files. As a result, some objects have disappeared, others have appeared. After that `git annex whereis .' displays stale information, it doesn't take the manual modifications to accound. `git annex fsck' seems to fix this, but it runs the rehashing of all new files, so it's slow. Is there a fast alternative, which notices all the file changes, trusts them, and just updates .git/annex/index? From afc40e012156d2ce09696d13a720cde81af237fd Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawnlotDRSLW2JVXY3SLSwhrcHteqUHhTtoY" Date: Tue, 10 Sep 2013 14:24:18 +0000 Subject: [PATCH 20/62] --- .../manual_update_of_.git__47__annex__47__objects.txt | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/doc/forum/manual_update_of_.git__47__annex__47__objects.txt b/doc/forum/manual_update_of_.git__47__annex__47__objects.txt index 5d67516a69..9de5779c12 100644 --- a/doc/forum/manual_update_of_.git__47__annex__47__objects.txt +++ b/doc/forum/manual_update_of_.git__47__annex__47__objects.txt @@ -1 +1,8 @@ -Let's suppose that I've manually modified files in .git/annex/objects, for example I ran an rsync or some other file synchronization software to copy files. As a result, some objects have disappeared, others have appeared. After that `git annex whereis .' displays stale information, it doesn't take the manual modifications to accound. `git annex fsck' seems to fix this, but it runs the rehashing of all new files, so it's slow. Is there a fast alternative, which notices all the file changes, trusts them, and just updates .git/annex/index? +Let's suppose that I've manually modified files in .git/annex/objects, +for example I ran an rsync or some other file synchronization software +to copy files. As a result, some objects have disappeared, others have +appeared. After that `git annex whereis .' displays stale information, +it doesn't take the manual modifications to accound. `git annex fsck' +seems to fix this, but it runs the rehashing of all new files, so it's +slow. Is there a fast alternative, which notices all the object file +changes, trusts them, and just updates .git/annex/index quickly? From d7f7bff71710f0b4893e27f5ab8c673d57ab24e0 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawkQrsXK-icnYXg6kJNd-jDjMgOixnhwE34" Date: Tue, 10 Sep 2013 15:36:57 +0000 Subject: [PATCH 21/62] Added a comment: Comment3 --- ...omment_3_70c22716fde60d14fd0c7e74acf4a224._comment | 11 +++++++++++ 1 file changed, 11 insertions(+) create mode 100644 doc/forum/Git_annex_on_Windows/comment_3_70c22716fde60d14fd0c7e74acf4a224._comment diff --git a/doc/forum/Git_annex_on_Windows/comment_3_70c22716fde60d14fd0c7e74acf4a224._comment b/doc/forum/Git_annex_on_Windows/comment_3_70c22716fde60d14fd0c7e74acf4a224._comment new file mode 100644 index 0000000000..772d9348a0 --- /dev/null +++ b/doc/forum/Git_annex_on_Windows/comment_3_70c22716fde60d14fd0c7e74acf4a224._comment @@ -0,0 +1,11 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawkQrsXK-icnYXg6kJNd-jDjMgOixnhwE34" + nickname="x" + subject="Comment3" + date="2013-09-10T15:36:57Z" + content=""" +In addition, because the file is actually symlink direct to the data stored in Amazon S3 +How could I get the URL, given the symlink for git annex. For example: +foo.nc -> ../../.git/annex/objects/jK/v7/WORM-s16009120-m1337754158--foo.nc/WORM-s16009120-m1337754158--foo.nc +Thank you. +"""]] From 61b528c49fbafad9dc4ac8133007fd5d1f4bfbae Mon Sep 17 00:00:00 2001 From: "https://id.koumbit.net/anarcat" Date: Tue, 10 Sep 2013 18:11:46 +0000 Subject: [PATCH 22/62] this may seem trivial, but i went freaking crazy with this --- ...disconnected_directories_to_git_annex.mdwn | 53 +++++++++++++++++++ 1 file changed, 53 insertions(+) create mode 100644 doc/tips/migrating_two_seperate_disconnected_directories_to_git_annex.mdwn diff --git a/doc/tips/migrating_two_seperate_disconnected_directories_to_git_annex.mdwn b/doc/tips/migrating_two_seperate_disconnected_directories_to_git_annex.mdwn new file mode 100644 index 0000000000..3d7e9dac45 --- /dev/null +++ b/doc/tips/migrating_two_seperate_disconnected_directories_to_git_annex.mdwn @@ -0,0 +1,53 @@ +Scenario +-------- + +You are a new git-annex user. You have already files spread around many computers and wish to migrate those into git-annex, without having to recopy all files all over the place. + +Let's say, for example, you have a server, named `marcos` and a workstation named `angela`. You have your audio collection stored in `/srv/mp3` in `marcos` and `~/mp3` on `angela`, but only `marcos` has all the files, and `angela` only has a subset. + +We also assume that `marcos` has an SSH server. + +How do you add all this stuff to git-annex? + +Create the biggest git-annex repository +--------------------------------------- + +Start with `marcos`, with the complete directory: + + cd /srv/mp3 + git init + git annex init + git annex add + +You may want to use [[direct mode]] if you want to avoid creating a forest of symlinks there, but this is generally error-prone and should be avoided. + +This will checksum all files and add them to the `git-annex` branch of the git repository. Wait for this process to complete. + +Create the smaller repo and synchronise +--------------------------------------- + +On `angela`, we want to synchronise the git annex metadata with `marcos`. We need to initialize a git repo with `marcos` as a remote: + + cd ~/mp3 + git init + git remote add marcos marcos.example.com:/srv/mp3 + git fetch marcos + git annex status # this should display the two repos + git annex add . + +This will, again, checksum all files and add them to git annex. Once that is done, you can verify that the files are really the same as marcos with `whereis`: + + git annex whereis + +This should display something like: + + whereis Orange Seeds/I remember.wav (2 copies) + b7802161-c984-4c9f-8d05-787a29c41cfe -- marcos (anarcat@marcos:/srv/mp3) + c2ca4a13-9a5f-461b-a44b-53255ed3e2f9 -- here (anarcat@angela) + ok + +Once you are sure things went on okay, you can synchronise this with `marcos`: + + git annex sync + +This will push the metadata information to marcos, so it knows which files are available on `angela`. From there on, you can freely get and move files between the two repos! From b74625944395834677e0c0bec8354450e3962f54 Mon Sep 17 00:00:00 2001 From: "https://id.koumbit.net/anarcat" Date: Tue, 10 Sep 2013 18:13:57 +0000 Subject: [PATCH 23/62] mention import --- ...eperate_disconnected_directories_to_git_annex.mdwn | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/doc/tips/migrating_two_seperate_disconnected_directories_to_git_annex.mdwn b/doc/tips/migrating_two_seperate_disconnected_directories_to_git_annex.mdwn index 3d7e9dac45..fd8eb6151f 100644 --- a/doc/tips/migrating_two_seperate_disconnected_directories_to_git_annex.mdwn +++ b/doc/tips/migrating_two_seperate_disconnected_directories_to_git_annex.mdwn @@ -17,7 +17,8 @@ Start with `marcos`, with the complete directory: cd /srv/mp3 git init git annex init - git annex add + git annex add . + git commit -m"git annex yay" You may want to use [[direct mode]] if you want to avoid creating a forest of symlinks there, but this is generally error-prone and should be avoided. @@ -51,3 +52,11 @@ Once you are sure things went on okay, you can synchronise this with `marcos`: git annex sync This will push the metadata information to marcos, so it knows which files are available on `angela`. From there on, you can freely get and move files between the two repos! + +Importing files from a third directory +-------------------------------------- + +Say that some files on `angela` are actually spread out outside of the `~/mp3` directory. You can use the `git annex import` command to add those extra directories: + + cd ~/mp3 + git annex import ~/music/ From 8a374a537f69c1cdf325ca8d89cdd8d5456ef0df Mon Sep 17 00:00:00 2001 From: "https://id.koumbit.net/anarcat" Date: Tue, 10 Sep 2013 18:35:15 +0000 Subject: [PATCH 24/62] how to deal with renamed files between the two repos --- ...eperate_disconnected_directories_to_git_annex.mdwn | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/doc/tips/migrating_two_seperate_disconnected_directories_to_git_annex.mdwn b/doc/tips/migrating_two_seperate_disconnected_directories_to_git_annex.mdwn index fd8eb6151f..9fde576b33 100644 --- a/doc/tips/migrating_two_seperate_disconnected_directories_to_git_annex.mdwn +++ b/doc/tips/migrating_two_seperate_disconnected_directories_to_git_annex.mdwn @@ -60,3 +60,14 @@ Say that some files on `angela` are actually spread out outside of the `~/mp3` d cd ~/mp3 git annex import ~/music/ + +Deleting deleted files +---------------------- + +It is quite possible some files were removed (or renamed!) on `marcos` but not on `angela`, since it was synchronised only some time ago. A good way to find out about those files is to use the `--not --in` argument, for example, on `angela`: + + git annex whereis --in here --not --in marcos + +This will show files that are on `angela` and not on `marcos`. They could be new files that were only added on `angela`, so be careful! A manual analysis is necessary, but let's say you are certain those files are not relevant anymore, you can delete them from `angela`: + + git annex drop From e172f87fd88e88ba9f994d35e370d370b77cc13d Mon Sep 17 00:00:00 2001 From: "https://id.koumbit.net/anarcat" Date: Tue, 10 Sep 2013 19:39:58 +0000 Subject: [PATCH 25/62] --- ...ting_two_seperate_disconnected_directories_to_git_annex.mdwn | 2 -- 1 file changed, 2 deletions(-) diff --git a/doc/tips/migrating_two_seperate_disconnected_directories_to_git_annex.mdwn b/doc/tips/migrating_two_seperate_disconnected_directories_to_git_annex.mdwn index 9fde576b33..b303c617e8 100644 --- a/doc/tips/migrating_two_seperate_disconnected_directories_to_git_annex.mdwn +++ b/doc/tips/migrating_two_seperate_disconnected_directories_to_git_annex.mdwn @@ -20,8 +20,6 @@ Start with `marcos`, with the complete directory: git annex add . git commit -m"git annex yay" -You may want to use [[direct mode]] if you want to avoid creating a forest of symlinks there, but this is generally error-prone and should be avoided. - This will checksum all files and add them to the `git-annex` branch of the git repository. Wait for this process to complete. Create the smaller repo and synchronise From 636563fd1b46722f4835e2377de7fbd9b5e976e9 Mon Sep 17 00:00:00 2001 From: "https://id.koumbit.net/anarcat" Date: Tue, 10 Sep 2013 19:51:00 +0000 Subject: [PATCH 26/62] mention --force --- ...ting_two_seperate_disconnected_directories_to_git_annex.mdwn | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/tips/migrating_two_seperate_disconnected_directories_to_git_annex.mdwn b/doc/tips/migrating_two_seperate_disconnected_directories_to_git_annex.mdwn index b303c617e8..9b71b4cf1f 100644 --- a/doc/tips/migrating_two_seperate_disconnected_directories_to_git_annex.mdwn +++ b/doc/tips/migrating_two_seperate_disconnected_directories_to_git_annex.mdwn @@ -69,3 +69,5 @@ It is quite possible some files were removed (or renamed!) on `marcos` but not o This will show files that are on `angela` and not on `marcos`. They could be new files that were only added on `angela`, so be careful! A manual analysis is necessary, but let's say you are certain those files are not relevant anymore, you can delete them from `angela`: git annex drop + +If the file is a renamed or modified version from the original, you may need to use `--force`, but be careful! If you delete the wrong file, it will be lost forever! From 27b6b50c70bc4d024d2ed76db5f7f6530c75742e Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawlZQcpoRmyZBlsFQ7TMMYlZZdmBG1TxMzA" Date: Tue, 10 Sep 2013 20:19:54 +0000 Subject: [PATCH 27/62] Added a comment: translation --- .../comment_1_12d60f767d90bea94974e1ff6b206d31._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/contact/comment_1_12d60f767d90bea94974e1ff6b206d31._comment diff --git a/doc/contact/comment_1_12d60f767d90bea94974e1ff6b206d31._comment b/doc/contact/comment_1_12d60f767d90bea94974e1ff6b206d31._comment new file mode 100644 index 0000000000..f72f3fbe52 --- /dev/null +++ b/doc/contact/comment_1_12d60f767d90bea94974e1ff6b206d31._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawlZQcpoRmyZBlsFQ7TMMYlZZdmBG1TxMzA" + nickname="Felipe" + subject="translation" + date="2013-09-10T20:19:53Z" + content=""" +I would like to translate this content of site, the tutorials, pages, etc, to portuguese. I think that can help many users in my country. How can I build a new pages here with git-annex content in portuguese? +"""]] From a1db6fb58175ae1f8fe5435220b182f7b9517cc0 Mon Sep 17 00:00:00 2001 From: "https://id.koumbit.net/anarcat" Date: Tue, 10 Sep 2013 21:39:15 +0000 Subject: [PATCH 28/62] make a real link --- doc/forum/Wishlist:_Bittorrent-like_transfers.mdwn | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/forum/Wishlist:_Bittorrent-like_transfers.mdwn b/doc/forum/Wishlist:_Bittorrent-like_transfers.mdwn index 8352a4edad..7ec6632169 100644 --- a/doc/forum/Wishlist:_Bittorrent-like_transfers.mdwn +++ b/doc/forum/Wishlist:_Bittorrent-like_transfers.mdwn @@ -1,4 +1,4 @@ -**EDIT: Mistakenly posted this thread in the forum. I created a new post in todo. Link: http://git-annex.branchable.com/todo/Bittorrent-like_features/?updated** +**EDIT: Mistakenly posted this thread in the forum. I created a new post in [[todo|todo/Bittorrent-like_features]] Do you think it would be possible to have bittorrent-like transfers between remotes, so that no one remote gets pegged too hard with transfers? It would be great if you distribute your files between multiple bandwidth-capped remotes, and want fast down speed. Obviously, this isn't a simple task, but the protocol is already there, it just needs to be adapted for the purpose (and re-written in Haskell...). Maybe some day in the future after the more important stuff gets taken care of? It could be an enticing stretch goal. From 0536b69e15d56d01a1deb5e0db8688de6d3e5fc9 Mon Sep 17 00:00:00 2001 From: "https://id.koumbit.net/anarcat" Date: Tue, 10 Sep 2013 21:40:30 +0000 Subject: [PATCH 29/62] fix link --- doc/todo/Bittorrent-like_features.mdwn | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/doc/todo/Bittorrent-like_features.mdwn b/doc/todo/Bittorrent-like_features.mdwn index 4ec991a65b..246b90f5e0 100644 --- a/doc/todo/Bittorrent-like_features.mdwn +++ b/doc/todo/Bittorrent-like_features.mdwn @@ -1,7 +1,4 @@ -I made an oops and created a wishlist thread in the forum regarding bittorrent-like behaviour. Sorry, my bad. - -Here's the original thread: -http://git-annex.branchable.com/forum/Wishlist:_Bittorrent-like_transfers/ +I made an oops and created a wishlist thread in the forum regarding bittorrent-like behaviour. Sorry, my bad. Here's the [[original thread|forum/Wishlist:_Bittorrent-like_transfers]]. I think I summed up pretty well what bittorrent-like features could be added to git-annex in one of the posts, so I'll copy and paste some of it here (with slight clarifications added in). From b048e13a8c8c9c275908b4d8de67c98f8b9df912 Mon Sep 17 00:00:00 2001 From: "https://id.koumbit.net/anarcat" Date: Tue, 10 Sep 2013 21:45:11 +0000 Subject: [PATCH 30/62] remove self-recriminations and keep the good stuff, thanks for this great idea! :) --- doc/todo/Bittorrent-like_features.mdwn | 40 ++++++++++++-------------- 1 file changed, 18 insertions(+), 22 deletions(-) diff --git a/doc/todo/Bittorrent-like_features.mdwn b/doc/todo/Bittorrent-like_features.mdwn index 246b90f5e0..511656d3b2 100644 --- a/doc/todo/Bittorrent-like_features.mdwn +++ b/doc/todo/Bittorrent-like_features.mdwn @@ -1,28 +1,24 @@ -I made an oops and created a wishlist thread in the forum regarding bittorrent-like behaviour. Sorry, my bad. Here's the [[original thread|forum/Wishlist:_Bittorrent-like_transfers]]. +The assistant as a bittorrent client +==================================== -I think I summed up pretty well what bittorrent-like features could be added to git-annex in one of the posts, so I'll copy and paste some of it here (with slight clarifications added in). +Having your remotes (optionally!) act like a swarm would be an awesome feature to have because you bring in a lot of new features that optimize storage, bandwidth, and overall traffic usage. This would be made a lot easier if parts of it were implemented in small steps that added a nifty feature. The best part is, each of these could be implemented by themselves, and they're all features that would be really useful. ->Disclaimer: I'm thinking out loud of what could make git-annex even more awesome. I don't expect this to be implemented any time soon. Please pardon any dumbassery. + 1. Concurrent downloads of a file from remotes. ->Having your remotes (optionally!) act like a swarm would be an awesome feature to have because you bring in a lot of new features that optimize storage, bandwidth, and overall traffic usage. This would be made a lot easier if parts of it were implemented in small steps that added a nifty feature. The best part is, each of these could be implemented by themselves, and they're all features that would be really useful. -> ->Step 1. Concurrent downloads of a file from remotes. -> ->This would make sense to have, it saves upload traffic on your remotes, and you also get faster DL speeds on the receiving end. -> ->Step 2. Implementing part of the super-seeding capabilities. -> ->You upload pieces of a file to different remotes from your laptop, and on your desktop you can download all those pieces and put them together again to get a complete file. If you really wanted to get fancy, you could build in redundancy (ala RAID) so if a remote or two gets lost, you don't lose the entire file. This would be a very efficient use of storage if you have a bunch of free cloud storage accounts (~1GB each) and some big files you want to back up. -> ->Step 3. Setting it up so that those remotes could talk to one another and share those pieces. -> ->This is where it gets more like bittorrent. Useful because you upload 1 copy and in a few hours, have say, 5 complete copies on 5 different remotes. You could add or remove remotes from a swarm locally, and push those changes to those remotes, which then adapt themselves to suit the new rules and share those with other remotes in the swarm (rules should be GPG-signed as a safety precaution). Also, if/when deltas get implemented, you could push that delta to the swarm and have all the remotes adopt it. This is cooler than regular bittorrent because the shared file can be updated. As a safety precaution, the delta could be GPG signed so a corrupt file doesn't contaminate the entire swarm. Each remote could have bandwidth/storage limits set in a dotfile. -> ->This is a high-level idea of how it might work, and it's also a HUGE set of features to add, but if implemented, you'd be saving a ton of resources, adding new use cases, and making git-annex more flexible. + This would make sense to have, it saves upload traffic on your remotes, and you also get faster DL speeds on the receiving end. -And this: + 2. Implementing part of the super-seeding capabilities. ->Obviously, Step 3 would only work on remotes that you have control of processes on, but if given login credentials to cloud storage remotes (potentially dangerous!) they could read/write to something like dropbox or rsync. -> ->Another thing, this would be completely trackerless. You just use remote groups (or create swarm definitions) and share those with your remotes. **It's completely decentralized!** + You upload pieces of a file to different remotes from your laptop, and on your desktop you can download all those pieces and put them together again to get a complete file. If you really wanted to get fancy, you could build in redundancy (ala RAID) so if a remote or two gets lost, you don't lose the entire file. This would be a very efficient use of storage if you have a bunch of free cloud storage accounts (~1GB each) and some big files you want to back up. + 3. Setting it up so that those remotes could talk to one another and share those pieces. + + This is where it gets more like bittorrent. Useful because you upload 1 copy and in a few hours, have say, 5 complete copies on 5 different remotes. You could add or remove remotes from a swarm locally, and push those changes to those remotes, which then adapt themselves to suit the new rules and share those with other remotes in the swarm (rules should be GPG-signed as a safety precaution). Also, if/when deltas get implemented, you could push that delta to the swarm and have all the remotes adopt it. This is cooler than regular bittorrent because the shared file can be updated. As a safety precaution, the delta could be GPG signed so a corrupt file doesn't contaminate the entire swarm. Each remote could have bandwidth/storage limits set in a dotfile. + +This is a high-level idea of how it might work, and it's also a HUGE set of features to add, but if implemented, you'd be saving a ton of resources, adding new use cases, and making git-annex more flexible. + +Obviously, Step 3 would only work on remotes that you have control of processes on, but if given login credentials to cloud storage remotes (potentially dangerous!) they could read/write to something like dropbox or rsync. + +Another thing, this would be completely trackerless. You just use remote groups (or create swarm definitions) and share those with your remotes. **It's completely decentralized!** + +This was originally posted [[as a forum opst|forum/Wishlist:_Bittorrent-like_transfers]]. From 2915124b122086ffebd5dd3738f45e7355a1d4a4 Mon Sep 17 00:00:00 2001 From: "https://id.koumbit.net/anarcat" Date: Tue, 10 Sep 2013 21:52:23 +0000 Subject: [PATCH 31/62] add another idea: use an external bittorrent client --- doc/todo/Bittorrent-like_features.mdwn | 24 +++++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/doc/todo/Bittorrent-like_features.mdwn b/doc/todo/Bittorrent-like_features.mdwn index 511656d3b2..dbf67b0be9 100644 --- a/doc/todo/Bittorrent-like_features.mdwn +++ b/doc/todo/Bittorrent-like_features.mdwn @@ -1,3 +1,10 @@ +There are two different possible ways git-annex could use bittorrent: + + 1. by downloading any file from multiple git-annex daemons, using the bittorrent protocol + 2. by downloading specific files using an external bittorrent client + +Let's describe those one by one. + The assistant as a bittorrent client ==================================== @@ -21,4 +28,19 @@ Obviously, Step 3 would only work on remotes that you have control of processes Another thing, this would be completely trackerless. You just use remote groups (or create swarm definitions) and share those with your remotes. **It's completely decentralized!** -This was originally posted [[as a forum opst|forum/Wishlist:_Bittorrent-like_transfers]]. +This was originally posted [[as a forum post|forum/Wishlist:_Bittorrent-like_transfers]] by [[users/GLITTAH]]. + +Using an external bittorrent client +=================================== + +The alternative to this would be to add `addurl` support for bittorrent files. The same way we can now add Youtube videos to a git-annex repository thanks to [[quvi]], we could also simply do: + + git annex addtorrent debian-live-7.0.0-amd64-standard.iso.torrent + +or even better: + + git annex addurl http://cdimage.debian.org/debian-cd/current-live/amd64/bt-hybrid/debian-live-7.0.0-amd64-standard.iso.torrent + +This way, a torrent would just become another source for a specific file. When we `get` the file, it fires up `$YOUR_FAVORITE_TORRENT_CLIENT` to download the file. + +That way we avoid the implementation complexity of shoving a complete bittorrent client within the assistant. The `get` operation would block until the torrent is downloaded, i guess... From 3c79cf2d1dffbfe752ba8caa6423a8c9bc30a053 Mon Sep 17 00:00:00 2001 From: "https://id.koumbit.net/anarcat" Date: Tue, 10 Sep 2013 21:54:52 +0000 Subject: [PATCH 32/62] clarify, simplify with toc --- doc/todo/Bittorrent-like_features.mdwn | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/doc/todo/Bittorrent-like_features.mdwn b/doc/todo/Bittorrent-like_features.mdwn index dbf67b0be9..d1645b6f8a 100644 --- a/doc/todo/Bittorrent-like_features.mdwn +++ b/doc/todo/Bittorrent-like_features.mdwn @@ -1,12 +1,11 @@ There are two different possible ways git-annex could use bittorrent: - 1. by downloading any file from multiple git-annex daemons, using the bittorrent protocol - 2. by downloading specific files using an external bittorrent client - Let's describe those one by one. -The assistant as a bittorrent client -==================================== +[[!toc]] + +Downloading files from multiple git-annex sources simultaneously +================================================================ Having your remotes (optionally!) act like a swarm would be an awesome feature to have because you bring in a lot of new features that optimize storage, bandwidth, and overall traffic usage. This would be made a lot easier if parts of it were implemented in small steps that added a nifty feature. The best part is, each of these could be implemented by themselves, and they're all features that would be really useful. @@ -30,8 +29,8 @@ Another thing, this would be completely trackerless. You just use remote groups This was originally posted [[as a forum post|forum/Wishlist:_Bittorrent-like_transfers]] by [[users/GLITTAH]]. -Using an external bittorrent client -=================================== +Using an external client (addurl torrent support) +================================================= The alternative to this would be to add `addurl` support for bittorrent files. The same way we can now add Youtube videos to a git-annex repository thanks to [[quvi]], we could also simply do: From 692794bc6aef6f561725fb6315277fa37ffa192b Mon Sep 17 00:00:00 2001 From: "https://id.koumbit.net/anarcat" Date: Tue, 10 Sep 2013 22:17:08 +0000 Subject: [PATCH 33/62] idea about direct mode --- ...ting_two_seperate_disconnected_directories_to_git_annex.mdwn | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/tips/migrating_two_seperate_disconnected_directories_to_git_annex.mdwn b/doc/tips/migrating_two_seperate_disconnected_directories_to_git_annex.mdwn index 9b71b4cf1f..42761b5b93 100644 --- a/doc/tips/migrating_two_seperate_disconnected_directories_to_git_annex.mdwn +++ b/doc/tips/migrating_two_seperate_disconnected_directories_to_git_annex.mdwn @@ -71,3 +71,5 @@ This will show files that are on `angela` and not on `marcos`. They could be new git annex drop If the file is a renamed or modified version from the original, you may need to use `--force`, but be careful! If you delete the wrong file, it will be lost forever! + +> (!) Maybe this wouldn't happen with [[direct mode]] and an fsck? --[[anarcat]] From 3476e9a46f7ed2a194a353df934662dd0d1d68bc Mon Sep 17 00:00:00 2001 From: "https://me.yahoo.com/a/2grhJvAC049fJnvALDXek.6MRZMTlg--#eec89" Date: Tue, 10 Sep 2013 23:11:28 +0000 Subject: [PATCH 34/62] Added a comment --- .../comment_2_d3e38d6f6bba179dab40d4d75ff061de._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/devblog/day_7__release_day/comment_2_d3e38d6f6bba179dab40d4d75ff061de._comment diff --git a/doc/devblog/day_7__release_day/comment_2_d3e38d6f6bba179dab40d4d75ff061de._comment b/doc/devblog/day_7__release_day/comment_2_d3e38d6f6bba179dab40d4d75ff061de._comment new file mode 100644 index 0000000000..96fb5dce30 --- /dev/null +++ b/doc/devblog/day_7__release_day/comment_2_d3e38d6f6bba179dab40d4d75ff061de._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="https://me.yahoo.com/a/2grhJvAC049fJnvALDXek.6MRZMTlg--#eec89" + nickname="John" + subject="comment 2" + date="2013-09-10T23:11:27Z" + content=""" +Everything is all setup now to build git-annex on both systems, at most once per hour (if there have been new Git commits), and only between 2am and 4pm. Just contact me on IRC or e-mail and I'll setup you up with credentials so that you can manipulate the jobs and access the build artifacts. +"""]] From 8d6c36a35ec147d17aaece1627c35f45650eb449 Mon Sep 17 00:00:00 2001 From: "https://id.koumbit.net/anarcat" Date: Tue, 10 Sep 2013 23:13:59 +0000 Subject: [PATCH 35/62] some classic shared repo configuration that is way too relevant to leave out of git-annex's documentation --- ...nnex_directory_between_multiple_users.mdwn | 32 +++++++++++++++++++ 1 file changed, 32 insertions(+) create mode 100644 doc/tips/shared_git_annex_directory_between_multiple_users.mdwn diff --git a/doc/tips/shared_git_annex_directory_between_multiple_users.mdwn b/doc/tips/shared_git_annex_directory_between_multiple_users.mdwn new file mode 100644 index 0000000000..5b240839eb --- /dev/null +++ b/doc/tips/shared_git_annex_directory_between_multiple_users.mdwn @@ -0,0 +1,32 @@ +Scenario +======== + +You have a server where you want to welcome other people to push files, say for a family photo album. People have their own user account, so by default they will not be able to read/write from each other's repositories, due to git-annex strict restrictions. + +Solution +======== + +Setup a shared git repository: + + git init shared ; cd shared # you can also do this on an existing git annex repo + git config core.sharedrepository group + chmod g+rwX -R . + chown -R :media . + +The idea here is to use the new (since [[news/version 4.20130909]]) support for git's `sharedRepository` configuration and restrict access to a specific group (instead of the default, a single user). You can also this to make the files accessible to all users on the system: + + git config core.sharedrepository world + chmod a+rwX -R . + +This will make sure that you anyone can operate that git annex repository remotely. + +Third party applications +------------------------ + +Now if another application that is not aware of git's `sharedRepository` configuration (say a [[bittorrent]] daemon) writes files there, you may want to make sure that the files created are also writable by everyone. This is more tricky, but one way of doing this is with the [[!wikipedia setgid]] bit: + + find -type d -exec chmod g+s {} \; + +You will also need to start the process with a proper [[!wikipedia umask]] (`002` instead of `022`). + +See also [[tips/setup a public repository on a web site]]. From be3cc3897108777cc31a8d4ce6e92784f7c32f07 Mon Sep 17 00:00:00 2001 From: "https://id.koumbit.net/anarcat" Date: Tue, 10 Sep 2013 23:14:58 +0000 Subject: [PATCH 36/62] --- .../shared_git_annex_directory_between_multiple_users.mdwn | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/doc/tips/shared_git_annex_directory_between_multiple_users.mdwn b/doc/tips/shared_git_annex_directory_between_multiple_users.mdwn index 5b240839eb..972539632a 100644 --- a/doc/tips/shared_git_annex_directory_between_multiple_users.mdwn +++ b/doc/tips/shared_git_annex_directory_between_multiple_users.mdwn @@ -29,4 +29,8 @@ Now if another application that is not aware of git's `sharedRepository` configu You will also need to start the process with a proper [[!wikipedia umask]] (`002` instead of `022`). -See also [[tips/setup a public repository on a web site]]. +See also +======== + + * [[tips/setup a public repository on a web site]] + * [[news/version 4.20130909]] From 1c0cb4f7db7606f251c8207a9ccdd0e26d0717f1 Mon Sep 17 00:00:00 2001 From: "https://id.koumbit.net/anarcat" Date: Tue, 10 Sep 2013 23:31:55 +0000 Subject: [PATCH 37/62] where is my documentation! :) --- .../cabal_install_fails_to_install_manpage.mdwn | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) create mode 100644 doc/bugs/cabal_install_fails_to_install_manpage.mdwn diff --git a/doc/bugs/cabal_install_fails_to_install_manpage.mdwn b/doc/bugs/cabal_install_fails_to_install_manpage.mdwn new file mode 100644 index 0000000000..c4a8c0325a --- /dev/null +++ b/doc/bugs/cabal_install_fails_to_install_manpage.mdwn @@ -0,0 +1,17 @@ +### Please describe the problem. + +The git-annex manual page is not installed when using [[install/cabal]] to install git-annex. + +### What steps will reproduce the problem? + +Follow [[install/cabal]] with `--bin-dir=/usr/local` and try `man git-annex`. + +I was expecting the manpage to be installed in `/usr/local/share/man`. + +### What version of git-annex are you using? On what operating system? + +git annex from a recent git build (4.20130909-g7f079d6) in Debian wheezy. + +### Please provide any additional information below. + +This could be just a documentation problem... --[[anarcat]] From 5766a2194eac5fc3e3a2eb6353a16ca6aa4b9829 Mon Sep 17 00:00:00 2001 From: "https://id.koumbit.net/anarcat" Date: Tue, 10 Sep 2013 23:36:08 +0000 Subject: [PATCH 38/62] index --- doc/users/anarcat.mdwn | 38 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/doc/users/anarcat.mdwn b/doc/users/anarcat.mdwn index 2ca221132b..e8a1d7f002 100644 --- a/doc/users/anarcat.mdwn +++ b/doc/users/anarcat.mdwn @@ -1,3 +1,41 @@ Miaou. See . + +[[!toc]] + +My tips +======= + +... or the ones I commented it, to be more precise. + +[[!inline pages="tips/* and and link(users/anarcat) and !bugs/*/*" sort=mtime feeds=no actions=yes archive=yes show=0]] + + +My todos +======== + +... same. + +[[!inline pages="todo/* and !todo/done and !link(todo/done) and +link(users/anarcat) and !todo/*/*" sort=mtime feeds=no actions=yes archive=yes show=0]] + +Done +---- + +[[!inline pages="todo/* and !todo/done and link(todo/done) and +link(users/anarcat) and !todo/*/*" feeds=no actions=yes archive=yes show=0]] + +My bugs +======= + +... same. + +[[!inline pages="bugs/* and !bugs/done and !link(bugs/done) and +link(users/anarcat) and !bugs/*/*" sort=mtime feeds=no actions=yes archive=yes show=0]] + +Fixed +----- + +[[!inline pages="bugs/* and !bugs/done and link(bugs/done) and +link(users/anarcat) and !bugs/*/*" feeds=no actions=yes archive=yes show=0]] From 84c206eabf39dec1d973de64d1df53fdae0e8655 Mon Sep 17 00:00:00 2001 From: "https://id.koumbit.net/anarcat" Date: Tue, 10 Sep 2013 23:37:25 +0000 Subject: [PATCH 39/62] sign --- doc/todo/Bittorrent-like_features.mdwn | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/todo/Bittorrent-like_features.mdwn b/doc/todo/Bittorrent-like_features.mdwn index d1645b6f8a..41988a422f 100644 --- a/doc/todo/Bittorrent-like_features.mdwn +++ b/doc/todo/Bittorrent-like_features.mdwn @@ -42,4 +42,4 @@ or even better: This way, a torrent would just become another source for a specific file. When we `get` the file, it fires up `$YOUR_FAVORITE_TORRENT_CLIENT` to download the file. -That way we avoid the implementation complexity of shoving a complete bittorrent client within the assistant. The `get` operation would block until the torrent is downloaded, i guess... +That way we avoid the implementation complexity of shoving a complete bittorrent client within the assistant. The `get` operation would block until the torrent is downloaded, i guess... --[[anarcat]] From 0a1dc3482fc3a7fb06159dbef19022ccb9c5b189 Mon Sep 17 00:00:00 2001 From: "https://id.koumbit.net/anarcat" Date: Tue, 10 Sep 2013 23:38:16 +0000 Subject: [PATCH 40/62] sign --- doc/tips/shared_git_annex_directory_between_multiple_users.mdwn | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/tips/shared_git_annex_directory_between_multiple_users.mdwn b/doc/tips/shared_git_annex_directory_between_multiple_users.mdwn index 972539632a..61f02ab3c1 100644 --- a/doc/tips/shared_git_annex_directory_between_multiple_users.mdwn +++ b/doc/tips/shared_git_annex_directory_between_multiple_users.mdwn @@ -29,6 +29,8 @@ Now if another application that is not aware of git's `sharedRepository` configu You will also need to start the process with a proper [[!wikipedia umask]] (`002` instead of `022`). +(!) I haven't actually tested this part. --[[anarcat]] + See also ======== From 8780e090b89572c5917316b262bee8c1903d39b2 Mon Sep 17 00:00:00 2001 From: "https://id.koumbit.net/anarcat" Date: Tue, 10 Sep 2013 23:43:30 +0000 Subject: [PATCH 41/62] sign --- doc/bugs/inconsistent_use_of_SI_prefixes.mdwn | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/bugs/inconsistent_use_of_SI_prefixes.mdwn b/doc/bugs/inconsistent_use_of_SI_prefixes.mdwn index 3168fd36c6..df7fc5fb2a 100644 --- a/doc/bugs/inconsistent_use_of_SI_prefixes.mdwn +++ b/doc/bugs/inconsistent_use_of_SI_prefixes.mdwn @@ -35,7 +35,7 @@ Notice `mebibytes` and `gigabytes`. It is unclear whether those are gigabytes (1 ### Please provide any additional information below. -nil. +nil. --[[anarcat]] > git-annex consistently uses the powers of ten units > for disk storage sizes. Its "gigabyte" is the SI gigabyte. From f2734ee1758da017136071162562d3ee18fab68f Mon Sep 17 00:00:00 2001 From: "https://id.koumbit.net/anarcat" Date: Tue, 10 Sep 2013 23:44:40 +0000 Subject: [PATCH 42/62] sign --- doc/bugs/cannot_determine_uuid_for_origin.mdwn | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/bugs/cannot_determine_uuid_for_origin.mdwn b/doc/bugs/cannot_determine_uuid_for_origin.mdwn index ce46e5733a..611a0f2368 100644 --- a/doc/bugs/cannot_determine_uuid_for_origin.mdwn +++ b/doc/bugs/cannot_determine_uuid_for_origin.mdwn @@ -119,7 +119,7 @@ Update: it seems te problem was that I had the following in my `.git/config`: annex-ignore = true fetch = +refs/heads/*:refs/remotes/marcos-bare/* -I have *no* idea how that `annex-ignore` got there, but that was the root of my problem. Removing it it allowed my to do `git annex copy`. I really don't know how this happened, but I guess this is [[done]], although I believe this error message is really confusing and could be improved. +I have *no* idea how that `annex-ignore` got there, but that was the root of my problem. Removing it it allowed my to do `git annex copy`. I really don't know how this happened, but I guess this is [[done]], although I believe this error message is really confusing and could be improved. --[[anarcat]] > `annex-ignore` is set automatically by git-annex if it fails to query > the uuid of a remote the first time it tries to use it. It will say From 80c6e9ab4921e8f5475a5ac017fe895bb42cb1ab Mon Sep 17 00:00:00 2001 From: "https://id.koumbit.net/anarcat" Date: Wed, 11 Sep 2013 00:06:48 +0000 Subject: [PATCH 43/62] silly tiny bug in the manpage generation --- .../manpage_has_slight_indentation_error.mdwn | 36 +++++++++++++++++++ 1 file changed, 36 insertions(+) create mode 100644 doc/bugs/manpage_has_slight_indentation_error.mdwn diff --git a/doc/bugs/manpage_has_slight_indentation_error.mdwn b/doc/bugs/manpage_has_slight_indentation_error.mdwn new file mode 100644 index 0000000000..fd6f5ea06e --- /dev/null +++ b/doc/bugs/manpage_has_slight_indentation_error.mdwn @@ -0,0 +1,36 @@ +### Please describe the problem. + +The "import" section of the manpage has an indentation error. + +### What steps will reproduce the problem? + +man git-annex + +### What version of git-annex are you using? On what operating system? + +anything after 20130827, i believe. + +### Please provide any additional information below. + +silly patch: + +[[!format diff """ +# If you can, paste a complete transcript of the problem occurring here. +# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log +diff --git a/Build/mdwn2man b/Build/mdwn2man +index ba5919b..b3ee404 100755 +--- a/Build/mdwn2man ++++ b/Build/mdwn2man +@@ -25,7 +25,7 @@ while (<>) { + $inlist=1; + $spippara=0; + } +- elsif (/.SH/) { ++ elsif (/^.SH/) { + $skippara=0; + $inlist=0; + } +# End of transcript or log. +"""]] + +-- [[anarcat]] From 33d092f36ed1741cb4eb7aa514e60cc2197d6bb6 Mon Sep 17 00:00:00 2001 From: "https://id.koumbit.net/anarcat" Date: Wed, 11 Sep 2013 00:10:42 +0000 Subject: [PATCH 44/62] --- doc/bugs/cabal_install_fails_to_install_manpage.mdwn | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/bugs/cabal_install_fails_to_install_manpage.mdwn b/doc/bugs/cabal_install_fails_to_install_manpage.mdwn index c4a8c0325a..b0de72e916 100644 --- a/doc/bugs/cabal_install_fails_to_install_manpage.mdwn +++ b/doc/bugs/cabal_install_fails_to_install_manpage.mdwn @@ -14,4 +14,4 @@ git annex from a recent git build (4.20130909-g7f079d6) in Debian wheezy. ### Please provide any additional information below. -This could be just a documentation problem... --[[anarcat]] +This could be just a documentation problem... Actually, the manpage can be read directly from the source tree in `doc/git-annex.mdwn`, but I still think it should be installed, for [[!wikipedia POLA]]'s sake. --[[anarcat]] From f23c980123c8a3a8ed798bd3c425d00fccb37f55 Mon Sep 17 00:00:00 2001 From: "https://id.koumbit.net/anarcat" Date: Wed, 11 Sep 2013 00:25:29 +0000 Subject: [PATCH 45/62] document some IRC discussions --- doc/todo/nicer_whereis_output.mdwn | 79 ++++++++++++++++++++++++++++++ 1 file changed, 79 insertions(+) create mode 100644 doc/todo/nicer_whereis_output.mdwn diff --git a/doc/todo/nicer_whereis_output.mdwn b/doc/todo/nicer_whereis_output.mdwn new file mode 100644 index 0000000000..22be93bd2e --- /dev/null +++ b/doc/todo/nicer_whereis_output.mdwn @@ -0,0 +1,79 @@ +We had some informal discussions on IRC about improving the output of the `whereis` command. + +[[!toc levels=2]] + +First version: columns +====================== + +[[mastensg]] started by implementing a [simple formatter](https://gist.github.com/mastensg/6500982) that would display things in columns [screenshot](http://www.ping.uio.no/~mastensg/whereis.png) + +Second version: Xs +================== + +After some suggestions from [[joey]], [[mastensg]] changed the format slightly ([screenshot](http://www.ping.uio.no/~mastensg/whereis2.png)): + +[[!format txt """ +17:01:34 foo +17:01:34 |bar +17:01:34 ||baz (untrusted) +17:01:34 ||| +17:01:34 XXx 3? img.png +17:01:36 _X_ 1! bigfile +17:01:37 XX_ 2 zort +17:01:39 __x 1?! maybemissing +17:02:09 * joeyh does a s/\?/+/ in the above +17:02:24 and decrements the counters for untrusted +17:03:37 __x 0+! maybemissing +"""]] + +Third version: incremental +========================== + +Finally, [[anarcat]] worked on making it run faster on large repositories, in a [fork](https://gist.github.com/anarcat/6502988) of that first gist. Then paging was added (so headers are repeated). + +Fourth version: tuning and blocked +================================== + +[[TobiasTheViking]] provided some bugfixes, and the next step was to implement the trusted/untrusted detection, and have a counter. + +This required more advanced parsing of the remotes, and instead of starting to do some JSON parsing, [[anarcat]] figured it was time to learn some Haskell instead. + +Current status: needs work +========================== + +So right now, the most recent version is in [anarcat's gist](https://gist.github.com/anarcat/6502988) and works reasonably well. However, it doesn't distinguish between trusted and untrusted repos and so on. + +Furthermore, we'd like to see this factored into the `whereis` command directly. + +Desired output +-------------- + +The output we're aiming for is: + + foo + |bar + ||baz (untrusted) + ||| + XXx 2+ img.png + _X_ 1! bigfile + XX_ 2 zort + __x 0+! maybemissing + +Legend: + + * `_` - file missing from repo + * `x` - file may be present in untrusted repo + * `X` - file is present in trusted repo + * `[0-9]` - number of copies present in trusted repos + * `+` - indicates there may be more copies present + * `!` - indicates only one copy is left + +Implementation notes +-------------------- + +[[!format txt """ +20:48:18 if someone writes me a headerWhereis :: [(RemoteName, TrustLevel)] -> String and a formatWhereis :: [(RemoteName, TrustLevel, UUID)] -> [UUD] -> FileName -> String , I can do the rest ;) +20:49:22 make that second one formatWhereis :: [(RemoteName, TrueLevel, Bool)] -> FileName -> String +20:49:37 gah, typos +20:49:45 suppose you don't need the RemoteName either +"""]] From 6625d0e024a73fc81fefd71d2110685c896b13a8 Mon Sep 17 00:00:00 2001 From: "https://id.koumbit.net/anarcat" Date: Wed, 11 Sep 2013 00:30:56 +0000 Subject: [PATCH 46/62] add the nicer whereis --- doc/related_software.mdwn | 1 + 1 file changed, 1 insertion(+) diff --git a/doc/related_software.mdwn b/doc/related_software.mdwn index 024a155e3b..e436092a39 100644 --- a/doc/related_software.mdwn +++ b/doc/related_software.mdwn @@ -10,3 +10,4 @@ designed to interoperate with it. * [sizes](http://hackage.haskell.org/package/sizes) is another du-like utility, with a `-A` switch that enables git-annex support. * Emacs Org mode can auto-commit attached files to git-annex. +* [[todo/nicer whereis output]] improves the output of the whereis command to be more human-readable. From 6a526c73354c0ae267b53bf1cd79a1fbea6e2a24 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Antoine=20Beaupr=C3=A9?= Date: Tue, 10 Sep 2013 20:47:13 -0400 Subject: [PATCH 47/62] nitpicking: make this more readable on the web this most options are now formatted as code. this has no effect on the manpage whatsoever (unfortunately) --- doc/git-annex.mdwn | 120 ++++++++++++++++++++++----------------------- 1 file changed, 60 insertions(+), 60 deletions(-) diff --git a/doc/git-annex.mdwn b/doc/git-annex.mdwn index 7afe5fd13c..07398d366e 100644 --- a/doc/git-annex.mdwn +++ b/doc/git-annex.mdwn @@ -64,7 +64,7 @@ subdirectories). Adds files in the path to the annex. Files that are already checked into git, or that git has been configured to ignore will be silently skipped. - (Use --force to add ignored files.) Dotfiles are skipped unless explicitly + (Use `--force` to add ignored files.) Dotfiles are skipped unless explicitly listed. * get [path ...] @@ -74,34 +74,34 @@ subdirectories). or transferring them from some kind of key-value store. Normally git-annex will choose which repository to copy the content from, - but you can override this using the --from option. + but you can override this using the `--from` option. * drop [path ...] Drops the content of annexed files from this repository. git-annex will refuse to drop content if it cannot verify it is - safe to do so. This can be overridden with the --force switch. + safe to do so. This can be overridden with the `--force` switch. - To drop content from a remote, specify --from. + To drop content from a remote, specify `--from`. * move [path ...] - When used with the --from option, moves the content of annexed files + When used with the `--from` option, moves the content of annexed files from the specified repository to the current one. - When used with the --to option, moves the content of annexed files from + When used with the `--to` option, moves the content of annexed files from the current repository to the specified one. * copy [path ...] - When used with the --from option, copies the content of annexed files + When used with the `--from` option, copies the content of annexed files from the specified repository to the current one. - When used with the --to option, copies the content of annexed files from + When used with the `--to` option, copies the content of annexed files from the current repository to the specified one. - To avoid contacting the remote to check if it has every file, specify --fast + To avoid contacting the remote to check if it has every file, specify `--fast` * unlock [path ...] @@ -125,10 +125,10 @@ subdirectories). Use this command when you want to synchronize the local repository with one or more of its remotes. You can specifiy the remotes to sync with; - the default is to sync with all remotes. Or specify --fast to sync with + the default is to sync with all remotes. Or specify `--fast` to sync with the remotes with the lowest annex-cost value. - The sync process involves first committing all local changes (git commit -a), + The sync process involves first committing all local changes (`git commit -a`), then fetching and merging the `synced/master` and the `git-annex` branch from the remote repositories and finally pushing the changes back to those branches on the remote repositories. You can use standard git @@ -162,9 +162,9 @@ subdirectories). This causes a destination repository to mirror a source repository. To use the local repository as the source repository, - specify mirror --to remote. + specify mirror `--to` remote. - To use a remote as the source repository, specify mirror --from remote. + To use a remote as the source repository, specify mirror `--from` remote. Each specified file in the source repository is mirrored to the destination repository. If a file's content is present in the source repository, it is @@ -179,18 +179,18 @@ subdirectories). Downloads each url to its own file, which is added to the annex. - To avoid immediately downloading the url, specify --fast. + To avoid immediately downloading the url, specify `--fast`. To avoid storing the size of the url's content, and accept whatever - is there at a future point, specify --relaxed. (Implies --fast.) + is there at a future point, specify `--relaxed`. (Implies `--fast`.) Normally the filename is based on the full url, so will look like "www.example.com_dir_subdir_bigfile". For a shorter filename, specify - --pathdepth=N. For example, --pathdepth=1 will use "dir/subdir/bigfile", - while --pathdepth=3 will use "bigfile". It can also be negative; - --pathdepth=-2 will use the last two parts of the url. + --pathdepth=N. For example, `--pathdepth=1` will use "dir/subdir/bigfile", + while `--pathdepth=3` will use "bigfile". It can also be negative; + `--pathdepth=-2` will use the last two parts of the url. - Or, to directly specify what file the url is added to, specify --file. + Or, to directly specify what file the url is added to, specify `--file`. This changes the behavior; now all the specified urls are recorded as alternate locations from which the file can be downloaded. In this mode, addurl can be used both to add new files, or to add urls to existing files. @@ -209,7 +209,7 @@ subdirectories). If a directory is specified, all files in it are imported, and any subdirectory structure inside it is preserved. - git annex import /media/camera/DCIM/ + git annex import /media/camera/DCIM/ By default, importing two files with the same contents from two different locations will result in both files being added to the repository. @@ -217,7 +217,7 @@ subdirectories). only one copy of the data will be stored.) To not delete files from the import location, use the - --duplicate option. This could allow importing the same files repeatedly + `--duplicate` option. This could allow importing the same files repeatedly to different locations in a repository. More likely, it could be used to import the same files to a number of different branches or separate git repositories. @@ -330,7 +330,7 @@ subdirectories). Example Amazon S3 remote: - git annex initremote mys3 type=S3 encryption=hybrid keyid=me@example.com datacenter=EU + git annex initremote mys3 type=S3 encryption=hybrid keyid=me@example.com datacenter=EU * enableremote name [param=value ...] @@ -361,7 +361,7 @@ subdirectories). One use-case of keyid-= is to replace a revoked key with a new key: - git annex enableremote mys3 keyid-=revokedkey keyid+=newkey + git annex enableremote mys3 keyid-=revokedkey keyid+=newkey Also, note that for encrypted special remotes using plain public-key encryption (encryption=pubkey), adding or removing a key has NO effect @@ -406,7 +406,7 @@ subdirectories). For example: - git annex content . "include=*.mp3 or include=*.ogg" + git annex content . "include=*.mp3 or include=*.ogg" Without an expression, displays the current preferred content setting of the repository. @@ -443,16 +443,16 @@ subdirectories). With parameters, only the specified files are checked. - To check a remote to fsck, specify --from. + To check a remote to fsck, specify `--from`. To avoid expensive checksum calculations (and expensive transfers when - fscking a remote), specify --fast. + fscking a remote), specify `--fast`. - To start a new incremental fsck, specify --incremental. Then - the next time you fsck, you can specify --more to skip over + To start a new incremental fsck, specify `--incremental`. Then + the next time you fsck, you can specify `--more` to skip over files that have already been checked, and continue where it left off. - The --incremental-schedule option makes a new incremental fsck be + The `--incremental-schedule` option makes a new incremental fsck be started a configurable time after the last incremental fsck was started. Once the current incremental fsck has completely finished, it causes a new one to start. @@ -462,25 +462,25 @@ subdirectories). have been fscked. And once it's done, you'd like a new fsck pass to start, but no more often than once a month. Then put this in a nightly cron job: - git annex fsck --incremental-schedule 30d --time-limit 5h + git annex fsck --incremental-schedule 30d --time-limit 5h To verify data integrity only while disregarding required number of copies, - use --numcopies=1. + use `--numcopies=1`. * unused Checks the annex for data that does not correspond to any files present in any tag or branch, and prints a numbered list of the data. - To only show unused temp and bad files, specify --fast. + To only show unused temp and bad files, specify `--fast`. - To check for annexed data on a remote, specify --from. + To check for annexed data on a remote, specify `--from`. - After running this command, you can use the --unused option to + After running this command, you can use the `--unused` option to operate on all the unused data that was found. For example, to move all unused data to origin: - git annex unused; git annex move --unused --to origin + git annex unused; git annex move --unused --to origin * dropunused [number|range ...] @@ -489,7 +489,7 @@ subdirectories). You can also specify ranges of numbers, such as "1-1000". - To drop the data from a remote, specify --from. + To drop the data from a remote, specify `--from.` * addunused [number|range ...] @@ -515,11 +515,11 @@ subdirectories). files used to be located. To also prune references to repositories that have been marked as dead, - specify --drop-dead. + specify `--drop-dead`. When this rewritten branch is merged into other clones of - the repository, git-annex will automatically perform the same rewriting - to their local git-annex branches. So the forgetfulness will automatically + the repository, `git-annex` will automatically perform the same rewriting + to their local `git-annex` branches. So the forgetfulness will automatically propigate out from its starting point until all repositories running git-annex have forgotten their old history. (You may need to force git to push the branch to any git repositories not running git-annex. @@ -537,12 +537,12 @@ subdirectories). By default, only lists annexed files whose content is currently present. This can be changed by specifying file matching options. To list all - annexed files, present or not, specify --include "*". To list all - annexed files whose content is not present, specify --not --in=here + annexed files, present or not, specify `--include "*"`. To list all + annexed files whose content is not present, specify `--not --in=here` To output filenames terminated with nulls, for use with xargs -0, - specify --print0. Or, a custom output formatting can be specified using - --format. The default output format is the same as --format='${file}\\n' + specify `--print0`. Or, a custom output formatting can be specified using + `--format`. The default output format is the same as `--format='${file}\\n'` These variables are available for use in formats: file, key, backend, bytesize, humansize @@ -558,18 +558,18 @@ subdirectories). showing each repository they were added to ("+") and removed from ("-"). To limit how far back to seach for location log changes, the options - --since, --after, --until, --before, and --max-count can be specified. - They are passed through to git log. For example, --since "1 month ago" + `--since`, `--after`, `--until`, `--before`, and `--max-count` can be specified. + They are passed through to git log. For example, `--since "1 month ago"` To generate output suitable for the gource visualisation program, - specify --gource. + specify `--gource`. * status [directory ...] Displays some statistics and other information, including how much data is in the annex and a list of all known repositories. - To only show the data that can be gathered quickly, use --fast. + To only show the data that can be gathered quickly, use `--fast`. When a directory is specified, shows only an abbreviated status display for that directory. In this mode, all of the file matching @@ -580,7 +580,7 @@ subdirectories). would first like to see how much disk space that will use. Then run: - git annex status . --not --in here + git annex status . --not --in here * map @@ -588,7 +588,7 @@ subdirectories). by going out and looking at all the ones it can get to, and generating a Graphviz file displaying it all. If the `dot` command is available, it is used to display the file to your screen (using x11 backend). (To disable - this display, specify --fast) + this display, specify `--fast`) This command only connects to hosts that the host it's run on can directly connect to. It does not try to tunnel through intermediate hosts. @@ -605,7 +605,7 @@ subdirectories). * migrate [path ...] Changes the specified annexed files to use the default key-value backend - (or the one specified with --backend). Only files whose content + (or the one specified with `--backend`). Only files whose content is currently available are migrated. Note that the content is also still available using the old key after @@ -626,7 +626,7 @@ subdirectories). Example: - git annex reinject /tmp/foo.iso foo.iso + git annex reinject /tmp/foo.iso foo.iso * unannex [path ...] @@ -638,7 +638,7 @@ subdirectories). file and don't want its contents any more. In that case you should use `git annex drop` instead, and you can also `git rm` the file. - In --fast mode, this command leaves content in the annex, simply making + In `--fast` mode, this command leaves content in the annex, simply making a hard link to it. * uninit @@ -673,7 +673,7 @@ subdirectories). Example: - git annex dropkey SHA1-s10-7da006579dd64330eb2456001fd01948430572f2 + git annex dropkey SHA1-s10-7da006579dd64330eb2456001fd01948430572f2 * transferkeys @@ -684,7 +684,7 @@ subdirectories). This plumbing-level command is similar to migrate, but you specify both the file, and the new key to use for it. - With --force, even files whose content is not currently available will + With `--force`, even files whose content is not currently available will be rekeyed. Use with caution. * test @@ -841,14 +841,14 @@ file contents are present at either of two repositories. Skips files matching the glob pattern. The glob is matched relative to the current directory. For example: - --exclude='*.mp3' --exclude='subdir/*' + --exclude='*.mp3' --exclude='subdir/*' * --include=glob Skips files not matching the glob pattern. (Same as --not --exclude.) For example, to include only mp3 and ogg files: - --include='*.mp3' --or --include='*.ogg' + --include='*.mp3' --or --include='*.ogg' * --in=repository @@ -903,7 +903,7 @@ file contents are present at either of two repositories. * --not Inverts the next file matching option. For example, to only act on - files with less than 3 copies, use --not --copies=3 + files with less than 3 copies, use `--not --copies=3` * --and @@ -937,7 +937,7 @@ For example: The main differences are that `exclude=` and `include=` always match relative to the top of the git repository, and that there is -no equivilant to --in. +no equivilant to `--in`. # CONFIGURATION @@ -980,7 +980,7 @@ Here are all the supported configuration settings. Example: - annex.largefiles = largerthan=100kb and not (include=*.c or include=*.h) + annex.largefiles = largerthan=100kb and not (include=*.c or include=*.h) * `annex.queuesize` @@ -1084,7 +1084,7 @@ Here are all the supported configuration settings. If set to `true`, prevents git-annex from storing file contents on this remote by default. - (You can still request it be used by the --from and --to options.) + (You can still request it be used by the `--from` and `--to` options.) This is, for example, useful if the remote is located somewhere without git-annex-shell. (For example, if it's on GitHub). From 252381c57dc9e8320d8bc2040e7023c0ae5bd259 Mon Sep 17 00:00:00 2001 From: "https://id.koumbit.net/anarcat" Date: Wed, 11 Sep 2013 00:54:35 +0000 Subject: [PATCH 48/62] --- doc/todo/mdwn2man:_make_backticks_bold.mdwn | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100644 doc/todo/mdwn2man:_make_backticks_bold.mdwn diff --git a/doc/todo/mdwn2man:_make_backticks_bold.mdwn b/doc/todo/mdwn2man:_make_backticks_bold.mdwn new file mode 100644 index 0000000000..87c228ab88 --- /dev/null +++ b/doc/todo/mdwn2man:_make_backticks_bold.mdwn @@ -0,0 +1,20 @@ +The traditionnal way of marking commandline flags in a manpage is with a `.B` (for Bold, I guess). It doesn't seem to be used by mdwn2man, which makes the manpage look a little more dull than it could. + +The following patch makes those options come out more obviously: + +[[!format diff """ +diff --git a/Build/mdwn2man b/Build/mdwn2man +index ba5919b..7f819ad 100755 +--- a/Build/mdwn2man ++++ b/Build/mdwn2man +@@ -8,6 +8,7 @@ print ".TH $prog $section\n"; + + while (<>) { + s{(\\?)\[\[([^\s\|\]]+)(\|[^\s\]]+)?\]\]}{$1 ? "[[$2]]" : $2}eg; ++ s/\`([^\`]*)\`/\\fB$1\\fP/g; + s/\`//g; + s/^\s*\./\\&./g; + if (/^#\s/) { +"""]] + +I tested it against the git-annex manpage and it seems to work well. --[[anarcat]] From c27e7fb3d1555e6e76e0da6fd2161447a5705b58 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawnxlx1UrzVhdy6_gFjzmF42x6QXxBUxg00" Date: Wed, 11 Sep 2013 00:58:09 +0000 Subject: [PATCH 49/62] Added a comment: pre-commit hook of git annex --- ...comment_1_6d47485728ea65a9b555f8be7159dea5._comment | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 doc/bugs/merge_causes_out_of_memory_on_large_repos/comment_1_6d47485728ea65a9b555f8be7159dea5._comment diff --git a/doc/bugs/merge_causes_out_of_memory_on_large_repos/comment_1_6d47485728ea65a9b555f8be7159dea5._comment b/doc/bugs/merge_causes_out_of_memory_on_large_repos/comment_1_6d47485728ea65a9b555f8be7159dea5._comment new file mode 100644 index 0000000000..f87a9a84d1 --- /dev/null +++ b/doc/bugs/merge_causes_out_of_memory_on_large_repos/comment_1_6d47485728ea65a9b555f8be7159dea5._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawnxlx1UrzVhdy6_gFjzmF42x6QXxBUxg00" + nickname="Jakukyo" + subject="pre-commit hook of git annex" + date="2013-09-11T00:58:09Z" + content=""" +The pre-commit hook of git annex throws an 'out of memory' error, too. + +As a workaround, I've deleted ~/.git/hooks/pre-commit, and use `git merge` manually. +"""]] From 8fbbf1123357bebe911d711f03a1cc80031f94fe Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Antoine=20Beaupr=C3=A9?= Date: Tue, 10 Sep 2013 21:00:59 -0400 Subject: [PATCH 50/62] missing backticks --- doc/git-annex.mdwn | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/doc/git-annex.mdwn b/doc/git-annex.mdwn index 07398d366e..6db91a05f8 100644 --- a/doc/git-annex.mdwn +++ b/doc/git-annex.mdwn @@ -186,7 +186,7 @@ subdirectories). Normally the filename is based on the full url, so will look like "www.example.com_dir_subdir_bigfile". For a shorter filename, specify - --pathdepth=N. For example, `--pathdepth=1` will use "dir/subdir/bigfile", + `--pathdepth=N`. For example, `--pathdepth=1` will use "dir/subdir/bigfile", while `--pathdepth=3` will use "bigfile". It can also be negative; `--pathdepth=-2` will use the last two parts of the url. @@ -223,14 +223,14 @@ subdirectories). repositories. To only import files whose content has not been seen before by git-annex, - use the --deduplicate option. Duplicate files will be deleted from the + use the `--deduplicate` option. Duplicate files will be deleted from the import location. - The --clean-duplicates option does not import any new files, but any files + The `--clean-duplicates` option does not import any new files, but any files found in the import location that are duplicates of content in the annex are deleted. - (Note that using --deduplicate or --clean-duplicates with the WORM + (Note that using `--deduplicate` or `--clean-duplicates` with the WORM backend does not look at file content, but filename and mtime.) * importfeed [url ...] @@ -238,13 +238,13 @@ subdirectories). Imports the contents of podcast feeds. Only downloads files whose urls have not already been added to the repository before, so you can delete, rename, etc the resulting files and repeated runs won't duplicate - them. (Use --force to force downloading urls it's seen before.) + them. (Use `--force` to force downloading urls it's seen before.) - Use --template to control where the files are stored. + Use `--template` to control where the files are stored. The default template is '${feedtitle}/${itemtitle}${extension}' (Other available variables: feedauthor, itemauthor, itemsummary, itemdescription, itemrights, itemid) - The --relaxed and --fast options behave the same as they do in addurl. + The `--relaxed` and `--fast` options behave the same as they do in addurl. * watch @@ -254,15 +254,15 @@ subdirectories). background, you no longer need to manually run git commands when manipulating your files. - To not daemonize, run with --foreground ; to stop a running daemon, - run with --stop + To not daemonize, run with `--foreground` ; to stop a running daemon, + run with `--stop` * assistant Like watch, but also automatically syncs changes to other remotes. Typically started at boot, or when you log in. - With the --autostart option, the assistant is started in any repositories + With the `--autostart` option, the assistant is started in any repositories it has created. These are listed in `~/.config/git-annex/autostart` * webapp @@ -273,7 +273,7 @@ subdirectories). By default, the webapp can only be accessed from localhost, and running it opens a browser window. - With the --listen=address[:port] option, the webapp can be made to listen + With the `--listen=address[:port]` option, the webapp can be made to listen for connections on the specified address. This disables running a local web browser, and outputs the url you can use to open the webapp from a remote computer. @@ -325,7 +325,7 @@ subdirectories). Note that with encryption enabled, a cryptographic key is created. This requires sufficient entropy. If initremote seems to hang or take a long time while generating the key, you may want to ctrl-c it and - re-run with --fast, which causes it to use a lower-quality source of + re-run with `--fast`, which causes it to use a lower-quality source of randomness. Example Amazon S3 remote: @@ -845,7 +845,7 @@ file contents are present at either of two repositories. * --include=glob - Skips files not matching the glob pattern. (Same as --not --exclude.) + Skips files not matching the glob pattern. (Same as `--not --exclude`.) For example, to include only mp3 and ogg files: --include='*.mp3' --or --include='*.ogg' @@ -858,7 +858,7 @@ file contents are present at either of two repositories. The repository should be specified using the name of a configured remote, or the UUID or description of a repository. For the current repository, - use --in=here + use `--in=here` * --copies=number From 375a942dc9d60a5dc8813d7dc0ff9bd19ae9fc7e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Antoine=20Beaupr=C3=A9?= Date: Tue, 10 Sep 2013 21:12:30 -0400 Subject: [PATCH 51/62] add backticks to all options --- doc/git-annex.mdwn | 184 ++++++++++++++++++++++----------------------- 1 file changed, 92 insertions(+), 92 deletions(-) diff --git a/doc/git-annex.mdwn b/doc/git-annex.mdwn index 6db91a05f8..adb21428fd 100644 --- a/doc/git-annex.mdwn +++ b/doc/git-annex.mdwn @@ -60,14 +60,14 @@ files in the directory. When no path is specified, most git-annex commands default to acting on all relevant files in the current directory (and subdirectories). -* add [path ...] +* `add [path ...]` Adds files in the path to the annex. Files that are already checked into git, or that git has been configured to ignore will be silently skipped. (Use `--force` to add ignored files.) Dotfiles are skipped unless explicitly listed. -* get [path ...] +* `get [path ...]` Makes the content of annexed files available in this repository. This will involve copying them from another repository, or downloading them, @@ -76,7 +76,7 @@ subdirectories). Normally git-annex will choose which repository to copy the content from, but you can override this using the `--from` option. -* drop [path ...] +* `drop [path ...]` Drops the content of annexed files from this repository. @@ -85,7 +85,7 @@ subdirectories). To drop content from a remote, specify `--from`. -* move [path ...] +* `move [path ...]` When used with the `--from` option, moves the content of annexed files from the specified repository to the current one. @@ -93,7 +93,7 @@ subdirectories). When used with the `--to` option, moves the content of annexed files from the current repository to the specified one. -* copy [path ...] +* `copy [path ...]` When used with the `--from` option, copies the content of annexed files from the specified repository to the current one. @@ -103,7 +103,7 @@ subdirectories). To avoid contacting the remote to check if it has every file, specify `--fast` -* unlock [path ...] +* `unlock [path ...]` Normally, the content of annexed files is protected from being changed. Unlocking a annexed file allows it to be modified. This replaces the @@ -111,17 +111,17 @@ subdirectories). You can then modify it and `git annex add` (or `git commit`) to inject it back into the annex. -* edit [path ...] +* `edit [path ...]` This is an alias for the unlock command. May be easier to remember, if you think of this as allowing you to edit an annexed file. -* lock [path ...] +* `lock [path ...]` Use this to undo an unlock command if you don't want to modify the files, or have made modifications you want to discard. -* sync [remote ...] +* `sync [remote ...]` Use this command when you want to synchronize the local repository with one or more of its remotes. You can specifiy the remotes to sync with; @@ -148,7 +148,7 @@ subdirectories). Note that sync does not transfer any file contents from or to the remote repositories. -* merge +* `merge` This performs the same merging that is done by the sync command, but without pushing or pulling any data. @@ -157,7 +157,7 @@ subdirectories). post-receive hook. Then any syncs to the repository will update its working copy automatically. -* mirror [path ...] +* `mirror [path ...]` This causes a destination repository to mirror a source repository. @@ -175,7 +175,7 @@ subdirectories). Note that mirror does not sync the git repository, but only the file contents. -* addurl [url ...] +* `addurl [url ...]` Downloads each url to its own file, which is added to the annex. @@ -198,11 +198,11 @@ subdirectories). When quvi is installed, urls are automatically tested to see if they are on a video hosting site, and the video is downloaded instead. -* rmurl file url +* `rmurl file url` Record that the file is no longer available at the url. -* import [path ...] +* `import [path ...]` Moves files from somewhere outside the git working copy, and adds them to the annex. Individual files to import can be specified. @@ -233,7 +233,7 @@ subdirectories). (Note that using `--deduplicate` or `--clean-duplicates` with the WORM backend does not look at file content, but filename and mtime.) -* importfeed [url ...] +* `importfeed [url ...]` Imports the contents of podcast feeds. Only downloads files whose urls have not already been added to the repository before, so you can @@ -246,7 +246,7 @@ subdirectories). The `--relaxed` and `--fast` options behave the same as they do in addurl. -* watch +* `watch` Watches for changes to files in the current directory and its subdirectories, and takes care of automatically adding new files, as well as dealing with @@ -257,7 +257,7 @@ subdirectories). To not daemonize, run with `--foreground` ; to stop a running daemon, run with `--stop` -* assistant +* `assistant` Like watch, but also automatically syncs changes to other remotes. Typically started at boot, or when you log in. @@ -265,7 +265,7 @@ subdirectories). With the `--autostart` option, the assistant is started in any repositories it has created. These are listed in `~/.config/git-annex/autostart` -* webapp +* `webapp` Opens a web app, that allows easy setup of a git-annex repository, and control of the git-annex assistant. @@ -281,7 +281,7 @@ subdirectories). # REPOSITORY SETUP COMMANDS -* init [description] +* `init [description]` Until a repository (or one of its remotes) has been initialized, git-annex will refuse to operate on it, to avoid accidentially @@ -291,7 +291,7 @@ subdirectories). of a repository with its own description. If you don't provide one, one will be generated. -* describe repository description +* `describe repository description` Changes the description of a repository. @@ -299,7 +299,7 @@ subdirectories). by uuid. To change the description of the current repository, use "here". -* initremote name [param=value ...] +* `initremote name [param=value ...]` Creates a new special remote, and adds it to `.git/config`. @@ -332,7 +332,7 @@ subdirectories). git annex initremote mys3 type=S3 encryption=hybrid keyid=me@example.com datacenter=EU -* enableremote name [param=value ...] +* `enableremote name [param=value ...]` Enables use of an existing special remote in the current repository, which may be a different repository than the one in which it was @@ -369,37 +369,37 @@ subdirectories). keyid+= and keyid-= with such remotes should be used with care, and make little sense except in cases like the revoked key example above. -* trust [repository ...] +* `trust [repository ...]` Records that a repository is trusted to not unexpectedly lose content. Use with care. To trust the current repository, use "here". -* untrust [repository ...] +* `untrust [repository ...]` Records that a repository is not trusted and could lose content at any time. -* semitrust [repository ...] +* `semitrust [repository ...]` Returns a repository to the default semi trusted state. -* dead [repository ...] +* `dead [repository ...]` Indicates that the repository has been irretrevably lost. (To undo, use semitrust.) -* group repository groupname +* `group repository groupname` Adds a repository to a group, such as "archival", "enduser", or "transfer". The groupname must be a single word. -* ungroup repository groupname +* `ungroup repository groupname` Removes a repository from a group. -* content repository [expression] +* `content repository [expression]` When run with an expression, configures the content that is preferred to be held in the archive. See PREFERRED CONTENT below. @@ -411,13 +411,13 @@ subdirectories). Without an expression, displays the current preferred content setting of the repository. -* vicfg +* `vicfg` Opens EDITOR on a temp file containing most of the above configuration settings, and when it exits, stores any changes made back to the git-annex branch. -* direct +* `direct` Switches a repository to use direct mode, where rather than symlinks to files, the files are directly present in the repository. @@ -428,7 +428,7 @@ subdirectories). use in direct mode repositories, and can result in data loss or other bad behavior. -* indirect +* `indirect` Switches a repository back from direct mode to the default, indirect mode. @@ -436,7 +436,7 @@ subdirectories). # REPOSITORY MAINTENANCE COMMANDS -* fsck [path ...] +* `fsck [path ...]` With no parameters, this command checks the whole annex for consistency, and warns about or fixes any problems found. @@ -467,7 +467,7 @@ subdirectories). To verify data integrity only while disregarding required number of copies, use `--numcopies=1`. -* unused +* `unused` Checks the annex for data that does not correspond to any files present in any tag or branch, and prints a numbered list of the data. @@ -482,7 +482,7 @@ subdirectories). git annex unused; git annex move --unused --to origin -* dropunused [number|range ...] +* `dropunused [number|range ...]` Drops the data corresponding to the numbers, as listed by the last `git annex unused` @@ -491,23 +491,23 @@ subdirectories). To drop the data from a remote, specify `--from.` -* addunused [number|range ...] +* `addunused [number|range ...]` Adds back files for the content corresponding to the numbers or ranges, as listed by the last `git annex unused`. The files will have names starting with "unused." -* fix [path ...] +* `fix [path ...]` Fixes up symlinks that have become broken to again point to annexed content. This is useful to run if you have been moving the symlinks around, but is done automatically when committing a change with git too. -* upgrade +* `upgrade` Upgrades the repository to current layout. -* forget +* `forget` Causes the git-annex branch to be rewritten, throwing away historical data about past locations of files. The resulting branch will use less @@ -526,11 +526,11 @@ subdirectories). # QUERY COMMANDS -* version +* `version` Shows the version of git-annex, as well as repository version information. -* find [path ...] +* `find [path ...]` Outputs a list of annexed files in the specified path. With no path, finds files in the current directory and its subdirectories. @@ -547,12 +547,12 @@ subdirectories). These variables are available for use in formats: file, key, backend, bytesize, humansize -* whereis [path ...] +* `whereis [path ...]` Displays a list of repositories known to contain the content of the specified file or files. -* log [path ...] +* `log [path ...]` Displays the location log for the specified file or files, showing each repository they were added to ("+") and removed from ("-"). @@ -564,7 +564,7 @@ subdirectories). To generate output suitable for the gource visualisation program, specify `--gource`. -* status [directory ...] +* `status [directory ...]` Displays some statistics and other information, including how much data is in the annex and a list of all known repositories. @@ -582,7 +582,7 @@ subdirectories). git annex status . --not --in here -* map +* `map` Helps you keep track of your repositories, and the connections between them, by going out and looking at all the ones it can get to, and generating a @@ -602,7 +602,7 @@ subdirectories). # UTILITY COMMANDS -* migrate [path ...] +* `migrate [path ...]` Changes the specified annexed files to use the default key-value backend (or the one specified with `--backend`). Only files whose content @@ -615,7 +615,7 @@ subdirectories). However, if a backend changes the information it uses to construct a key, this can also be used to migrate files to use the new key format. -* reinject src dest +* `reinject src dest` Moves the src file into the annex as the content of the dest file. This can be useful if you have obtained the content of a file from @@ -628,7 +628,7 @@ subdirectories). git annex reinject /tmp/foo.iso foo.iso -* unannex [path ...] +* `unannex [path ...]` Use this to undo an accidental `git annex add` command. You can use `git annex unannex` to move content out of the annex at any point, @@ -641,7 +641,7 @@ subdirectories). In `--fast` mode, this command leaves content in the annex, simply making a hard link to it. -* uninit +* `uninit` Use this to stop using git annex. It will unannex every file in the repository, and remove all of git-annex's other data, leaving you with a @@ -649,7 +649,7 @@ subdirectories). # PLUMBING COMMANDS -* pre-commit [path ...] +* `pre-commit [path ...]` Fixes up symlinks that are staged as part of a commit, to ensure they point to annexed content. Also handles injecting changes to unlocked @@ -658,12 +658,12 @@ subdirectories). This is meant to be called from git's pre-commit hook. `git annex init` automatically creates a pre-commit hook using this. -* fromkey key file +* `fromkey key file` This plumbing-level command can be used to manually set up a file in the git repository to link to a specified key. -* dropkey [key ...] +* `dropkey [key ...]` This plumbing-level command drops the annexed data for the specified keys from this repository. @@ -675,11 +675,11 @@ subdirectories). git annex dropkey SHA1-s10-7da006579dd64330eb2456001fd01948430572f2 -* transferkeys +* `transferkeys` This plumbing-level command is used by the assistant to transfer data. -* rekey [file key ...] +* `rekey [file key ...]` This plumbing-level command is similar to migrate, but you specify both the file, and the new key to use for it. @@ -687,34 +687,34 @@ subdirectories). With `--force`, even files whose content is not currently available will be rekeyed. Use with caution. -* test +* `test` This runs git-annex's built-in test suite. -* xmppgit +* `xmppgit` This command is used internally to perform git pulls over XMPP. # OPTIONS -* --force +* `--force` Force unsafe actions, such as dropping a file's content when no other source of it can be verified to still exist, or adding ignored files. Use with care. -* --fast +* `--fast` Enables less expensive, but also less thorough versions of some commands. What is avoided depends on the command. -* --auto +* `--auto` Enables automatic mode. Commands that get, drop, or move file contents will only do so when needed to help satisfy the setting of annex.numcopies, and preferred content configuration. -* --all +* `--all` Operate on all data that has been stored in the git annex, including old versions of files. This is the default behavior when @@ -722,56 +722,56 @@ subdirectories). normal behavior is to only operate on specified files in the working tree. -* --unused +* `--unused` Operate on all data that has been determined to be unused by a previous run of `git-annex unused`. -* --quiet +* `--quiet` Avoid the default verbose display of what is done; only show errors and progress displays. -* --verbose +* `--verbose` Enable verbose display. -* --json +* `--json` Rather than the normal output, generate JSON. This is intended to be parsed by programs that use git-annex. Each line of output is a JSON object. Note that json output is only usable with some git-annex commands, like status and find. -* --debug +* `--debug` Show debug messages. -* --no-debug +* `--no-debug` Disable debug messages. -* --from=repository +* `--from=repository` Specifies a repository that content will be retrieved from, or that should otherwise be acted on. It should be specified using the name of a configured remote. -* --to=repository +* `--to=repository` Specifies a repository that content will be sent to. It should be specified using the name of a configured remote. -* --numcopies=n +* `--numcopies=n` Overrides the `annex.numcopies` setting, forcing git-annex to ensure the specified number of copies exist. Note that setting numcopies to 0 is very unsafe. -* --time-limit=time +* `--time-limit=time` Limits how long a git-annex command runs. The time can be something like "5h", or "30m" or even "45s" or "10d". @@ -782,16 +782,16 @@ subdirectories). Also, note that if the time limit prevents git-annex from doing all it was asked to, it will exit with a special code, 101. -* --trust=repository -* --semitrust=repository -* --untrust=repository +* `--trust=repository` +* `--semitrust=repository` +* `--untrust=repository` Overrides trust settings for a repository. May be specified more than once. The repository should be specified using the name of a configured remote, or the UUID or description of a repository. -* --trust-glacier-inventory +* `--trust-glacier-inventory` Amazon Glacier inventories take hours to retrieve, and may not represent the current state of a repository. So git-annex does not trust that @@ -802,14 +802,14 @@ subdirectories). removed a file from Glacier. If you try to drop the only other copy of the file, and this switch is enabled, you could lose data! -* --backend=name +* `--backend=name` Specifies which key-value backend to use. This can be used when adding a file to the annex, or migrating a file. Once files are in the annex, their backend is known and this option is not necessary. -* --format=value +* `--format=value` Specifies a custom output format. The value is a format string, in which '${var}' is expanded to the value of a variable. To right-justify @@ -819,7 +819,7 @@ subdirectories). Also, '\\n' is a newline, '\\000' is a NULL, etc. -* -c name=value +* `-c name=value` Used to override git configuration settings. May be specified multiple times. @@ -836,21 +836,21 @@ For example: The above example prevents git-annex from working on mp3 files whose file contents are present at either of two repositories. -* --exclude=glob +* `--exclude=glob` Skips files matching the glob pattern. The glob is matched relative to the current directory. For example: --exclude='*.mp3' --exclude='subdir/*' -* --include=glob +* `--include=glob` Skips files not matching the glob pattern. (Same as `--not --exclude`.) For example, to include only mp3 and ogg files: --include='*.mp3' --or --include='*.ogg' -* --in=repository +* `--in=repository` Matches only files that git-annex believes have their contents present in a repository. Note that it does not check the repository to verify @@ -860,13 +860,13 @@ file contents are present at either of two repositories. or the UUID or description of a repository. For the current repository, use `--in=here` -* --copies=number +* `--copies=number` Matches only files that git-annex believes to have the specified number of copies, or more. Note that it does not check remotes to verify that the copies still exist. -* --copies=trustlevel:number +* `--copies=trustlevel:number` Matches only files that git-annex believes have the specified number of copies, on remotes with the specified trust level. For example, @@ -875,24 +875,24 @@ file contents are present at either of two repositories. To match any trust level at or higher than a given level, use 'trustlevel+'. For example, "--copies=semitrusted+:2" -* --copies=groupname:number +* `--copies=groupname:number` Matches only files that git-annex believes have the specified number of copies, on remotes in the specified group. For example, "--copies=archive:2" -* --inbackend=name +* `--inbackend=name` Matches only files whose content is stored using the specified key-value backend. -* --inallgroup=groupname +* `--inallgroup=groupname` Matches only files that git-annex believes are present in all repositories in the specified group. -* --smallerthan=size -* --largerthan=size +* `--smallerthan=size` +* `--largerthan=size` Matches only files whose content is smaller than, or larger than the specified size. @@ -900,25 +900,25 @@ file contents are present at either of two repositories. The size can be specified with any commonly used units, for example, "0.5 gb" or "100 KiloBytes" -* --not +* `--not` Inverts the next file matching option. For example, to only act on files with less than 3 copies, use `--not --copies=3` -* --and +* `--and` Requires that both the previous and the next file matching option matches. The default. -* --or +* `--or` Requires that either the previous, or the next file matching option matches. -* -( +* `-(` Opens a group of file matching options. -* -) +* `-)` Closes a group of file matching options. From 5174785ea24998961c2917a020f30cced34904f1 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawnxlx1UrzVhdy6_gFjzmF42x6QXxBUxg00" Date: Wed, 11 Sep 2013 01:21:11 +0000 Subject: [PATCH 52/62] Syncing with a huge repo on a small partition runs out of inodes. --- doc/forum/Running_out_of__inodes.mdwn | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) create mode 100644 doc/forum/Running_out_of__inodes.mdwn diff --git a/doc/forum/Running_out_of__inodes.mdwn b/doc/forum/Running_out_of__inodes.mdwn new file mode 100644 index 0000000000..417de18de0 --- /dev/null +++ b/doc/forum/Running_out_of__inodes.mdwn @@ -0,0 +1,17 @@ +When syncing with a huge git annex repository on usb disk, my small laptop partition runs out of inodes. + +Any workaround for this? + +- Use bare repository? + + Some git annex command are not supported. + This makes managing (particularly adding) files difficult. + +- Use a loop file partition with tiny block size and large inode numbers? + + Operations on a huge git repository are slow. + On a loop file partition will be slower. + +- Maybe shrink the partition and make room for a specific partition for git annex repository? + +Any opinions? From 1b9de447db3fac94548635b3b778206f5296edee Mon Sep 17 00:00:00 2001 From: "https://id.koumbit.net/anarcat" Date: Wed, 11 Sep 2013 05:20:05 +0000 Subject: [PATCH 53/62] prototype ready for review --- doc/todo/nicer_whereis_output.mdwn | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/doc/todo/nicer_whereis_output.mdwn b/doc/todo/nicer_whereis_output.mdwn index 22be93bd2e..b6352d737e 100644 --- a/doc/todo/nicer_whereis_output.mdwn +++ b/doc/todo/nicer_whereis_output.mdwn @@ -38,12 +38,12 @@ Fourth version: tuning and blocked This required more advanced parsing of the remotes, and instead of starting to do some JSON parsing, [[anarcat]] figured it was time to learn some Haskell instead. -Current status: needs work -========================== +Current status: needs review +============================ -So right now, the most recent version is in [anarcat's gist](https://gist.github.com/anarcat/6502988) and works reasonably well. However, it doesn't distinguish between trusted and untrusted repos and so on. +So right now, the most recent version of the python script is in [anarcat's gist](https://gist.github.com/anarcat/6502988) and works reasonably well. However, it doesn't distinguish between trusted and untrusted repos and so on. -Furthermore, we'd like to see this factored into the `whereis` command directly. +Furthermore, we'd like to see this factored into the `whereis` command directly. A [raw.hs](http://codepad.org/miVJb5oK) file has been programmed by `mastensg`, and is now available in the above gist. It fits the desired output and prototypes. Desired output -------------- From 497b9d152cd918fb790f10866811475dd8cab16a Mon Sep 17 00:00:00 2001 From: eezec Date: Wed, 11 Sep 2013 06:12:20 +0000 Subject: [PATCH 54/62] Added a comment --- ...ent_3_4bf55320439de152a65e2f21d4a0604b._comment | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 doc/forum/git-annex:_status:_1_failed/comment_3_4bf55320439de152a65e2f21d4a0604b._comment diff --git a/doc/forum/git-annex:_status:_1_failed/comment_3_4bf55320439de152a65e2f21d4a0604b._comment b/doc/forum/git-annex:_status:_1_failed/comment_3_4bf55320439de152a65e2f21d4a0604b._comment new file mode 100644 index 0000000000..0c3604ad6e --- /dev/null +++ b/doc/forum/git-annex:_status:_1_failed/comment_3_4bf55320439de152a65e2f21d4a0604b._comment @@ -0,0 +1,14 @@ +[[!comment format=mdwn + username="eezec" + ip="93.21.246.236" + subject="comment 3" + date="2013-09-11T06:12:20Z" + content=""" +Joey .. moving transfer didn't work .. i have the same message when doing + + git annex status + .... + getFileStatus: does not exist (No such file or directory) + .... + git-annex: status: 1 failed +"""]] From 53ef391cc13544eaa199a5ded0e94d6de90f69a3 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawmkBwMWvNKZZCge_YqobCSILPMeK6xbFw8" Date: Wed, 11 Sep 2013 07:37:38 +0000 Subject: [PATCH 55/62] --- doc/bugs/box.com_never_stops_syncing..mdwn | 61 ++++++++++++++++++++++ 1 file changed, 61 insertions(+) create mode 100644 doc/bugs/box.com_never_stops_syncing..mdwn diff --git a/doc/bugs/box.com_never_stops_syncing..mdwn b/doc/bugs/box.com_never_stops_syncing..mdwn new file mode 100644 index 0000000000..0346816ec2 --- /dev/null +++ b/doc/bugs/box.com_never_stops_syncing..mdwn @@ -0,0 +1,61 @@ +### Please describe the problem. +Git-annex will constantly sync most(if not all) my files to box.com + +### What steps will reproduce the problem? +1 - Use git-annex instead of Dropbox at work +2 - Boot computer. +3 - Watch it sync everything to box.com (even files i believe it has transferred each and every day for the last few months) + +### What version of git-annex are you using? On what operating system? +git-annex version: 4.20130827 + +But i have never seen it work satisfactory in any version. + +### Please provide any additional information below. + +I am going to add more debug to this bug constantly. (I intend to do a full 'git annex copy --to box.com --not --in box.com' daily, and see if the same files are transfered again and again) + + +For now, i see a few different issues already: + + +[[!format sh """ +# If you can, paste a complete transcript of the problem occurring here. +# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log + + +tou@DSK1049:~/work-annex$ git annex copy --to box.com --not --in box.com 2>&1 | tee ../work-annex-copy-to-box.com-not-in-box.com-run1.log +[2013-09-11 09:24:53 CEST] read: git ["--git-dir=/home/tou/work-annex/.git","--work-tree=/home/tou/work-annex","show-ref","git-annex"] +[2013-09-11 09:24:53 CEST] read: git ["--git-dir=/home/tou/work-annex/.git","--work-tree=/home/tou/work-annex","show-ref","--hash","refs/heads/git-annex"] +[2013-09-11 09:24:53 CEST] read: git ["--git-dir=/home/tou/work-annex/.git","--work-tree=/home/tou/work-annex","log","refs/heads/git-annex..dbe8b1cfa5f84126c45a39fdc7c7f26e272c71cc","--oneline","-n1"] +[2013-09-11 09:24:53 CEST] read: git ["--git-dir=/home/tou/work-annex/.git","--work-tree=/home/tou/work-annex","log","refs/heads/git-annex..45e279375897a2cd7f5b893402e0ec25c1b23436","--oneline","-n1"] +[2013-09-11 09:24:54 CEST] read: git ["--git-dir=/home/tou/work-annex/.git","--work-tree=/home/tou/work-annex","log","refs/heads/git-annex..c4921be4434f751493fce1c932ac759214abacd4","--oneline","-n1"] +[2013-09-11 09:24:54 CEST] read: git ["--git-dir=/home/tou/work-annex/.git","--work-tree=/home/tou/work-annex","log","refs/heads/git-annex..d591398dc1cac824a5fc5bdacdcb82301a9b15a3","--oneline","-n1"] +[2013-09-11 09:24:54 CEST] chat: git ["--git-dir=/home/tou/work-annex/.git","--work-tree=/home/tou/work-annex","cat-file","--batch"] +[2013-09-11 09:24:54 CEST] read: git ["config","--null","--list"] +[2013-09-11 09:24:54 CEST] read: git ["--git-dir=/home/tou/work-annex/.git","--work-tree=/home/tou/work-annex","ls-files","--cached","-z","--"] +[2013-09-11 09:24:54 CEST] chat: git ["--git-dir=/home/tou/work-annex/.git","--work-tree=/home/tou/work-annex","cat-file","--batch"] +copy Documents/Gamle catillo overførsler/Rapport b-bm.odt (gpg) (checking box.com...) ok +copy Documents/Gamle catillo overførsler/Rapport brandt skorstensfejeren.odt (checking box.com...) (failed to read https://www.box.com/dav/work-annex/4a5/18e/GPGHMACSHA1--5f8660edac93899cf9adc5fadcc480ddc2992bb1/GPGHMACSHA1--5f8660edac93899cf9adc5fadcc480ddc2992bb1.chunkcount) failed +copy Documents/Gamle catillo overførsler/Rapport teamkoege.odt (checking box.com...) (failed to read https://www.box.com/dav/work-annex/98d/ae7/GPGHMACSHA1--5253241407527aa6c980f1174fdbc32713c54c44/GPGHMACSHA1--5253241407527aa6c980f1174fdbc32713c54c44.chunkcount) failed +copy Documents/Gamle catillo overførsler/Rapport vikarborsen.odt (checking box.com...) ok +copy Documents/Gamle catillo overførsler/Rapport-terrariemesteren.odt (checking box.com...) ok +copy Documents/Gamle catillo overførsler/catillo-efhandel.odt (checking box.com...) (failed to read https://www.box.com/dav/work-annex/9d9/aea/GPGHMACSHA1--1516eac1ec7b4ceaa840faebabde1f50f5db0a52/GPGHMACSHA1--1516eac1ec7b4ceaa840faebabde1f50f5db0a52.chunkcount) failed +copy Documents/Nøgeordsanalyse catillo104.xlsx (checking box.com...) (ResponseTimeout) failed +copy Documents/Søgeordsliste.txt (checking box.com...) ok +copy Documents/catillo guide.odt (checking box.com...) ok +copy Documents/guide bruger oprettelse.odt (checking box.com...) (failed to read https://www.box.com/dav/work-annex/49e/175/GPGHMACSHA1--2b47737f8de7faac7704eaa322785edad63a921c/GPGHMACSHA1--2b47737f8de7faac7704eaa322785edad63a921c.chunkcount) failed +copy Documents/guide skift backup bånd.odt (checking box.com...) ok +copy Documents/lilletest.csv (checking box.com...) (failed to read https://www.box.com/dav/work-annex/915/373/GPGHMACSHA1--49ba3d5f63c012ae2cd2c0fc3e729178b1023c33/GPGHMACSHA1--49ba3d5f63c012ae2cd2c0fc3e729178b1023c33.chunkcount) failed +copy Dropbox/adams-scraper/CommonFunctions.py (checking box.com...) (ResponseTimeout) failed +copy Dropbox/adams-scraper/__pycache__/CommonFunctions.cpython-33.pyc (checking box.com...) (to box.com...) [2013-09-11 09:28:30 CEST] chat: gpg ["--batch","--no-tty","--use-agent","--quiet","--trust-model","always","--batch","--passphrase-fd","14","--symmetric","--force-mdc"] + +100% 0.0 B/s 0sResponseTimeout +ResponseTimeout +failed + + +# End of transcript or log. +"""]] + +More to come(full log) From 8a18236df8666aaa9bbc366a70e95381110f76a1 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawmkBwMWvNKZZCge_YqobCSILPMeK6xbFw8" Date: Wed, 11 Sep 2013 07:38:52 +0000 Subject: [PATCH 56/62] --- doc/bugs/box.com_never_stops_syncing..mdwn | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/bugs/box.com_never_stops_syncing..mdwn b/doc/bugs/box.com_never_stops_syncing..mdwn index 0346816ec2..42b2eaf1aa 100644 --- a/doc/bugs/box.com_never_stops_syncing..mdwn +++ b/doc/bugs/box.com_never_stops_syncing..mdwn @@ -11,6 +11,8 @@ git-annex version: 4.20130827 But i have never seen it work satisfactory in any version. +Also, i have seen this is 10+ different clean git-annexes. So it isn't annex specific. + ### Please provide any additional information below. I am going to add more debug to this bug constantly. (I intend to do a full 'git annex copy --to box.com --not --in box.com' daily, and see if the same files are transfered again and again) From 915dd2bdb212a63618aaf4e21b8ad8e5f7b08148 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawmkBwMWvNKZZCge_YqobCSILPMeK6xbFw8" Date: Wed, 11 Sep 2013 07:42:29 +0000 Subject: [PATCH 57/62] --- doc/users/tobiastheviking.mdwn | 12 ++++++++++++ 1 file changed, 12 insertions(+) create mode 100644 doc/users/tobiastheviking.mdwn diff --git a/doc/users/tobiastheviking.mdwn b/doc/users/tobiastheviking.mdwn new file mode 100644 index 0000000000..fd0f7e8015 --- /dev/null +++ b/doc/users/tobiastheviking.mdwn @@ -0,0 +1,12 @@ +Tobias Ussing + +See: + +https://github.com/TobiasTheViking/flickrannex/ +https://github.com/TobiasTheViking/imapannex +https://github.com/TobiasTheViking/dropboxannex +https://github.com/TobiasTheViking/skydriveannex +https://github.com/TobiasTheViking/googledriveannex +https://github.com/TobiasTheViking/owncloudannex +https://github.com/TobiasTheViking/megaannex +http://git-annex.branchable.com/forum/nntp__47__usenet_special_remote/ From 8fff54d569c4fc2e5d89b978907578743ca08a56 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawmkBwMWvNKZZCge_YqobCSILPMeK6xbFw8" Date: Wed, 11 Sep 2013 07:43:49 +0000 Subject: [PATCH 58/62] --- doc/users/tobiastheviking.mdwn | 24 ++++++++++++++++-------- 1 file changed, 16 insertions(+), 8 deletions(-) diff --git a/doc/users/tobiastheviking.mdwn b/doc/users/tobiastheviking.mdwn index fd0f7e8015..0629e34a9a 100644 --- a/doc/users/tobiastheviking.mdwn +++ b/doc/users/tobiastheviking.mdwn @@ -2,11 +2,19 @@ Tobias Ussing See: -https://github.com/TobiasTheViking/flickrannex/ -https://github.com/TobiasTheViking/imapannex -https://github.com/TobiasTheViking/dropboxannex -https://github.com/TobiasTheViking/skydriveannex -https://github.com/TobiasTheViking/googledriveannex -https://github.com/TobiasTheViking/owncloudannex -https://github.com/TobiasTheViking/megaannex -http://git-annex.branchable.com/forum/nntp__47__usenet_special_remote/ +[[https://github.com/TobiasTheViking/flickrannex/]] + +[[https://github.com/TobiasTheViking/imapannex]] + +[[https://github.com/TobiasTheViking/dropboxannex]] + +[[https://github.com/TobiasTheViking/skydriveannex]] + +[[https://github.com/TobiasTheViking/googledriveannex]] + +[[https://github.com/TobiasTheViking/owncloudannex]] + +[[https://github.com/TobiasTheViking/megaannex]] + +[[http://git-annex.branchable.com/forum/nntp__47__usenet_special_remote/]] + From fe61003a89404ea045809eff48a7720a4479cd91 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawmkBwMWvNKZZCge_YqobCSILPMeK6xbFw8" Date: Wed, 11 Sep 2013 07:46:08 +0000 Subject: [PATCH 59/62] Added a comment --- ...ment_2_48736ed17c98ffcfb13ec00b901b2dd6._comment | 13 +++++++++++++ 1 file changed, 13 insertions(+) create mode 100644 doc/forum/nntp__47__usenet_special_remote/comment_2_48736ed17c98ffcfb13ec00b901b2dd6._comment diff --git a/doc/forum/nntp__47__usenet_special_remote/comment_2_48736ed17c98ffcfb13ec00b901b2dd6._comment b/doc/forum/nntp__47__usenet_special_remote/comment_2_48736ed17c98ffcfb13ec00b901b2dd6._comment new file mode 100644 index 0000000000..90b264f7b6 --- /dev/null +++ b/doc/forum/nntp__47__usenet_special_remote/comment_2_48736ed17c98ffcfb13ec00b901b2dd6._comment @@ -0,0 +1,13 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawmkBwMWvNKZZCge_YqobCSILPMeK6xbFw8" + nickname="develop" + subject="comment 2" + date="2013-09-11T07:46:07Z" + content=""" +Only played a tiny bit with this since the initial post. + +I've decided that sql backend should be dropped entirely. Mostly the SQL backend was there because the code does something else entirely, and this was just a test. + +But since there has been no interest it will probably take ages before i actually get this finished :) + +"""]] From e9d265c51b699a0d6d6ef7d9d967c621c88b167d Mon Sep 17 00:00:00 2001 From: "http://hamf.myopenid.com/" Date: Wed, 11 Sep 2013 11:52:22 +0000 Subject: [PATCH 60/62] --- ...git-annex_on_archlinuxarm__44___armv6.mdwn | 20 +++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100644 doc/forum/git-annex_on_archlinuxarm__44___armv6.mdwn diff --git a/doc/forum/git-annex_on_archlinuxarm__44___armv6.mdwn b/doc/forum/git-annex_on_archlinuxarm__44___armv6.mdwn new file mode 100644 index 0000000000..edfd0ad4eb --- /dev/null +++ b/doc/forum/git-annex_on_archlinuxarm__44___armv6.mdwn @@ -0,0 +1,20 @@ +Hi! + +I can't get git-annex to run on my archlinuxARM sytem. It fails with + + $ git annex test + error: git-annex died of signal 11 + + Its a RaspberryPi (model B I think) so its an ARM11 (armv6h if I'm not mistaken). I use archlinuxarm because I am familiar with the distribution and you can have a very lean system. There is a very crude package for git-annex arm and armv7 that is build like this: https://github.com/archlinuxarm/PKGBUILDs/blob/master/alarm/git-annex/PKGBUILD . Crude, because it just takes the debian package, changes the version of libraries where apropriate and packs it as archlinux package. Two problems: 1) It's not been tested on armv6. 2) The 4.20130417 version does not exist anymore on the debian mirrors. + +1. The debian package is not marked as armv5 or armv7 specific and is runs on the RaspberryPi with Raspbian (but in an pretty old version if I remember correctly). So I would have imagined it runs on armv6h in general.\\ +2. I just did the same magic as in the linked PKGBUILD with a more current version to build the package but it still crashes. Can I get the old version anywhere to check if it works with that? + + +Cheers, + Hannes + +Background for whoever's interested :) +I tried to set up a RaspberryPi to manage my files, contacts and calendars. That shouts out loud for owncloud because of its convenience (caldav, carddav, webdav and a nice interface bundled together). However, compared to git-annex owncloud is unbearably slow (on an RaspberryPi) and lacks git-annex's easy way to keep your files sorted even if you don't have them on your local disk. + +For these reasons I tried to install a hybrid git-annex/owncloud system. There, basically, the files are served by owncloud but at the same time also managed (and served) by git-annex (+ssh). As far as I can see that should not have any noticeable side effects in my use case (mostly single user). I have some external hard drives connected to the RaspberryPi of which I will just symlink the appropriate folders to the owncloud 'files' folder. This way I can also just take the drives with me and still have the same (UUID) git annex repository on them. Any comments? From 67f118169495bc7df80c00ab299b4350a6dc972c Mon Sep 17 00:00:00 2001 From: "http://droggl.myopenid.com/" Date: Wed, 11 Sep 2013 11:55:27 +0000 Subject: [PATCH 61/62] Added a comment --- ...nt_1_7a553ad559519f3b3e3cd20115b4c44e._comment | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 doc/design/assistant/blog/day_18__merging/comment_1_7a553ad559519f3b3e3cd20115b4c44e._comment diff --git a/doc/design/assistant/blog/day_18__merging/comment_1_7a553ad559519f3b3e3cd20115b4c44e._comment b/doc/design/assistant/blog/day_18__merging/comment_1_7a553ad559519f3b3e3cd20115b4c44e._comment new file mode 100644 index 0000000000..94261f81eb --- /dev/null +++ b/doc/design/assistant/blog/day_18__merging/comment_1_7a553ad559519f3b3e3cd20115b4c44e._comment @@ -0,0 +1,15 @@ +[[!comment format=mdwn + username="http://droggl.myopenid.com/" + ip="2001:638:602:1181:a6ba:dbff:fedd:8041" + subject="comment 1" + date="2013-09-11T11:55:27Z" + content=""" +Nice! +However I didnt find any documentation yet on how to actually resolve the situation as a user (sorry if I'm being blind). +To be more specific: Say a conflict occurs on foo.txt and i now have two files foo.variant-dead.txt and foo.variant-beef.txt. +Now what? Say I used my favorite merge diffmerge tool to create a merged version foo-merged.txt, should I rename that to foo.txt? +Should I delete the variant files? Or drop them? Or would that lead to another merge for systems that still have the old foo.txt? +(You see I'm kind of confused of what happens here ;)) + +TIA for any helpers +"""]] From a20b460fdd8fb15279f58ce4d03070161c2cc008 Mon Sep 17 00:00:00 2001 From: "https://id.koumbit.net/anarcat" Date: Wed, 11 Sep 2013 15:13:20 +0000 Subject: [PATCH 62/62] status update: code has been reviewed, needs merge --- doc/todo/nicer_whereis_output.mdwn | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/doc/todo/nicer_whereis_output.mdwn b/doc/todo/nicer_whereis_output.mdwn index b6352d737e..79c4ba02ce 100644 --- a/doc/todo/nicer_whereis_output.mdwn +++ b/doc/todo/nicer_whereis_output.mdwn @@ -38,12 +38,14 @@ Fourth version: tuning and blocked This required more advanced parsing of the remotes, and instead of starting to do some JSON parsing, [[anarcat]] figured it was time to learn some Haskell instead. -Current status: needs review -============================ +Current status: needs merge +=========================== So right now, the most recent version of the python script is in [anarcat's gist](https://gist.github.com/anarcat/6502988) and works reasonably well. However, it doesn't distinguish between trusted and untrusted repos and so on. -Furthermore, we'd like to see this factored into the `whereis` command directly. A [raw.hs](http://codepad.org/miVJb5oK) file has been programmed by `mastensg`, and is now available in the above gist. It fits the desired output and prototypes. +Furthermore, we'd like to see this factored into the `whereis` command directly. A [raw.hs](http://codepad.org/miVJb5oK) file has been programmed by `mastensg`, and is now available in the above gist. It fits the desired output and prototypes, and has been `haskellized` thanks to [[guilhem]]. + +Now we just need to merge those marvelous functions in `Whereis.hs` - but I can't quite figure out where to throw that code, so I'll leave it to someone more familiar with the internals of git-annex. The most recent version is still in [anarcat's gist](https://gist.github.com/anarcat/6502988). --[[anarcat]] Desired output --------------