From 192120ffb79d72f4c717b56c2e43752190c20824 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawktbkKjilg70XC9XBFpIgVhtfLYH-0UMHY" Date: Mon, 27 Oct 2014 21:27:49 +0000 Subject: [PATCH 01/66] --- doc/forum/Using_git-annex.mdwn | 41 ++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) create mode 100644 doc/forum/Using_git-annex.mdwn diff --git a/doc/forum/Using_git-annex.mdwn b/doc/forum/Using_git-annex.mdwn new file mode 100644 index 0000000000..e9f49adc5a --- /dev/null +++ b/doc/forum/Using_git-annex.mdwn @@ -0,0 +1,41 @@ +Hi, + +I have created a git annex repo, added data. I then went to check it out in another location in the following way (my goal is to checkout origin, add a test file, push it back to origin). + git clone ../test_repo/ + 282 17:19 cd test_repo/ + 283 17:19 ls + 284 17:19 git status + 285 17:22 git annex init DEV + 286 17:22 touch test.txt + 287 17:22 vi test.txt + 288 17:22 git annex merge + 289 17:22 git annex add test.txt + 290 17:22 git commit -am "test" + 291 17:23 git push origin master git-annex + +However I am getting the following error + +Counting objects: 3, done. +Delta compression using up to 48 threads. +Compressing objects: 100% (3/3), done. +Writing objects: 100% (3/3), 364 bytes | 0 bytes/s, done. +Total 3 (delta 1), reused 0 (delta 0) +remote: error: refusing to update checked out branch: refs/heads/master +remote: error: By default, updating the current branch in a non-bare repository +remote: error: is denied, because it will make the index and work tree inconsistent +remote: error: with what you pushed, and will require 'git reset --hard' to match +remote: error: the work tree to HEAD. +remote: error: +remote: error: You can set 'receive.denyCurrentBranch' configuration variable to +remote: error: 'ignore' or 'warn' in the remote repository to allow pushing into +remote: error: its current branch; however, this is not recommended unless you +remote: error: arranged to update its work tree to match what you pushed in some +remote: error: other way. +remote: error: +remote: error: To squelch this message and still keep the default behaviour, set +remote: error: 'receive.denyCurrentBranch' configuration variable to 'refuse'. +To /test_repo/ + ! [remote rejected] master -> master (branch is currently checked out) +error: failed to push some refs to '/test_repo/' + +What am I missing? From 287e961e08f50c26fc07ecebefa1050af4f5a6f8 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawktbkKjilg70XC9XBFpIgVhtfLYH-0UMHY" Date: Mon, 27 Oct 2014 21:31:40 +0000 Subject: [PATCH 02/66] --- doc/forum/Using_git-annex.mdwn | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/doc/forum/Using_git-annex.mdwn b/doc/forum/Using_git-annex.mdwn index e9f49adc5a..0e2be4e81b 100644 --- a/doc/forum/Using_git-annex.mdwn +++ b/doc/forum/Using_git-annex.mdwn @@ -2,40 +2,72 @@ Hi, I have created a git annex repo, added data. I then went to check it out in another location in the following way (my goal is to checkout origin, add a test file, push it back to origin). git clone ../test_repo/ + 282 17:19 cd test_repo/ + 283 17:19 ls + 284 17:19 git status + 285 17:22 git annex init DEV + 286 17:22 touch test.txt + 287 17:22 vi test.txt + 288 17:22 git annex merge + 289 17:22 git annex add test.txt + 290 17:22 git commit -am "test" + 291 17:23 git push origin master git-annex However I am getting the following error Counting objects: 3, done. + Delta compression using up to 48 threads. + Compressing objects: 100% (3/3), done. + Writing objects: 100% (3/3), 364 bytes | 0 bytes/s, done. + Total 3 (delta 1), reused 0 (delta 0) + remote: error: refusing to update checked out branch: refs/heads/master + remote: error: By default, updating the current branch in a non-bare repository + remote: error: is denied, because it will make the index and work tree inconsistent + remote: error: with what you pushed, and will require 'git reset --hard' to match + remote: error: the work tree to HEAD. + remote: error: + remote: error: You can set 'receive.denyCurrentBranch' configuration variable to + remote: error: 'ignore' or 'warn' in the remote repository to allow pushing into + remote: error: its current branch; however, this is not recommended unless you + remote: error: arranged to update its work tree to match what you pushed in some + remote: error: other way. + remote: error: + remote: error: To squelch this message and still keep the default behaviour, set + remote: error: 'receive.denyCurrentBranch' configuration variable to 'refuse'. + To /test_repo/ + ! [remote rejected] master -> master (branch is currently checked out) + error: failed to push some refs to '/test_repo/' + What am I missing? From f586507026c31384ad2075c2ce2a6eb967d2d19b Mon Sep 17 00:00:00 2001 From: aardo Date: Tue, 28 Oct 2014 08:12:02 +0000 Subject: [PATCH 03/66] --- doc/forum/Deduplication_in_direct_mode.mdwn | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) create mode 100644 doc/forum/Deduplication_in_direct_mode.mdwn diff --git a/doc/forum/Deduplication_in_direct_mode.mdwn b/doc/forum/Deduplication_in_direct_mode.mdwn new file mode 100644 index 0000000000..076d0ab5a2 --- /dev/null +++ b/doc/forum/Deduplication_in_direct_mode.mdwn @@ -0,0 +1,18 @@ +Hi, + +I'm using git-annex across a number of (indirect) repositories, making heavy use of deduplication for organizing files according to various different aspects. + +Now I want to keep part of the files also on a VFAT device, which doesn't let me use indirect mode. In direct mode, however, git-annex "get" or "copy" places a separate copy of each file in the repository, whereas in indirect mode, it would just keep a single copy and maintain a number of (inexpensive) symbolic links. Since space on the VFAT drive is limited, I would like to just keep one, specific copy, not caring about the others. If I "drop" an unneeded copy of the file, it also gets replaced by the ASCII "link" in all other places that contained the same file. Therefore, I can either have multiple copies of the same data or none at all. + +Imagine you have a bunch of photos sorted into a directories in meant to make it easy to find them (same file name means same file content): + +./photo1.jpg +./photo2.jpg +./by-date/2014-10-27/photo1.jpg +./by-date/2014-10-28/photo2.jpg +./by-event/holiday-by-the-sea/photo1.jpg +./by-event/her-birthday/photo2.jpg + +I want to keep a copy of ./photo?.jpg in the VFAT repository, but not the other (identical) files. How do I do that? Or is there really no way of doing this? + +Thanks. From 31a1bb31bfb38c239277decce439e5926745d731 Mon Sep 17 00:00:00 2001 From: "http://alerque.com/" Date: Tue, 28 Oct 2014 10:58:38 +0000 Subject: [PATCH 04/66] the plain git-annex package does not work ATM, has a 0 release number, and is not the recomended install method on Arch, use git-annex-bin unless you know what you are doing (I'm the maintainer of both packages) --- doc/install.mdwn | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/install.mdwn b/doc/install.mdwn index 478044593b..618f32dfaf 100644 --- a/doc/install.mdwn +++ b/doc/install.mdwn @@ -10,7 +10,7 @@ detailed instructions | quick install   [[Ubuntu]] | `apt-get install git-annex`   [[Fedora]] | `yum install git-annex`   [[FreeBSD]] | `pkg_add -r hs-git-annex` -  [[ArchLinux]] | `yaourt -Sy git-annex` +  [[ArchLinux]] | `yaourt -Sy git-annex-bin`   [[NixOS]] | `nix-env -i git-annex`   [[Gentoo]] | `emerge git-annex`   [[ScientificLinux5]] | From c6df013e6602232113b66ffa533faab9473ee8da Mon Sep 17 00:00:00 2001 From: "http://alerque.com/" Date: Tue, 28 Oct 2014 10:59:27 +0000 Subject: [PATCH 05/66] removed --- ...comment_5_cbc960cd78bf5b90e3bb6cb605d6d970._comment | 10 ---------- 1 file changed, 10 deletions(-) delete mode 100644 doc/install/ArchLinux/comment_5_cbc960cd78bf5b90e3bb6cb605d6d970._comment diff --git a/doc/install/ArchLinux/comment_5_cbc960cd78bf5b90e3bb6cb605d6d970._comment b/doc/install/ArchLinux/comment_5_cbc960cd78bf5b90e3bb6cb605d6d970._comment deleted file mode 100644 index b3ed0846b6..0000000000 --- a/doc/install/ArchLinux/comment_5_cbc960cd78bf5b90e3bb6cb605d6d970._comment +++ /dev/null @@ -1,10 +0,0 @@ -[[!comment format=mdwn - username="http://alerque.com/" - nickname="Caleb" - subject="Out of date" - date="2013-12-28T22:04:24Z" - content=""" -The AUR package you reference is woefully out of date. I have updated the standalone variant so it might be worth using that until the maintainer catches up. - - yaourt -Sy git-annex-standalone -"""]] From b8c33a4caa0d95f8674d1b459eb09561fbd8c18f Mon Sep 17 00:00:00 2001 From: "http://alerque.com/" Date: Tue, 28 Oct 2014 10:59:44 +0000 Subject: [PATCH 06/66] removed --- .../comment_1_da5919c986d2ae187bc2f73de9633978._comment | 8 -------- 1 file changed, 8 deletions(-) delete mode 100644 doc/install/ArchLinux/comment_1_da5919c986d2ae187bc2f73de9633978._comment diff --git a/doc/install/ArchLinux/comment_1_da5919c986d2ae187bc2f73de9633978._comment b/doc/install/ArchLinux/comment_1_da5919c986d2ae187bc2f73de9633978._comment deleted file mode 100644 index d4db232922..0000000000 --- a/doc/install/ArchLinux/comment_1_da5919c986d2ae187bc2f73de9633978._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="https://www.google.com/accounts/o8/id?id=AItOawlwYMdU0H7P7MMlD0v_BcczO-ZkYHY4zuY" - nickname="Morris" - subject="Arch Linux" - date="2012-10-17T13:21:24Z" - content=""" -For Arch Linux there should be the AUR package [git-annex-bin](https://aur.archlinux.org/packages.php?ID=63503) mentioned, because it's easier to install (no haskell dependencies to be installed) and is based on the prebuild linux binary tarball. -"""]] From 96434881069d4a285a7b356ffcf2300570a48358 Mon Sep 17 00:00:00 2001 From: "http://alerque.com/" Date: Tue, 28 Oct 2014 11:14:58 +0000 Subject: [PATCH 07/66] update notes on install methods --- doc/install/ArchLinux.mdwn | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/doc/install/ArchLinux.mdwn b/doc/install/ArchLinux.mdwn index 3dd7271c44..43b7d132e7 100644 --- a/doc/install/ArchLinux.mdwn +++ b/doc/install/ArchLinux.mdwn @@ -1,10 +1,10 @@ -There are three non non-official packages for git-annex in the Archlinux User Repository. Any of these may be installed manually per [AUR guidelines](https://wiki.archlinux.org/index.php/AUR_User_Guidelines#Installing_packages) or using a wrapper such as [`yaourt`](https://wiki.archlinux.org/index.php/yaourt) shown below. +There are four non non-official packages for git-annex in the Arch Linux User Repository. Any of these may be installed manually per [AUR guidelines](https://wiki.archlinux.org/index.php/AUR_User_Guidelines#Installing_packages) or using a wrapper such as [`yaourt`](https://wiki.archlinux.org/index.php/yaourt) shown below. 1. The simplest method is to use the [git-annex-bin](https://aur.archlinux.org/packages/git-annex-bin/) package based on the [prebuilt Linux tarballs](http://downloads.kitenet.net/git-annex/linux/current/). This package includes many of the binary shims from the pre-built package. Although common Linux system utilities have been stripped in favor of normal dependencies, the pre-configured Haskell libraries included out of the box make this an easy install. The disadvantage is the resulting installation is a bit on the heavy side at nearly 100M. $ yaourt -Sy git-annex-bin -2. A more traditional source package is available at [git-annex](https://aur.archlinux.org/packages/git-annex/). This depends on a large number of Haskell packages available from a third party repository or through Cabal. This has been historically a bit problematic and the package frequently sits flagged out of date. The state of dependencies also varies, so some intervention may be required to get this option to work. +2. A more traditional source package is available at [git-annex](https://aur.archlinux.org/packages/git-annex/). This depends on a large number of Haskell packages available from a third party repository or through Cabal. You must either enable a 3rd party repo that has the dependencies or have a working Cabal installation. Unless you know what you are doing this is a bit problematic and some intervention may be required to get this option to work. The state of available dependency versions also varies so this may not work at all times. $ yaourt -Sy git-annex @@ -12,7 +12,11 @@ There are three non non-official packages for git-annex in the Archlinux User Re $ yaourt -Sy git-annex-git -Finally you may choose to forgo the Archlinux package system and install git-annex directly through cabal. +4. A Cabal sandbox build is also available + + $ yaourt -Sy git-annex-cabal + +Finally you may choose to forgo the Arch Linux package system entirely and install git-annex directly through cabal. $ pacman -S git rsync curl wget gnupg openssh cabal-install $ cabal update From dcaf89da2fc544631ff4214b136f078d31c089f7 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 28 Oct 2014 13:06:24 -0400 Subject: [PATCH 08/66] aws library now supports multipart; initial design --- ..._74b2a392a537dde1c28089f1deed940c._comment | 31 +++++++++++++++++++ 1 file changed, 31 insertions(+) create mode 100644 doc/bugs/S3_upload_not_using_multipart/comment_9_74b2a392a537dde1c28089f1deed940c._comment diff --git a/doc/bugs/S3_upload_not_using_multipart/comment_9_74b2a392a537dde1c28089f1deed940c._comment b/doc/bugs/S3_upload_not_using_multipart/comment_9_74b2a392a537dde1c28089f1deed940c._comment new file mode 100644 index 0000000000..b965ff0ab1 --- /dev/null +++ b/doc/bugs/S3_upload_not_using_multipart/comment_9_74b2a392a537dde1c28089f1deed940c._comment @@ -0,0 +1,31 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 9""" + date="2014-10-28T16:42:21Z" + content=""" +The aws library now supports multipart uploads, using its +S3.Commands.Multipart module. + +I don't think that multipart and chunking fit together: Typically the +chunks are too small to need multipart for individual chunks. And the +chunks shouldn't be combined together into a complete object at the end (at +least not if we care about using chunking to obscure object size). +Individual chunks sizes can vary when encryption is used, so combining them +all into one file wouldn't work. + +Also, multipart uploads require at least 3 http calls, so there's no point +using it for small objects, as it would only add overhead. + +So, multipart uploads should be used when not chunking, when the object to +upload exceeds some size, which should probably defaut to something in the +range of 100 mb to 1 gb. + +It might be possible to support resuming of interrupted multipart uploads. +It seems that git-annex would need to store, locally, the UploadId, +as well as the list of uploaded parts, including the Etag for the upload +(which is needed when completing the multipart upload too). + +Also it should probably set Expires when initiating the multipart upload, +so that incomplete ones get cleaned up after some period of time. +Otherwise, users would probably be billed for them. +"""]] From 6246514ef545dfb666bf339567544147059c4f01 Mon Sep 17 00:00:00 2001 From: poelzi Date: Tue, 28 Oct 2014 18:23:27 +0000 Subject: [PATCH 09/66] --- ...th_sync_error:_syntax_or_usage_error_.mdwn | 85 +++++++++++++++++++ 1 file changed, 85 insertions(+) create mode 100644 doc/bugs/rsync_fails_with_sync_error:_syntax_or_usage_error_.mdwn diff --git a/doc/bugs/rsync_fails_with_sync_error:_syntax_or_usage_error_.mdwn b/doc/bugs/rsync_fails_with_sync_error:_syntax_or_usage_error_.mdwn new file mode 100644 index 0000000000..09c79d78dd --- /dev/null +++ b/doc/bugs/rsync_fails_with_sync_error:_syntax_or_usage_error_.mdwn @@ -0,0 +1,85 @@ +### Please describe the problem. + +i tried to sync my home music annex to a server. Half the files worked nicely, but on the rest it fails with rsync errors. + +### What steps will reproduce the problem? + +git annex -d sync --content + +### What version of git-annex are you using? On what operating system? + +Client: + +ubuntu trusty +Version: 5.20140517.4 + +Server: + +ubuntu trusty +Version: 5.20140412ubuntu1 + + +### Please provide any additional information below. + +[[!format sh """ +dirus-dom:/music$ git annex -d sync --content +[2014-10-28 19:18:28 CET] read: git ["--git-dir=/exports/music/.git","--work-tree=/exports/music","show-ref","git-annex"] +[2014-10-28 19:18:28 CET] read: git ["--git-dir=/exports/music/.git","--work-tree=/exports/music","show-ref","--hash","refs/heads/git-annex"] +[2014-10-28 19:18:29 CET] read: git ["--git-dir=/exports/music/.git","--work-tree=/exports/music","log","refs/heads/git-annex..1599d29eba7a0ec50217c2b4a3f4cc1ecc8b2116","--oneline","-n1"] +[2014-10-28 19:18:29 CET] read: git ["--git-dir=/exports/music/.git","--work-tree=/exports/music","log","refs/heads/git-annex..27b47301dcb5007705d1dcd5a414df964b840467","--oneline","-n1"] +[2014-10-28 19:18:29 CET] read: git ["--git-dir=/exports/music/.git","--work-tree=/exports/music","log","refs/heads/git-annex..a95ca0282fefcd774ec8a00b6f33c11f08f789d8","--oneline","-n1"] +[2014-10-28 19:18:29 CET] chat: git ["--git-dir=/exports/music/.git","--work-tree=/exports/music","cat-file","--batch"] +commit [2014-10-28 19:18:29 CET] read: git ["--git-dir=/exports/music/.git","--work-tree=/exports/music","commit","-a","-m","git-annex automatic sync"] +ok +[2014-10-28 19:18:31 CET] read: git ["--git-dir=/exports/music/.git","--work-tree=/exports/music","symbolic-ref","HEAD"] +[2014-10-28 19:18:31 CET] read: git ["--git-dir=/exports/music/.git","--work-tree=/exports/music","show-ref","refs/heads/master"] +[2014-10-28 19:18:31 CET] call: git ["--git-dir=/exports/music/.git","--work-tree=/exports/music","show-ref","--verify","-q","refs/heads/synced/master"] +[2014-10-28 19:18:31 CET] read: git ["--git-dir=/exports/music/.git","--work-tree=/exports/music","log","refs/heads/master..refs/heads/synced/master","--oneline","-n1"] +pull sync.poelzi.org__music +[2014-10-28 19:18:31 CET] read: ssh ["-O","stop","-S","poelzigit@sync.poelzi.org","-o","ControlMaster=auto","-o","ControlPersist=yes","localhost"] +[2014-10-28 19:18:31 CET] call: git ["--git-dir=/exports/music/.git","--work-tree=/exports/music","fetch","sync.poelzi.org__music"] +[2014-10-28 19:18:33 CET] call: git ["--git-dir=/exports/music/.git","--work-tree=/exports/music","show-ref","--verify","-q","refs/remotes/sync.poelzi.org__music/master"] +[2014-10-28 19:18:33 CET] read: git ["--git-dir=/exports/music/.git","--work-tree=/exports/music","log","refs/heads/master..refs/remotes/sync.poelzi.org__music/master","--oneline","-n1"] +[2014-10-28 19:18:33 CET] call: git ["--git-dir=/exports/music/.git","--work-tree=/exports/music","show-ref","--verify","-q","refs/remotes/sync.poelzi.org__music/synced/master"] +[2014-10-28 19:18:33 CET] read: git ["--git-dir=/exports/music/.git","--work-tree=/exports/music","log","refs/heads/synced/master..refs/remotes/sync.poelzi.org__music/synced/master","--oneline","-n1"] +ok +[2014-10-28 19:18:33 CET] read: git ["--git-dir=/exports/music/.git","--work-tree=/exports/music","show-ref","git-annex"] +[2014-10-28 19:18:33 CET] read: git ["--git-dir=/exports/music/.git","--work-tree=/exports/music","show-ref","--hash","refs/heads/git-annex"] +[2014-10-28 19:18:33 CET] read: git ["--git-dir=/exports/music/.git","--work-tree=/exports/music","log","refs/heads/git-annex..1599d29eba7a0ec50217c2b4a3f4cc1ecc8b2116","--oneline","-n1"] +[2014-10-28 19:18:33 CET] read: git ["--git-dir=/exports/music/.git","--work-tree=/exports/music","log","refs/heads/git-annex..27b47301dcb5007705d1dcd5a414df964b840467","--oneline","-n1"] +[2014-10-28 19:18:33 CET] read: git ["--git-dir=/exports/music/.git","--work-tree=/exports/music","log","refs/heads/git-annex..a95ca0282fefcd774ec8a00b6f33c11f08f789d8","--oneline","-n1"] +[2014-10-28 19:18:33 CET] read: git ["--git-dir=/exports/music/.git","--work-tree=/exports/music","ls-files","--cached","-z","--"] +[2014-10-28 19:18:33 CET] chat: git ["--git-dir=/exports/music/.git","--work-tree=/exports/music","check-attr","-z","--stdin","annex.backend","annex.numcopies","--"] +copy Alan Parsons Project/Eye In The Sky/.07 - Psychobabble.mood copy Alan Parsons Project/Eye In The Sky/.07 - Psychobabble.mood (checking sync.poelzi.org__music...) [2014-10-28 19:18:43 CET] call: ssh ["-S",".git/annex/ssh/poelzigit@sync.poelzi.org","-o","ControlMaster=auto","-o","ControlPersist=yes","-T","poelzigit@sync.poelzi.org","git-annex-shell 'inannex' '/music/' 'SHA256E-s3000--da8a3336a484a171a438c99660260cc35cbd37c339dd2c18447cd025064bc00b.mood' --uuid 35a89672-4ff5-4d9a-9bf2-cedb272bb7cb"] +(to sync.poelzi.org__music...) +[2014-10-28 19:18:43 CET] read: rsync ["--progress","--inplace","--perms","-e","'ssh' '-S' '.git/annex/ssh/poelzigit@sync.poelzi.org' '-o' 'ControlMaster=auto' '-o' 'ControlPersist=yes' '-T' 'poelzigit@sync.poelzi.org' 'git-annex-shell ''recvkey'' ''/music/'' ''SHA256E-s3000--da8a3336a484a171a438c99660260cc35cbd37c339dd2c18447cd025064bc00b.mood'' --uuid 35a89672-4ff5-4d9a-9bf2-cedb272bb7cb ''--'' ''remoteuuid=97a3cd71-ee6c-4437-8740-253cde0d32ae'' ''direct='' ''associatedfile=Alan Parsons Project/Eye In The Sky/.07 - Psychobabble.mood'' ''--'''","--","/exports/music/.git/annex/objects/20/Z4/SHA256E-s3000--da8a3336a484a171a438c99660260cc35cbd37c339dd2c18447cd025064bc00b.mood/SHA256E-s3000--da8a3336a484a171a438c99660260cc35cbd37c339dd2c18447cd025064bc00b.mood","dummy:"] +rsync error: syntax or usage error (code 1) at main.c(1183) [sender=3.1.1] + + rsync failed -- run git annex again to resume file transfer +failed +copy Alessandro Scarlatti/Motets - Gérard Lesne, Veronique Gens (1993) [300]/06 - Infirmata, Vulnerata - VI Semper Gratus.ogg copy Alessandro Scarlatti/Motets - Gérard Lesne, Veronique Gens (1993) [300]/06 - Infirmata, Vulnerata - VI Semper Gratus.ogg (checking sync.poelzi.org__music...) [2014-10-28 19:18:48 CET] call: ssh ["-S",".git/annex/ssh/poelzigit@sync.poelzi.org","-o","ControlMaster=auto","-o","ControlPersist=yes","-T","poelzigit@sync.poelzi.org","git-annex-shell 'inannex' '/music/' 'SHA256E-s3847396--05c5498f08c727645ba84270cb8d82da69a3c9bede35520aa3128b938d003a3d.ogg' --uuid 35a89672-4ff5-4d9a-9bf2-cedb272bb7cb"] +(to sync.poelzi.org__music...) +[2014-10-28 19:18:48 CET] read: rsync ["--progress","--inplace","--perms","-e","'ssh' '-S' '.git/annex/ssh/poelzigit@sync.poelzi.org' '-o' 'ControlMaster=auto' '-o' 'ControlPersist=yes' '-T' 'poelzigit@sync.poelzi.org' 'git-annex-shell ''recvkey'' ''/music/'' ''SHA256E-s3847396--05c5498f08c727645ba84270cb8d82da69a3c9bede35520aa3128b938d003a3d.ogg'' --uuid 35a89672-4ff5-4d9a-9bf2-cedb272bb7cb ''--'' ''remoteuuid=97a3cd71-ee6c-4437-8740-253cde0d32ae'' ''direct='' ''associatedfile=Alessandro Scarlatti/Motets - G\233rard Lesne, Veronique Gens (1993) [300]/06 - Infirmata, Vulnerata - VI Semper Gratus.ogg'' ''--'''","--","/exports/music/.git/annex/objects/XJ/f9/SHA256E-s3847396--05c5498f08c727645ba84270cb8d82da69a3c9bede35520aa3128b938d003a3d.ogg/SHA256E-s3847396--05c5498f08c727645ba84270cb8d82da69a3c9bede35520aa3128b938d003a3d.ogg","dummy:"] +rsync error: syntax or usage error (code 1) at main.c(1183) [sender=3.1.1] + + rsync failed -- run git annex again to resume file transfer +failed +^C + + +Calling this through python gives: + +In [5]: subprocess.call(["rsync", "--debug=all", "--progress","--inplace","--perms","-e","'ssh' '-S' '.git/annex/ssh/poelzigit@sync.poelzi.org' '-o' 'ControlMaster=auto' '-o' 'ControlPersist=yes' '-T' 'poelzigit@sync.poelzi.org' 'git-annex-shell ''recvkey'' ''/music/'' ''SHA256E-s3000--da8a3336a484a171a438c99660260cc35cbd37c339dd2c18447cd025064bc00b.mood'' --uuid 35a89672-4ff5-4d9a-9bf2-cedb272bb7cb ''--'' ''remoteuuid=97a3cd71-ee6c-4437-8740-253cde0d32ae'' ''direct='' ''associatedfile=Alan Parsons Project/Eye In The Sky/.07 - Psychobabble.mood'' ''--'''","--","/exports/music/.git/annex/objects/20/Z4/SHA256E-s3000--da8a3336a484a171a438c99660260cc35cbd37c339dd2c18447cd025064bc00b.mood/SHA256E-s3000--da8a3336a484a171a438c99660260cc35cbd37c339dd2c18447cd025064bc00b.mood","dummy:"]) +opening connection using: ssh -S .git/annex/ssh/poelzigit@sync.poelzi.org -o ControlMaster=auto -o ControlPersist=yes -T poelzigit@sync.poelzi.org "git-annex-shell 'recvkey' '/music/' 'SHA256E-s3000--da8a3336a484a171a438c99660260cc35cbd37c339dd2c18447cd025064bc00b.mood' --uuid 35a89672-4ff5-4d9a-9bf2-cedb272bb7cb '--' 'remoteuuid=97a3cd71-ee6c-4437-8740-253cde0d32ae' 'direct=' 'associatedfile=Alan Parsons Project/Eye In The Sky/.07 - Psychobabble.mood' '--'" dummy rsync --server -pe.Lsfx --log-format=X --debug=ALL --inplace . . (19 args) +(Client) Protocol versions: remote=31, negotiated=31 +[sender] change_dir(/exports/music/.git/annex/objects/20/Z4/SHA256E-s3000--da8a3336a484a171a438c99660260cc35cbd37c339dd2c18447cd025064bc00b.mood) +send_files starting +send_files phase=1 +send_files phase=2 +send files finished +total: matches=0 hash_hits=0 false_alarms=0 data=0 +rsync error: syntax or usage error (code 1) at main.c(1183) [sender=3.1.1] +[sender] _exit_cleanup(code=1, file=main.c, line=1183): about to call exit(1) + + +"""]] From 2bcea7366ba2af2a3cff101e21855faf03bd54e7 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 28 Oct 2014 14:43:07 -0400 Subject: [PATCH 10/66] WIP branch --- .../comment_9_1f5578a9100f0f087a558e5e5968d753._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/bugs/S3_upload_not_using_multipart/comment_9_1f5578a9100f0f087a558e5e5968d753._comment diff --git a/doc/bugs/S3_upload_not_using_multipart/comment_9_1f5578a9100f0f087a558e5e5968d753._comment b/doc/bugs/S3_upload_not_using_multipart/comment_9_1f5578a9100f0f087a558e5e5968d753._comment new file mode 100644 index 0000000000..10cab3da9c --- /dev/null +++ b/doc/bugs/S3_upload_not_using_multipart/comment_9_1f5578a9100f0f087a558e5e5968d753._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 9""" + date="2014-10-28T18:25:04Z" + content=""" +I have a WIP branch `aws-s3-multipart`. I stopped when I got blocked +by a bad API in the aws library: +"""]] From 7f5b5b460df73e60b0f359f91c7499342202b689 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 28 Oct 2014 14:43:43 -0400 Subject: [PATCH 11/66] this TODO item is obsolete; S3 supports chunking and there's a different bug to support S3 multipart uploads --- doc/todo/support_S3_multipart_uploads.mdwn | 14 -------------- 1 file changed, 14 deletions(-) delete mode 100644 doc/todo/support_S3_multipart_uploads.mdwn diff --git a/doc/todo/support_S3_multipart_uploads.mdwn b/doc/todo/support_S3_multipart_uploads.mdwn deleted file mode 100644 index 711ac41b2a..0000000000 --- a/doc/todo/support_S3_multipart_uploads.mdwn +++ /dev/null @@ -1,14 +0,0 @@ -Did not know of this when I wrote S3 support. Ability to resume large -uploads would be good. - - - -Also allows supporting files > 5 gb, a S3 limit I was not aware of. - -NB: It would work just as well to split the object and upload the N parts -to S3, but not bother with S3's paperwork to rejoin them into one object. -Only reasons not to do that are a) backwards compatability with -the existing S3 remote and b) this would not allow accessing the content -in S3 w/o using git-annex, which could be useful in some scenarios. - ---[[Joey]] From 91188992573f7724970937cdca6e5d616262a6bd Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 28 Oct 2014 15:25:13 -0400 Subject: [PATCH 12/66] comment --- ...omment_1_a3e6fc93669ec76656c84793d1f59149._comment | 11 +++++++++++ 1 file changed, 11 insertions(+) create mode 100644 doc/forum/Recovering_deleted_file_in_direct_mode/comment_1_a3e6fc93669ec76656c84793d1f59149._comment diff --git a/doc/forum/Recovering_deleted_file_in_direct_mode/comment_1_a3e6fc93669ec76656c84793d1f59149._comment b/doc/forum/Recovering_deleted_file_in_direct_mode/comment_1_a3e6fc93669ec76656c84793d1f59149._comment new file mode 100644 index 0000000000..0b23bbc784 --- /dev/null +++ b/doc/forum/Recovering_deleted_file_in_direct_mode/comment_1_a3e6fc93669ec76656c84793d1f59149._comment @@ -0,0 +1,11 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 1""" + date="2014-10-28T19:22:54Z" + content=""" +> the probability of something going wrong during the conversion is quite high (see other bugs reported during import) + +I don't know what bugs you speak of. If the probability of something going +wrong is quite high, then you must have a reproducible test case. Submit a +bug with such a test case, and I can fix it. +"""]] From 886ba31c382240d02cb3f682b1d69cc3d514e407 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 28 Oct 2014 15:54:40 -0400 Subject: [PATCH 13/66] comment --- ...omment_1_6a598b2a0fc6c241aa051f663dca9842._comment | 11 +++++++++++ 1 file changed, 11 insertions(+) create mode 100644 doc/forum/Copy_contents_of_directory_to_rsync_remotes_possible__63__/comment_1_6a598b2a0fc6c241aa051f663dca9842._comment diff --git a/doc/forum/Copy_contents_of_directory_to_rsync_remotes_possible__63__/comment_1_6a598b2a0fc6c241aa051f663dca9842._comment b/doc/forum/Copy_contents_of_directory_to_rsync_remotes_possible__63__/comment_1_6a598b2a0fc6c241aa051f663dca9842._comment new file mode 100644 index 0000000000..6e4186622b --- /dev/null +++ b/doc/forum/Copy_contents_of_directory_to_rsync_remotes_possible__63__/comment_1_6a598b2a0fc6c241aa051f663dca9842._comment @@ -0,0 +1,11 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 1""" + date="2014-10-28T19:26:01Z" + content=""" +git-annex has to tell rsync to transfer the "24863" directory, +otherwise rsync would not create that directory, and would then +fail to create "24863/2fe" since its parent directory doesn't exist. + +Might be able to use use ACLS, I suppose, but they bring a lot of complexity. +"""]] From 39dd5a2ac3b2371a250667624a67e9272f4724fa Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 28 Oct 2014 15:54:44 -0400 Subject: [PATCH 14/66] improve uuid mismatch message --- Remote/GCrypt.hs | 2 +- ..._1bb55c9a35d9447a31237c8bfa24dba9._comment | 21 +++++++++++++++++++ 2 files changed, 22 insertions(+), 1 deletion(-) create mode 100644 doc/bugs/problems_with_enableremote_on_gcrypt_remote___40__hosted_with_gitolite__41__/comment_2_1bb55c9a35d9447a31237c8bfa24dba9._comment diff --git a/Remote/GCrypt.hs b/Remote/GCrypt.hs index 995c3e8389..9aa70d57e6 100644 --- a/Remote/GCrypt.hs +++ b/Remote/GCrypt.hs @@ -202,7 +202,7 @@ gCryptSetup mu _ c = go $ M.lookup "gitrepo" c method <- setupRepo gcryptid =<< inRepo (Git.Construct.fromRemoteLocation gitrepo) gitConfigSpecialRemote u c' "gcrypt" (fromAccessMethod method) return (c', u) - else error $ "uuid mismatch " ++ show (u, mu, gcryptid) + else error $ "uuid mismatch; expected " ++ show mu ++ " but remote gitrepo has " ++ show u ++ " (" ++ show gcryptid ++ ")" {- Sets up the gcrypt repository. The repository is either a local - repo, or it is accessed via rsync directly, or it is accessed over ssh diff --git a/doc/bugs/problems_with_enableremote_on_gcrypt_remote___40__hosted_with_gitolite__41__/comment_2_1bb55c9a35d9447a31237c8bfa24dba9._comment b/doc/bugs/problems_with_enableremote_on_gcrypt_remote___40__hosted_with_gitolite__41__/comment_2_1bb55c9a35d9447a31237c8bfa24dba9._comment new file mode 100644 index 0000000000..c26b0a390d --- /dev/null +++ b/doc/bugs/problems_with_enableremote_on_gcrypt_remote___40__hosted_with_gitolite__41__/comment_2_1bb55c9a35d9447a31237c8bfa24dba9._comment @@ -0,0 +1,21 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 2""" + date="2014-10-28T19:40:37Z" + content=""" +The uuid mismatch message tells me that you have tried to enable a special +remote that has uuid 984e0333-3327-5f21-87a1-35d30f37f337. However, when +it checked the gcrypt repository, it found that it had gcrypt id +":id:8sucFsBZIGQKXFv5ecSW", which means its uuid should be +78104a6f-16a9-504b-8e8a-d8a3c59351e8. I have improved that message to be +more clear. + +Since `git annex info` doesn't list that +78104a6f-16a9-504b-8e8a-d8a3c59351e8 uuid at all, it seems that +this gcrypt repository has not been initialized using `git-annex +initremote`. It's not a gcrypt special remote, but instead is a gcrypt +repository, that was probably created using `git push`. You can convert it +to a gcrypt special remote by running `git-annex initremote` and providing +its repourl. Once that's done and pushed, you will be able to `git annex +enableremote` it elsewhere. +"""]] From 5451273be6f2e4d8707acc8b5c75b962438eba72 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 28 Oct 2014 16:02:42 -0400 Subject: [PATCH 15/66] remove unused imports --- Assistant/DeleteRemote.hs | 1 - Assistant/WebApp/Gpg.hs | 1 - 2 files changed, 2 deletions(-) diff --git a/Assistant/DeleteRemote.hs b/Assistant/DeleteRemote.hs index 968b214c10..9b1f974ffe 100644 --- a/Assistant/DeleteRemote.hs +++ b/Assistant/DeleteRemote.hs @@ -17,7 +17,6 @@ import Logs.Location import Assistant.DaemonStatus import qualified Remote import Remote.List -import qualified Git.Remote import qualified Git.Remote.Remove import Logs.Trust import qualified Annex diff --git a/Assistant/WebApp/Gpg.hs b/Assistant/WebApp/Gpg.hs index fa20db2f4b..f487a693b6 100644 --- a/Assistant/WebApp/Gpg.hs +++ b/Assistant/WebApp/Gpg.hs @@ -13,7 +13,6 @@ import Assistant.WebApp.Common import Assistant.Gpg import Utility.Gpg import qualified Git.Command -import qualified Git.Remote import qualified Git.Remote.Remove import qualified Git.Construct import qualified Annex.Branch From c6f0a6583236fca8c4aaca075435f5ef3c459000 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 28 Oct 2014 16:03:45 -0400 Subject: [PATCH 16/66] close --- ...emote_on_gcrypt_remote___40__hosted_with_gitolite__41__.mdwn | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/bugs/problems_with_enableremote_on_gcrypt_remote___40__hosted_with_gitolite__41__.mdwn b/doc/bugs/problems_with_enableremote_on_gcrypt_remote___40__hosted_with_gitolite__41__.mdwn index 5dcf732a63..4d6e041bb5 100644 --- a/doc/bugs/problems_with_enableremote_on_gcrypt_remote___40__hosted_with_gitolite__41__.mdwn +++ b/doc/bugs/problems_with_enableremote_on_gcrypt_remote___40__hosted_with_gitolite__41__.mdwn @@ -103,3 +103,5 @@ git-annex: uuid mismatch (UUID "78104a6f-16a9-504b-8e8a-d8a3c59351e8",Just (UUID # End of transcript or log. """]] + +> [[done]]; see comments. --[[Joey]] From 2d77bb310f16330de28404e858d794943379ae14 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 28 Oct 2014 16:03:57 -0400 Subject: [PATCH 17/66] info: avoid strange display of here repo when it has NoUUID This happens when info is run in a git repo before git annex init. --- Command/Info.hs | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/Command/Info.hs b/Command/Info.hs index 96b7eb6d79..9356830e05 100644 --- a/Command/Info.hs +++ b/Command/Info.hs @@ -221,7 +221,8 @@ repository_mode = simpleStat "repository mode" $ lift $ remote_list :: TrustLevel -> Stat remote_list level = stat n $ nojson $ lift $ do - us <- M.keys <$> (M.union <$> uuidMap <*> remoteMap Remote.name) + us <- filter (/= NoUUID) . M.keys + <$> (M.union <$> uuidMap <*> remoteMap Remote.name) rs <- fst <$> trustPartition level us s <- prettyPrintUUIDs n rs return $ if null s then "0" else show (length rs) ++ "\n" ++ beginning s From 913c2bf31208d9f1b179b3f87a89659cde6a00ab Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 28 Oct 2014 16:20:03 -0400 Subject: [PATCH 18/66] comment --- .../comment_11_3b946519c9f52569ce60ac04bdc3380a._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/forum/Restricting_SSH_+_supply_key/comment_11_3b946519c9f52569ce60ac04bdc3380a._comment diff --git a/doc/forum/Restricting_SSH_+_supply_key/comment_11_3b946519c9f52569ce60ac04bdc3380a._comment b/doc/forum/Restricting_SSH_+_supply_key/comment_11_3b946519c9f52569ce60ac04bdc3380a._comment new file mode 100644 index 0000000000..6283053771 --- /dev/null +++ b/doc/forum/Restricting_SSH_+_supply_key/comment_11_3b946519c9f52569ce60ac04bdc3380a._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="joey" + subject="""fixed""" + date="2014-10-28T20:19:04Z" + content=""" +I seem to have forgotten to follow up here, but I think I fixed +this problem some time ago, in [[!commit 5afc8b28e03f4d242fa81a9a93384714d12d4e5c]]. +"""]] From e9615aec5fea6ad48113f84cda5292c0aa8aa9c5 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 28 Oct 2014 16:24:41 -0400 Subject: [PATCH 19/66] comment --- .../comment_7_413593ea0663874196305c9eb602e7c4._comment | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100644 doc/forum/Standard_groups__47__preferred_contents/comment_7_413593ea0663874196305c9eb602e7c4._comment diff --git a/doc/forum/Standard_groups__47__preferred_contents/comment_7_413593ea0663874196305c9eb602e7c4._comment b/doc/forum/Standard_groups__47__preferred_contents/comment_7_413593ea0663874196305c9eb602e7c4._comment new file mode 100644 index 0000000000..32bfa6bc00 --- /dev/null +++ b/doc/forum/Standard_groups__47__preferred_contents/comment_7_413593ea0663874196305c9eb602e7c4._comment @@ -0,0 +1,7 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 7""" + date="2014-10-28T20:23:55Z" + content=""" +See [[bugs/present_files__47__directories_are_dropped_after_a_sync]] +"""]] From 92f35df226e5ae40fa74277765aeaaf54dfd0d89 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 28 Oct 2014 16:26:52 -0400 Subject: [PATCH 20/66] comment --- .../comment_5_07348ed61ccdd93417365f1525a05bf2._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/forum/Ssh_remote_on_NAS/comment_5_07348ed61ccdd93417365f1525a05bf2._comment diff --git a/doc/forum/Ssh_remote_on_NAS/comment_5_07348ed61ccdd93417365f1525a05bf2._comment b/doc/forum/Ssh_remote_on_NAS/comment_5_07348ed61ccdd93417365f1525a05bf2._comment new file mode 100644 index 0000000000..c8e631226f --- /dev/null +++ b/doc/forum/Ssh_remote_on_NAS/comment_5_07348ed61ccdd93417365f1525a05bf2._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 5""" + date="2014-10-28T20:26:08Z" + content=""" +git-upload-pack is a git command, which is not present in PATH +on your NAS when git sshes in and tries to run it. +"""]] From a4dfb63be88f515935fa6a7c3e8bd6e162f365a2 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 28 Oct 2014 16:32:59 -0400 Subject: [PATCH 21/66] comment --- ..._b0a091e11c18c8640888b0aedfbff5c5._comment | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) create mode 100644 doc/forum/Central_git_annex_server_that_always_keeps_one_copy/comment_2_b0a091e11c18c8640888b0aedfbff5c5._comment diff --git a/doc/forum/Central_git_annex_server_that_always_keeps_one_copy/comment_2_b0a091e11c18c8640888b0aedfbff5c5._comment b/doc/forum/Central_git_annex_server_that_always_keeps_one_copy/comment_2_b0a091e11c18c8640888b0aedfbff5c5._comment new file mode 100644 index 0000000000..35778ad9ab --- /dev/null +++ b/doc/forum/Central_git_annex_server_that_always_keeps_one_copy/comment_2_b0a091e11c18c8640888b0aedfbff5c5._comment @@ -0,0 +1,19 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 2""" + date="2014-10-28T20:28:26Z" + content=""" +If you configure the central repository to be in the "full backup" group, +then the assistant and `git annex sync --content` etc will send the +contents of all files to it, and never remove the contents of files from +it. + +This doesn't prevent someone manually running `git annex drop --from +$centralrepo --force`, or similar. But then, nothing prevents users from +using git to push annexed files to the central repo, and never sending the +contents of the files along to it. + +It would be possible to make `git-annex-shell` have a config setting that +would prevent ever dropping keys. But I'd like to talk about this use case +more, to make sure that really makes sense. +"""]] From 67a88b9bdd77b08de400f386c26a35e5f1da8a42 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 28 Oct 2014 16:40:17 -0400 Subject: [PATCH 22/66] comment --- .../comment_1_f1eca38ca0c1edb66e64ac64f0321829._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/forum/How_to_get_detailed_information_on_special_remotes__63__/comment_1_f1eca38ca0c1edb66e64ac64f0321829._comment diff --git a/doc/forum/How_to_get_detailed_information_on_special_remotes__63__/comment_1_f1eca38ca0c1edb66e64ac64f0321829._comment b/doc/forum/How_to_get_detailed_information_on_special_remotes__63__/comment_1_f1eca38ca0c1edb66e64ac64f0321829._comment new file mode 100644 index 0000000000..84b67f75e1 --- /dev/null +++ b/doc/forum/How_to_get_detailed_information_on_special_remotes__63__/comment_1_f1eca38ca0c1edb66e64ac64f0321829._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 1""" + date="2014-10-28T20:39:41Z" + content=""" +The latest version of git-annex adds the ability to `git annex info +$remote` and get all the info you asked for. Enjoy! +"""]] From 552f7dbe88048819473f772f50a56f91ab0fbfed Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Tue, 28 Oct 2014 16:40:57 -0400 Subject: [PATCH 23/66] devblog --- doc/devblog/day_229__S3_multipart.mdwn | 11 +++++++++++ 1 file changed, 11 insertions(+) create mode 100644 doc/devblog/day_229__S3_multipart.mdwn diff --git a/doc/devblog/day_229__S3_multipart.mdwn b/doc/devblog/day_229__S3_multipart.mdwn new file mode 100644 index 0000000000..caccda07ab --- /dev/null +++ b/doc/devblog/day_229__S3_multipart.mdwn @@ -0,0 +1,11 @@ +Some progress on the [[bugs/S3_upload_not_using_multipart]] bug. The aws +library now includes the multipart API. However, when I dug into it, it +looks like the API needs some changes to get the ETAG of each uploaded +part. Once that's fixed, git-annex should be able to support S3 multipart +uploads, although I think that git-annex's own chunking is better in most +situations -- it supports resuming uploads and downloads better. The main +use case for S3 multipart seems to be using git-annex to publish large +files. + +Also, managed to get the backlog down from 100 to just 65 messages, +including catching up on quite old parts of backlog. From a08f99dd9cd04a356388ab14604ef289d7cae4f3 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawmKyeW2G4jjSdnL1m6kPPtAiGFUBsnYCfY" Date: Wed, 29 Oct 2014 03:34:10 +0000 Subject: [PATCH 24/66] --- ...disable__47__enable_expiry_after_X_days__34___mean__63__.mdwn | 1 + 1 file changed, 1 insertion(+) create mode 100644 doc/forum/What_does___34__disable__47__enable_expiry_after_X_days__34___mean__63__.mdwn diff --git a/doc/forum/What_does___34__disable__47__enable_expiry_after_X_days__34___mean__63__.mdwn b/doc/forum/What_does___34__disable__47__enable_expiry_after_X_days__34___mean__63__.mdwn new file mode 100644 index 0000000000..e95e27a755 --- /dev/null +++ b/doc/forum/What_does___34__disable__47__enable_expiry_after_X_days__34___mean__63__.mdwn @@ -0,0 +1 @@ +Git-annex assistant configuration has an item saying "disable/enable expiry after X days". What does it mean? Am I the only one confused by this? From c0ad5d1c770f40946a222f68d2e15a8f11ec6ce2 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawlog_5wIICaMcrKTexlFNA6IO6UTp323aE" Date: Wed, 29 Oct 2014 18:51:29 +0000 Subject: [PATCH 25/66] Added a comment --- ...comment_5_52010f21a15d76d68986aa1fba29aaf1._comment | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 doc/forum/git_annex_sync:_only_git-annex/comment_5_52010f21a15d76d68986aa1fba29aaf1._comment diff --git a/doc/forum/git_annex_sync:_only_git-annex/comment_5_52010f21a15d76d68986aa1fba29aaf1._comment b/doc/forum/git_annex_sync:_only_git-annex/comment_5_52010f21a15d76d68986aa1fba29aaf1._comment new file mode 100644 index 0000000000..cee2a9bd22 --- /dev/null +++ b/doc/forum/git_annex_sync:_only_git-annex/comment_5_52010f21a15d76d68986aa1fba29aaf1._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawlog_5wIICaMcrKTexlFNA6IO6UTp323aE" + nickname="Torkaly" + subject="comment 5" + date="2014-10-29T18:51:29Z" + content=""" +Hi, + +thank you for your response. I just want to control my branches (master, dev and so on ...) by myself, without sync/master or sync/dev and without merging it automatically. But the git-annex branch should be populated between the repositories \"magically\" (some kind of \"git annex syncannex\"). As annex can't deliver such a basic functionality i assumed, that it was not designed to work with existing \"real\" git repositories. +"""]] From f1d7a591fd0def816deecaaaf6931949392a38a5 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Wed, 29 Oct 2014 16:37:42 -0400 Subject: [PATCH 26/66] thoughts --- doc/todo/direct_mode_undo.mdwn | 76 ++++++++++++++++++++++++++++++++++ 1 file changed, 76 insertions(+) create mode 100644 doc/todo/direct_mode_undo.mdwn diff --git a/doc/todo/direct_mode_undo.mdwn b/doc/todo/direct_mode_undo.mdwn new file mode 100644 index 0000000000..969678d234 --- /dev/null +++ b/doc/todo/direct_mode_undo.mdwn @@ -0,0 +1,76 @@ +A fairly common request is that a repo is using direct mode, and the user +has made some change, and now wants to undo it. Since direct mode doesn't +allow using `git revert`, the repo would need to be switched to indirect +mode first, which can range from annoying to really annoying to impossible +(on eg FAT). + +## general approach + +`git annex foo $gitcmd` could: + +1. check out a local clone of the repo +2. run "git $gitcmd" inside the cline +3. merge any changes from the clone back into the direct mode repo + and update the work tree the same as is done by `git annex merge`. + +This is a general bypass for the direct mode guard. It should work anywhere +(even on FAT). It avoids problems like `git commit -a` being unsafe in +direct mode, since running such a command in a local clone, which does not +use direct mode is always safe. + +One problem with it is that it can only operate on changes that have been +committed. If you've just accidentially deleted a file and want to undo +that, and haven't run `git annex sync` to commit it, you can't revert it. + +Also, while this is general, I don't know if the generality is useful. +What other changes, than revert, would it make sense to do with such a +command? It could be used to check out some other branch, but step 3 above +would merge that branch back into the direct mode repo. + +## git annex undo + +I don't want to recapitulate all of the git commands in git-annex for +direct mode. So I don't want to add `git annex revert` and `git annex +branch` etc, etc. + +So, adding `git annex undo` feels like a step down a slippery slope. But it +might be justified as providing just enough functionality to make direct +mode a lot more useful, without trying to recapitulate all the flexability +of git. Like `git annex merge` and `git annex sync` also do. + +Another use case is binding `git annex undo $file` to an action in a file +manager. + +Here's a design for undo: + +1. Can be passed one or more files. Which may or may not exist in the work tree. +2. First, commits the current state of the files as staged in the index, + or in the working tree. This may involve checksumming modified files. +3. Then, for each file, looks back through git history, to find the commit + just before the most recent change that was made to that file. + Stage the version of the file as it was in that commit. +4. Updates work tree, and leaves the changes staged + but not committed. (To allow the user to bundle up multiple undos in a + single commit). +6. Does not get or drop content. The content may even be completely + missing after an undo. + +Note that undoing an undo should get back to the original state. This is +why #2 commits changes first. This way, if a file has a staged change, +it gets committed, and then that commit is reverted, resulting in another +commit. Which a later run of undo can in turn revert. If it didn't commit, +the history about the staged change that was reverted would be lost. + +What about undoing changes to a whole directory? It first would +need to look through the git history to find every file under that +directory. And then it would behave as if it were passed all those +files. This would be useful for reverting `rm -rf`. But it could be very +expensive. And it could lead to surprising results, like undoing a lot +of unrelated changes when running on a bunch of files in a directory +that were changed at different times. + +Maybe instead of letting a directory be passed, make undo with no +parameters revert all changes made in the most recent commit. + +Also, --depth could make undo look for an older commit than the most +recent one to affect the specified file. From bc0830af0d1559a8a0b3ffcda536368d058e5a81 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawn7OQLFQ6yc9sK5O6ekgYhLrZhqtm4JZBM" Date: Wed, 29 Oct 2014 22:28:14 +0000 Subject: [PATCH 27/66] --- ...ectiveUserID_not_supported_on_Windows.mdwn | 22 +++++++++++++++++++ 1 file changed, 22 insertions(+) create mode 100644 doc/bugs/getEffectiveUserID_not_supported_on_Windows.mdwn diff --git a/doc/bugs/getEffectiveUserID_not_supported_on_Windows.mdwn b/doc/bugs/getEffectiveUserID_not_supported_on_Windows.mdwn new file mode 100644 index 0000000000..c3d123fb3f --- /dev/null +++ b/doc/bugs/getEffectiveUserID_not_supported_on_Windows.mdwn @@ -0,0 +1,22 @@ +### Please describe the problem. +After a clean install of the latest git-annex for Windows (https://downloads.kitenet.net/git-annex/windows/current/), most git-annex commands fail with the following error message: + + git-annex: System.PosixCompat.User.getEffectiveUserID: not supported: illegal operation + +### What steps will reproduce the problem? + + git init + git annex init + +Running for example ´git annex version´ or ´git annex info´ gives the same error message. + + +### What version of git-annex are you using? On what operating system? + +git-annex version should be 5.20141024, but the installer does not specify version and ´git annex info´ does not work, so it is hard to tell for sure. +Running Windows 7, x64. Also tried running as administrator and in a cmd.exe shell as well as Powershell. + + +### Please provide any additional information below. + +The ´git annex test´ command does work, and all 84 tests passes. From 6a9c044caad0e01aeead9309032122ee0de31bd0 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawn7OQLFQ6yc9sK5O6ekgYhLrZhqtm4JZBM" Date: Wed, 29 Oct 2014 22:29:29 +0000 Subject: [PATCH 28/66] --- doc/bugs/getEffectiveUserID_not_supported_on_Windows.mdwn | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/doc/bugs/getEffectiveUserID_not_supported_on_Windows.mdwn b/doc/bugs/getEffectiveUserID_not_supported_on_Windows.mdwn index c3d123fb3f..458c18b901 100644 --- a/doc/bugs/getEffectiveUserID_not_supported_on_Windows.mdwn +++ b/doc/bugs/getEffectiveUserID_not_supported_on_Windows.mdwn @@ -8,15 +8,15 @@ After a clean install of the latest git-annex for Windows (https://downloads.kit git init git annex init -Running for example ´git annex version´ or ´git annex info´ gives the same error message. +Running for example `git annex version` or `git annex info` gives the same error message. ### What version of git-annex are you using? On what operating system? -git-annex version should be 5.20141024, but the installer does not specify version and ´git annex info´ does not work, so it is hard to tell for sure. +git-annex version should be 5.20141024, but the installer does not specify version and `git annex info` does not work, so it is hard to tell for sure. Running Windows 7, x64. Also tried running as administrator and in a cmd.exe shell as well as Powershell. ### Please provide any additional information below. -The ´git annex test´ command does work, and all 84 tests passes. +The `git annex test` command does work, and all 84 tests passes. From 723653ab69d0f04608c8ea6a1203897aca8f6d72 Mon Sep 17 00:00:00 2001 From: anarcat Date: Thu, 30 Oct 2014 14:21:20 +0000 Subject: [PATCH 29/66] weird multiple password prompts problem --- ...flooding_me_with_ssh_password_prompts.mdwn | 34 +++++++++++++++++++ 1 file changed, 34 insertions(+) create mode 100644 doc/bugs/flooding_me_with_ssh_password_prompts.mdwn diff --git a/doc/bugs/flooding_me_with_ssh_password_prompts.mdwn b/doc/bugs/flooding_me_with_ssh_password_prompts.mdwn new file mode 100644 index 0000000000..6e6be07900 --- /dev/null +++ b/doc/bugs/flooding_me_with_ssh_password_prompts.mdwn @@ -0,0 +1,34 @@ +### Please describe the problem. + +When I login to my session, git-annex starts a few daemons, which is fine, but then it prompts me for tons of passwords, which is really annoying. + +It is strange because one of the things I do when I start my session is to input my keys in the SSH agent. But then git-annex still prompts me: + +~~~~ +21503 ? Sl 3:04 git-annex assistant --startdelay=5s +21517 ? S 0:03 \_ git --git-dir=/srv/musique/anarcat/books/.git --work-tree=/srv/musique/anarcat/books cat-file --batch +21612 ? SNl 0:00 \_ git-annex remotedaemon +21706 ? S 0:00 \_ git --git-dir=/srv/musique/anarcat/books/.git --work-tree=/srv/musique/anarcat/books check-ignore -z --stdin --verbose --non-matching +21898 ? SN 0:00 \_ git --git-dir=/srv/musique/anarcat/books/.git --work-tree=/srv/musique/anarcat/books check-attr -z --stdin annex.backend annex.numcopies -- + 6712 ? SNl 0:00 \_ git-annex transferkeys + 6719 ? SN 0:00 \_ git --git-dir=/srv/musique/anarcat/books/.git --work-tree=/srv/musique/anarcat/books cat-file --batch + 6720 ? SN 0:00 \_ rsync --progress --inplace --perms -e 'ssh' '-S' '.git/annex/ssh/anarc.at' '-o' 'ControlMaster=auto' '-o' 'ControlPersist=yes' '-T' 'anarc.at' 'git-annex-shell ''sendkey'' ''/srv/books'' ''SHA256E-s17781587--f204e6ef1f14b624e222d6ad73ed41edf65c29c93afad9a1e4e1954ad68d1753.pdf'' --uuid a75cbbf7-e055-423e-b375-443e0552c9e2 ''--'' ''remoteuuid=aa500f29-42d9-4777-ae02-4a2c3d47db44'' ''direct='' ''associatedfile=Garde cotiere canadienne/Navigation dans les glaces en eaux Canadiennes (1112)/Navigation dans les glaces en eaux Canadie - Garde cotiere canadienne.pdf'' ''--''' -- dummy: /srv/musique/anarcat/books/.git/annex/tmp/SHA256E-s17781587--f204e6ef1f14b624e222d6ad73ed41edf65c29c93afad9a1e4e1954ad68d1753.pdf + 6721 ? SN 0:00 \_ ssh -S .git/annex/ssh/anarc.at -o ControlMaster=auto -o ControlPersist=yes -T anarc.at git-annex-shell 'sendkey' '/srv/books' 'SHA256E-s17781587--f204e6ef1f14b624e222d6ad73ed41edf65c29c93afad9a1e4e1954ad68d1753.pdf' --uuid a75cbbf7-e055-423e-b375-443e0552c9e2 '--' 'remoteuuid=aa500f29-42d9-4777-ae02-4a2c3d47db44' 'direct=' 'associatedfile=Garde cotiere canadienne/Navigation dans les glaces en eaux Canadiennes (1112)/Navigation dans les glaces en eaux Canadie - Garde cotiere canadienne.pdf' '--' dummy rsync --server --sender -vpe.Lsf --inplace . + 6722 ? SN 0:00 \_ /usr/bin/ssh-askpass anarcat@anarc.at's password: +~~~~ + +Yet I can login to `shell.anarc.at` without a passphrase without problems. + +Interestingly enough, my main session and git-annex do not seem to share the same `SSH_AGENT` environment variable. It's unclear to me why. git-annex's SSH_AGENT environment variable seems to refer to a process that disappeared, actually. So it could be there's something wrong with my session. + +Still, when a situation like this occurs, it seems to me that it should generate in this noisy concert of ssh prompts that basically blocks all work until i hit "escape" often enough. In fact, that it is rather problematic to have random password prompts show up like this without an explanation: git-annex should tell me it's the source of this password prompt or not prompt at all, because there's no way i'll start entering random passphrases into random pinentry dialogs that show up... + +Doesn't git-annex deploy its own ssh keys once it has established a connexion with an SSH remote? + +### What steps will reproduce the problem? + +It's unclear - i guess you need to setup git-annex to autostart and sync with remote ssh annexes. You may also need to have to use XFCE with Awesome to reproduce the problem. I can provide more details on my session setup on request. --[[anarcat]] + +### What version of git-annex are you using? On what operating system? + +5.20140927~bpo70+3 on debian wheezy. From c87e84fc905dd50d758568cc67bcaa95bd516d59 Mon Sep 17 00:00:00 2001 From: anarcat Date: Thu, 30 Oct 2014 14:33:46 +0000 Subject: [PATCH 30/66] add more information --- doc/bugs/flooding_me_with_ssh_password_prompts.mdwn | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/doc/bugs/flooding_me_with_ssh_password_prompts.mdwn b/doc/bugs/flooding_me_with_ssh_password_prompts.mdwn index 6e6be07900..191ad02507 100644 --- a/doc/bugs/flooding_me_with_ssh_password_prompts.mdwn +++ b/doc/bugs/flooding_me_with_ssh_password_prompts.mdwn @@ -25,6 +25,14 @@ Still, when a situation like this occurs, it seems to me that it should generate Doesn't git-annex deploy its own ssh keys once it has established a connexion with an SSH remote? +> After more investigation, it turns out this peculiar git-annex daemon was some left-overs from a previous session i had logged out of. It is unclear why git-annex was still running, but there were also pulseaudio and redshift programs lying around so I suspect it wasn't git-annex specific. +> +> However, this problem remains on login. When I start a new session, there's a race condition between git-annex asking passwords and ssh-add asking me to unlock my private key. It's a nice festival of password prompts as I struggle to type in my ssh key faster than git-annex asks me for the remote host's password. +> +> In general, I think git-annex should set `PasswordAuthentication=no` (or allow me to configure it as such), especially if it knows it was able to login without a password at some point. It should especially do that if there's no UI attached informing the user it will be prompting for a password. `NumberOfPasswordPrompts=1` would also be a welcome improvement, but I really don't think I should be seeing those password prompts, especially since I *can* login to the server with my existing SSH key. +> +> Also note that I use Monkeysphere to input my private key into the SSH agent so it could explain why this doesn't usually happen to other people: Monkeysphere doesn't automatically get started from `ssh` if the key is missing, and instead `ssh` will revert to a regular password-based authentication which competes with the Monkeysphere password prompt. + ### What steps will reproduce the problem? It's unclear - i guess you need to setup git-annex to autostart and sync with remote ssh annexes. You may also need to have to use XFCE with Awesome to reproduce the problem. I can provide more details on my session setup on request. --[[anarcat]] From 4af58cd4461fc7b3146fec0f80cb86f44404c0e3 Mon Sep 17 00:00:00 2001 From: anarcat Date: Thu, 30 Oct 2014 15:15:57 +0000 Subject: [PATCH 31/66] Added a comment --- .../comment_2_b9236ea884a2934dd0c8e511a80f4fda._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/todo/show_readonly_removable_drives_in_the_webapp/comment_2_b9236ea884a2934dd0c8e511a80f4fda._comment diff --git a/doc/todo/show_readonly_removable_drives_in_the_webapp/comment_2_b9236ea884a2934dd0c8e511a80f4fda._comment b/doc/todo/show_readonly_removable_drives_in_the_webapp/comment_2_b9236ea884a2934dd0c8e511a80f4fda._comment new file mode 100644 index 0000000000..9aa4c179e6 --- /dev/null +++ b/doc/todo/show_readonly_removable_drives_in_the_webapp/comment_2_b9236ea884a2934dd0c8e511a80f4fda._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="anarcat" + ip="70.83.139.100" + subject="comment 2" + date="2014-10-30T15:15:57Z" + content=""" +yep, that would be fine. +"""]] From 680b056405b3b526f36c5f121944c0961fb2abf1 Mon Sep 17 00:00:00 2001 From: anarcat Date: Thu, 30 Oct 2014 15:22:32 +0000 Subject: [PATCH 32/66] Added a comment --- ...comment_4_2491fabe2eb7a14bfef0e388b4b9c4c7._comment | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 doc/bugs/annex_get_fails_from_read-only_filesystem/comment_4_2491fabe2eb7a14bfef0e388b4b9c4c7._comment diff --git a/doc/bugs/annex_get_fails_from_read-only_filesystem/comment_4_2491fabe2eb7a14bfef0e388b4b9c4c7._comment b/doc/bugs/annex_get_fails_from_read-only_filesystem/comment_4_2491fabe2eb7a14bfef0e388b4b9c4c7._comment new file mode 100644 index 0000000000..8543aaad7d --- /dev/null +++ b/doc/bugs/annex_get_fails_from_read-only_filesystem/comment_4_2491fabe2eb7a14bfef0e388b4b9c4c7._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="anarcat" + ip="70.83.139.100" + subject="comment 4" + date="2014-10-30T15:22:32Z" + content=""" +the problem here is that it upgrading the repo will not work on a readonly filesystem, so we can't upgrade, we can't get files, so we can't sync. + +we should be able to sync anyways - can't we try our best shot at getting files even without upgrading the metadata? i mean i'm looking for something in .git/annex/objects/, maybe i don't care about tracking so much - i just want to recover some files... +"""]] From 0fbcfe09cf7f70b0b5cdbbb5da932e86043b3143 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawmBUR4O9mofxVbpb8JV9mEbVfIYv670uJo" Date: Thu, 30 Oct 2014 18:07:28 +0000 Subject: [PATCH 33/66] Added a comment --- ...ent_1_dba769cf0fca540f50f5e4b732a7e353._comment | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 doc/bugs/flooding_me_with_ssh_password_prompts/comment_1_dba769cf0fca540f50f5e4b732a7e353._comment diff --git a/doc/bugs/flooding_me_with_ssh_password_prompts/comment_1_dba769cf0fca540f50f5e4b732a7e353._comment b/doc/bugs/flooding_me_with_ssh_password_prompts/comment_1_dba769cf0fca540f50f5e4b732a7e353._comment new file mode 100644 index 0000000000..1cfa8763ce --- /dev/null +++ b/doc/bugs/flooding_me_with_ssh_password_prompts/comment_1_dba769cf0fca540f50f5e4b732a7e353._comment @@ -0,0 +1,14 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawmBUR4O9mofxVbpb8JV9mEbVfIYv670uJo" + nickname="Justin" + subject="comment 1" + date="2014-10-30T18:07:28Z" + content=""" +I've ran into this as well using wheezy+awesome. It definitely looks like a race condition, I ended up doing a + + killall git-annex + ssh-add + git-annex webapp + +to get things going again. +"""]] From 0da54c7dddef6e2f52a40fe568cea772399c6086 Mon Sep 17 00:00:00 2001 From: "http://jaen.saul.ee/id/" Date: Thu, 30 Oct 2014 19:04:57 +0000 Subject: [PATCH 34/66] Added a comment: Re: why this happened --- ...comment_5_07f612d7059a9b561c48d63e471d5ed7._comment | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 doc/bugs/annex_get_fails_from_read-only_filesystem/comment_5_07f612d7059a9b561c48d63e471d5ed7._comment diff --git a/doc/bugs/annex_get_fails_from_read-only_filesystem/comment_5_07f612d7059a9b561c48d63e471d5ed7._comment b/doc/bugs/annex_get_fails_from_read-only_filesystem/comment_5_07f612d7059a9b561c48d63e471d5ed7._comment new file mode 100644 index 0000000000..24b7dd1180 --- /dev/null +++ b/doc/bugs/annex_get_fails_from_read-only_filesystem/comment_5_07f612d7059a9b561c48d63e471d5ed7._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="http://jaen.saul.ee/id/" + nickname="Jaen" + subject="Re: why this happened" + date="2014-10-30T19:04:56Z" + content=""" +I agree that at least read-only support should be there. This error was from an filesystem type that is impossible to mount read-write. + +Now that I am looking at it again with fresh eyes, I suppose it's possible to get around this by mounting an union filesystem with a read-write temp layer on top of the read-only one (not that a regular user would ever figure that out...) +"""]] From a4810a47577f46fe502b01f2f32929d3e4122209 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Fri, 31 Oct 2014 11:27:05 -0400 Subject: [PATCH 35/66] better organization and a few wording tweaks --- doc/internals.mdwn | 32 +++++++++++++++----------------- 1 file changed, 15 insertions(+), 17 deletions(-) diff --git a/doc/internals.mdwn b/doc/internals.mdwn index 71d1b03800..ca2b7b41f0 100644 --- a/doc/internals.mdwn +++ b/doc/internals.mdwn @@ -2,7 +2,9 @@ In the world of git, we're not scared about internal implementation details, and sometimes we like to dive in and tweak things by hand. Here's some documentation to that end. -## `.git/annex/objects/aa/bb/*/*` +## The .git/ directory + +### `.git/annex/objects/aa/bb/*/*` This is where locally available file contents are actually stored. Files added to the annex get a symlink checked into git that points @@ -27,11 +29,11 @@ Also in [[direct_mode]], some additional data is stored in these directories. changed, and `.map` files contain a list of file(s) in the work directory that contain the key. -# `.git/annex/tmp/` +### `.git/annex/tmp/` This directory contains partially transferred objects. -# `.git/annex/misctmp/` +### `.git/annex/misctmp/` This is a temp directory for miscellaneous other temp files. @@ -39,26 +41,26 @@ While .git/annex/objects and .git/annex/tmp can be put on different filesystems if desired, .git/annex/misctmp has to be on the same filesystem as the work tree and git repository. -# `.git/annex/bad/` +### `.git/annex/bad/` git-annex fsck puts any bad objects it finds in here. -# `.git/annex/transfers/` +### `.git/annex/transfers/` Contains information files for uploads and downloads that are in progress, as well as any that have failed. Used especially by the assistant. It is safe to delete these files. -# `.git/annex/ssh/` +### `.git/annex/ssh/` ssh connection caching files are written in here. -# `.git/annex/index` +### `.git/annex/index` -This is a git index file which git-annex uses for commits to the git-annex -branch. +This is a git index file which git-annex uses to stage files +when preparing commits to the git-annex branch. -# `.git/annex/journal/` +### `.git/annex/journal/` git-annex uses this to journal changes to the git-annex branch, before committing a set of changes. @@ -67,13 +69,9 @@ before committing a set of changes. This branch is managed by git-annex, with the contents listed below. -The file `.git/annex/index` is a separate git index file it uses -to accumulate changes for the git-annex branch. -Also, `.git/annex/journal/` is used to record changes before they -are added to git. - -This branch operates on objects exclusively. No file names will ever -be stored in this branch. +This branch is not connected to your master, etc branches. It it used for +internal tracking of information about git-annex repositories and annexed +objects. The files stored in this branch are all designed to be auto-merged using git's [[union merge driver|git-union-merge]]. So each line From 8f072c8db713852516f788d49786971aae0c1c1d Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Fri, 31 Oct 2014 12:39:50 -0400 Subject: [PATCH 36/66] add example multiline data constructor --- doc/coding_style.mdwn | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/doc/coding_style.mdwn b/doc/coding_style.mdwn index 101ac4f581..6968c99586 100644 --- a/doc/coding_style.mdwn +++ b/doc/coding_style.mdwn @@ -72,6 +72,14 @@ that line up with the open and close punctuation. , address = "baz" } +Similarly, data structures line up the leading `=` with the following `|` + + data Foo + = Bar + | Baz + | Quux Foo + deriving (Eq, Ord) + Module imports are separated into two blocks, one for third-party modules, and one for modules that are part of git-annex. (Additional blocks can be used if it makes sense.) From 86b326782a93cf5c3476a39b70659f1f5ec9ada4 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Fri, 31 Oct 2014 15:46:44 -0400 Subject: [PATCH 37/66] getUserEntryForID does not work on windows Throw an error instead, mentioning the relevant env vars. --- Utility/UserInfo.hs | 11 +++++++---- ...omment_6_c469f1fe8ae6981a038be7f8723a07b1._comment | 7 +++++++ 2 files changed, 14 insertions(+), 4 deletions(-) create mode 100644 doc/bugs/annex_get_fails_from_read-only_filesystem/comment_6_c469f1fe8ae6981a038be7f8723a07b1._comment diff --git a/Utility/UserInfo.hs b/Utility/UserInfo.hs index 617c3e945b..b3e191bbfc 100644 --- a/Utility/UserInfo.hs +++ b/Utility/UserInfo.hs @@ -48,8 +48,11 @@ myUserGecos = myVal [] userGecos #endif myVal :: [String] -> (UserEntry -> String) -> IO String -myVal envvars extract = maybe (extract <$> getpwent) return =<< check envvars +myVal envvars extract = go envvars where - check [] = return Nothing - check (v:vs) = maybe (check vs) (return . Just) =<< getEnv v - getpwent = getUserEntryForID =<< getEffectiveUserID +#ifndef mingw32_HOST_OS + go [] = extract <$> (getUserEntryForID =<< getEffectiveUserID) +#else + go [] = error $ "environment not set: " ++ show envvars +#endif + go (v:vs) = maybe (go vs) return =<< getEnv v diff --git a/doc/bugs/annex_get_fails_from_read-only_filesystem/comment_6_c469f1fe8ae6981a038be7f8723a07b1._comment b/doc/bugs/annex_get_fails_from_read-only_filesystem/comment_6_c469f1fe8ae6981a038be7f8723a07b1._comment new file mode 100644 index 0000000000..fac28c24d0 --- /dev/null +++ b/doc/bugs/annex_get_fails_from_read-only_filesystem/comment_6_c469f1fe8ae6981a038be7f8723a07b1._comment @@ -0,0 +1,7 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 6""" + date="2014-10-31T16:55:47Z" + content=""" +An upgrade could move the annexed objects around. +"""]] From 8c622ec10077562165e44b24d0e268c7b952efc3 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Fri, 31 Oct 2014 15:48:52 -0400 Subject: [PATCH 38/66] commnt --- ..._1_26cd277e6bbfcac920ca0a2a47acfcfd._comment | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) create mode 100644 doc/bugs/getEffectiveUserID_not_supported_on_Windows/comment_1_26cd277e6bbfcac920ca0a2a47acfcfd._comment diff --git a/doc/bugs/getEffectiveUserID_not_supported_on_Windows/comment_1_26cd277e6bbfcac920ca0a2a47acfcfd._comment b/doc/bugs/getEffectiveUserID_not_supported_on_Windows/comment_1_26cd277e6bbfcac920ca0a2a47acfcfd._comment new file mode 100644 index 0000000000..f9412487c4 --- /dev/null +++ b/doc/bugs/getEffectiveUserID_not_supported_on_Windows/comment_1_26cd277e6bbfcac920ca0a2a47acfcfd._comment @@ -0,0 +1,17 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 1""" + date="2014-10-31T16:57:05Z" + content=""" +This code path should only be called when USERPROFILE or +one of USERNAME/USER/LOGNAME is not set. Which should not usually be the case +when using git-annex on windows. + +However, since we know that won't work on Windows it might as well not even +try.. So I've made it instead print an error that indicates which +environment variables need to be set. + +Can you please download the windows daily build from +, and see what this new, +better error message says? +"""]] From 0f6aaf8012b3dce4ba4668b3f1e6f387fe8c63b3 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Fri, 31 Oct 2014 16:14:12 -0400 Subject: [PATCH 39/66] Windows: Fix crash when user.name is not set in git config. --- Annex/Environment.hs | 2 +- Utility/UserInfo.hs | 9 +++++---- debian/changelog | 6 ++++++ ...ffectiveUserID_not_supported_on_Windows.mdwn | 6 ++++++ ..._1_26cd277e6bbfcac920ca0a2a47acfcfd._comment | 17 ----------------- 5 files changed, 18 insertions(+), 22 deletions(-) delete mode 100644 doc/bugs/getEffectiveUserID_not_supported_on_Windows/comment_1_26cd277e6bbfcac920ca0a2a47acfcfd._comment diff --git a/Annex/Environment.hs b/Annex/Environment.hs index 1ddd2b2384..ffdd071575 100644 --- a/Annex/Environment.hs +++ b/Annex/Environment.hs @@ -32,7 +32,7 @@ checkEnvironment = do liftIO checkEnvironmentIO checkEnvironmentIO :: IO () -checkEnvironmentIO = whenM (null <$> myUserGecos) $ do +checkEnvironmentIO = whenM (isNothing <$> myUserGecos) $ do username <- myUserName ensureEnv "GIT_AUTHOR_NAME" username ensureEnv "GIT_COMMITTER_NAME" username diff --git a/Utility/UserInfo.hs b/Utility/UserInfo.hs index b3e191bbfc..1a557c9a52 100644 --- a/Utility/UserInfo.hs +++ b/Utility/UserInfo.hs @@ -40,11 +40,12 @@ myUserName = myVal env userName env = ["USERNAME", "USER", "LOGNAME"] #endif -myUserGecos :: IO String -#ifdef __ANDROID__ -myUserGecos = return "" -- userGecos crashes on Android +myUserGecos :: IO (Maybe String) +-- userGecos crashes on Android and is not available on Windows. +#if defined(__ANDROID__) || defined(mingw32_HOST_OS) +myUserGecos = return Nothing #else -myUserGecos = myVal [] userGecos +myUserGecos = Just <$> myVal [] userGecos #endif myVal :: [String] -> (UserEntry -> String) -> IO String diff --git a/debian/changelog b/debian/changelog index 8af44b3bb3..efaba2d950 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,9 @@ +git-annex (5.20141025) UNRELEASED; urgency=medium + + * Windows: Fix crash when user.name is not set in git config. + + -- Joey Hess Fri, 31 Oct 2014 16:13:43 -0400 + git-annex (5.20141024) unstable; urgency=medium * vicfg: Deleting configurations now resets to the default, where diff --git a/doc/bugs/getEffectiveUserID_not_supported_on_Windows.mdwn b/doc/bugs/getEffectiveUserID_not_supported_on_Windows.mdwn index 458c18b901..e8271ff94b 100644 --- a/doc/bugs/getEffectiveUserID_not_supported_on_Windows.mdwn +++ b/doc/bugs/getEffectiveUserID_not_supported_on_Windows.mdwn @@ -20,3 +20,9 @@ Running Windows 7, x64. Also tried running as administrator and in a cmd.exe she ### Please provide any additional information below. The `git annex test` command does work, and all 84 tests passes. + +> [[fixed|done]]; I was able to reproduce this bug, and it was crashing +> trying to look up the geckos username. I don't understand why this worked +> before; some change exposed this code path on Windows. In any case, I've +> fixed the crash, and I will be updating the windows builds with this bug +> fix. --[[Joey]] diff --git a/doc/bugs/getEffectiveUserID_not_supported_on_Windows/comment_1_26cd277e6bbfcac920ca0a2a47acfcfd._comment b/doc/bugs/getEffectiveUserID_not_supported_on_Windows/comment_1_26cd277e6bbfcac920ca0a2a47acfcfd._comment deleted file mode 100644 index f9412487c4..0000000000 --- a/doc/bugs/getEffectiveUserID_not_supported_on_Windows/comment_1_26cd277e6bbfcac920ca0a2a47acfcfd._comment +++ /dev/null @@ -1,17 +0,0 @@ -[[!comment format=mdwn - username="joey" - subject="""comment 1""" - date="2014-10-31T16:57:05Z" - content=""" -This code path should only be called when USERPROFILE or -one of USERNAME/USER/LOGNAME is not set. Which should not usually be the case -when using git-annex on windows. - -However, since we know that won't work on Windows it might as well not even -try.. So I've made it instead print an error that indicates which -environment variables need to be set. - -Can you please download the windows daily build from -, and see what this new, -better error message says? -"""]] From d1fe814a81549ebd61753ae0f285aac0d9fbd08c Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Fri, 31 Oct 2014 16:19:23 -0400 Subject: [PATCH 40/66] comment --- ..._cb8eaa8c19f5bb9e145a5e4c8995a8ca._comment | 22 +++++++++++++++++++ 1 file changed, 22 insertions(+) create mode 100644 doc/forum/Deduplication_in_direct_mode/comment_1_cb8eaa8c19f5bb9e145a5e4c8995a8ca._comment diff --git a/doc/forum/Deduplication_in_direct_mode/comment_1_cb8eaa8c19f5bb9e145a5e4c8995a8ca._comment b/doc/forum/Deduplication_in_direct_mode/comment_1_cb8eaa8c19f5bb9e145a5e4c8995a8ca._comment new file mode 100644 index 0000000000..63367fcc14 --- /dev/null +++ b/doc/forum/Deduplication_in_direct_mode/comment_1_cb8eaa8c19f5bb9e145a5e4c8995a8ca._comment @@ -0,0 +1,22 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 1""" + date="2014-10-31T20:15:58Z" + content=""" +There is really no way to do this. + +We could consider hard-linking the files, but then modifying one would +modify the other, which is likely to be confusing. And, FAT doesn't support +hard links anyway. + +I don't want to complicate git-annex's notion of whether an object is +present or not with the possibility that it might be present for some +files but not for others. For example, `git annex get` would then need +to make a copy of content that was already locally present, while +currently it knows that if the file is locally present, it has nothing to +do. + +I think that the solution is to use either a better filesystem +which can support the suprerior indirect mode, or to switch your +repository to use the WORM backend which does not do deduplication. +"""]] From b67500d806b69822f17f091895dcc1437b1ba1cb Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Fri, 31 Oct 2014 16:24:52 -0400 Subject: [PATCH 41/66] comment --- ..._face45c85e4713325bb23bbca7b07502._comment | 28 +++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100644 doc/forum/What_does___34__disable__47__enable_expiry_after_X_days__34___mean__63__/comment_1_face45c85e4713325bb23bbca7b07502._comment diff --git a/doc/forum/What_does___34__disable__47__enable_expiry_after_X_days__34___mean__63__/comment_1_face45c85e4713325bb23bbca7b07502._comment b/doc/forum/What_does___34__disable__47__enable_expiry_after_X_days__34___mean__63__/comment_1_face45c85e4713325bb23bbca7b07502._comment new file mode 100644 index 0000000000..adae1861a1 --- /dev/null +++ b/doc/forum/What_does___34__disable__47__enable_expiry_after_X_days__34___mean__63__/comment_1_face45c85e4713325bb23bbca7b07502._comment @@ -0,0 +1,28 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 1""" + date="2014-10-31T20:21:52Z" + content=""" +Well, let's see.. You've gone into the Configuration and picked "Unused +files". The full text in front of you is: + +
+Managing unused files
+
+Old versions of files and deleted files can be preserved inside this repository.
+
+This might be useful, if you ever need to access those old or deleted files. But they'll also use up disk space. There are three ways to deal with this
+
+1. Set up a backup or archive repository, on a removable drive or in the cloud, and the unused files will be moved to it, freeing up space. 
+   [Add a new repository]
+2. Or, you can let unused files expire after a period of time. 
+   [Disable expiry / Enable expiry] after [7] days. [Save changes]
+
+ +So yes, if I read only item #2, and possibly skip the first sentence of it +and look at only the "Disable expiry / Enable expiry", +it seems a litte confusing, possibly, but if I read the 2 paragraphs +of explaination above it, not so much? + +What do you find confusing about this form as a whole? +"""]] From 746b1cb7bcf6e30d52df467edb38e8c9bd0dee7e Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Fri, 31 Oct 2014 16:31:00 -0400 Subject: [PATCH 42/66] sssheeesh --- ...mment_4_cc3cebf5bc403b490e31e63af964a823._comment | 12 ------------ ...mment_6_df94154fbbc4edbf7ff658f61bde48b5._comment | 11 +++++++++++ 2 files changed, 11 insertions(+), 12 deletions(-) delete mode 100644 doc/forum/git_annex_sync:_only_git-annex/comment_4_cc3cebf5bc403b490e31e63af964a823._comment create mode 100644 doc/forum/git_annex_sync:_only_git-annex/comment_6_df94154fbbc4edbf7ff658f61bde48b5._comment diff --git a/doc/forum/git_annex_sync:_only_git-annex/comment_4_cc3cebf5bc403b490e31e63af964a823._comment b/doc/forum/git_annex_sync:_only_git-annex/comment_4_cc3cebf5bc403b490e31e63af964a823._comment deleted file mode 100644 index 134ea87e27..0000000000 --- a/doc/forum/git_annex_sync:_only_git-annex/comment_4_cc3cebf5bc403b490e31e63af964a823._comment +++ /dev/null @@ -1,12 +0,0 @@ -[[!comment format=mdwn - username="http://joeyh.name/" - ip="209.250.56.96" - subject="comment 4" - date="2014-10-22T16:18:50Z" - content=""" -I struggle to see how you could draw that conclusion from what I said. - -git-annex will work fine in an existing git repository. You can mix regular git commands like `git add`, `git push`, `git pull`, `git merge` with git-annex commands like `git annex add`, `git annex copy --to origin`, `git annex get`, `git annex merge`, in the same repository. - -The `git annex sync` command effcetively runs `git commit; git pull; git annex merge; git push; git annex copy --to origin; git annex get`. If you don't want to run all those commands at once, you don't want to run `git annex sync`. That will not prevent you from using git-annex in any way. -"""]] diff --git a/doc/forum/git_annex_sync:_only_git-annex/comment_6_df94154fbbc4edbf7ff658f61bde48b5._comment b/doc/forum/git_annex_sync:_only_git-annex/comment_6_df94154fbbc4edbf7ff658f61bde48b5._comment new file mode 100644 index 0000000000..8772255eb4 --- /dev/null +++ b/doc/forum/git_annex_sync:_only_git-annex/comment_6_df94154fbbc4edbf7ff658f61bde48b5._comment @@ -0,0 +1,11 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 1""" + date="2014-10-31T20:25:40Z" + content=""" +I have explained clearly in comment #1 above how to do what you want to do, +using git-annex, so it is a pity if still think that "annex +can't deliver such a basic functionality". + +It can. I have explained how. I don't know how to explain any better. +"""]] From 1970563dd9ba8d39bf83a170755f56f5aa3c3354 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Fri, 31 Oct 2014 16:39:19 -0400 Subject: [PATCH 43/66] comment and associated todo --- doc/todo/find_unused_in_any_commit.mdwn | 14 ++++++++++++++ ...ent_3_5f973eb7c265015fcd400b6b0bbdf235._comment | 10 ++++++++++ 2 files changed, 24 insertions(+) create mode 100644 doc/todo/find_unused_in_any_commit.mdwn create mode 100644 doc/walkthrough/unused_data/comment_3_5f973eb7c265015fcd400b6b0bbdf235._comment diff --git a/doc/todo/find_unused_in_any_commit.mdwn b/doc/todo/find_unused_in_any_commit.mdwn new file mode 100644 index 0000000000..11bbbd7260 --- /dev/null +++ b/doc/todo/find_unused_in_any_commit.mdwn @@ -0,0 +1,14 @@ +`git annex unused` only looks at tags and branches. Some users would like +to drop any objects that are not pointed to by any commit, but keep any +object that any commit ever referenced. + +This could be a switch, like --ever. + +The implementation would need to walk the history of all branches and check +all commits. This would tend to be slow. It could look at tags+branches as +it does now as a first pass, and only do the slow part if there are objects +not referred to by the tags+branches. And, it could stop looking through +the whole commit history if there were no more objects to check. Still, +gonna be slooow. Another optimisation would be to get only the objects +changed by the commit, and not look at the whole tree as it appeared on +each commit. --[[Joey]] diff --git a/doc/walkthrough/unused_data/comment_3_5f973eb7c265015fcd400b6b0bbdf235._comment b/doc/walkthrough/unused_data/comment_3_5f973eb7c265015fcd400b6b0bbdf235._comment new file mode 100644 index 0000000000..dcca0b0959 --- /dev/null +++ b/doc/walkthrough/unused_data/comment_3_5f973eb7c265015fcd400b6b0bbdf235._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 3""" + date="2014-10-31T20:31:39Z" + content=""" +`git-annex unused` looks at what data is used by git branches and tags, but +not by other commits. It's a reasonable request and I have made a todo for +it: [[todo/find_unused_in_any_commit]] .. But I am unure if it can be +implemented to run fast enough to be usable. +"""]] From f1ef94fd051ca288496691f247521fbb66650cf8 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Fri, 31 Oct 2014 16:42:49 -0400 Subject: [PATCH 44/66] comment --- ...t_1_ac386ca07e9aaf009c264f3deed83832._comment | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) create mode 100644 doc/forum/Using_git-annex/comment_1_ac386ca07e9aaf009c264f3deed83832._comment diff --git a/doc/forum/Using_git-annex/comment_1_ac386ca07e9aaf009c264f3deed83832._comment b/doc/forum/Using_git-annex/comment_1_ac386ca07e9aaf009c264f3deed83832._comment new file mode 100644 index 0000000000..9e0e8ff665 --- /dev/null +++ b/doc/forum/Using_git-annex/comment_1_ac386ca07e9aaf009c264f3deed83832._comment @@ -0,0 +1,16 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 1""" + date="2014-10-31T20:40:18Z" + content=""" +git does not allow pushing into a non-bare repsitory. This is why +bare repositories exist. + +You have basically 3 choices: + +1. Go to the first repo you made, and `git pull` from the other repo + into it. Pulling in this direction will work. +2. Set up a bare repo that both the non-bare repos use. +3. Use `git annex sync`, which avoids this problem. You need to run + it in both repos in order to get them to sync with one-another. +"""]] From bfac20c7448ba4ea94ad148cefa08423736d2efa Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Fri, 31 Oct 2014 17:23:02 -0400 Subject: [PATCH 45/66] comment --- ..._464d733de18f6ca438ebd84e88b8cee2._comment | 29 +++++++++++++++++++ 1 file changed, 29 insertions(+) create mode 100644 doc/bugs/rsync_fails_with_sync_error:_syntax_or_usage_error_/comment_1_464d733de18f6ca438ebd84e88b8cee2._comment diff --git a/doc/bugs/rsync_fails_with_sync_error:_syntax_or_usage_error_/comment_1_464d733de18f6ca438ebd84e88b8cee2._comment b/doc/bugs/rsync_fails_with_sync_error:_syntax_or_usage_error_/comment_1_464d733de18f6ca438ebd84e88b8cee2._comment new file mode 100644 index 0000000000..ea9db7bc69 --- /dev/null +++ b/doc/bugs/rsync_fails_with_sync_error:_syntax_or_usage_error_/comment_1_464d733de18f6ca438ebd84e88b8cee2._comment @@ -0,0 +1,29 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 1""" + date="2014-10-31T20:45:41Z" + content=""" +This is a rsync protocol level error; one side is sending something +that the other side fails to deal with. We can see in your log that the +two rsync are communicating successfully over the ssh connection +at first. + +This could be something not clean about your ssh connection, or some incompatability +in the versions of rsync or git-annex between the client and the server. +It probably wouldn't hurt to make sure client and server have the same rsync +version, and perhaps upgrade them both to the newest git-annex in case this +problem is somehow fixed there. + +Then, seems to me that the next step is to get git-annex-shell out of the +picture and see if you can still reproduce the problem. You can find the rsync +command that git-annex-shell runs by passing --debug to it. The just replace +the git-annex-shell command in your python code with the rsync command it runs. + +Here's a shell command line I came up with by doing that. It will have +different paths for your repo, and localhost will need to be changed to your +server's name. + +
+rsync --progress --inplace --perms --debug=all -e 'ssh  -4 -T localhost "rsync --server -t --inplace -e.Lsf . //home/joey/annex/.git/annex/tmp/SHA256E-s30--bdce956a335681853344fce6f1f940a5c8b7061007398661a3b14f2037843744" dummy rsync --server -pe.Lsfx --log-format=X --debug=ALL --inplace . .'  /tmp/annex/.git/annex/objects/Wx/Mf/SHA256E-s30--bdce956a335681853344fce6f1f940a5c8b7061007398661a3b14f2037843744/SHA256E-s30--bdce956a335681853344fce6f1f940a5c8b7061007398661a3b14f2037843744 dummy:
+
+"""]] From a029e558781ad2ef87b96ac0cb7d28ab89aa57f9 Mon Sep 17 00:00:00 2001 From: "https://renaud.casenave.fr/" Date: Sat, 1 Nov 2014 01:57:53 +0000 Subject: [PATCH 46/66] --- doc/bugs/git_annex_assistant_leaks_a_lot_of_memory.mdwn | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 doc/bugs/git_annex_assistant_leaks_a_lot_of_memory.mdwn diff --git a/doc/bugs/git_annex_assistant_leaks_a_lot_of_memory.mdwn b/doc/bugs/git_annex_assistant_leaks_a_lot_of_memory.mdwn new file mode 100644 index 0000000000..3f701b788b --- /dev/null +++ b/doc/bugs/git_annex_assistant_leaks_a_lot_of_memory.mdwn @@ -0,0 +1,9 @@ +Hi, + +I have a git-annex repository on a server managed by git annex assistant in indirect mode and as a backup group serving my org-mode files (organizer module for emacs) which are plain text files no more than a 100kb. + +I also have 2 clients in direct mode to sync the files across my computers. + +I save my files fairly often so the assistant might be a bit overwhelmed but I think it can managed that. At least it used to. + +Since maybe a couple months, the assistant running on the server has been leaking nearly 1Gb each day and I suspect it rises whenever I push files to it. From 624ae956de13bcdf2446d1c751a3631d8f9669b6 Mon Sep 17 00:00:00 2001 From: "http://svario.it/gioele" Date: Sat, 1 Nov 2014 14:12:35 +0000 Subject: [PATCH 47/66] new forum post "Why is git annex status slow?" --- .../Why_is_git_annex_status_slow__63__.mdwn | 20 +++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100644 doc/forum/Why_is_git_annex_status_slow__63__.mdwn diff --git a/doc/forum/Why_is_git_annex_status_slow__63__.mdwn b/doc/forum/Why_is_git_annex_status_slow__63__.mdwn new file mode 100644 index 0000000000..5c659b61d4 --- /dev/null +++ b/doc/forum/Why_is_git_annex_status_slow__63__.mdwn @@ -0,0 +1,20 @@ +It seems that `git annex status` is much slower than `git status`, at least in direct mode. The man page does not give any hint about why it should be slower. + +Does `git annex status` do something that `git status` does not? + +Here is an example in a repo with 8000+ files in direct mode and with no modified files: + + + $ time git -c core.bare=false status --porcelain > /dev/null + + real 0m0.096s + user 0m0.042s + sys 0m0.071s + + $ time git annex status + + real 0m17.144s + user 0m10.555s + sys 0m1.934s + +It is strange to see that `git annex status` is ~200 times slower than the bare `git status`. From 7447ef29fb27a7ac4cfaf7c71169e66ed008169f Mon Sep 17 00:00:00 2001 From: "http://svario.it/gioele" Date: Sat, 1 Nov 2014 16:40:56 +0000 Subject: [PATCH 48/66] new wish "--dry-run option for all commands" --- doc/todo/wishlist:_--dry-run_option_for_all_commands.mdwn | 3 +++ 1 file changed, 3 insertions(+) create mode 100644 doc/todo/wishlist:_--dry-run_option_for_all_commands.mdwn diff --git a/doc/todo/wishlist:_--dry-run_option_for_all_commands.mdwn b/doc/todo/wishlist:_--dry-run_option_for_all_commands.mdwn new file mode 100644 index 0000000000..fbea1b6faf --- /dev/null +++ b/doc/todo/wishlist:_--dry-run_option_for_all_commands.mdwn @@ -0,0 +1,3 @@ +Could a --dry-run option be added to the git annex commands? Or, at least, to the most common ones like `git annex add`. + +Given that there is no undo command, it would be nice to have the ability to simulate what git annex will do. From e99809528bd91623438d99311f85323bb0685ca5 Mon Sep 17 00:00:00 2001 From: "http://svario.it/gioele" Date: Sat, 1 Nov 2014 16:53:07 +0000 Subject: [PATCH 49/66] new bug "git annex status does not report added files" --- ...ex_status_does_not_report_added_files.mdwn | 23 +++++++++++++++++++ 1 file changed, 23 insertions(+) create mode 100644 doc/bugs/git_annex_status_does_not_report_added_files.mdwn diff --git a/doc/bugs/git_annex_status_does_not_report_added_files.mdwn b/doc/bugs/git_annex_status_does_not_report_added_files.mdwn new file mode 100644 index 0000000000..cd2de5bce5 --- /dev/null +++ b/doc/bugs/git_annex_status_does_not_report_added_files.mdwn @@ -0,0 +1,23 @@ +### Please describe the problem. + +`git annex status` does not report the fact that some files have been added but not yet committed. + +### What steps will reproduce the problem? + + $ # alwayscommit = false + $ echo "new" > new-file + $ git annex status + ? new-file + $ git annex add + add new-file + $ git annex status + $ + +Using the `git status` command directly will show the added files + + $ git -c core.bare=false status --porcelain | grep -v '^ T' + AT new-file + +### What version of git-annex are you using? On what operating system? + +git-annex version: 5.20141024-g613f396 From 8e7dd00353a7855f362b44170683aead72088cf9 Mon Sep 17 00:00:00 2001 From: "https://id.koumbit.net/lavamind" Date: Sun, 2 Nov 2014 00:25:53 +0000 Subject: [PATCH 50/66] --- .../webapp_does_not_start_on_android.mdwn | 20 +++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100644 doc/bugs/webapp_does_not_start_on_android.mdwn diff --git a/doc/bugs/webapp_does_not_start_on_android.mdwn b/doc/bugs/webapp_does_not_start_on_android.mdwn new file mode 100644 index 0000000000..2788a47bc9 --- /dev/null +++ b/doc/bugs/webapp_does_not_start_on_android.mdwn @@ -0,0 +1,20 @@ +### Please describe the problem. + +The webapp does not start automatically and instead this message is printed in the terminal: + + Falling back to hardcoded app location; cannot find expected files in /data/app-lib + git annex webapp + COLUMNS=0;LINES=0;export COLUMNS LINES; + +After typing `git annex webapp`, the webapp starts normally. + +If I click the WebApp entry in the terminal menu, nothing happens. + +### What steps will reproduce the problem? + +Launch Git Annex from the applications menu. + +### What version of git-annex are you using? On what operating system? + +Using 4.3/4.4 daily build (nov 1st 2014) apk. +CyanogemMod 11 (M10) From 3630a1242059c671d11e5e4b656a95c740667ed7 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawmp1ThsNNAbSn46ju-gwFELfStlhl8usJo" Date: Sun, 2 Nov 2014 13:03:49 +0000 Subject: [PATCH 51/66] --- ...hinks_remote__39__s_name_is_a_command.mdwn | 98 +++++++++++++++++++ 1 file changed, 98 insertions(+) create mode 100644 doc/bugs/git_annex_sync_thinks_remote__39__s_name_is_a_command.mdwn diff --git a/doc/bugs/git_annex_sync_thinks_remote__39__s_name_is_a_command.mdwn b/doc/bugs/git_annex_sync_thinks_remote__39__s_name_is_a_command.mdwn new file mode 100644 index 0000000000..c4788c2bb9 --- /dev/null +++ b/doc/bugs/git_annex_sync_thinks_remote__39__s_name_is_a_command.mdwn @@ -0,0 +1,98 @@ +### Please describe the problem. +git annex sync fails with "Unknown command 'i3'". i3 is the name of one annex. + + +### What steps will reproduce the problem? + +git annex clone i3:PATH annex + +git annex sync + +### What version of git-annex are you using? On what operating system? +git-annex version: 5.20140920-gb0c4300 +build flags: Assistant Webapp Webapp-secure Pairing Testsuite S3 WebDAV Inotify DBus DesktopNotify XMPP DNS Feeds Quvi TDFA CryptoHash +key/value backends: SHA256E SHA1E SHA512E SHA224E SHA384E SKEIN256E SKEIN512E SHA256 SHA1 SHA512 SHA224 SHA384 SKEIN256 SKEIN512 WORM URL +remote types: git gcrypt S3 bup directory rsync web webdav tahoe glacier ddar hook external + +on Linux Mint Qiana + +### Please provide any additional information below. + +[[!format sh """ +# If you can, paste a complete transcript of the problem occurring here. +# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log + + git annex sync +(merging origin/git-annex origin/synced/git-annex into git-annex...) +(Recording state in git...) +commit ok +pull origin +git-annex: Unknown command 'i3' + +Did you mean one of these? + fix + edit + init + find + add + get + vicfg + log + map + drop + move + copy + lock + sync + import + uninit + dead + fsck + direct + help + test + +fatal: Could not read from remote repository. + +Please make sure you have the correct access rights +and the repository exists. +failed +push origin +git-annex: Unknown command 'i3' + +Did you mean one of these? + fix + edit + init + find + add + get + vicfg + log + map + drop + move + copy + lock + sync + import + uninit + dead + fsck + direct + help + test + +fatal: Could not read from remote repository. + +Please make sure you have the correct access rights +and the repository exists. + + Pushing to origin failed. + + (non-fast-forward problems can be solved by setting receive.denyNonFastforwards to false in the remote's git config) +failed +git-annex: sync: 2 failed + +# End of transcript or log. +"""]] From 1b6b8a06334e766caf04553065a0a29393f1f1f2 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawlUGPPMvAP5Hu0NyNqeRMPC4pANeJNHZ0o" Date: Sun, 2 Nov 2014 17:46:57 +0000 Subject: [PATCH 52/66] Added a comment: why worry about modifications ? --- ...comment_2_ad51dced0c0146e1867830b93b520118._comment | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 doc/todo/wishlist:_use_hardlinks_for_local_clones/comment_2_ad51dced0c0146e1867830b93b520118._comment diff --git a/doc/todo/wishlist:_use_hardlinks_for_local_clones/comment_2_ad51dced0c0146e1867830b93b520118._comment b/doc/todo/wishlist:_use_hardlinks_for_local_clones/comment_2_ad51dced0c0146e1867830b93b520118._comment new file mode 100644 index 0000000000..8476b42207 --- /dev/null +++ b/doc/todo/wishlist:_use_hardlinks_for_local_clones/comment_2_ad51dced0c0146e1867830b93b520118._comment @@ -0,0 +1,10 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawlUGPPMvAP5Hu0NyNqeRMPC4pANeJNHZ0o" + nickname="Sylvain" + subject="why worry about modifications ?" + date="2014-11-02T17:46:57Z" + content=""" +I don't understand why you worry about having one remote modify the file locally which would get propagated to the other because of the hardlinks. When using checksumming, the repos would be pretty screwed if someone would do that anyways. + +In other words, hardlinking the SHA-checksummed files in .git/annex/objects/ across repos and/or between e.g. a directory remote and a repo would sound a pretty safe and nice optimization to me. I am like the OP, deduplication that does not require brtfs would be a great git annex feature. +"""]] From 3fd6a3171bcc0326e356ce5acfdad2e3ff03c747 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawk9SYh6N-JUMkYkW4aOk55zC3Vr9KonDV4" Date: Sun, 2 Nov 2014 19:19:28 +0000 Subject: [PATCH 53/66] --- .../Button_to_cancel_conistency_checks.mdwn | 21 +++++++++++++++++++ 1 file changed, 21 insertions(+) create mode 100644 doc/bugs/Button_to_cancel_conistency_checks.mdwn diff --git a/doc/bugs/Button_to_cancel_conistency_checks.mdwn b/doc/bugs/Button_to_cancel_conistency_checks.mdwn new file mode 100644 index 0000000000..2fd63dd2a8 --- /dev/null +++ b/doc/bugs/Button_to_cancel_conistency_checks.mdwn @@ -0,0 +1,21 @@ +### Please describe the problem. + +Sometimes a consistency check is undesired, e.g. when one wants to unmount the external drive. Add a button (along Configure) to the bubble that says Consistency check in progress to the assistent. + +### What steps will reproduce the problem? + +Run an automatic conistency check. + +### What version of git-annex are you using? On what operating system? + +20141024 + +### Please provide any additional information below. + +[[!format sh """ +# If you can, paste a complete transcript of the problem occurring here. +# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log + + +# End of transcript or log. +"""]] From 06903f6c2cd6c185ab99a94765e8040edacf79f3 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawlVB7wf1TFFrNPvM62fg78X-simVV-R1Cg" Date: Mon, 3 Nov 2014 02:21:17 +0000 Subject: [PATCH 54/66] Added a comment --- .../comment_2_50c0fd1e9dda9ccfff08765e892ad3c0._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/forum/Copy_contents_of_directory_to_rsync_remotes_possible__63__/comment_2_50c0fd1e9dda9ccfff08765e892ad3c0._comment diff --git a/doc/forum/Copy_contents_of_directory_to_rsync_remotes_possible__63__/comment_2_50c0fd1e9dda9ccfff08765e892ad3c0._comment b/doc/forum/Copy_contents_of_directory_to_rsync_remotes_possible__63__/comment_2_50c0fd1e9dda9ccfff08765e892ad3c0._comment new file mode 100644 index 0000000000..c4aceed19c --- /dev/null +++ b/doc/forum/Copy_contents_of_directory_to_rsync_remotes_possible__63__/comment_2_50c0fd1e9dda9ccfff08765e892ad3c0._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawlVB7wf1TFFrNPvM62fg78X-simVV-R1Cg" + nickname="Joe" + subject="comment 2" + date="2014-11-03T02:21:17Z" + content=""" +I was thinking that a call to `rsync /blah/.git/annex/tmp/rsynctmp/24863/2fe blah@blah:/home/shared-xfer/24863/` would work just as well, creating the necessary directories. But this requires knowing all the subdirectories of `24863`. Not sure if `git-annex` knows this. +"""]] From 0e4b6a65bdea9aafebb17d6345dadee88f48a6e6 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawlVB7wf1TFFrNPvM62fg78X-simVV-R1Cg" Date: Mon, 3 Nov 2014 02:21:32 +0000 Subject: [PATCH 55/66] Added a comment --- .../comment_3_09b7f1dd1fa4589b3fe0be0086849bf3._comment | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/forum/Copy_contents_of_directory_to_rsync_remotes_possible__63__/comment_3_09b7f1dd1fa4589b3fe0be0086849bf3._comment diff --git a/doc/forum/Copy_contents_of_directory_to_rsync_remotes_possible__63__/comment_3_09b7f1dd1fa4589b3fe0be0086849bf3._comment b/doc/forum/Copy_contents_of_directory_to_rsync_remotes_possible__63__/comment_3_09b7f1dd1fa4589b3fe0be0086849bf3._comment new file mode 100644 index 0000000000..1d6e912bc9 --- /dev/null +++ b/doc/forum/Copy_contents_of_directory_to_rsync_remotes_possible__63__/comment_3_09b7f1dd1fa4589b3fe0be0086849bf3._comment @@ -0,0 +1,8 @@ +[[!comment format=mdwn + username="https://www.google.com/accounts/o8/id?id=AItOawlVB7wf1TFFrNPvM62fg78X-simVV-R1Cg" + nickname="Joe" + subject="comment 3" + date="2014-11-03T02:21:32Z" + content=""" +I was thinking that a call to `rsync /blah/.git/annex/tmp/rsynctmp/24863/2fe blah@blah:/home/shared-xfer/24863/` would work just as well, creating the necessary directories. But this requires knowing all the subdirectories of `24863`. Not sure if `git-annex` knows this. +"""]] From e83416496d32aba37655f002da06bf128fab8316 Mon Sep 17 00:00:00 2001 From: "https://www.google.com/accounts/o8/id?id=AItOawlVB7wf1TFFrNPvM62fg78X-simVV-R1Cg" Date: Mon, 3 Nov 2014 02:21:58 +0000 Subject: [PATCH 56/66] removed --- .../comment_2_50c0fd1e9dda9ccfff08765e892ad3c0._comment | 8 -------- 1 file changed, 8 deletions(-) delete mode 100644 doc/forum/Copy_contents_of_directory_to_rsync_remotes_possible__63__/comment_2_50c0fd1e9dda9ccfff08765e892ad3c0._comment diff --git a/doc/forum/Copy_contents_of_directory_to_rsync_remotes_possible__63__/comment_2_50c0fd1e9dda9ccfff08765e892ad3c0._comment b/doc/forum/Copy_contents_of_directory_to_rsync_remotes_possible__63__/comment_2_50c0fd1e9dda9ccfff08765e892ad3c0._comment deleted file mode 100644 index c4aceed19c..0000000000 --- a/doc/forum/Copy_contents_of_directory_to_rsync_remotes_possible__63__/comment_2_50c0fd1e9dda9ccfff08765e892ad3c0._comment +++ /dev/null @@ -1,8 +0,0 @@ -[[!comment format=mdwn - username="https://www.google.com/accounts/o8/id?id=AItOawlVB7wf1TFFrNPvM62fg78X-simVV-R1Cg" - nickname="Joe" - subject="comment 2" - date="2014-11-03T02:21:17Z" - content=""" -I was thinking that a call to `rsync /blah/.git/annex/tmp/rsynctmp/24863/2fe blah@blah:/home/shared-xfer/24863/` would work just as well, creating the necessary directories. But this requires knowing all the subdirectories of `24863`. Not sure if `git-annex` knows this. -"""]] From 6ed2654adabcdf15df1a7a92f43a9b583e2253b2 Mon Sep 17 00:00:00 2001 From: "https://gregy.mojeid.cz/#RQxWK6xlii" Date: Mon, 3 Nov 2014 10:04:36 +0000 Subject: [PATCH 57/66] --- ...nnex_add_ignores_some_files__47__directories.mdwn | 12 ++++++++++++ 1 file changed, 12 insertions(+) create mode 100644 doc/bugs/git_annex_add_ignores_some_files__47__directories.mdwn diff --git a/doc/bugs/git_annex_add_ignores_some_files__47__directories.mdwn b/doc/bugs/git_annex_add_ignores_some_files__47__directories.mdwn new file mode 100644 index 0000000000..699ac7652e --- /dev/null +++ b/doc/bugs/git_annex_add_ignores_some_files__47__directories.mdwn @@ -0,0 +1,12 @@ +### Please describe the problem. +After I run git annex add some folders in current directory remain unannexed and git shows them as untracked. Even git annex add "problemDirName" does nothing. git annex add --force doesn't help either. + +### What steps will reproduce the problem? +No idea :( sometimes it happens and sometimes not. + +### What version of git-annex are you using? On what operating system? +5.20141013 on Debian testing + +### Please provide any additional information below. + +git add works fine. It correctly stages the files. From fb1a3d2867be71fe7f1dbf012aec3cb02670cce0 Mon Sep 17 00:00:00 2001 From: "https://gregy.mojeid.cz/#RQxWK6xlii" Date: Mon, 3 Nov 2014 10:08:00 +0000 Subject: [PATCH 58/66] --- doc/bugs/git_annex_add_ignores_some_files__47__directories.mdwn | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/bugs/git_annex_add_ignores_some_files__47__directories.mdwn b/doc/bugs/git_annex_add_ignores_some_files__47__directories.mdwn index 699ac7652e..35113574db 100644 --- a/doc/bugs/git_annex_add_ignores_some_files__47__directories.mdwn +++ b/doc/bugs/git_annex_add_ignores_some_files__47__directories.mdwn @@ -1,5 +1,5 @@ ### Please describe the problem. -After I run git annex add some folders in current directory remain unannexed and git shows them as untracked. Even git annex add "problemDirName" does nothing. git annex add --force doesn't help either. +After I run "git annex add" some folders (not empty) in current directory remain unannexed and git shows them as untracked. Even git annex add "problemDirName" does nothing. git annex add --force doesn't help either. ### What steps will reproduce the problem? No idea :( sometimes it happens and sometimes not. From 8887b6df7842115b86548435da9aa9ee013bd01b Mon Sep 17 00:00:00 2001 From: "https://gregy.mojeid.cz/#RQxWK6xlii" Date: Mon, 3 Nov 2014 10:10:59 +0000 Subject: [PATCH 59/66] --- doc/bugs/git_annex_add_ignores_some_files__47__directories.mdwn | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/bugs/git_annex_add_ignores_some_files__47__directories.mdwn b/doc/bugs/git_annex_add_ignores_some_files__47__directories.mdwn index 35113574db..6d947542ff 100644 --- a/doc/bugs/git_annex_add_ignores_some_files__47__directories.mdwn +++ b/doc/bugs/git_annex_add_ignores_some_files__47__directories.mdwn @@ -10,3 +10,5 @@ No idea :( sometimes it happens and sometimes not. ### Please provide any additional information below. git add works fine. It correctly stages the files. + +This is regular repository (not direct). From ec2c4bf67cfd67b7c77de45d3305be986dc7f0d1 Mon Sep 17 00:00:00 2001 From: "https://gregy.mojeid.cz/#RQxWK6xlii" Date: Mon, 3 Nov 2014 10:21:31 +0000 Subject: [PATCH 60/66] --- .../git_annex_add_ignores_some_files__47__directories.mdwn | 3 +++ 1 file changed, 3 insertions(+) diff --git a/doc/bugs/git_annex_add_ignores_some_files__47__directories.mdwn b/doc/bugs/git_annex_add_ignores_some_files__47__directories.mdwn index 6d947542ff..bf554b877d 100644 --- a/doc/bugs/git_annex_add_ignores_some_files__47__directories.mdwn +++ b/doc/bugs/git_annex_add_ignores_some_files__47__directories.mdwn @@ -12,3 +12,6 @@ No idea :( sometimes it happens and sometimes not. git add works fine. It correctly stages the files. This is regular repository (not direct). + +EDIT: +It seems that all the problem directories have either .git directory in them (I understand that git cannot manage .git but what about all the other files in there?) or are full of symlinks (git annex cannot manage symlinks?) From 8a66df78488717a607d4b55912457fca670c466f Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Mon, 3 Nov 2014 12:48:57 -0400 Subject: [PATCH 61/66] comment --- ...nt_1_03bf493d5a7f957339f9aa388ba85ef8._comment | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 doc/todo/wishlist:__--dry*/comment_1_03bf493d5a7f957339f9aa388ba85ef8._comment diff --git a/doc/todo/wishlist:__--dry*/comment_1_03bf493d5a7f957339f9aa388ba85ef8._comment b/doc/todo/wishlist:__--dry*/comment_1_03bf493d5a7f957339f9aa388ba85ef8._comment new file mode 100644 index 0000000000..2e314afbc5 --- /dev/null +++ b/doc/todo/wishlist:__--dry*/comment_1_03bf493d5a7f957339f9aa388ba85ef8._comment @@ -0,0 +1,15 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 1""" + date="2014-11-03T16:44:45Z" + content=""" +This would add a lot of complexity; it's not like I could switch off +running all external commands, since many external commands are run to +query state to decide what to do. And then there's large chunks of code +that actually do stuff and would have to all be guarded to not run. + +I don't see the benefit to justify this work. `git annex add` is entirely +predictable; it's very similar to `git add`. Which itself lacks a dry-run +option. And like `git add`, you can certianly undo the effects of `git +annex add`. +"""]] From 2819a466c7d40bc8b2f2b8d8ea24915c2020073d Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Mon, 3 Nov 2014 12:50:44 -0400 Subject: [PATCH 62/66] close non-bug --- .../git_annex_add_ignores_some_files__47__directories.mdwn | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/doc/bugs/git_annex_add_ignores_some_files__47__directories.mdwn b/doc/bugs/git_annex_add_ignores_some_files__47__directories.mdwn index bf554b877d..45af8d76a5 100644 --- a/doc/bugs/git_annex_add_ignores_some_files__47__directories.mdwn +++ b/doc/bugs/git_annex_add_ignores_some_files__47__directories.mdwn @@ -15,3 +15,9 @@ This is regular repository (not direct). EDIT: It seems that all the problem directories have either .git directory in them (I understand that git cannot manage .git but what about all the other files in there?) or are full of symlinks (git annex cannot manage symlinks?) + +> You should `git add` symlinks. They are not large files, so +> are out of scope for git-annex. +> +> Git repositories cannot contain other git repositories. [[done]] +> --[[Joey]] From e57745b8d6406835d235152874d012183584d71f Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Mon, 3 Nov 2014 13:02:31 -0400 Subject: [PATCH 63/66] followup --- ..._sync_thinks_remote__39__s_name_is_a_command.mdwn | 2 ++ ...mment_1_dd9490e8c9488a78475ccee1ffef08f5._comment | 12 ++++++++++++ 2 files changed, 14 insertions(+) create mode 100644 doc/bugs/git_annex_sync_thinks_remote__39__s_name_is_a_command/comment_1_dd9490e8c9488a78475ccee1ffef08f5._comment diff --git a/doc/bugs/git_annex_sync_thinks_remote__39__s_name_is_a_command.mdwn b/doc/bugs/git_annex_sync_thinks_remote__39__s_name_is_a_command.mdwn index c4788c2bb9..bdecdcae7e 100644 --- a/doc/bugs/git_annex_sync_thinks_remote__39__s_name_is_a_command.mdwn +++ b/doc/bugs/git_annex_sync_thinks_remote__39__s_name_is_a_command.mdwn @@ -96,3 +96,5 @@ git-annex: sync: 2 failed # End of transcript or log. """]] + +[[!tag moreinfo]] diff --git a/doc/bugs/git_annex_sync_thinks_remote__39__s_name_is_a_command/comment_1_dd9490e8c9488a78475ccee1ffef08f5._comment b/doc/bugs/git_annex_sync_thinks_remote__39__s_name_is_a_command/comment_1_dd9490e8c9488a78475ccee1ffef08f5._comment new file mode 100644 index 0000000000..8446926d8d --- /dev/null +++ b/doc/bugs/git_annex_sync_thinks_remote__39__s_name_is_a_command/comment_1_dd9490e8c9488a78475ccee1ffef08f5._comment @@ -0,0 +1,12 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 1""" + date="2014-11-03T16:55:45Z" + content=""" +I cannot reproduce this. (I made the hostname i3 point to localhost to test +it.) + +Please paste the output of `git annex sync --debug` and +`git config --list`. If you have any executable hooks in .git/hooks, please +check that they don't try to run "git annex i3" for some reason. +"""]] From 4f80dcc02a78905b1b7528eb2adb55fe46a22fdc Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Mon, 3 Nov 2014 13:05:21 -0400 Subject: [PATCH 64/66] comment --- ...omment_1_3874f5da5a300b7d443e1ad6373b4099._comment | 11 +++++++++++ 1 file changed, 11 insertions(+) create mode 100644 doc/bugs/git_annex_status_does_not_report_added_files/comment_1_3874f5da5a300b7d443e1ad6373b4099._comment diff --git a/doc/bugs/git_annex_status_does_not_report_added_files/comment_1_3874f5da5a300b7d443e1ad6373b4099._comment b/doc/bugs/git_annex_status_does_not_report_added_files/comment_1_3874f5da5a300b7d443e1ad6373b4099._comment new file mode 100644 index 0000000000..7e7cc139b1 --- /dev/null +++ b/doc/bugs/git_annex_status_does_not_report_added_files/comment_1_3874f5da5a300b7d443e1ad6373b4099._comment @@ -0,0 +1,11 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 1""" + date="2014-11-03T17:03:34Z" + content=""" +It's documentation says it shows files that have been deleted/modified/are +not checked into git. Not staged files. + +So, this is not a bug report, it's a request to make git annex status list +more files. +"""]] From 46180eab4da3a0f95676e826d2bb46760c858935 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Mon, 3 Nov 2014 13:06:26 -0400 Subject: [PATCH 65/66] todo item --- .../make_status_show_staged_files.mdwn} | 0 .../comment_1_3874f5da5a300b7d443e1ad6373b4099._comment | 0 2 files changed, 0 insertions(+), 0 deletions(-) rename doc/{bugs/git_annex_status_does_not_report_added_files.mdwn => todo/make_status_show_staged_files.mdwn} (100%) rename doc/{bugs/git_annex_status_does_not_report_added_files => todo/make_status_show_staged_files}/comment_1_3874f5da5a300b7d443e1ad6373b4099._comment (100%) diff --git a/doc/bugs/git_annex_status_does_not_report_added_files.mdwn b/doc/todo/make_status_show_staged_files.mdwn similarity index 100% rename from doc/bugs/git_annex_status_does_not_report_added_files.mdwn rename to doc/todo/make_status_show_staged_files.mdwn diff --git a/doc/bugs/git_annex_status_does_not_report_added_files/comment_1_3874f5da5a300b7d443e1ad6373b4099._comment b/doc/todo/make_status_show_staged_files/comment_1_3874f5da5a300b7d443e1ad6373b4099._comment similarity index 100% rename from doc/bugs/git_annex_status_does_not_report_added_files/comment_1_3874f5da5a300b7d443e1ad6373b4099._comment rename to doc/todo/make_status_show_staged_files/comment_1_3874f5da5a300b7d443e1ad6373b4099._comment From bf57edbdd6b0926bef42c9401dc888bab41a3457 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Mon, 3 Nov 2014 13:17:55 -0400 Subject: [PATCH 66/66] comment --- Command/Status.hs | 2 +- ...1_a5139c5a9fc94486cf62fc9a9fa9f8d9._comment | 18 ++++++++++++++++++ 2 files changed, 19 insertions(+), 1 deletion(-) create mode 100644 doc/forum/Why_is_git_annex_status_slow__63__/comment_1_a5139c5a9fc94486cf62fc9a9fa9f8d9._comment diff --git a/Command/Status.hs b/Command/Status.hs index 0d3efa8405..206b6a25e4 100644 --- a/Command/Status.hs +++ b/Command/Status.hs @@ -65,7 +65,7 @@ statusDirect f = checkstatus =<< liftIO (catchMaybeIO $ getFileStatus f) where checkstatus Nothing = return $ Just DeletedFile checkstatus (Just s) - -- Git thinks that present direct mode files modifed, + -- Git thinks that present direct mode files are modifed, -- so have to check. | not (isSymbolicLink s) = checkkey s =<< catKeyFile f | otherwise = Just <$> checkNew f diff --git a/doc/forum/Why_is_git_annex_status_slow__63__/comment_1_a5139c5a9fc94486cf62fc9a9fa9f8d9._comment b/doc/forum/Why_is_git_annex_status_slow__63__/comment_1_a5139c5a9fc94486cf62fc9a9fa9f8d9._comment new file mode 100644 index 0000000000..f21fcac488 --- /dev/null +++ b/doc/forum/Why_is_git_annex_status_slow__63__/comment_1_a5139c5a9fc94486cf62fc9a9fa9f8d9._comment @@ -0,0 +1,18 @@ +[[!comment format=mdwn + username="joey" + subject="""comment 1""" + date="2014-11-03T17:08:50Z" + content=""" +`git status` looks at the index and work tree. In an indirect mode +repository, `git annex status` does too, and is not significantly slower. + +In direct mode, `git annex status` has to look up from git the key +that corresponds to each file in the work tree. This is the main +thing that slows it down. + +(See the code for details, it's quite clear.) + +The best workaround is proably to pass git-annex status a subdirectory +that you're interested in, so it can only look at the contents of that one +directory. +"""]]