Merge branch 'master' of ssh://git-annex.branchable.com

This commit is contained in:
Joey Hess 2014-07-15 17:35:58 -04:00
commit 019ed359e6
13 changed files with 266 additions and 0 deletions

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="http://joeyh.name/"
ip="209.250.56.2"
subject="comment 7"
date="2014-07-15T19:15:13Z"
content="""
This bug and the other one are fixed in 5.20140709. I assume that your `git log` dates from an earlier version.
"""]]

View file

@ -0,0 +1,72 @@
[[!comment format=mdwn
username="divB"
ip="128.12.90.218"
subject="comment 2"
date="2014-07-15T20:51:07Z"
content="""
Unfortunately not. I did that already. At least in my opinion nothing special.
These are the latest lines:
[[!format sh \"\"\"
Please make sure you have the correct access rights
and the repository exists.
[2014-07-15 13:34:32 Pacific Daylight Time] read: git [\"--git-dir=c:\\data\\annex\\.git\",\"--work-tree=c:\\data\\annex\",\"-c\",\"core.bare=false\",\"show-ref\",\"git-annex\"]
[2014-07-15 13:34:32 Pacific Daylight Time] read: git [\"--git-dir=c:\\data\\annex\\.git\",\"--work-tree=c:\\data\\annex\",\"-c\",\"core.bare=false\",\"show-ref\",\"--hash\",\"refs/heads/git-annex\"]
[2014-07-15 13:34:32 Pacific Daylight Time] read: git [\"--git-dir=c:\\data\\annex\\.git\",\"--work-tree=c:\\data\\annex\",\"-c\",\"core.bare=false\",\"log\",\"refs/heads/git-annex..8ea0bbec209e1ec1240aa94ba2d6975023d83ac5\",\"-n1\",\"--pretty=%H\"]
[2014-07-15 13:34:32 Pacific Daylight Time] read: git [\"--git-dir=c:\\data\\annex\\.git\",\"--work-tree=c:\\data\\annex\",\"-c\",\"core.bare=false\",\"log\",\"refs/heads/git-annex..528b6b61318d138db5df2262ee501d6895b859d0\",\"-n1\",\"--pretty=%H\"]
[2014-07-15 13:34:32 Pacific Daylight Time] read: git [\"--git-dir=c:\\data\\annex\\.git\",\"--work-tree=c:\\data\\annex\",\"-c\",\"core.bare=false\",\"log\",\"refs/heads/git-annex..45aff729d5c9e60d925fad4c7fd75497ff5301fa\",\"-n1\",\"--pretty=%H\"]
[2014-07-15 13:34:32 Pacific Daylight Time] call: git [\"--git-dir=c:\\data\\annex\\.git\",\"--work-tree=c:\\data\\annex\",\"-c\",\"core.bare=false\",\"show-ref\",\"--verify\",\"-q\",\"refs/remotes/server/annex/direct/master\"]
[2014-07-15 13:34:32 Pacific Daylight Time] read: git [\"--git-dir=c:\\data\\annex\\.git\",\"--work-tree=c:\\data\\annex\",\"-c\",\"core.bare=false\",\"log\",\"refs/heads/annex/direct/master..refs/remotes/server/annex/direct/master\",\"-n1\",\"--pretty=%H\"]
[2014-07-15 13:34:32 Pacific Daylight Time] call: git [\"--git-dir=c:\\data\\annex\\.git\",\"--work-tree=c:\\data\\annex\",\"-c\",\"core.bare=false\",\"show-ref\",\"--verify\",\"-q\",\"refs/remotes/server/synced/master\"]
[2014-07-15 13:34:32 Pacific Daylight Time] read: git [\"--git-dir=c:\\data\\annex\\.git\",\"--work-tree=c:\\data\\annex\",\"-c\",\"core.bare=false\",\"log\",\"refs/heads/synced/master..refs/remotes/server/synced/master\",\"-n1\",\"--pretty=%H\"]
[2014-07-15 13:34:32 Pacific Daylight Time] Pusher: pushing to [Remote { name =\"server\" }]
[2014-07-15 13:34:32 Pacific Daylight Time] call: git [\"--git-dir=c:\\data\\annex\\.git\",\"--work-tree=c:\\data\\annex\",\"-c\",\"core.bare=false\",\"push\",\"server\",\"+git-annex:synced/git-annex\",\"annex/direct/master:synced/master\"]
ssh: Could not resolve hostname annex: hostname nor servname provided, or not known
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
[2014-07-15 13:34:34 Pacific Daylight Time] read: git [\"--git-dir=c:\\data\\annex\\.git\",\"--work-tree=c:\\data\\annex\",\"-c\",\"core.bare=false\",\"push\",\"server\",\"master\"]
ssh: Could not resolve hostname annex: hostname nor servname provided, or not known
[2014-07-15 13:34:35 Pacific Daylight Time] RemoteControl: DISCONNECTED ssh://test@annex/srv/data/Shared
[2014-07-15 13:34:35 Pacific Daylight Time] RemoteControl: fromList []
[2014-07-15 13:34:36 Pacific Daylight Time] Pusher: fallback pushing to [Remote { name =\"server\" }]
[2014-07-15 13:34:36 Pacific Daylight Time] call: git [\"--git-dir=c:\\data\\annex\\.git\",\"--work-tree=c:\\data\\annex\",\"-c\",\"core.bare=false\",\"push\",\"server\",\"+git-annex:refs/synced/91347741-60cd-4bca-a64a-90adee8d1910/git-annex\",\"refs/heads/annex/direct/master:refs/synced/91347741-60cd-4bca-a64a-90adee8d1910/annex/direct/master\"]
ssh: Could not resolve hostname annex: hostname nor servname provided, or not known
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
[2014-07-15 13:34:51 Pacific Daylight Time] chat: ssh [\"-T\",\"test@annex\",\"git-annex-shell 'notifychanges' '/srv/data/Shared' --uuid df1c0b77-f9b6-4ed7-96f3-3e7f010c4fb6\"]
ssh: Could not resolve hostname annex: hostname nor servname provided, or not known
[2014-07-15 13:35:03 Pacific Daylight Time] RemoteControl: DISCONNECTED ssh://test@annex/srv/data/Shared
[2014-07-15 13:35:03 Pacific Daylight Time] RemoteControl: fromList []
[2014-07-15 13:35:35 Pacific Daylight Time] chat: ssh [\"-T\",\"test@annex\",\"git-annex-shell 'notifychanges' '/srv/data/Shared' --uuid df1c0b77-f9b6-4ed7-96f3-3e7f010c4fb6\"]
ss[2014-07-15 13:35:47 Pacific Daylight Time] RemoteControl: DISCONNECTED ssh://test@annex/srv/data/Shared
[2014-07-15 13:35:47 Pacific Daylight Time] RemoteControl: fromList []
[2014-07-15 13:36:51 Pacific Daylight Time] chat: ssh [\"-T\",\"test@annex\",\"git-annex-shell 'notifychanges' '/srv/data/Shared' --uuid df1c0b77-f9b6-4ed7-96f3-3e7f010c4fb6\"]
ssh: Could not resolve hostname annex: hostname nor servname provided, or not known
[2014-07-15 13:37:02 Pacific Daylight Time] RemoteControl: DISCONNECTED ssh://test@annex/srv/data/Shared
[2014-07-15 13:37:02 Pacific Daylight Time] RemoteControl: fromList []
[2014-07-15 13:39:10 Pacific Daylight Time] chat: ssh [\"-T\",\"test@annex\",\"git-annex-shell 'notifychanges' '/srv/data/Shared' --uuid df1c0b77-f9b6-4ed7-96f3-3e7f010c4fb6\"]
ssh: Co[2014-07-15 13:39:21 Pacific Daylight Time] RemoteControl: DISCONNECTED ssh://test@annex/srv/data/Shared
[2014-07-15 13:39:21 Pacific Daylight Time] RemoteControl: fromList []
[2014-07-15 13:43:37 Pacific Daylight Time] chat: ssh [\"-T\",\"test@annex\",\"git-annex-shell 'notifychanges' '/srv/data/Shared' --uuid df1c0b77-f9b6-4ed7-96f3-3e7f010c4fb6\"]
ssh: Could not resolve hostname an[2014-07-15 13:43:49 Pacific Daylight Time] RemoteControl: DISCONNECTED ssh://test@annex/srv/data/Shared
[2014-07-15 13:43:49 Pacific Daylight Time] RemoteControl: fromList []
(Recording state in git...)
git-annex: <stdin>: hGetLine: invalid argument (Bad file descriptor)
failed
git-annex: assistant: 1 failed
\"\"\"]]
Maybe it's something different but what could it be?
"""]]

View file

@ -0,0 +1,20 @@
[[!comment format=mdwn
username="http://joeyh.name/"
ip="209.250.56.2"
subject="comment 3"
date="2014-07-15T20:42:15Z"
content="""
So let's think about whether gcrypt using gpg -R makes sense from the git-annex perspective.
Without -R, an attacker who can look at the remote, encrypted git repository can easily see the gpg keys of participants. This could allow them to perform other, more targeted attacks to get at the unencrypted repository.
If the user is using eg, github to store the gcrypt repo, an attacker can easily find out who owns the repo anyway, so they know who to attack then, even when -R is used. The -R can still prevent them from finding out when it's encrypted to additional users than the owner.
As far as the assistant goes, it only ever sets up a gcrypt repo with one participant. Using either an existing gpg key of the user, or generating a new one (which doesn't even have their name on it). Adding more participants to a gcrypt repo is tricky, and the assistant doesn't currently support it. But I'd like to have it support setting up multiple participants eventually. So the current limitations of the assistant are not a sufficient reason to avoid using -R.
So, it seems to come down to the question of whether it's a reasonable goal for git-annex, when used with gcrypt, to hide the identities of people who use a repository. And whether such a goal is worth the security/usability tradeoff of the user getting gpg passphrase prompts for other keys.
Hmm, when git-annex encrypts files to send them to a special remote, if it's using encryption=pubkey, it does not use -R. So an attacker can get the list of participants that way. Although the more common approach is for git-annex to encrypt using encryption=shared/hybrid, which uses a symmetric cipher, which avoids that problem. There are plenty of other things that git-annex can do that would leak identity.
It kind of seems to me that if you want to prevent anyone learning who uses your repository, you are going to need to be very cautious (using tor etc) and the most git-annex can do is be open about how it works and avoid adding any obstacles. Looking at it this way, such a user, or group of users, would be well-served by using gpg keys that don't have their names on them...
"""]]

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="http://joeyh.name/"
ip="209.250.56.2"
subject="comment 4"
date="2014-07-15T21:06:23Z"
content="""
@lex, it shouldn't fail due to passphrase if you have gpg-agent set up. You'll only get some unncessary passphrase prompts.
"""]]

View file

@ -0,0 +1,75 @@
Hello,
I am trying to wrap my head around annex still. I'm creating a source local git repo, editing an annex file, and then syncing in a second repo. In that second repo I'm trying to figure out how I can get a status notifying me that a file is out of date.
If I use 'annex sync --content' the files are all up to date, as expected, but what I kind of expected is 'annex status' to say something like 'files out of date, blah blah'. I am spelling out my example below.
Annex version is 5.20140613
## I create a source and target repo, and I add a git managed file and an annex managed file
$ mkdir source target
$ cd source
$ git init
$ git annex init
$ cat > gitfile
hi this is my git file
$ cat > annexfile
hi this is my annex file
$ git add gitfile
$ git annex add annexfile
$ git commit -m 'init commit'
$ cd ../target
$ git clone ../source/ .
$ ls -al
total 16
drwxr-xr-x 170 Jul 15 15:55 .
drwxr-xr-x 136 Jul 15 15:54 ..
drwxr-xr-x 442 Jul 15 15:55 .git
lrwxr-xr-x 180 Jul 15 15:55 annexfile -> .git/annex/objects/Qp/F0/SHA256E-s25--045cf30cb201c6723cb6fad9ca539f639de7f242b87775b876ef9ccb1f577ccf/SHA256E-s25--045cf30cb201c6723cb6fad9ca539f639de7f242b87775b876ef9ccb1f577ccf
-rw-r--r-- 23 Jul 15 15:55 gitfile
$ git annex sync
...
To [base path]/target/../source/
* [new branch] git-annex -> synced/git-annex
* [new branch] master -> synced/master
ok
$ cat gitfile
hi this is my git file
$ cat annexfile
cat: annexfile: No such file or directory
$ git annex sync --content
$ cat annexfile
hi this is my annex file
## So far so good. Now I'm going to edit my annexed file in source repo and see if I can find out that the file was edited in the target
$ cd ..
$ cd source
$ git annex edit annexfile
$ cat > annexfile
wow I changed my annex file
$ git annex add annexfile
$ git commit -m 'changed an annex file'
$ cd ..
$ cd target
$ git pull
$ cat annexfile
cat: annexfile: No such file or directory
$ git annex sync
$ ls
annexfile gitfile
$ cat annexfile
cat: annexfile: No such file or directory
$ git annex status
[no output]
## Here is where I'd expect something saying 'annexfile is out of date', etc.
## I can infer it because the link is missing, but I'm sure there's a more logical way.
$ git annex sync --content
$ cat annexfile
wow I changed my annex file
## After I synced content all is well.

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="http://joeyh.name/"
ip="209.250.56.2"
subject="comment 6"
date="2014-07-15T19:20:13Z"
content="""
msgit seems to install it in both places, I am not sure why. Either one works ok when I try to use them. The msgit installer puts \"c:\program files (x86)\git\cmd\\" into PATH so I assume that's the one you're supposed to use.
"""]]

View file

@ -0,0 +1,16 @@
[[!comment format=mdwn
username="divB"
ip="128.12.90.218"
subject="comment 7"
date="2014-07-15T20:33:07Z"
content="""
Thanks Joey,
The problem is more that git-annex assistant takes the wrong (which is in \"bin\" rather than \"cmd\"). I think this is a bug. Because this way the connection does not work in git-annex assistant the same way it does not work with the version in \"bin\" ...
I think when git-annex assistant just calls the git.exe from path (which should be cmd/git.exe) then it should work.
Regards
Niki
"""]]

View file

@ -0,0 +1,11 @@
[[!comment format=mdwn
username="http://joeyh.name/"
ip="209.250.56.2"
subject="comment 3"
date="2014-07-15T18:55:58Z"
content="""
I don't think it was a good idea to remove git from the NAS. To set up a git remote, you necessarily need to have git installed on the remote.
`git annex` should work as long as git-annex is somewhere in PATH -- unless your build of git is very strange and does not check PATH for git-foo commands.
Even if this was the case, you could run `git-annex` instead.
"""]]

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="http://joeyh.name/"
ip="209.250.56.2"
subject="comment 3"
date="2014-07-15T18:56:31Z"
content="""
It's there to represent the world wide web when using eg, `git annex addurl`
"""]]

View file

@ -0,0 +1,10 @@
[[!comment format=mdwn
username="http://joeyh.name/"
ip="209.250.56.2"
subject="comment 1"
date="2014-07-15T18:50:29Z"
content="""
It's normal for git-repair to take a long time.
git-annex uses checksums to detect if a file gets corrupted.
"""]]

View file

@ -0,0 +1,8 @@
[[!comment format=mdwn
username="http://joeyh.name/"
ip="209.250.56.2"
subject="comment 1"
date="2014-07-15T18:53:47Z"
content="""
Well, it's easy enough to make symlinks to content in a git-annex repository yourself if you want to. I don't see why this belongs in `git annex import`, which is too complicated already.
"""]]

View file

@ -0,0 +1,12 @@
[[!comment format=mdwn
username="http://joeyh.name/"
ip="209.250.56.2"
subject="comment 4"
date="2014-07-15T19:01:24Z"
content="""
So the goal is to inject any known objects from the dump into the local annex to avoid needing to re-transfer them.
It seems to me that in this case, you would not even want to create new symlinks in the git repository.
`git annex reinject` might be a better place to put code to handle this than `git annex import`.
"""]]

View file

@ -0,0 +1,10 @@
[[!comment format=mdwn
username="http://joeyh.name/"
ip="209.250.56.2"
subject="comment 5"
date="2014-07-15T19:13:05Z"
content="""
A fundamental problem with this idea is that git-annex's keys can use any of many checksumming backends. So, which checksum should it try? Running every possible checksum on a file is going to re-read it repeatedly and be expensive.
`git annex import` avoids this problem by using whatever the default backend is configured to be for the filename it's importing. This is good enough to make repeated runs of `git annex import` work ok, but when we get into trying to reinject whole directory trees like this, I don't think that's good enough.
"""]]