Merge branch 'master' of ssh://git-annex.branchable.com
This commit is contained in:
Joey Hess
commit
2a82567b6e
11 changed files with 277 additions and 1 deletions
|
|
@ -0,0 +1,8 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://www.google.com/accounts/o8/id?id=AItOawkdiXn5s1wX2Zncy1S7_h-yPTv2gt0bmmY"
|
||||
nickname="Jacob"
|
||||
subject="Same problem on ubuntu"
|
||||
date="2013-12-15T14:22:55Z"
|
||||
content="""
|
||||
Same problem, I even manually setup ssh keys between the two servers but it just loops and keeps asking for a password every sync.
|
||||
"""]]
|
||||
11
doc/bugs/OSX_build_broken.mdwn
Normal file
11
doc/bugs/OSX_build_broken.mdwn
Normal file
|
|
@ -0,0 +1,11 @@
|
|||
### Please describe the problem.
|
||||
|
||||
The OSX webapp doesn't start on the current OSX 10.7.5 builds.
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
|
||||
Download git-annex from http://downloads.kitenet.net/git-annex/OSX/current/10.7.5_Lion/git-annex.dmg and install it by copying it to Application. Try to launch it then. The result is a message in the system log saying "git-annex unknown command webapp" and then the usage information.
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
|
||||
The current version as downloaded from http://downloads.kitenet.net/git-annex/OSX/current/10.7.5_Lion/git-annex.dmg on OSX 10.7.5
|
||||
|
|
@ -0,0 +1,132 @@
|
|||
### Please describe the problem.
|
||||
|
||||
Since I last tested git-annex (version 4.20130516.1) my setup seems to no longer work in current versions (5.20131130)
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
|
||||
[[!format sh """
|
||||
$ mkdir 1 2
|
||||
$ cd 1; git init; git-annex init; git annex direct; echo "file added to 1" > file_from_1; cd ..
|
||||
Initialized empty Git repository in /home/pedrocr/testannexbug/1/.git/
|
||||
init ok
|
||||
(Recording state in git...)
|
||||
commit
|
||||
# On branch master
|
||||
#
|
||||
# Initial commit
|
||||
#
|
||||
nothing to commit (create/copy files and use "git add" to track)
|
||||
ok
|
||||
direct ok
|
||||
$ cd 2; git init; git-annex init; git annex direct; echo "file added to 2" > file_from_2; cd ..
|
||||
Initialized empty Git repository in /home/pedrocr/testannexbug/2/.git/
|
||||
init ok
|
||||
(Recording state in git...)
|
||||
commit
|
||||
# On branch master
|
||||
#
|
||||
# Initial commit
|
||||
#
|
||||
nothing to commit (create/copy files and use "git add" to track)
|
||||
ok
|
||||
direct ok
|
||||
$ cd 1; git remote add 2 ssh://localhost/~pedrocr/Hacks/test-git-annex/2; git annex assistant; cd ..
|
||||
$ cd 2; git remote add 1 ssh://localhost/~pedrocr/Hacks/test-git-annex/1; git annex assistant; cd ..
|
||||
$ ls -la 1
|
||||
total 20
|
||||
drwxrwxr-x 3 pedrocr pedrocr 4096 Dec 15 11:33 .
|
||||
drwxrwxr-x 4 pedrocr pedrocr 4096 Dec 15 11:33 ..
|
||||
-rw-rw-r-- 1 pedrocr pedrocr 16 Dec 15 11:33 file_from_1
|
||||
-rw-r--r-- 1 pedrocr pedrocr 16 Dec 15 11:33 file_from_2
|
||||
drwxrwxr-x 9 pedrocr pedrocr 4096 Dec 15 11:33 .git
|
||||
$ ls -la 2
|
||||
total 20
|
||||
drwxrwxr-x 3 pedrocr pedrocr 4096 Dec 15 11:33 .
|
||||
drwxrwxr-x 4 pedrocr pedrocr 4096 Dec 15 11:33 ..
|
||||
-rw-r--r-- 1 pedrocr pedrocr 16 Dec 15 11:33 file_from_1
|
||||
-rw-rw-r-- 1 pedrocr pedrocr 16 Dec 15 11:33 file_from_2
|
||||
drwxrwxr-x 9 pedrocr pedrocr 4096 Dec 15 11:33 .git
|
||||
"""]]
|
||||
|
||||
so far so good
|
||||
|
||||
[[!format sh """
|
||||
$ date | tee 1/file_from_1
|
||||
Sun Dec 15 11:53:18 WET 2013
|
||||
$ date | tee 2/file_from_2
|
||||
Sun Dec 15 11:53:22 WET 2013
|
||||
$ cat 1/*
|
||||
Sun Dec 15 11:53:18 WET 2013
|
||||
file added to 2
|
||||
$ cat 2/*
|
||||
file added to 1
|
||||
Sun Dec 15 11:53:22 WET 2013
|
||||
$
|
||||
"""]]
|
||||
|
||||
But then syncing doesn't work. There doesn't seem to be anything special in the logs
|
||||
|
||||
[[!format sh """
|
||||
$ tail -n 20 1/.git/annex/daemon.log
|
||||
(Recording state in git...)
|
||||
add file_from_1 (checksum...) [2013-12-15 11:53:18 WET] Committer: Committing changes to git
|
||||
[2013-12-15 11:53:18 WET] Pusher: Syncing with 2
|
||||
|
||||
file_from_1
|
||||
29 100% 0.00kB/s 0:00:00 (xfer#1, to-check=0/1)
|
||||
|
||||
sent 104 bytes received 31 bytes 270.00 bytes/sec
|
||||
total size is 29 speedup is 0.21
|
||||
[2013-12-15 11:53:18 WET] Transferrer: Uploaded file_from_1
|
||||
Already up-to-date.
|
||||
To ssh://localhost/~pedrocr/Hacks/test-git-annex/2
|
||||
f025062..431ac37 git-annex -> synced/git-annex
|
||||
d393bb3..fef60ca annex/direct/master -> synced/master
|
||||
Already up-to-date.
|
||||
[2013-12-15 11:53:21 WET] Pusher: Syncing with 2
|
||||
To ssh://localhost/~pedrocr/Hacks/test-git-annex/2
|
||||
431ac37..3cc7045 git-annex -> synced/git-annex
|
||||
|
||||
|
||||
$ tail -n 20 2/.git/annex/daemon.log
|
||||
(Recording state in git...)
|
||||
add file_from_2 (checksum...) [2013-12-15 11:53:22 WET] Committer: Committing changes to git
|
||||
[2013-12-15 11:53:22 WET] Pusher: Syncing with 1
|
||||
|
||||
file_from_2
|
||||
29 100% 0.00kB/s 0:00:00 (xfer#1, to-check=0/1)
|
||||
Already up-to-date.
|
||||
|
||||
sent 104 bytes received 31 bytes 270.00 bytes/sec
|
||||
total size is 29 speedup is 0.21
|
||||
[2013-12-15 11:53:22 WET] Transferrer: Uploaded file_from_2
|
||||
To ssh://localhost/~pedrocr/Hacks/test-git-annex/1
|
||||
ebbcd4c..dae7a77 git-annex -> synced/git-annex
|
||||
ffd5a90..67ad7c1 annex/direct/master -> synced/master
|
||||
Already up-to-date.
|
||||
[2013-12-15 11:53:25 WET] Pusher: Syncing with 1
|
||||
To ssh://localhost/~pedrocr/Hacks/test-git-annex/1
|
||||
dae7a77..976c899 git-annex -> synced/git-annex
|
||||
"""]]
|
||||
|
||||
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
|
||||
I'm running git-annex from the Ubuntu PPA on Ubuntu 12.04 LTS.
|
||||
|
||||
[[!format sh """
|
||||
$ lsb_release -v
|
||||
No LSB modules are available.
|
||||
$ lsb_release -a
|
||||
No LSB modules are available.
|
||||
Distributor ID: Ubuntu
|
||||
Description: Ubuntu 12.04.3 LTS
|
||||
Release: 12.04
|
||||
Codename: precise
|
||||
$ git annex version
|
||||
git-annex version: 5.20131130
|
||||
build flags: Assistant Webapp Pairing S3 WebDAV Inotify DBus XMPP Feeds Quvi TDFA
|
||||
key/value backends: SHA256E SHA1E SHA512E SHA224E SHA384E SHA256 SHA1 SHA512 SHA224 SHA384 WORM URL
|
||||
remote types: git gcrypt S3 bup directory rsync web webdav glacier hook
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,27 @@
|
|||
[[!comment format=mdwn
|
||||
username="TroisSinges"
|
||||
ip="82.227.207.5"
|
||||
subject="Almost done"
|
||||
date="2013-12-15T10:13:50Z"
|
||||
content="""
|
||||
Hi!
|
||||
|
||||
Thank you for the fix, it's much better now. However, in some case, it doesn't work. For example, when I try to add inject a file in the direct repository (rm symlinkedFile ; mv /tmp/myFile symlinkedFile ; git annex add symlinkedFile), it tries to map each file in memory (which takes a very long time through the network).
|
||||
|
||||
# git annex add bigFile --debug
|
||||
[2013-12-15 11:00:00 CET] read: git [\"--git-dir=/Volumes/Video/Annex/.git\",\"--work-tree=/Volumes/Video/Annex\",\"-c\",\"core.bare=false\",\"ls-files\",\"--others\",\"--exclude-standard\",\"-z\",\"--\",\"bigFile\"]
|
||||
[2013-12-15 11:00:00 CET] read: git [\"--git-dir=/Volumes/Video/Annex/.git\",\"--work-tree=/Volumes/Video/Annex\",\"-c\",\"core.bare=false\",\"ls-files\",\"--modified\",\"-z\",\"--\",\"bigFile\"]
|
||||
|
||||
If I use lsof, each another file in repository is mapped in memory:
|
||||
|
||||
# lsof -p 30130
|
||||
git 30130 troissinges 3r REG 46,11 4519526640 3093837953332665982 /Volumes/Video/Annex/AnotherBigFile
|
||||
|
||||
My version of software:
|
||||
|
||||
git annex version
|
||||
git-annex version: 5.20131213
|
||||
build flags: Assistant Webapp Pairing Testsuite S3 WebDAV FsEvents XMPP DNS Feeds Quvi TDFA CryptoHash
|
||||
key/value backends: SHA256E SHA1E SHA512E SHA224E SHA384E SKEIN256E SKEIN512E SHA256 SHA1 SHA512 SHA224 SHA384 SKEIN256 SKEIN512 WORM URL
|
||||
remote types: git gcrypt S3 bup directory rsync web webdav glacier hook
|
||||
"""]]
|
||||
27
doc/bugs/feature_request:_addhash.mdwn
Normal file
27
doc/bugs/feature_request:_addhash.mdwn
Normal file
|
|
@ -0,0 +1,27 @@
|
|||
### Use case 1
|
||||
|
||||
I have a big repo using a SHA256E back-end. Along comes a new shiny SKEIN512E back-end and I would like to transition to using that, because it's faster and moves from ridiculously to ludicrously low risk of collisions.
|
||||
|
||||
I can set `.gitattributes` to use the new back-end for any new files added, but then I when I import some arbitrary mix of existing and new files to the repo it will not deduplicate any more, it will add all the files under the new hash scheme.
|
||||
|
||||
### Use case 2
|
||||
|
||||
I have a big repo of files I have added using `addurl --fast`. I download the files, and they are in the repo.
|
||||
|
||||
- I cannot verify later that none of them have been damaged.
|
||||
- If I come across an offline collection of some of the files, I cannot easily get them into the annex by a simple import.
|
||||
|
||||
### Workaround
|
||||
|
||||
In both these cases, what I can do is unlock (maybe?) or unannex (definitely) all of the files, and then re-add them using the new hash. In both cases this means I now risk having duplicates in various clones of the repo, and would have to clean them up with `drop-unused` -- after first having re-copied them from a repo that has them under the new hash. I also lose the continuity of the history of that object.
|
||||
|
||||
In use case 2 I also lose the URLs of the files and would have to re-add them using `addurl`.
|
||||
|
||||
... which brings me to the proposed feature.
|
||||
|
||||
### addhash
|
||||
|
||||
Symmetrical to `addurl`, which in one form can take an existing hashed (or URL-sourced) file and add an URL source to it, `addhash` can take an existing URL-sourced (or hashed) file and add a hash reference to it (given that the file is in the annex so that the hash may be calculated) -- an alias under which it may also be identified, in addition to the existing URL or hash.
|
||||
|
||||
- Any file added to the annex after `addhash` will use the symlink name of the original hash if their hash matches the `addhash`ed one.
|
||||
- An `fsck` run will use one of the available hashes to verify the integrity of the file, maybe according to some internal order of preference, or possibly a configurable one.
|
||||
|
|
@ -0,0 +1,32 @@
|
|||
### Please describe the problem.
|
||||
git annex get fileName- should perform a hash check on the file content before adding to the local repository
|
||||
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
Two scenarios:
|
||||
1) Malicious user and owner of repository being pulled from can edit his/her local .git/annex/objects directory
|
||||
to alter the file content. For src code, this could be to insert a bug, insert a backdoor, or for example
|
||||
to replace an image file artifact for a website, with a pornographic image.
|
||||
The user pulling the file content with "git annex get fileName" might not be aware of the file contents
|
||||
until they actually examine the file or perform an fsck or commit locally.
|
||||
In the meantime a kiddy porn image could be sitting in their repository or a src code backdoor can get incorporated and deployed etc.
|
||||
2) a file could also simply get corrupted during download. An inherent hash check during the 'annex get' would
|
||||
point out the problem immediately.
|
||||
To reproduce: create repoNum1, and clone it to create repoNum2. manual edit/replace content in repoNum1/.git/annex/objects/...
|
||||
then perform a 'git annex get <fileName>' from repoNum2 on the file that has been manipulated
|
||||
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
3.2012112ubuntu2 on running linux mint
|
||||
|
||||
|
||||
### Please provide any additional information below.
|
||||
|
||||
Aside: Thanks Joey - this is fantastic work you are doing. You have really improved git. The ability to checkout
|
||||
an entire tree - but selectively get only the content actually needed is a real killer feature.
|
||||
Kudos and again many many thanks
|
||||
M.
|
||||
|
||||
|
||||
# End of transcript or log.
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://www.google.com/accounts/o8/id?id=AItOawniayrgSdVLUc3c6bf93VbO-_HT4hzxmyo"
|
||||
nickname="Tobias"
|
||||
subject="comment 8"
|
||||
date="2013-12-14T13:15:13Z"
|
||||
content="""
|
||||
The master branch was checkout out (I don't know why). Checking out the mentioned branch fixed the syncing issue: `git checkout annex/direct/master`
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://me.yahoo.com/a/FHnTlSBo1eCGJRwueeKeB6.RCaPbGMPr5jxx8A--#ce0d8"
|
||||
nickname="Hamza"
|
||||
subject="comment 3"
|
||||
date="2013-12-14T09:05:26Z"
|
||||
content="""
|
||||
remote info is recorded in git once you clone the git repo it knows about machine a and s3 repo.
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://www.google.com/accounts/o8/id?id=AItOawnfC02DTOZygFDDvvg0maRciYYMhEIFgkw"
|
||||
nickname="Utku"
|
||||
subject="comment 10"
|
||||
date="2013-12-15T09:51:28Z"
|
||||
content="""
|
||||
unfortunately, it is not working on the computer where ssh was not working
|
||||
I click an got annex and nothing seems to open
|
||||
I try to go on the system logs but I see nothing really obvious. The only log that may be related would be :
|
||||
15/12/13 10:46:23,230 Dock[166]: find_shared_window: WID 21425
|
||||
15/12/13 10:46:23,230 Dock[166]: CGSGetWindowTags: Invalid window 0x53b1
|
||||
|
||||
if you tell me where to look at I can give you something more useful.
|
||||
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
[[!comment format=mdwn
|
||||
username="http://joeyh.name/"
|
||||
ip="209.250.56.87"
|
||||
subject="comment 9"
|
||||
date="2013-12-13T18:52:40Z"
|
||||
content="""
|
||||
As a workaround, I have removed ssh-keygen and ssh from the OSX DMG. AIUI, OSX ships ssh by default anyway.
|
||||
"""]]
|
||||
|
|
@ -34,7 +34,7 @@ the app for OSX Lion.
|
|||
|
||||
<pre>
|
||||
brew update
|
||||
brew install haskell-platform git ossp-uuid md5sha1sum coreutils libgsasl gnutls libidn libgsasl pkg-config libxml2
|
||||
brew install haskell-platform git ossp-uuid md5sha1sum coreutils gnutls libidn gsasl pkg-config libxml2
|
||||
brew link libxml2 --force
|
||||
cabal update
|
||||
mkdir $HOME/bin
|
||||
|
|
|
|||
Loading…
Reference in a new issue