Merge branch 'master' of ssh://git-annex.branchable.com
This commit is contained in:
Joey Hess
commit
2a82567b6e
11 changed files with 277 additions and 1 deletions
|
|
@ -0,0 +1,8 @@
|
||||||
|
[[!comment format=mdwn
|
||||||
|
username="https://www.google.com/accounts/o8/id?id=AItOawkdiXn5s1wX2Zncy1S7_h-yPTv2gt0bmmY"
|
||||||
|
nickname="Jacob"
|
||||||
|
subject="Same problem on ubuntu"
|
||||||
|
date="2013-12-15T14:22:55Z"
|
||||||
|
content="""
|
||||||
|
Same problem, I even manually setup ssh keys between the two servers but it just loops and keeps asking for a password every sync.
|
||||||
|
"""]]
|
||||||
11
doc/bugs/OSX_build_broken.mdwn
Normal file
11
doc/bugs/OSX_build_broken.mdwn
Normal file
|
|
@ -0,0 +1,11 @@
|
||||||
|
### Please describe the problem.
|
||||||
|
|
||||||
|
The OSX webapp doesn't start on the current OSX 10.7.5 builds.
|
||||||
|
|
||||||
|
### What steps will reproduce the problem?
|
||||||
|
|
||||||
|
Download git-annex from http://downloads.kitenet.net/git-annex/OSX/current/10.7.5_Lion/git-annex.dmg and install it by copying it to Application. Try to launch it then. The result is a message in the system log saying "git-annex unknown command webapp" and then the usage information.
|
||||||
|
|
||||||
|
### What version of git-annex are you using? On what operating system?
|
||||||
|
|
||||||
|
The current version as downloaded from http://downloads.kitenet.net/git-annex/OSX/current/10.7.5_Lion/git-annex.dmg on OSX 10.7.5
|
||||||
|
|
@ -0,0 +1,132 @@
|
||||||
|
### Please describe the problem.
|
||||||
|
|
||||||
|
Since I last tested git-annex (version 4.20130516.1) my setup seems to no longer work in current versions (5.20131130)
|
||||||
|
|
||||||
|
### What steps will reproduce the problem?
|
||||||
|
|
||||||
|
[[!format sh """
|
||||||
|
$ mkdir 1 2
|
||||||
|
$ cd 1; git init; git-annex init; git annex direct; echo "file added to 1" > file_from_1; cd ..
|
||||||
|
Initialized empty Git repository in /home/pedrocr/testannexbug/1/.git/
|
||||||
|
init ok
|
||||||
|
(Recording state in git...)
|
||||||
|
commit
|
||||||
|
# On branch master
|
||||||
|
#
|
||||||
|
# Initial commit
|
||||||
|
#
|
||||||
|
nothing to commit (create/copy files and use "git add" to track)
|
||||||
|
ok
|
||||||
|
direct ok
|
||||||
|
$ cd 2; git init; git-annex init; git annex direct; echo "file added to 2" > file_from_2; cd ..
|
||||||
|
Initialized empty Git repository in /home/pedrocr/testannexbug/2/.git/
|
||||||
|
init ok
|
||||||
|
(Recording state in git...)
|
||||||
|
commit
|
||||||
|
# On branch master
|
||||||
|
#
|
||||||
|
# Initial commit
|
||||||
|
#
|
||||||
|
nothing to commit (create/copy files and use "git add" to track)
|
||||||
|
ok
|
||||||
|
direct ok
|
||||||
|
$ cd 1; git remote add 2 ssh://localhost/~pedrocr/Hacks/test-git-annex/2; git annex assistant; cd ..
|
||||||
|
$ cd 2; git remote add 1 ssh://localhost/~pedrocr/Hacks/test-git-annex/1; git annex assistant; cd ..
|
||||||
|
$ ls -la 1
|
||||||
|
total 20
|
||||||
|
drwxrwxr-x 3 pedrocr pedrocr 4096 Dec 15 11:33 .
|
||||||
|
drwxrwxr-x 4 pedrocr pedrocr 4096 Dec 15 11:33 ..
|
||||||
|
-rw-rw-r-- 1 pedrocr pedrocr 16 Dec 15 11:33 file_from_1
|
||||||
|
-rw-r--r-- 1 pedrocr pedrocr 16 Dec 15 11:33 file_from_2
|
||||||
|
drwxrwxr-x 9 pedrocr pedrocr 4096 Dec 15 11:33 .git
|
||||||
|
$ ls -la 2
|
||||||
|
total 20
|
||||||
|
drwxrwxr-x 3 pedrocr pedrocr 4096 Dec 15 11:33 .
|
||||||
|
drwxrwxr-x 4 pedrocr pedrocr 4096 Dec 15 11:33 ..
|
||||||
|
-rw-r--r-- 1 pedrocr pedrocr 16 Dec 15 11:33 file_from_1
|
||||||
|
-rw-rw-r-- 1 pedrocr pedrocr 16 Dec 15 11:33 file_from_2
|
||||||
|
drwxrwxr-x 9 pedrocr pedrocr 4096 Dec 15 11:33 .git
|
||||||
|
"""]]
|
||||||
|
|
||||||
|
so far so good
|
||||||
|
|
||||||
|
[[!format sh """
|
||||||
|
$ date | tee 1/file_from_1
|
||||||
|
Sun Dec 15 11:53:18 WET 2013
|
||||||
|
$ date | tee 2/file_from_2
|
||||||
|
Sun Dec 15 11:53:22 WET 2013
|
||||||
|
$ cat 1/*
|
||||||
|
Sun Dec 15 11:53:18 WET 2013
|
||||||
|
file added to 2
|
||||||
|
$ cat 2/*
|
||||||
|
file added to 1
|
||||||
|
Sun Dec 15 11:53:22 WET 2013
|
||||||
|
$
|
||||||
|
"""]]
|
||||||
|
|
||||||
|
But then syncing doesn't work. There doesn't seem to be anything special in the logs
|
||||||
|
|
||||||
|
[[!format sh """
|
||||||
|
$ tail -n 20 1/.git/annex/daemon.log
|
||||||
|
(Recording state in git...)
|
||||||
|
add file_from_1 (checksum...) [2013-12-15 11:53:18 WET] Committer: Committing changes to git
|
||||||
|
[2013-12-15 11:53:18 WET] Pusher: Syncing with 2
|
||||||
|
|
||||||
|
file_from_1
|
||||||
|
29 100% 0.00kB/s 0:00:00 (xfer#1, to-check=0/1)
|
||||||
|
|
||||||
|
sent 104 bytes received 31 bytes 270.00 bytes/sec
|
||||||
|
total size is 29 speedup is 0.21
|
||||||
|
[2013-12-15 11:53:18 WET] Transferrer: Uploaded file_from_1
|
||||||
|
Already up-to-date.
|
||||||
|
To ssh://localhost/~pedrocr/Hacks/test-git-annex/2
|
||||||
|
f025062..431ac37 git-annex -> synced/git-annex
|
||||||
|
d393bb3..fef60ca annex/direct/master -> synced/master
|
||||||
|
Already up-to-date.
|
||||||
|
[2013-12-15 11:53:21 WET] Pusher: Syncing with 2
|
||||||
|
To ssh://localhost/~pedrocr/Hacks/test-git-annex/2
|
||||||
|
431ac37..3cc7045 git-annex -> synced/git-annex
|
||||||
|
|
||||||
|
|
||||||
|
$ tail -n 20 2/.git/annex/daemon.log
|
||||||
|
(Recording state in git...)
|
||||||
|
add file_from_2 (checksum...) [2013-12-15 11:53:22 WET] Committer: Committing changes to git
|
||||||
|
[2013-12-15 11:53:22 WET] Pusher: Syncing with 1
|
||||||
|
|
||||||
|
file_from_2
|
||||||
|
29 100% 0.00kB/s 0:00:00 (xfer#1, to-check=0/1)
|
||||||
|
Already up-to-date.
|
||||||
|
|
||||||
|
sent 104 bytes received 31 bytes 270.00 bytes/sec
|
||||||
|
total size is 29 speedup is 0.21
|
||||||
|
[2013-12-15 11:53:22 WET] Transferrer: Uploaded file_from_2
|
||||||
|
To ssh://localhost/~pedrocr/Hacks/test-git-annex/1
|
||||||
|
ebbcd4c..dae7a77 git-annex -> synced/git-annex
|
||||||
|
ffd5a90..67ad7c1 annex/direct/master -> synced/master
|
||||||
|
Already up-to-date.
|
||||||
|
[2013-12-15 11:53:25 WET] Pusher: Syncing with 1
|
||||||
|
To ssh://localhost/~pedrocr/Hacks/test-git-annex/1
|
||||||
|
dae7a77..976c899 git-annex -> synced/git-annex
|
||||||
|
"""]]
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
### What version of git-annex are you using? On what operating system?
|
||||||
|
|
||||||
|
I'm running git-annex from the Ubuntu PPA on Ubuntu 12.04 LTS.
|
||||||
|
|
||||||
|
[[!format sh """
|
||||||
|
$ lsb_release -v
|
||||||
|
No LSB modules are available.
|
||||||
|
$ lsb_release -a
|
||||||
|
No LSB modules are available.
|
||||||
|
Distributor ID: Ubuntu
|
||||||
|
Description: Ubuntu 12.04.3 LTS
|
||||||
|
Release: 12.04
|
||||||
|
Codename: precise
|
||||||
|
$ git annex version
|
||||||
|
git-annex version: 5.20131130
|
||||||
|
build flags: Assistant Webapp Pairing S3 WebDAV Inotify DBus XMPP Feeds Quvi TDFA
|
||||||
|
key/value backends: SHA256E SHA1E SHA512E SHA224E SHA384E SHA256 SHA1 SHA512 SHA224 SHA384 WORM URL
|
||||||
|
remote types: git gcrypt S3 bup directory rsync web webdav glacier hook
|
||||||
|
"""]]
|
||||||
|
|
@ -0,0 +1,27 @@
|
||||||
|
[[!comment format=mdwn
|
||||||
|
username="TroisSinges"
|
||||||
|
ip="82.227.207.5"
|
||||||
|
subject="Almost done"
|
||||||
|
date="2013-12-15T10:13:50Z"
|
||||||
|
content="""
|
||||||
|
Hi!
|
||||||
|
|
||||||
|
Thank you for the fix, it's much better now. However, in some case, it doesn't work. For example, when I try to add inject a file in the direct repository (rm symlinkedFile ; mv /tmp/myFile symlinkedFile ; git annex add symlinkedFile), it tries to map each file in memory (which takes a very long time through the network).
|
||||||
|
|
||||||
|
# git annex add bigFile --debug
|
||||||
|
[2013-12-15 11:00:00 CET] read: git [\"--git-dir=/Volumes/Video/Annex/.git\",\"--work-tree=/Volumes/Video/Annex\",\"-c\",\"core.bare=false\",\"ls-files\",\"--others\",\"--exclude-standard\",\"-z\",\"--\",\"bigFile\"]
|
||||||
|
[2013-12-15 11:00:00 CET] read: git [\"--git-dir=/Volumes/Video/Annex/.git\",\"--work-tree=/Volumes/Video/Annex\",\"-c\",\"core.bare=false\",\"ls-files\",\"--modified\",\"-z\",\"--\",\"bigFile\"]
|
||||||
|
|
||||||
|
If I use lsof, each another file in repository is mapped in memory:
|
||||||
|
|
||||||
|
# lsof -p 30130
|
||||||
|
git 30130 troissinges 3r REG 46,11 4519526640 3093837953332665982 /Volumes/Video/Annex/AnotherBigFile
|
||||||
|
|
||||||
|
My version of software:
|
||||||
|
|
||||||
|
git annex version
|
||||||
|
git-annex version: 5.20131213
|
||||||
|
build flags: Assistant Webapp Pairing Testsuite S3 WebDAV FsEvents XMPP DNS Feeds Quvi TDFA CryptoHash
|
||||||
|
key/value backends: SHA256E SHA1E SHA512E SHA224E SHA384E SKEIN256E SKEIN512E SHA256 SHA1 SHA512 SHA224 SHA384 SKEIN256 SKEIN512 WORM URL
|
||||||
|
remote types: git gcrypt S3 bup directory rsync web webdav glacier hook
|
||||||
|
"""]]
|
||||||
27
doc/bugs/feature_request:_addhash.mdwn
Normal file
27
doc/bugs/feature_request:_addhash.mdwn
Normal file
|
|
@ -0,0 +1,27 @@
|
||||||
|
### Use case 1
|
||||||
|
|
||||||
|
I have a big repo using a SHA256E back-end. Along comes a new shiny SKEIN512E back-end and I would like to transition to using that, because it's faster and moves from ridiculously to ludicrously low risk of collisions.
|
||||||
|
|
||||||
|
I can set `.gitattributes` to use the new back-end for any new files added, but then I when I import some arbitrary mix of existing and new files to the repo it will not deduplicate any more, it will add all the files under the new hash scheme.
|
||||||
|
|
||||||
|
### Use case 2
|
||||||
|
|
||||||
|
I have a big repo of files I have added using `addurl --fast`. I download the files, and they are in the repo.
|
||||||
|
|
||||||
|
- I cannot verify later that none of them have been damaged.
|
||||||
|
- If I come across an offline collection of some of the files, I cannot easily get them into the annex by a simple import.
|
||||||
|
|
||||||
|
### Workaround
|
||||||
|
|
||||||
|
In both these cases, what I can do is unlock (maybe?) or unannex (definitely) all of the files, and then re-add them using the new hash. In both cases this means I now risk having duplicates in various clones of the repo, and would have to clean them up with `drop-unused` -- after first having re-copied them from a repo that has them under the new hash. I also lose the continuity of the history of that object.
|
||||||
|
|
||||||
|
In use case 2 I also lose the URLs of the files and would have to re-add them using `addurl`.
|
||||||
|
|
||||||
|
... which brings me to the proposed feature.
|
||||||
|
|
||||||
|
### addhash
|
||||||
|
|
||||||
|
Symmetrical to `addurl`, which in one form can take an existing hashed (or URL-sourced) file and add an URL source to it, `addhash` can take an existing URL-sourced (or hashed) file and add a hash reference to it (given that the file is in the annex so that the hash may be calculated) -- an alias under which it may also be identified, in addition to the existing URL or hash.
|
||||||
|
|
||||||
|
- Any file added to the annex after `addhash` will use the symlink name of the original hash if their hash matches the `addhash`ed one.
|
||||||
|
- An `fsck` run will use one of the available hashes to verify the integrity of the file, maybe according to some internal order of preference, or possibly a configurable one.
|
||||||
|
|
@ -0,0 +1,32 @@
|
||||||
|
### Please describe the problem.
|
||||||
|
git annex get fileName- should perform a hash check on the file content before adding to the local repository
|
||||||
|
|
||||||
|
|
||||||
|
### What steps will reproduce the problem?
|
||||||
|
Two scenarios:
|
||||||
|
1) Malicious user and owner of repository being pulled from can edit his/her local .git/annex/objects directory
|
||||||
|
to alter the file content. For src code, this could be to insert a bug, insert a backdoor, or for example
|
||||||
|
to replace an image file artifact for a website, with a pornographic image.
|
||||||
|
The user pulling the file content with "git annex get fileName" might not be aware of the file contents
|
||||||
|
until they actually examine the file or perform an fsck or commit locally.
|
||||||
|
In the meantime a kiddy porn image could be sitting in their repository or a src code backdoor can get incorporated and deployed etc.
|
||||||
|
2) a file could also simply get corrupted during download. An inherent hash check during the 'annex get' would
|
||||||
|
point out the problem immediately.
|
||||||
|
To reproduce: create repoNum1, and clone it to create repoNum2. manual edit/replace content in repoNum1/.git/annex/objects/...
|
||||||
|
then perform a 'git annex get <fileName>' from repoNum2 on the file that has been manipulated
|
||||||
|
|
||||||
|
|
||||||
|
### What version of git-annex are you using? On what operating system?
|
||||||
|
3.2012112ubuntu2 on running linux mint
|
||||||
|
|
||||||
|
|
||||||
|
### Please provide any additional information below.
|
||||||
|
|
||||||
|
Aside: Thanks Joey - this is fantastic work you are doing. You have really improved git. The ability to checkout
|
||||||
|
an entire tree - but selectively get only the content actually needed is a real killer feature.
|
||||||
|
Kudos and again many many thanks
|
||||||
|
M.
|
||||||
|
|
||||||
|
|
||||||
|
# End of transcript or log.
|
||||||
|
"""]]
|
||||||
|
|
@ -0,0 +1,8 @@
|
||||||
|
[[!comment format=mdwn
|
||||||
|
username="https://www.google.com/accounts/o8/id?id=AItOawniayrgSdVLUc3c6bf93VbO-_HT4hzxmyo"
|
||||||
|
nickname="Tobias"
|
||||||
|
subject="comment 8"
|
||||||
|
date="2013-12-14T13:15:13Z"
|
||||||
|
content="""
|
||||||
|
The master branch was checkout out (I don't know why). Checking out the mentioned branch fixed the syncing issue: `git checkout annex/direct/master`
|
||||||
|
"""]]
|
||||||
|
|
@ -0,0 +1,8 @@
|
||||||
|
[[!comment format=mdwn
|
||||||
|
username="https://me.yahoo.com/a/FHnTlSBo1eCGJRwueeKeB6.RCaPbGMPr5jxx8A--#ce0d8"
|
||||||
|
nickname="Hamza"
|
||||||
|
subject="comment 3"
|
||||||
|
date="2013-12-14T09:05:26Z"
|
||||||
|
content="""
|
||||||
|
remote info is recorded in git once you clone the git repo it knows about machine a and s3 repo.
|
||||||
|
"""]]
|
||||||
|
|
@ -0,0 +1,15 @@
|
||||||
|
[[!comment format=mdwn
|
||||||
|
username="https://www.google.com/accounts/o8/id?id=AItOawnfC02DTOZygFDDvvg0maRciYYMhEIFgkw"
|
||||||
|
nickname="Utku"
|
||||||
|
subject="comment 10"
|
||||||
|
date="2013-12-15T09:51:28Z"
|
||||||
|
content="""
|
||||||
|
unfortunately, it is not working on the computer where ssh was not working
|
||||||
|
I click an got annex and nothing seems to open
|
||||||
|
I try to go on the system logs but I see nothing really obvious. The only log that may be related would be :
|
||||||
|
15/12/13 10:46:23,230 Dock[166]: find_shared_window: WID 21425
|
||||||
|
15/12/13 10:46:23,230 Dock[166]: CGSGetWindowTags: Invalid window 0x53b1
|
||||||
|
|
||||||
|
if you tell me where to look at I can give you something more useful.
|
||||||
|
|
||||||
|
"""]]
|
||||||
|
|
@ -0,0 +1,8 @@
|
||||||
|
[[!comment format=mdwn
|
||||||
|
username="http://joeyh.name/"
|
||||||
|
ip="209.250.56.87"
|
||||||
|
subject="comment 9"
|
||||||
|
date="2013-12-13T18:52:40Z"
|
||||||
|
content="""
|
||||||
|
As a workaround, I have removed ssh-keygen and ssh from the OSX DMG. AIUI, OSX ships ssh by default anyway.
|
||||||
|
"""]]
|
||||||
|
|
@ -34,7 +34,7 @@ the app for OSX Lion.
|
||||||
|
|
||||||
<pre>
|
<pre>
|
||||||
brew update
|
brew update
|
||||||
brew install haskell-platform git ossp-uuid md5sha1sum coreutils libgsasl gnutls libidn libgsasl pkg-config libxml2
|
brew install haskell-platform git ossp-uuid md5sha1sum coreutils gnutls libidn gsasl pkg-config libxml2
|
||||||
brew link libxml2 --force
|
brew link libxml2 --force
|
||||||
cabal update
|
cabal update
|
||||||
mkdir $HOME/bin
|
mkdir $HOME/bin
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue