Merge branch 'master' of ssh://git-annex.branchable.com
This commit is contained in:
Joey Hess
commit
5cc406572b
12 changed files with 282 additions and 15 deletions
|
|
@ -0,0 +1,8 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://www.google.com/accounts/o8/id?id=AItOawmUJBh1lYmvfCCiGr3yrdx-QhuLCSRnU5c"
|
||||
nickname="Justin"
|
||||
subject="comment 5"
|
||||
date="2014-11-30T22:18:56Z"
|
||||
content="""
|
||||
This is getting in my way, but I'm concerned about using a mostly-untested branch (and also, compiling for ARM sounds really unpleasant). Can you elaborate on the risks here? Can I perhaps donate developer time to get this merged into mainline?
|
||||
"""]]
|
||||
22
doc/bugs/Uninstalling_removes_libcrypto.dll.mdwn
Normal file
22
doc/bugs/Uninstalling_removes_libcrypto.dll.mdwn
Normal file
|
|
@ -0,0 +1,22 @@
|
|||
### Please describe the problem.
|
||||
Just feedback to you:
|
||||
After I uninstalled git-annex (Windows), traditional git was not able to push changes to an origin server via ssh.
|
||||
Instead I got an error "libcrypto.dll is missing".
|
||||
Installing git once again after uninstalling git-annex solved the problem.
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
Installing and uninstalling git-annex.
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
File "git-annex-installer.exe" downloaded 2014-11-09.
|
||||
Windows 8.
|
||||
|
||||
### Please provide any additional information below.
|
||||
|
||||
[[!format sh """
|
||||
# If you can, paste a complete transcript of the problem occurring here.
|
||||
# If the problem is with the git-annex assistant, paste in .git/annex/daemon.log
|
||||
|
||||
|
||||
# End of transcript or log.
|
||||
"""]]
|
||||
35
doc/bugs/fsck_of_special_remotes_incorrectly_reports_ok.mdwn
Normal file
35
doc/bugs/fsck_of_special_remotes_incorrectly_reports_ok.mdwn
Normal file
|
|
@ -0,0 +1,35 @@
|
|||
### Please describe the problem.
|
||||
|
||||
I am trying to verify that my files are where they should after having updated my remotes to use chunking. I do get the following two error messages, but then git-annex reports that everything is Ok. I find it slightly scary and thus wonder if the fsck command in all cases do a conservative enough check. All other files (except two other ones where it noticed that they where in fact not on the remote do not report any errors.)
|
||||
|
||||
When running the fsck command for all files I got the following slightly worring message for one file.
|
||||
|
||||
fsck Ekonomi/2014-06-02_Elavräkning.pdf (checking box.com5...) (checking box.com5...) (checking box.com5...)
|
||||
79% 23.2KB/s 26s
|
||||
Error_Packet "partial packet: expecting 16448 bytes, got: 891"
|
||||
ok
|
||||
|
||||
Reading the message it seems that the file was not correctly downloaded, but still deemed Ok enough by git-annex.
|
||||
|
||||
Further while writing this report I get the following message from a remote using the external protocol:
|
||||
|
||||
fsck Deklaration/archive/2012/DE/Steuerklärung Formular.pdf
|
||||
93% 221.9KB/s 12s
|
||||
/Users/carlmod/Annex/.git/annex/tmp/GPGHMACSHA1--0e7e4951e2650ac636bc3fc17c673f1f2dfb3071: openBinaryFile: does not exist (No such file or directory)
|
||||
ok
|
||||
|
||||
Source code for this remote is not yet published online, as i would want to build it slightly more reliable before releasing it. It however passes the remote test suite without any problem.
|
||||
### What steps will reproduce the problem?
|
||||
|
||||
Not really sure, but rerunning the commands show that the openBinaryFile happens here for this particular file.
|
||||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
|
||||
$ git annex version
|
||||
git-annex version: 5.20141125
|
||||
build flags: Assistant Webapp Webapp-secure Pairing Testsuite S3 WebDAV FsEvents XMPP DNS Feeds Quvi TDFA CryptoHash
|
||||
key/value backends: SHA256E SHA1E SHA512E SHA224E SHA384E SKEIN256E SKEIN512E SHA256 SHA1 SHA512 SHA224 SHA384 SKEIN256 SKEIN512 WORM URL
|
||||
remote types: git gcrypt S3 bup directory rsync web webdav tahoe glacier ddar hook external
|
||||
|
||||
built using Homebrew (i.e. in a cabal sandbox) on OS X Yosemite 10.10.1
|
||||
|
||||
|
|
@ -1,6 +1,6 @@
|
|||
### Please describe the problem.
|
||||
|
||||
The new ''git annex proxy'' command fails every command in the 2014-11-28 daily build for Windows.
|
||||
The new ''git annex proxy'' command fails every call in the 2014-11-28 daily Windows build.
|
||||
|
||||
### What steps will reproduce the problem?
|
||||
|
||||
|
|
@ -15,7 +15,7 @@ The new ''git annex proxy'' command fails every command in the 2014-11-28 daily
|
|||
|
||||
### What version of git-annex are you using? On what operating system?
|
||||
|
||||
Daily Windows build 5.20141128-g70f997e
|
||||
Daily Windows build 5.20141128-g70f997e on Windows 7 x64
|
||||
|
||||
### Please provide any additional information below.
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,10 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://www.google.com/accounts/o8/id?id=AItOawmWBvsZvSsAL8P2ye3F0OBStjFCVnOImzM"
|
||||
nickname="Jarno"
|
||||
subject="Not a bug"
|
||||
date="2014-11-28T18:25:40Z"
|
||||
content="""
|
||||
This was, in fact, just an unclear error message. The command was missing \"git\", and so should have been:
|
||||
|
||||
$ git annex proxy -- git pull
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,45 @@
|
|||
The current command fails (git-annex 5.20141125, debian sid, DAV server: radicale 0.9-1):
|
||||
|
||||
$ WEBDAV_USERNAME=user WEBDAV_PASSWORD=password git annex --debug initremote dav.example.com type=webdav url=https://dav.example.com encryption=none
|
||||
[2014-11-28 16:53:29 CET] read: git ["--git-dir=/home/user/test-annex/.git","--work-tree=/home/user/test-annex","show-ref","git-annex"]
|
||||
[2014-11-28 16:53:29 CET] read: git ["--git-dir=/home/user/test-annex/.git","--work-tree=/home/user/test-annex","show-ref","--hash","refs/heads/git-annex"]
|
||||
[2014-11-28 16:53:29 CET] read: git ["--git-dir=/home/user/test-annex/.git","--work-tree=/home/user/test-annex","log","refs/heads/git-annex..f60365a0d5333e1aefe80a3fe747e2cb3d5c8162","-n1","--pretty=%H"]
|
||||
[2014-11-28 16:53:29 CET] chat: git ["--git-dir=/home/user/test-annex/.git","--work-tree=/home/user/test-annex","cat-file","--batch"]
|
||||
initremote dav.example.com (testing WebDAV server...)
|
||||
|
||||
git-annex: WebDAV test failed: StatusCodeException (Status {statusCode = 500, statusMessage = "Internal Server Error"}) [("Server","nginx/1.2.1"),("Date","Fri, 28 Nov 2014 15:53:37 GMT"),("Content-Type","text/plain"),("Content-Length","59"),("Connection","keep-alive"),("Keep-Alive","timeout=75"),("X-Response-Body-Start","A server error occurred. Please contact the administrator."),("X-Request-URL","MKCOL https://dav.example.com:443/tmp")] (CJ {expose = []}): user error
|
||||
failed
|
||||
git-annex: initremote: 1 failed
|
||||
|
||||
server log:
|
||||
|
||||
::ffff:192.168.1.6 - user [28/Nov/2014:16:53:17 +0100] "MKCOL / HTTP/1.1" 500 59 "-" "hDav-using application"
|
||||
::ffff:192.168.1.6 - user [28/Nov/2014:16:53:18 +0100] "PROPFIND /tmp HTTP/1.1" 500 59 "-" "hDav-using application"
|
||||
::ffff:192.168.1.6 - user [28/Nov/2014:16:53:21 +0100] "MKCOL /tmp HTTP/1.1" 500 59 "-" "hDav-using application"
|
||||
::ffff:192.168.1.6 - user [28/Nov/2014:16:53:32 +0100] "MKCOL / HTTP/1.1" 500 59 "-" "hDav-using application"
|
||||
::ffff:192.168.1.6 - user [28/Nov/2014:16:53:34 +0100] "PROPFIND /tmp HTTP/1.1" 500 59 "-" "hDav-using application"
|
||||
::ffff:192.168.1.6 - user [28/Nov/2014:16:53:37 +0100] "MKCOL /tmp HTTP/1.1" 500 59 "-" "hDav-using application"
|
||||
::ffff:192.168.1.6 - user [28/Nov/2014:16:58:31 +0100] "PROPFIND / HTTP/1.1" 207 1035 "-" "cadaver/0.23.3 neon/0.30.1"
|
||||
::ffff:192.168.1.6 - user [28/Nov/2014:16:59:19 +0100] "MKCOL / HTTP/1.1" 500 59 "-" "hDav-using application"
|
||||
::ffff:192.168.1.6 - user [28/Nov/2014:16:59:21 +0100] "PROPFIND /tmp HTTP/1.1" 500 59 "-" "hDav-using application"
|
||||
::ffff:192.168.1.6 - user [28/Nov/2014:16:59:23 +0100] "MKCOL /tmp HTTP/1.1" 500 59 "-" "hDav-using application"
|
||||
|
||||
|
||||
cadaver works:
|
||||
|
||||
$ cadaver https://dav.example.com
|
||||
Authentication required for Radicale - Password Required on server `dav.example.com':
|
||||
Username: user
|
||||
Password:
|
||||
dav:/> ls
|
||||
Listing collection `/': succeeded.
|
||||
Coll: user 0 Jan 1 1970
|
||||
|
||||
server log:
|
||||
|
||||
::ffff:192.168.1.6 - - [28/Nov/2014:17:01:31 +0100] "OPTIONS / HTTP/1.1" 200 0 "-" "cadaver/0.23.3 neon/0.30.1"
|
||||
::ffff:192.168.1.6 - - [28/Nov/2014:17:01:31 +0100] "PROPFIND / HTTP/1.1" 401 0 "-" "cadaver/0.23.3 neon/0.30.1"
|
||||
::ffff:192.168.1.6 - user [28/Nov/2014:17:01:42 +0100] "PROPFIND / HTTP/1.1" 207 579 "-" "cadaver/0.23.3 neon/0.30.1"
|
||||
::ffff:192.168.1.6 - user [28/Nov/2014:17:01:42 +0100] "PROPFIND / HTTP/1.1" 207 579 "-" "cadaver/0.23.3 neon/0.30.1"
|
||||
::ffff:192.168.1.6 - user [28/Nov/2014:17:02:01 +0100] "PROPFIND / HTTP/1.1" 207 1035 "-" "cadaver/0.23.3 neon/0.30.1"
|
||||
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
My PC froze while git-annex was in the process of importing a file. It was in the final stage of updating git status when the freeze occurred. git status shows the imported file under "changes to be committed" as "new file."
|
||||
|
||||
I don't know how git-annex works under the hood, so I don't know how the freeze might've affected the repo. I don't want to lose my files. What do you recommend is the best way of recovering from this and making sure there are no errors?
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://www.google.com/accounts/o8/id?id=AItOawmWBvsZvSsAL8P2ye3F0OBStjFCVnOImzM"
|
||||
nickname="Jarno"
|
||||
subject="How about with the 'proxy' command?"
|
||||
date="2014-11-28T18:47:17Z"
|
||||
content="""
|
||||
Why does this not work?
|
||||
|
||||
git annex proxy -- git reset HEAD
|
||||
|
||||
"""]]
|
||||
|
|
@ -1,13 +0,0 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://www.google.com/accounts/o8/id?id=AItOawmWBvsZvSsAL8P2ye3F0OBStjFCVnOImzM"
|
||||
nickname="Jarno"
|
||||
subject="How about with the 'proxy' command?"
|
||||
date="2014-11-27T22:40:17Z"
|
||||
content="""
|
||||
With the proxy command, would the following be an appropriate way to discard all changes (mods, adds, dels) since last sync in a direct mode repo?
|
||||
|
||||
git annex proxy -- git clean -df
|
||||
git annex proxy -- git checkout -f .
|
||||
|
||||
Also, is it necessary to run annex sync or other direct mode housekeeping commands afterwards?
|
||||
"""]]
|
||||
54
doc/forum/repair_stuck_on_ls-tree_command.mdwn
Normal file
54
doc/forum/repair_stuck_on_ls-tree_command.mdwn
Normal file
|
|
@ -0,0 +1,54 @@
|
|||
Hi,
|
||||
|
||||
I have run the git annex repair command on one of my repositories and it has been running for 38h hours now. I have monitored the process and its sons with htop and it seems that git annex repair is stuck.
|
||||
|
||||
It repeatedly launches the following command:
|
||||
|
||||
git --git-dir=/home/vincent/photos2/.git --work-tree=/home/vincent/photos2 ls-tree --full-tree -z -r -- 5fe5193e079631c0ceac0688ae0a6c1636491b61
|
||||
|
||||
I have tried to execute it by hand and it produces a lot of output. I have redirected it into some file in order to count the content produced by this command.
|
||||
|
||||
vincent@berlioz:~/photos2/.git$ git --git-dir=/home/vincent/photos2/.git --work-tree=/home/vincent/photos2 ls-tree --full-tree -z -r -- 5fe5193e079631c0ceac0688ae0a6c1636491b61 > /tmp/ls-tree
|
||||
vincent@berlioz:/tmp$ wc ls-tree
|
||||
0 232525 11611220 ls-tree
|
||||
|
||||
Could this cause some troubles to git-annex ? Maybe my repository is just too big and it demands some time to deal with the whole thing?
|
||||
|
||||
Before doing the git annex repair command I have tried to delete a distant annex (hosted on a raspberry pi, running an arm raspbian) from this one, using the git annex webapp website. Since it was kind of stuck I asked webapp to shutdown. I ran git annex repair for some time. It looked like stuck. I stopped it, then tried git fsck. It ran well. I tried git annex fsck, it took all my RAM and swap. The process was eating 14 Giga bytes of memory. I tried git gc and the same memory problem occured. I ran git annex repair again and it has been running for 38 hours now. I am lost. Please help.
|
||||
|
||||
vincent@berlioz:~$ LANG=C apt-cache policy git-annex
|
||||
git-annex:
|
||||
Installed: 5.20141125
|
||||
Candidate: 5.20141125
|
||||
Package pin: 5.20141125
|
||||
Version table:
|
||||
*** 5.20141125 950
|
||||
400 http://ftp.fr.debian.org/debian/ testing/main amd64 Packages
|
||||
50 http://ftp.fr.debian.org/debian/ unstable/main amd64 Packages
|
||||
100 /var/lib/dpkg/status
|
||||
3.20120629 950
|
||||
900 http://ftp.fr.debian.org/debian/ wheezy/main amd64 Packages
|
||||
|
||||
vincent@berlioz:~/photos2$ du -sh .
|
||||
152G .
|
||||
|
||||
Info gathered from another client annex:
|
||||
|
||||
vincent@tigrou:/mnt/mars/images/photos2$ git annex info
|
||||
repository mode: indirect
|
||||
trusted repositories: 0
|
||||
semitrusted repositories: 11
|
||||
8<------------skipped repositories list------------->8
|
||||
untrusted repositories: 1
|
||||
a357bb9e-7a9d-4112-b74c-13707e2c7f85 -- vincent@rhett:~/photos [gateway]
|
||||
transfers in progress: none
|
||||
available local disk space: 945.91 gigabytes (+1 megabyte reserved)
|
||||
bad keys size: 368 bytes (clean up with git-annex unused)
|
||||
local annex keys: 76070
|
||||
local annex size: 383.91 gigabytes
|
||||
annexed files in working tree: 59117
|
||||
size of annexed files in working tree: 319.73 gigabytes
|
||||
bloom filter size: 16 mebibytes (15.2% full)
|
||||
backend usage:
|
||||
SHA256E: 101902
|
||||
SHA256: 33285
|
||||
|
|
@ -0,0 +1,37 @@
|
|||
[[!comment format=mdwn
|
||||
username="vho"
|
||||
subject="Update"
|
||||
date="2014-12-01T15:47:23Z"
|
||||
content="""
|
||||
It seems I could sort things of.
|
||||
|
||||
I have run git annex sync from another annex. It took 30 minutes but could synchronize with the slow annex. Then I launched git annex sync on the slow annex (the one which seemed to had troubles with the repair command). It took 278 minutes, used all my ram and swap, but managed to end successfully.
|
||||
|
||||
I launched git annex sync again today, another 279 minutes:
|
||||
|
||||
vincent@berlioz:~/photos2$ time git annex sync
|
||||
commit ok
|
||||
pull tigrou
|
||||
Error reading response length from authentication socket.
|
||||
Compression automatique du dépôt en tâche de fond pour optimiser les performances.
|
||||
Voir \"git help gc\" pour toute information sur le nettoyage manuel.
|
||||
ok
|
||||
pull gateway
|
||||
Compression automatique du dépôt en tâche de fond pour optimiser les performances.
|
||||
Voir \"git help gc\" pour toute information sur le nettoyage manuel.
|
||||
ok
|
||||
push gateway
|
||||
fatal: The remote end hung up unexpectedly
|
||||
|
||||
Pushing to gateway failed.
|
||||
|
||||
(non-fast-forward problems can be solved by setting receive.denyNonFastforwards to false in the remote's git config)
|
||||
failed
|
||||
git-annex: sync: 1 failed
|
||||
|
||||
real 279m27.760s
|
||||
user 7m15.159s
|
||||
sys 3m4.340s
|
||||
|
||||
I will try to do some git gc.
|
||||
"""]]
|
||||
55
doc/tips/publishing_your_files_to_the_public.mdwn
Normal file
55
doc/tips/publishing_your_files_to_the_public.mdwn
Normal file
|
|
@ -0,0 +1,55 @@
|
|||
# Creating a special S3 remote to hold files shareable by URL
|
||||
|
||||
(In this example, I'll assume you'll be creating a bucket in S3 named **public-annex** and a special remote in git-annex, which will store its files in the previous bucket, named **public-s3**, but change these names if you are going to do the thing for real)
|
||||
|
||||
First, in the AWS dashboard, go to (or create) the bucket you will use at S3 and add a public get policy to it:
|
||||
|
||||
{
|
||||
"Version": "2008-10-17",
|
||||
"Statement": [
|
||||
{
|
||||
"Sid": "AllowPublicRead",
|
||||
"Effect": "Allow",
|
||||
"Principal": {
|
||||
"AWS": "*"
|
||||
},
|
||||
"Action": "s3:GetObject",
|
||||
"Resource": "arn:aws:s3:::public-annex/*"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
Then set up your special [S3](http://git-annex.branchable.com/special_remotes/S3/) remote with (at least) these options:
|
||||
|
||||
git annex initremote public-s3 type=s3 encryption=none bucket=public-annex chunk=0
|
||||
|
||||
This way git-annex will upload the files to this repo, (when you call `git annex copy [FILES...] --to public-s3`) without encrypting them and without chunking them, and, because of the policy of the bucket, they will be accessible by anyone with the link.
|
||||
|
||||
Following the example, the files will be accessible at `http://public-annex.s3.amazonaws.com/KEY` where `KEY` is the file key created by git-annex and which you can discover running
|
||||
|
||||
git annex lookupkey FILEPATH
|
||||
|
||||
This way you can share a link to each file you have at your S3 remote.
|
||||
|
||||
___________________
|
||||
|
||||
## Sharing all links in a folder
|
||||
|
||||
To share all the links in a given folder, for example, you can go to that folder and run (this is an example with the _fish_ shell, but I'm sure you can do the same in _bash_, I just don't know exactly):
|
||||
|
||||
for filename in (ls)
|
||||
echo $filename": https://public-annex.s3.amazonaws.com/"(git annex lookupkey $filename)
|
||||
end
|
||||
|
||||
## Sharing all links matching certain metadata
|
||||
|
||||
The same applies to all the filters you can do with git-annex.
|
||||
|
||||
For example, let's share links to all the files whose _author_'s name starts with "Mario" and are, in fact, stored at your public-s3 remote.
|
||||
However, instead of just a list of links we will output a markdown-formatted list of the filenames linked to their S3 files:
|
||||
|
||||
for filename in (git annex find --metadata "author=Mario*" --and --in public-s3)
|
||||
echo "* ["$filename"](https://public-annex.s3.amazonaws.com/"(git annex lookupkey $filename)")"
|
||||
end
|
||||
|
||||
Very useful.
|
||||
Loading…
Add table
Add a link
Reference in a new issue