Merge branch 'master' of ssh://git-annex.branchable.com
This commit is contained in:
Joey Hess
commit
e8b4eb0cdb
10 changed files with 279 additions and 0 deletions
|
|
@ -0,0 +1,40 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://www.google.com/accounts/o8/id?id=AItOawmXSkgjC_ypUQafVwvHTLsStrkiXH8CfHU"
|
||||
nickname="Matthias"
|
||||
subject="comment 2"
|
||||
date="2014-06-13T18:30:25Z"
|
||||
content="""
|
||||
Thanks for your fast reply, Joey. I appended the shortened config below. Wanted and groupwanted were untouched and therefore still commented out. I removed the hash from wanted to explicitly set it standard. Now the assistant behaves as expected. However I’m wondering:
|
||||
|
||||
* Shouldn’t ‘standard’ be the default rule to apply for wanted/groupwanted in case both are not explicitly set in the config? I assumed this to be the case.
|
||||
* Shouldn’t “find --want-get” list the identical set of files that is also requested by the assistant? Especially this one confused me.
|
||||
|
||||
[[!format sh \"\"\"
|
||||
# Repository trust configuration
|
||||
# (for macbook)
|
||||
#trust 3938865a-ce40-4166-a918-2012078846c7 = semitrusted
|
||||
|
||||
# Repository groups
|
||||
# (for macbook)
|
||||
group 3938865a-ce40-4166-a918-2012078846c7 = manual
|
||||
|
||||
# Repository preferred contents
|
||||
# (for macbook)
|
||||
#wanted 3938865a-ce40-4166-a918-2012078846c7 = standard
|
||||
|
||||
# Group preferred contents
|
||||
# (Used by repositories with \"groupwanted\" in their preferred contents)
|
||||
(config defaults - all commented out)
|
||||
|
||||
# Standard preferred contents
|
||||
(config defaults - all commented out)
|
||||
|
||||
# Repository required contents
|
||||
# (for macbook)
|
||||
#required 3938865a-ce40-4166-a918-2012078846c7 =
|
||||
|
||||
# Scheduled activities
|
||||
# (for macbook)
|
||||
#schedule 3938865a-ce40-4166-a918-2012078846c7 =
|
||||
\"\"\"]]
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
[[!comment format=mdwn
|
||||
username="http://joeyh.name/"
|
||||
ip="108.236.230.124"
|
||||
subject="comment 3"
|
||||
date="2014-06-13T18:35:53Z"
|
||||
content="""
|
||||
I need to look into why --want-get didn't behave as expected.
|
||||
|
||||
It would probably make sense to make standard be the default. There could be upgrade concerns though.
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,13 @@
|
|||
[[!comment format=mdwn
|
||||
username="http://joeyh.name/"
|
||||
ip="108.236.230.124"
|
||||
subject="comment 1"
|
||||
date="2014-06-16T15:10:42Z"
|
||||
content="""
|
||||
Reproduced this as described.
|
||||
|
||||
Only happens when using the assistant in the second repository, which is a ssh remote of the first, and when there is not also a remote going the other way (so does not affect pairing).
|
||||
|
||||
As I thought, the problem is that the master branch is not updated by the assistant when in direct mode, and while annex/direct/master is pulled by the other assistant, it does not merge it.
|
||||
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://www.google.com/accounts/o8/id?id=AItOawnwfqF4wL6l_O26RyzoBowUMvQ_955Vpao"
|
||||
nickname="Markus"
|
||||
subject="comment 7"
|
||||
date="2014-06-14T06:29:55Z"
|
||||
content="""
|
||||
FWIW: Removing the [raspi-copies-and-fills](https://github.com/simonjhall/copies-and-fills) package ('apt-get purge raspi-copies-and-fills') stops annex from printing the message over and over again. According to the package's description though, you suffer a performance penalty without the package.
|
||||
"""]]
|
||||
26
doc/forum/View_performance_with_7__44__000_files.mdwn
Normal file
26
doc/forum/View_performance_with_7__44__000_files.mdwn
Normal file
|
|
@ -0,0 +1,26 @@
|
|||
I've imported about half of my photos into an annex on an external HDD,
|
||||
using metadata extensively for EXIF info, including place names.
|
||||
|
||||
Checking out a new view is slower than I expected, at ~6 minutes.
|
||||
Is this expected behavior, am I pushing the limits of file count already?
|
||||
|
||||
Is there anything I can do to speed things up?
|
||||
|
||||
% ls -1 | wc -l
|
||||
7050
|
||||
|
||||
% git branch -vv
|
||||
git-annex 4e590d4 update
|
||||
* master 985ba54 add jen's phone backups
|
||||
views/Year=_;Month=_;Address=_ 795a58b refs/heads/views/Year=_;Month=_;Address=_
|
||||
|
||||
% /usr/bin/time -p git annex view "Year=*" "Address=*"
|
||||
view (searching...)
|
||||
|
||||
Checking out files: 100% (12789/12789), done.
|
||||
Switched to branch 'views/Year=_;Address=_'
|
||||
ok
|
||||
real 376.80
|
||||
user 17.98
|
||||
sys 9.66
|
||||
|
||||
|
|
@ -0,0 +1,34 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://www.google.com/accounts/o8/id?id=AItOawmvzzyDA8uXFz8yokeCrepbh8PwWe_WrjE"
|
||||
nickname="Michael"
|
||||
subject="debug output"
|
||||
date="2014-06-16T14:17:06Z"
|
||||
content="""
|
||||
So after looking for a debug flag, I see that it's spending all its time in cat-file, which I guess is probably not a surprise:
|
||||
|
||||
% /usr/bin/time -p git annex view \"Year=*\" \"Address=*\" -d
|
||||
view [2014-06-16 06:29:12 PDT] read: git [\"--git-dir=/Volumes/Four TB Backup/Photos/.git\",\"--work-tree=/Volumes/Four TB Backup/Photos\",\"symbolic-ref\",\"HEAD\"]
|
||||
[2014-06-16 06:29:12 PDT] read: git [\"--git-dir=/Volumes/Four TB Backup/Photos/.git\",\"--work-tree=/Volumes/Four TB Backup/Photos\",\"show-ref\",\"refs/heads/master\"]
|
||||
[2014-06-16 06:29:12 PDT] read: git [\"--git-dir=/Volumes/Four TB Backup/Photos/.git\",\"--work-tree=/Volumes/Four TB Backup/Photos\",\"symbolic-ref\",\"HEAD\"]
|
||||
[2014-06-16 06:29:12 PDT] read: git [\"--git-dir=/Volumes/Four TB Backup/Photos/.git\",\"--work-tree=/Volumes/Four TB Backup/Photos\",\"show-ref\",\"refs/heads/master\"]
|
||||
(searching...)
|
||||
[2014-06-16 06:29:12 PDT] read: git [\"--git-dir=/Volumes/Four TB Backup/Photos/.git\",\"--work-tree=/Volumes/Four TB Backup/Photos\",\"ls-files\",\"--cached\",\"-z\",\"--\",\"/Volumes/Four TB Backup/Photos\"]
|
||||
[2014-06-16 06:29:12 PDT] feed: git [\"--git-dir=/Volumes/Four TB Backup/Photos/.git\",\"--work-tree=/Volumes/Four TB Backup/Photos\",\"update-index\",\"-z\",\"--index-info\"]
|
||||
[2014-06-16 06:29:12 PDT] chat: git [\"--git-dir=/Volumes/Four TB Backup/Photos/.git\",\"--work-tree=/Volumes/Four TB Backup/Photos\",\"hash-object\",\"-w\",\"--stdin-paths\",\"--no-filters\"]
|
||||
[2014-06-16 06:29:12 PDT] read: git [\"--git-dir=/Volumes/Four TB Backup/Photos/.git\",\"--work-tree=/Volumes/Four TB Backup/Photos\",\"show-ref\",\"git-annex\"]
|
||||
[2014-06-16 06:29:12 PDT] read: git [\"--git-dir=/Volumes/Four TB Backup/Photos/.git\",\"--work-tree=/Volumes/Four TB Backup/Photos\",\"show-ref\",\"--hash\",\"refs/heads/git-annex\"]
|
||||
[2014-06-16 06:29:12 PDT] read: git [\"--git-dir=/Volumes/Four TB Backup/Photos/.git\",\"--work-tree=/Volumes/Four TB Backup/Photos\",\"log\",\"refs/heads/git-annex..4e590d433e01886e2823c3316d18b7e3cbafe227\",\"--oneline\",\"-n1\"]
|
||||
[2014-06-16 06:29:12 PDT] chat: git [\"--git-dir=/Volumes/Four TB Backup/Photos/.git\",\"--work-tree=/Volumes/Four TB Backup/Photos\",\"cat-file\",\"--batch\"]
|
||||
[2014-06-16 06:34:54 PDT] read: git [\"--git-dir=/Volumes/Four TB Backup/Photos/.git\",\"--work-tree=/Volumes/Four TB Backup/Photos\",\"write-tree\"]
|
||||
[2014-06-16 06:34:55 PDT] chat: git [\"--git-dir=/Volumes/Four TB Backup/Photos/.git\",\"--work-tree=/Volumes/Four TB Backup/Photos\",\"commit-tree\",\"c0fd1ea9ccae70a7353a2e787089b88115a42b7a\"]
|
||||
[2014-06-16 06:34:55 PDT] call: git [\"--git-dir=/Volumes/Four TB Backup/Photos/.git\",\"--work-tree=/Volumes/Four TB Backup/Photos\",\"update-ref\",\"refs/heads/views/Year=_;Address=_\",\"249c37d78e45d1fccfcb00f40337e05cb69fb64c\"]
|
||||
|
||||
[2014-06-16 06:34:55 PDT] call: git [\"--git-dir=/Volumes/Four TB Backup/Photos/.git\",\"--work-tree=/Volumes/Four TB Backup/Photos\",\"checkout\",\"views/Year=_;Address=_\"]
|
||||
Checking out files: 100% (12789/12789), done.
|
||||
Switched to branch 'views/Year=_;Address=_'
|
||||
ok
|
||||
real 362.14
|
||||
user 18.16
|
||||
sys 7.70
|
||||
|
||||
"""]]
|
||||
76
doc/forum/downloading_from_moodle.mdwn
Normal file
76
doc/forum/downloading_from_moodle.mdwn
Normal file
|
|
@ -0,0 +1,76 @@
|
|||
my school uses moodle for our classes. We have to sign in and then manually click to download each file, assignment and video uploaded. I asked the school's tech administrator if there was a direct way I could access the videos through the ssh access they've given us to one of the servers, but he said it wasn't possible.
|
||||
|
||||
when I click on the link shown, the location I see is: http://moodle.jct.ac.il/mod/resource/view.php?id=135374
|
||||
Inspect element gives more information. the response from the server is:
|
||||
|
||||
Remote Address:147.161.6.59:80
|
||||
Request URL:http://moodle.jct.ac.il/mod/resource/view.php?id=135374
|
||||
Request Method:GET
|
||||
Status Code:303 See Other
|
||||
Request Headersview source
|
||||
Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
|
||||
Accept-Encoding:gzip,deflate,sdch
|
||||
Accept-Language:en-US,en;q=0.8,he;q=0.6
|
||||
Connection:keep-alive
|
||||
Cookie:visid_incap_97364=qJx2WaKqQfGidGf9VfM6QWrnlFIAAAAAQUIPAAAAAAC5EXcbt00vqNu9jdVDwEDN; __utma=98014340.1409421308.1381214363.1381214363.1390387318.2; __utmz=98014340.1381214363.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); MoodleSession5771=7s1cfqfo4ahdtmna5h7vserg97; MOODLEID1_5771=%257F%25D39%2522N%25B4%25AFY
|
||||
DNT:1
|
||||
Host:moodle.jct.ac.il
|
||||
Referer:http://moodle.jct.ac.il/course/view.php?id=20151
|
||||
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.67 Safari/537.36
|
||||
Query String Parametersview sourceview URL encoded
|
||||
id:135374
|
||||
Response Headersview source
|
||||
Cache-Control:no-store, no-cache, must-revalidate, post-check=0, pre-check=0
|
||||
Connection:Keep-Alive
|
||||
Content-Encoding:gzip
|
||||
Content-Language:he
|
||||
Content-Length:503
|
||||
Content-Type:text/html
|
||||
Date:Mon, 16 Jun 2014 12:31:22 GMT
|
||||
Expires:Thu, 19 Nov 1981 08:52:00 GMT
|
||||
Keep-Alive:timeout=15, max=100
|
||||
Location:http://moodle.jct.ac.il/pluginfile.php/288409/mod_resource/content/0/movie%205773/150151.5773.week1.wmv?forcedownload=1
|
||||
Pragma:no-cache
|
||||
Server:Apache/2.2.14 (Ubuntu)
|
||||
Vary:Accept-Encoding
|
||||
X-Powered-By:PHP/5.3.2-1ubuntu4.24
|
||||
|
||||
this then pulls the following:
|
||||
|
||||
Remote Address:147.161.6.59:80
|
||||
Request URL:http://moodle.jct.ac.il/pluginfile.php/288409/mod_resource/content/0/movie%205773/150151.5773.week1.wmv?forcedownload=1
|
||||
Request Method:GET
|
||||
Status Code:200 OK
|
||||
Request Headersview source
|
||||
Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
|
||||
Accept-Encoding:gzip,deflate,sdch
|
||||
Accept-Language:en-US,en;q=0.8,he;q=0.6
|
||||
Connection:keep-alive
|
||||
Cookie:visid_incap_97364=qJx2WaKqQfGidGf9VfM6QWrnlFIAAAAAQUIPAAAAAAC5EXcbt00vqNu9jdVDwEDN; __utma=98014340.1409421308.1381214363.1381214363.1390387318.2; __utmz=98014340.1381214363.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); MoodleSession5771=7s1cfqfo4ahdtmna5h7vserg97; MOODLEID1_5771=%257F%25D39%2522N%25B4%25AFY
|
||||
DNT:1
|
||||
Host:moodle.jct.ac.il
|
||||
Referer:http://moodle.jct.ac.il/course/view.php?id=20151
|
||||
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.67 Safari/537.36
|
||||
Query String Parametersview sourceview URL encoded
|
||||
forcedownload:1
|
||||
Response Headersview source
|
||||
Accept-Ranges:bytes
|
||||
Cache-Control:max-age=86400
|
||||
Connection:Keep-Alive
|
||||
Content-Disposition:attachment; filename="150151.5773.week1.wmv"
|
||||
Content-Length:1353673203
|
||||
Content-Type:application/x-forcedownload
|
||||
Date:Mon, 16 Jun 2014 12:31:23 GMT
|
||||
ETag:675e7d2cffd7a79afd8686c59ff2533f9e3508b7
|
||||
Expires:Tue, 17 Jun 2014 12:31:23 GMT
|
||||
Keep-Alive:timeout=15, max=99
|
||||
Last-Modified:Fri, 19 Jul 2013 17:06:54 GMT
|
||||
Pragma:
|
||||
Server:Apache/2.2.14 (Ubuntu)
|
||||
X-Powered-By:PHP/5.3.2-1ubuntu4.24
|
||||
|
||||
when I right-click on the second one in the inspect element window, I can select "copy as cURL" i get:
|
||||
|
||||
curl 'http://moodle.jct.ac.il/pluginfile.php/288409/mod_resource/content/0/movie%205773/150151.5773.week1.wmv?forcedownload=1' -H 'DNT: 1' -H 'Accept-Encoding: gzip,deflate,sdch' -H 'Accept-Language: en-US,en;q=0.8,he;q=0.6' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.67 Safari/537.36' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' -H 'Referer: http://moodle.jct.ac.il/course/view.php?id=20151' -H 'Cookie: visid_incap_97364=qJx2WaKqQfGidGf9VfM6QWrnlFIAAAAAQUIPAAAAAAC5EXcbt00vqNu9jdVDwEDN; __utma=98014340.1409421308.1381214363.1381214363.1390387318.2; __utmz=98014340.1381214363.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); MoodleSession5771=7s1cfqfo4ahdtmna5h7vserg97; MOODLEID1_5771=%257F%25D39%2522N%25B4%25AFY' -H 'Connection: keep-alive' --compressed
|
||||
|
||||
if I append " > week1.wmv" to the end of that output it downloads fine. How can I add this file to a git-annex repo?
|
||||
7
doc/forum/recovery_from_failed_merge.mdwn
Normal file
7
doc/forum/recovery_from_failed_merge.mdwn
Normal file
|
|
@ -0,0 +1,7 @@
|
|||
Starting the assistant version 5.20140613 on my repository (~60GB), it performed a merge
|
||||
with an offline-repository, deleting a large part of the files.
|
||||
Since the repo is in direct mode, I cannot do a git revert. The other repo is not available anymore
|
||||
|
||||
Any way of getting them back?
|
||||
Also, du shows me that the .git/annex/objects folder has approximately the size of my repo before the incident.
|
||||
|
||||
|
|
@ -0,0 +1,43 @@
|
|||
[[!comment format=mdwn
|
||||
username="http://olivier.mehani.name/"
|
||||
nickname="olivier-mehani"
|
||||
subject="Manual solution"
|
||||
date="2014-06-14T13:59:38Z"
|
||||
content="""
|
||||
My problem stems from the fact that I manually git clone the git-annex repo, which prevents the assistant from creating the setup to use passwordless keys. I just reverse-engineered a working setup to work up what I was missing. I jot it down here for reference, but I guess the bottomline is that if you want to use the assistant with a repo, do it from the start.
|
||||
|
||||
I assume that the client has a clone of the git(-annex) repo of the server.
|
||||
|
||||
client$ git clone server:annex
|
||||
|
||||
Our goal is to let git-annex on the client know that there is a specific key to use when connecting to server that will let it access the git-annex-shell (without a password). We first create the key.
|
||||
|
||||
client:~$ ssh-keygen -t rsa -f ~/.ssh/git-annex/key.git-annex-server-user_annex
|
||||
[enter an empty passphrase]
|
||||
|
||||
We can then create a virtual SSH host on the client that will use this key to connect to the server, in client:~/.ssh/config:
|
||||
|
||||
# Added manually for git-annex
|
||||
Host git-annex-server-user_annex
|
||||
Hostname server
|
||||
Port 22
|
||||
IdentityFile ~/.ssh/git-annex/key.git-annex-server-user_annex
|
||||
IdentitiesOnly yes
|
||||
StrictHostKeyChecking yes
|
||||
|
||||
(git-annex seems to use .2F (%2F) to encode path separators in the filenames.)
|
||||
|
||||
The server then needs to know to let the key in, but only for git-annex in the specific folder. This is done in server:.ssh/authorized_keys:
|
||||
|
||||
command=\"GIT_ANNEX_SHELL_DIRECTORY='annex' ~/.ssh/git-annex-shell\",no-agent-forwarding,no-port-forwarding,no-X11-forwarding,no-pty ssh-rsa AAAA... user@client
|
||||
|
||||
The bit starting with ssh-rsa is the public key created in client:.ssh/git-annex/key.git-annex-server-user_annex.pub at the same time as the private key.
|
||||
|
||||
Finally, all that remains is to change the remote in the client clone to use the virtual SSH host.
|
||||
|
||||
client:~/annex $ git remote set-url origin ssh://user@git-annex-server-user_annex/~/annex
|
||||
client:~/annex $ git remote set-url origin --push ssh://user@git-annex-server-user_annex/~/annex
|
||||
|
||||
If everything worked, a sync from the client should now work without asking for a password, and starting the assistant will not either.
|
||||
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,22 @@
|
|||
[[!comment format=mdwn
|
||||
username="http://olivier.mehani.name/"
|
||||
nickname="olivier-mehani"
|
||||
subject="comment 3"
|
||||
date="2014-06-14T14:15:55Z"
|
||||
content="""
|
||||
After having done that on my first test repo, git-annex could sync, but failed to get the files.
|
||||
|
||||
client:~/annex$ git annex get file
|
||||
get file (not available)
|
||||
Try making some of these repositories available:
|
||||
12345678-90ab-cdef-1234567890abcdef1 -- user@server:~/annex [origin]
|
||||
|
||||
(Note that these git remotes have annex-ignore set: origin)
|
||||
failed
|
||||
git-annex: get: 1 failed
|
||||
|
||||
The note helps: the problem is with the origin remote having annex-ignore set. git-annex therefore ignores it. This is easily fixed by just setting the flag to false.
|
||||
|
||||
client:~/annex$ git config remote.origin.annex-ignore false
|
||||
|
||||
"""]]
|
||||
Loading…
Reference in a new issue