Merge branch 'master' into assistant
This commit is contained in:
Joey Hess
commit
6de38a2ca8
45 changed files with 877 additions and 8 deletions
|
|
@ -0,0 +1,8 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://www.google.com/accounts/o8/id?id=AItOawkSq2FDpK2n66QRUxtqqdbyDuwgbQmUWus"
|
||||
nickname="Jimmy"
|
||||
subject="comment 1"
|
||||
date="2012-06-19T06:53:26Z"
|
||||
content="""
|
||||
heh, yea, it's detecting changes on OSX ;)
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://www.google.com/accounts/o8/id?id=AItOawkSq2FDpK2n66QRUxtqqdbyDuwgbQmUWus"
|
||||
nickname="Jimmy"
|
||||
subject="comment 1"
|
||||
date="2012-06-19T07:01:26Z"
|
||||
content="""
|
||||
issues with the watch command on OSX, it seems that there is a race condition somewhere. I dumped a few iso's into an annex and it only annexed the smaller files (checksums) and the bigger ones (the iso's) just got made read only. also do you want these bugs to be logged here or in the bugs section?
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://www.google.com/accounts/o8/id?id=AItOawmBUR4O9mofxVbpb8JV9mEbVfIYv670uJo"
|
||||
nickname="Justin"
|
||||
subject="comment 1"
|
||||
date="2012-06-27T12:46:31Z"
|
||||
content="""
|
||||
can X and Y be the names of the git-annex remotes?
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
[[!comment format=mdwn
|
||||
username="http://joeyh.name/"
|
||||
ip="4.153.2.189"
|
||||
subject="comment 2"
|
||||
date="2012-07-26T17:27:39Z"
|
||||
content="""
|
||||
That's a good question. Unfortunatly they cannot; X and Y need to be stable across repositories, and git remotes can have different names in different repositories.
|
||||
|
||||
Even using the description that git-annex stores for each repository for X and Y is problimatic, since that description can change, and so could be different in two repos that are each trying to resolve the same merge conflict.
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://www.google.com/accounts/o8/id?id=AItOawkSq2FDpK2n66QRUxtqqdbyDuwgbQmUWus"
|
||||
nickname="Jimmy"
|
||||
subject="comment 2"
|
||||
date="2012-07-19T18:43:30Z"
|
||||
content="""
|
||||
Joey, yes dbus is available from macports and homebrew, it's not installed by default (or as a dependancy) for most packages in macports.
|
||||
"""]]
|
||||
83
doc/design/assistant/blog/day_44__webapp_basics.mdwn
Normal file
83
doc/design/assistant/blog/day_44__webapp_basics.mdwn
Normal file
|
|
@ -0,0 +1,83 @@
|
|||
After an all-nighter, I have `git annex webapp` launching a WebApp!
|
||||
|
||||
It doesn't do anything useful yet, just uses Yesod to display a couple of
|
||||
hyperlinked pages and a favicon, securely.
|
||||
|
||||
The binary size grew rather alarmingly, BTW. :) Indeed, it's been growing
|
||||
for months..
|
||||
|
||||
-rwxr-xr-x 1 root root 9.4M Jul 21 16:59 git-annex-no-assistant-stripped
|
||||
-rwxr-xr-x 1 joey joey 12M Jul 25 20:54 git-annex-no-webapp-stripped
|
||||
-rwxr-xr-x 1 joey joey 17M Jul 25 20:52 git-annex-with-webapp-stripped
|
||||
|
||||
----
|
||||
|
||||
Along the way, some Not Invented Here occurred:
|
||||
|
||||
I didn't use the yesod scaffolded site, because it's a lot of what
|
||||
seems mostly to be cruft in this use case. And because I don't like
|
||||
code generated from templates that people are then expected to edit. Ugh.
|
||||
That's my least favorite part of Yesod. This added some pain, since
|
||||
I had to do everything the hard way.
|
||||
|
||||
I didn't use [wai-handler-launch](http://hackage.haskell.org/package/wai-handler-launch)
|
||||
because:
|
||||
|
||||
* It seems broken on IPv6 capable machines (it always opens
|
||||
`http://127.0.0.1:port/` even though it apparently doesn't always
|
||||
listen there.. I think it was listening on my machine's ipv6 address
|
||||
instead. I know, I know; I should file a bug about this..)
|
||||
* It always uses port 4587, which is **insane**. What if you have two
|
||||
webapps?
|
||||
* It requires javascript in the web browser, which
|
||||
is used to ping the server, and shut it down when the web browser closes
|
||||
(which behavior is wrong for git-annex anyway, since the daemon should
|
||||
stay running across browser closes).
|
||||
* It opens the webapp on web server startup, which is wrong for git-annex;
|
||||
instead the command `git annex webapp` will open the webapp,
|
||||
after `git annex assistant` started the web server.
|
||||
|
||||
Instead, I rolled my own WAI webapp laucher, that binds to any free port
|
||||
on localhost, It does use `xdg-open` to launch the web browser,
|
||||
like wai-handler-launch (or just `open` on OS X).
|
||||
|
||||
Also, I wrote my own WAI logger, which logs using System.Log.Logger,
|
||||
instead of to stdout, like `runDebug` does.
|
||||
|
||||
----
|
||||
|
||||
The webapp only listens for connections from localhost, but that's
|
||||
not sufficient "security". Instead, I added a secret token to
|
||||
every url in the webapp, that only `git annex webapp` knows about.
|
||||
|
||||
But, if that token is passed to `xdg-open` on its command line,
|
||||
it will be briefly visible to local attackers in the parameters of
|
||||
`xdg-open`.. And if the web browser's not already running, it'll run
|
||||
with it as a parameter, and be *very* visible.
|
||||
|
||||
So instead, I used a nasty hack. On startup, the assistant
|
||||
will create a html file, readably only by the user, that redirects
|
||||
the user to the real site url. Then `git annex webapp` will run
|
||||
xdg-open on that file.
|
||||
|
||||
----
|
||||
|
||||
Making Yesod check the `auth=` parameter (to verify that the secret token
|
||||
is right) is when using Yesod started to pay off. Yesod has a simple
|
||||
`isAuthorized` method that can be overridden to do your own authentication
|
||||
like this.
|
||||
|
||||
But Yesod really started to shine when I went to add the `auth=` parameter
|
||||
to every url in the webapp. There's a `joinPath` method can can be used
|
||||
to override the default url builder. And every type-safe url in the
|
||||
application goes through there, so it's perfect for this.
|
||||
|
||||
I just had to be careful to make it not add `auth=` to the url for the
|
||||
favicon, which is included in the "Permission Denied" error page. That'd be
|
||||
an amusing security hole..
|
||||
|
||||
----
|
||||
|
||||
Next up: Doing some AJAX to get a dynamic view of the state of the daemon,
|
||||
including currently running transfers, in the webapp. AKA stuff I've never
|
||||
done before, and that, unlike all this heavy Haskell Yesod, scares me. :)
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://www.google.com/accounts/o8/id?id=AItOawn7Oyqusvn0oONFtVhCx5gRAcvPjyRMcBI"
|
||||
nickname="Michaël"
|
||||
subject="is ftp an option?"
|
||||
date="2012-05-30T10:44:12Z"
|
||||
content="""
|
||||
for people only having ftp-access to there storage.
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
[[!comment format=mdwn
|
||||
username="http://www.klomp.eu/"
|
||||
ip="95.91.241.82"
|
||||
subject="Watch also possible with git?"
|
||||
date="2012-06-15T17:25:30Z"
|
||||
content="""
|
||||
Hi,
|
||||
|
||||
it seems that you put a lot of efforts in handling race conditions. Thats great. I wonder if the watch can also be used with git (i.e. changes are commited into git and not as annex)? I know that other projects follow this idea but why using different tools if the git-annex assistant could handle both...
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,10 @@
|
|||
[[!comment format=mdwn
|
||||
username="http://wiggy.net/"
|
||||
nickname="Wichert"
|
||||
subject="macports"
|
||||
date="2012-06-12T13:00:34Z"
|
||||
content="""
|
||||
The average OSX user has a) no idea what macports is, and b) will not be able to install it. Anything that requires a user to do anything with a commandline (or really anything other than using a GUI installer) is effectively a dealbreaker. For our use cases OSX is definitely a requirement, but it must only use standard OSX installation methods in order to be usable. Being in the appstore would be ideal, but standard dmg/pkg installers are still common enough that they are also acceptable.
|
||||
|
||||
FWIW this is the same reason many git GUIs were not usable for our OSX users: they required separate installation of the git commandline tools.
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://www.google.com/accounts/o8/id?id=AItOawkSq2FDpK2n66QRUxtqqdbyDuwgbQmUWus"
|
||||
nickname="Jimmy"
|
||||
subject="comment 5"
|
||||
date="2012-06-17T21:42:59Z"
|
||||
content="""
|
||||
okay, I've gotten gitbuilder to poll the git repo every minute for changes, gitbuilder doesn't build every commit. It doesn't work like that, it checks out the master and builds that. If there is a failure it automatically bisects to find out where the problem first got introduced. Hope the change to the builder helps!
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,9 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://www.google.com/accounts/o8/id?id=AItOawnV2c63kDc6X21a1H81me1mIenUCScd2Gs"
|
||||
nickname="Emanuele"
|
||||
subject="watch branch?"
|
||||
date="2012-06-01T19:19:17Z"
|
||||
content="""
|
||||
Hello there? Where can I find more info about this git watch branch?
|
||||
Keep up the good work!
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://www.google.com/accounts/o8/id?id=AItOawnBl7cA6wLDxVNUyLIHvAyCkf8ir3alYpk"
|
||||
nickname="Tyson"
|
||||
subject="Bridging LANs"
|
||||
date="2012-07-10T10:20:59Z"
|
||||
content="""
|
||||
Why rely on the cloud when you can instead use XMPP and jingle to perform NAT traversal for you? AFAIKT, it also means that traffic won't leave your router if the two endpoints are behind the same router.
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,80 @@
|
|||
[[!comment format=mdwn
|
||||
username="https://www.google.com/accounts/o8/id?id=AItOawkSq2FDpK2n66QRUxtqqdbyDuwgbQmUWus"
|
||||
nickname="Jimmy"
|
||||
subject="comment 1"
|
||||
date="2012-07-03T08:26:43Z"
|
||||
content="""
|
||||
On \"git syncing\" point number 9, on OSX you could potentially do this on a semi-regular basis
|
||||
|
||||
<pre>
|
||||
system_profiler SPNetworkVolumeDataType
|
||||
Volumes:
|
||||
|
||||
net:
|
||||
|
||||
Type: autofs
|
||||
Mount Point: /net
|
||||
Mounted From: map -hosts
|
||||
Automounted: Yes
|
||||
|
||||
home:
|
||||
|
||||
Type: autofs
|
||||
Mount Point: /home
|
||||
Mounted From: map auto_home
|
||||
Automounted: Yes
|
||||
</pre>
|
||||
|
||||
and
|
||||
|
||||
<pre>
|
||||
x00:~ jtang$ system_profiler SPUSBDataType
|
||||
USB:
|
||||
|
||||
USB High-Speed Bus:
|
||||
|
||||
Host Controller Location: Built-in USB
|
||||
Host Controller Driver: AppleUSBEHCI
|
||||
PCI Device ID: 0x0aa9
|
||||
PCI Revision ID: 0x00b1
|
||||
PCI Vendor ID: 0x10de
|
||||
Bus Number: 0x26
|
||||
|
||||
Hub:
|
||||
|
||||
Product ID: 0x2504
|
||||
Vendor ID: 0x0424 (SMSC)
|
||||
Version: 0.01
|
||||
Speed: Up to 480 Mb/sec
|
||||
Location ID: 0x26200000 / 3
|
||||
Current Available (mA): 500
|
||||
Current Required (mA): 2
|
||||
|
||||
USB to ATA/ATAPI Bridge:
|
||||
|
||||
Capacity: 750.16 GB (750,156,374,016 bytes)
|
||||
Removable Media: Yes
|
||||
Detachable Drive: Yes
|
||||
BSD Name: disk1
|
||||
Product ID: 0x2338
|
||||
Vendor ID: 0x152d (JMicron Technology Corp.)
|
||||
Version: 1.00
|
||||
Serial Number: 313541813001
|
||||
Speed: Up to 480 Mb/sec
|
||||
Manufacturer: JMicron
|
||||
Location ID: 0x26240000 / 5
|
||||
Current Available (mA): 500
|
||||
Current Required (mA): 2
|
||||
Partition Map Type: MBR (Master Boot Record)
|
||||
S.M.A.R.T. status: Not Supported
|
||||
Volumes:
|
||||
Porta-Disk:
|
||||
Capacity: 750.16 GB (750,156,341,760 bytes)
|
||||
Available: 668.42 GB (668,424,208,384 bytes)
|
||||
Writable: Yes
|
||||
File System: ExFAT
|
||||
....
|
||||
</pre>
|
||||
|
||||
I think its possible to programatically get this information either from the CLI (it dumps out XML output if required) or some development library. There is also DBUS in macports, but I have never had much interaction with it, so I don't know if its good or bad on OSX.
|
||||
"""]]
|
||||
|
|
@ -7,6 +7,9 @@ The webapp is a web server that displays a shiny interface.
|
|||
token. This guards against other users on the same system. **done**
|
||||
(I would like to avoid passwords or other authentication methods,
|
||||
it's your local system.)
|
||||
* Don't pass the url with secret token directly to the web browser,
|
||||
as that exposes it to `ps`. Instead, write a html file only the user can read,
|
||||
that redirects to the webapp. **done**
|
||||
* Alternative for Linux at least would be to write a small program using
|
||||
GTK+ Webkit, that runs the webapp, and can know what user ran it, avoiding
|
||||
needing authentication.
|
||||
|
|
@ -28,11 +31,12 @@ The webapp is a web server that displays a shiny interface.
|
|||
|
||||
## implementation
|
||||
|
||||
Hope to use Yesod.
|
||||
|
||||
TODO: Ensure that Yesod will work on arm. Necessary for later Android port.
|
||||
Will its template haskell cause a problem? Does new GHC support TH on ARM?
|
||||
Will it use too much memory or be too slow?
|
||||
|
||||
Hopefully Yesod comes with some good UI widgets. Otherwise, need to use
|
||||
Jquery or similar.
|
||||
* use `addStaticContent` to make /favicon.ico work. Return `Right (route, query)`
|
||||
and I think the route can be `favicon_ico`.
|
||||
* perhaps define a custom `errorHandler`, which could avoid the potential
|
||||
of leaking auth tokens on error pages
|
||||
* possibly lose the ugly auth= token past the first page,
|
||||
and use a client-side session. It could be encrypted using the token
|
||||
as the `encryptKey`. Note: Would need to set the session duration
|
||||
to infinite (how?)
|
||||
* look up "server-sent events" sent using `sendWaiResponse`
|
||||
|
|
|
|||
|
|
@ -0,0 +1,10 @@
|
|||
[[!comment format=mdwn
|
||||
username="yatesa"
|
||||
ip="171.25.193.21"
|
||||
subject="Secret URL token"
|
||||
date="2012-06-19T03:41:16Z"
|
||||
content="""
|
||||
> Instruct the user's web browser to open an url that contains a secret token. This guards against other users on the same system.
|
||||
|
||||
How will you implement that? Running \"sensible-browser URL\" would be the obvious way, but the secret URL would show up in a well timed ps listing. (And depending on the browser, ps may show the URL the entire time it's running.)
|
||||
"""]]
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
[[!comment format=mdwn
|
||||
username="jtang"
|
||||
ip="79.97.135.214"
|
||||
subject="comment 3"
|
||||
date="2012-07-26T17:35:18Z"
|
||||
content="""
|
||||
Using twitter-bootstrap for the webapp - this might be a wishlist item, but would it be possible to ensure that the webapp's css uses twitter-bootstrap classes. It would make theming much easier in the long run and it would give you a nice modern look with a low amount of effort.
|
||||
"""]]
|
||||
Loading…
Add table
Add a link
Reference in a new issue