From 6b5649a64871b8f646b6b08d646f334b5f74481c Mon Sep 17 00:00:00 2001 From: Geoffrey McRae Date: Tue, 12 Dec 2017 04:23:27 +1100 Subject: [PATCH] [misc] removed README.md before release as it needs rewriting --- README.md | 216 ------------------------------------------------------ 1 file changed, 216 deletions(-) delete mode 100644 README.md diff --git a/README.md b/README.md deleted file mode 100644 index e222e3dd..00000000 --- a/README.md +++ /dev/null @@ -1,216 +0,0 @@ -Looking Glass -============= - -Purpose -------- - -To provide a low latency KVM Guest client that relies on video capture rather -than the output of a virtual VGA device. The goal of this project is to allow -lossless zero latency display of a guest on the host on the local host. - -The Problem ------------ - -Exising solutions such as Spice are designed to work with a virtual VGA -adaptor, but when a physical VGA device is passed into the guest VM the -display can not be presented to the host, instead a physical monitor must be -attached to the physcal VGA device. While this may be acceptible to some it -requires a physical switch between the host and guest VM preventing the VM -from being inlined into the host window manager. - -Existing Solutions ------------------- - -Some have reported that they are able to stream using Steam InHouse Streaming -to stream the desktop back to the host by launching a program such as notepad -in the guest, and then tabbing to desktop. While this works Steam is designed -to stream this over a local area network and as such employs technologies such -as compression and packetization for network transmission, there are several -issues with this solution: - - * Latency from guest to client is generally no less then 50-80ms, this is - very noticable when using a cursor input or running games that demand high - precision and fast input. - - * Compressing the stream adds additional CPU/GPU overhead that can degrade - performance. - - * Compressing converts to the YUV422P colorspace which degrades quality - quite substantially. - - * The stream is tuned for transmission over a network which not only adds - additional overheads, it also adds additional buffers that increase - latency. - -Our Solution ------------- - -Qemu contains a virtual device that allows mapping of shared memory between -host and guest called 'ivshmem'. This device has been used for very specific -cases where extremely high bandwidth low latency networking/firewalls have -been required and as such a windows driver was never written for this device. - -We developed a Windows driver for this device and have worked with Red Hat to -have it included into the official [VirtIO windows driver repository][1]. - -The shared memory segment is intialized by the guest with a strcture that -describes the format of the frame and the current mouse coordinates. Frame data -is appended to the buffer after this header. - -We have decided upon he name "KVM FrameRelay", or "KVMFR" for this technology. - -### Video Transfer - -The guest runs a program that uses the ivshmem device as a direct transfer -between the guest and host. This application uses accelerated capture methods -such as NvFBC or DXGI to capture the guests framebuffer. This captured frame is -then copied into the shared memory region. After the frame has been copied a -doorbell is set in ivshmem which the host client application is waiting on, -after which the client application waits for an IRQ from the host to state -that the frame has been processed. - -After the host client application is notified there is a new frame it maps the -frame to the host video card by means of a SDL streaming texture. Once a copy -of the frame has been completed the host client then notifies the guest by means -of raising the IRQ the guest is waiting on. - -At this time the guest still requires a monitor or dummy device attached that -presents valid EDID information for the resolutions that are desired. There -may be a way around this but that is not the goal of this project at this time -and is thus out of scope. This author found it easiest to just use the -secondary input to one of his monitors to present the EDID information. - -### Keyboard and Mouse Input - -The host client implements the Spice protocol and is able to send mouse and -keyboard events over the existing communications channels provided by qemu. - -If using libvirt the virtual tablet device must be removed from the host as -this application expects to and requires to use relative positioning for all -interactions. If the tablet device exists qemu will give priority to it for -click events which are unreliable when used in this way. - -There are also some issues with the qemu PS2 controller that will cause mouse -input lockups when keyboard events and mouse events occur at the same time. -The virtual USB keyboard and mouse inputs while are somewhat more reliable also -suffer random dropouts at high rates due to USB hub emulation timing issues. - -At the time of writing this we have found the best solution is to use the PS2 -mouse and the VirtIO Keyboard device. - -The guest program returns the current mouse position as part of each frame, -this is so that it is possible to syncronise the host mouse position in the -window to that reported by the guest. - -We recommend that the guest have the [mouse acceleration disabled][2] so that -the guest's mouse position is 1:1 with the host. This makes for a seemless -experience when the cursor is moved inside and outside of the client -application on the host. - -### Audio - -This client doesn't implement any form of audio, the guest needs to be -configured to use the hosts for audio directly. At this time we recommend using -pulseaudio as the ALSA implementation seems to be problematic and requires -some attention. - -Testing -------- - -Our test scenario involved running a physical monitor along side the guest -client, at this time we have only tested NvFBC as the capture technology on the -guest. - -### Hardware - -| | | -|-----------|------------------------------------------------------------| -| MB | Asrock AB350 Pro4 | -| CPU | AMD Ryzen 7 1700 @ 3.8GHz | -| RAM | 16GB DDR4 | -| Host GPU | NVidia GTX 1080 Ti running 4 monitors @ 1920x1200 | -| Guest GPU | NVidia GTX 680 [modified to a Quadro K5000][3] @ 1920x1200 | - -### Software - -| | | -|-------|----------------------------------------------------| -| Host | Debian 9 + [NPT patch][4] | -| Guest | Windows 10 running in KVM on the i440 device tree. | - -Results -------- - -Latency has not yet been measured, but in practice to this author it is -extremely hard to notice any latency between the physical display and host -client output when streaming 24bit RGB. When streaming YUV422P there no -percevable latency at all, this indicates that the memory copy is the slowest -part of the process. To combat this multi-threaded memory copy has been -implemented on both the host and guest ends of the transfer. - -At this time we are waiting for a signed ivshmem driver by Red Hat to be -released but since this may take quite some time we are currently in the -process of obtaining a driver signing certificate. - -Status ------- - -The host software is considered 90% complete for devices that are capable of -NvFBC and has been written in such as way to allow for other capture interfaces -to be easily added to support other platforms. - -The client software is considered 90% complete, features such as copy & paste -between client and host are missing and it would be nice if it could also -serve as the ivshmem-server. The client is able to support a variaty of pixel -formats types depending on the capability and performance of the capture API -that is in use. - -Security --------- - -This project has not been written with security of the shared memory in mind, -it is certainly possible at this time for the client to abuse the headers and -input invalid information that could be used to compromise the host system. - -At this time this is not a concern as our use case of this application at this -time is in a completely trusted environment, but before it is more widely put -into use this will need to be addressed. - -Improvements ------------- - -It may be possible to avoid the additional copy by handing the guest a block -of memory directly from the hosts graphics device. This would allow the guest -to stream directly into the texture on the host. This however could be another -avenue of escape for malicious code on the guest and would need very careful -consideration before any attempt is made to implement this. - -To achieve the above it would be required to modify qemu integrating the -features of this client directly so that a SDL\_Texture's RAM could be mapped -directly into the guest. - -License -------- - -~~~ -KVMGFX Client - A KVM Client for VGA Passthrough -Copyright (C) 2017 Geoffrey McRae - -This program is free software; you can redistribute it and/or modify it under -the terms of the GNU General Public License as published by the Free Software -Foundation; either version 2 of the License, or (at your option) any later -version. - -This program is distributed in the hope that it will be useful, but WITHOUT ANY -WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A -PARTICULAR PURPOSE. See the GNU General Public License for more details. - -You should have received a copy of the GNU General Public License along with -this program; if not, write to the Free Software Foundation, Inc., 59 Temple -Place, Suite 330, Boston, MA 02111-1307 USA -~~~ - -[1]: https://github.com/virtio-win/kvm-guest-drivers-windows/tree/master/ivshmem -[2]: http://donewmouseaccel.blogspot.com.au/2010/03/markc-windows-7-mouse-acceleration-fix.html -[3]: http://www.eevblog.com/forum/chat/hacking-nvidia-cards-into-their-professional-counterparts/ -[4]: https://patchwork.kernel.org/patch/10027525/