virtio-gpu and libvirt

libvirt has no support for virtio-vga yet. There is a bug for virtio-vga support in libvirt. But for the time being we need to play some tricks. Luckily libvirt has some special xml syntax to pass command line options to qemu. So here we go:

<domain type='kvm' xmlns:qemu=''>
  [ ... ]
    [ ... ]
      <model type='cirrus' vram='16384' heads='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    [ ... ]
    <qemu:arg value='-set'/>
    <qemu:arg value='device.video0.driver=virtio-vga'/>

The idea is to tell libvirt we wanna have a cirrus vga. That one has no config options, thats why it works best for our purposes. Then use the -set switch to change the emulation driver from cirrus to virtio-vga. video0 is the id libvirt assigns to the (first and only) video device.

Next level is turning on virgl & opengl support. Initially sdl and gtk user interfaces will be supported (qemu 2.5 most likely). Spice will follow later. So lets pick gtk. Here we go:

<domain type='kvm' xmlns:qemu=''>
  [ ... ]
    [ ... ]
    <graphics type='sdl' display=':0' xauth='/path/to/.Xauthority'/>
      <model type='cirrus' vram='16384' heads='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    [ ... ]
    <qemu:arg value='-set'/>
    <qemu:arg value='device.video0.driver=virtio-vga'/>
    <qemu:arg value='-display'/>
    <qemu:arg value='gtk,gl=on'/>

Simliar approach: We ask libvirt for sdl support. Picking that one because libvirt will take care to pass xauth info to qemu so it gets access to the X11 display. Then use the -display switch to override the libvirt display configuration with gtk, and turn on opengl.

One final remark: On modern linux systems xauth info is stored in /run/gdm/auth-for-$USER-$RANDOM/database instead of $HOME/.Xauthority. I have a little script to copy the xauth info to a fixed place and also make it readable so libvirt can access it:

if test "$XAUTHORITY" != "$HOME/.Xauthority"; then
    xauth extract "$HOME/.Xauthority" "$DISPLAY"
    chmod +r $HOME/.Xauthority

Note that making .Xauthority world-readable effectively gives every user on your machine access to your X11 display. On your private laptop where you are the only user that should be ok, but on a shared machine you probably want do something more secure instead.

Running Fedora on the Raspberry PI 2

The new Raspberry PI B 2 model got an armv7 core (the older ones are armv6), which makes it much easier to run Fedora on it. All Fedora packages are built for armv7, so they work as-is. Only things needed additionally are (a) the firmware and (b) a kernel. So I’ve packaged up the stuff. Firmware is just the binary stuff canned into rpms. Kernel is build from rapsberrypi git repo.

Package repository is here:
Images are here:

Install & usage

Just unpack the image and write it to a memory card. On first boot a rc.local script configures some things, then reboots the pi. root password is “pi”. Network is configured use dhcp on both eth0 and wlan0, using systemd-networkd. wireless needs configuration though, use wpa_passphrase "$ssid" "$psk" >> /etc/wpa_supplicant/wpa_supplicant.conf for that. kernel-rpi packages have a postinstall script updating /boot/config.txt, so kernel updates work as usual.


There is a uboot-rpi2 package as I’ve played around with u-boot. Didn’t found it that useful though. The firmware passes some command line args and a device tree to the kernel. With u-boot as boot loader inbetween that gets lost. So I’ve found myself facing problems by adding uboot to the mix, with no real advantages. Therefore the images don’t use u-boot. But you can install the rpm and play around with it if you want.


Didn’t came very far. Booting a kvm-enabled kernel in hyp mode isn’t that difficuilt. The more serve problem is that there is no GIC which kvm needs too. Someone hacked up vgic emulation to solve that issue. But the patches are old and unmaintained. They don’t rebase easily. And they also don’t solve the problem that the machine has problems accepting hardware interrupts in guest mode. Which can be worked around with isolcpus & taskset. But after all it doesn’t look like worth bothering …

New member in the virtio family: input devices.


If you have build a fresh 4.1-rc kernel you might have already noticed the new CONFIG_VIRTIO_INPUT config option. So, here is some background information on what this is and how to use it.

virtio-input basically sends linux evdev events over virtio. The device capability bits (available via ioctl for evdev) live in virtio device config space. The events themself are sent over virtio queues. They look exactly like evdev events, except that the struct fields are always in little endian byte order. So this allows the host to emulate pretty much any input device the linux kernel’s input layer is able to handle.

On the guest side the only thing you need is a kernel with CONFIG_VIRTIO_INPUT=m.

On the host side you need pretty recent qemu master branch. The first release with virtio-input support will be 2.4.

emulated devices

qemu has three virtual devices right now:

  • -device virtio-keyboard-pci : virtual keyboard, simliar to the ps/2 keyboard.
  • -device virtio-mouse-pci : virtual mouse, simliar to the ps/2 mouse.
  • -device virtio-tablet-pci : virtual tablet, simliar to usb-tablet, but without the usb overhead.

The tablet is probably the most useful one in practice. The pseries guys might be interested in the keyboard too as usb-keyboard replacement.

device pass-though

Not yet merged, but the patch should land upstream in time for the 2.4 release: There also is support for passing through host evdev devices to the guest:

-device virtio-input-host-pci,evdev=/dev/input/eventn

Take care: The guest gets exclusive access to the input device then. So better don’t try that with your one and only keyboard 😉

virtio-gpu on the way upstream

We are making progress, time for the next status report …

virtio-gpu, 2d mode

It’s on the way upstream, both guest and host side. The qemu code landed in git master already and will be shipped with the upcoming 2.4 release. The linux kms driver is in drm-next right now and will most likely land upstream in the 4.2 merge window.

virtio-gpu, 3d mode

With 2d mode being almost completed now the focus shifts to put 3d/virgl into shape.

The qemu code is in a new virgl branch now, the vga and ui bits are no longer separated.

Current state: SDL2 display is working. egl display (headless, see below) is working.

TODO list: Make gtk diplay (egl mode) work. Add support for native opengl (available in 3.16+, needed to run on wayland) to gtk display. Add spice support.

Building qemu with virgl:
You need virglrenderer. Copr has packages. If you want try egl display too you need pretty cutting edge mesa and libepoxy packages, for dma-buf export and import support. Fedora 22 will do for mesa. For libepoxy you need the rawhide/F23 version.

Linux guest bits for virgl:
The virtio-gpu branch in my linux repo has the kernel bits you need. The Copr repos have a “mesa.git” package which you need to install. This isn’t full mesa, only the guest driver for virtio-gpu. I suggest to use Fedora 22 as guest as I’m usually testing with F22 too.

You might need to put selinux (at the host) into permissive mode when playing with this. svirt doesn’t allow qemu access the gpu.

headless egl display

In this mode qemu doesn’t need access to your X11 (or wayland) display. It’ll open a drm render node instead and use that for opengl operations. The guest display is rendered into a dma-buf. A simple standalone app can connect to qemu, get access to the dma-bufs via file descriptor passing and blit the dma-buf to your screen.

Start qemu: qemu-system-x86_64 -display egl -name $name
Start viewer: qemu-eglview $name

The viewer is proof-of-concept for the dma-buf passing and it is pretty simple. No pointer shape support. No relative pointer support, so a tablet device is mandatory for the guest if you want a working pointer.

Not sure yet how the long-term future will look like for qemu-eglview. Right now it is a useful hacking and testing tool. But spice will some day operate in a very simliar way and additionally provide all the extra goodies spice has to offer (sound, usb, smartcard, webdav, …), so it probably isn’t that useful to turn it into a full-blown viewer app …

virtio-gpu and virgl status update

Just received a email asking where we are in terms of virgl support in qemu, partly because the links from last years kvm talk are outdated meanwhile. I suspect there might be more folks wondering, so it is probably a good idea to write up an update in public. So, here we go with the news …

virglrenderer isn’t a branch in Dave’s mesa repo any more, it got factored out into its own git repo.

The mesa guest driver still is in Dave’s mesa repo (branch virgl-mesa-driver).

The xorg guest driver is not needed any more because xorg can do 2d acceleration via opengl/mesa now.

The guest kernel driver is in my virtio-gpu branch.

The qemu bits are splitted over two branches at the moment: The sdl2 and opengl bits are in my sdl branch, whereas the virtio-gpu bits are in the vga branch. In theory making things work should be as easy as merging these two branches. I didn’t test this for a while though as I’m busy bringing the first batch of virtio-gpu patches (with 2d support only) into shape for upstream merging.

Upstream merge is stalled atm because the virtio-gpu patches depend on virtio 1.0 support, which in turn is blocked by qemu being in freeze for the 2.3 release right now. Things should start moving forward very soon though, once qemu 2.3 is released the master branch will be opened for 2.4 development.

qemu: using cirrus considered harmful

Cirrus is a hardware design from the 90ies and simply can’t keep up with todays needs.

  • Want a Full HD display? No way with cirrus.
  • See funky glitches in your X11 display? Yep, that’s because cirrus uses 24 instead of 32 bits per pixel for truecolor. Modern video hardware stopped doing at least a decade ago, so these code paths in Xorg are largely unused and do bitrot.

And because the cirrus emulation must mimic the behaviour of the real hardware there is no way to fix this in qemu. For historic and compatibility reasons cirrus is the default vga in qemu, but luckily there are other choices.

Cirrus may have its valid use cases with guests which are from the 90ies too, such as Windows NT4. With modern guests you are much better off using stdvga (-vga std) or qxl (-vga qxl) instead.

If you are using spice the best choice is qxl. Install guest drivers for best performance, without guest drivers qxl offers no benefits over stdvga. Linux distros should have the qxl driver packaged up and possibly even installed by default, so that is easy. Windows drivers are here.

stdvga on linux: In version 3.14 the kernel finally got a kms driver for the qemu stdvga. If you have that: Make sure CONFIG_DRM_BOCHS is enabled (most likely your distro did that already for you), be happy. Xorg runs the stdvga with the modesetting driver. xrandr works as it should. If you are not happy with the predefined list of modes just add your own via xrandr --newmode or xorg.conf. If highres modes don’t work use more video memory: qemu -vga std -global VGA.vgamem_mb=32 doubles the default size.

With older linux kernels Xorg will use the vesa driver for stdvga. It is less comfortable to use than modesetting. You are limited to the modes exposed by vgabios, but you still get more modes than with cirrus. You have to create a xorg.conf snippet with monitor refresh rates to convince Xorg that it can use higher resolution without blowing the virtual monitor. Nevertheless it is still the easy way out if you suffer cirrus display glitches.

stdvga on windows: Windows XP (which just went out of support) is the last windows release shipping with cirrus drivers. With anything more recent there is zero benefit in preferring cirrus over stdvga. In both cases windows uses the vgabios (VBE 2.0) for modeswitching and a simple framebuffer without any acceleration support. But stdvga has more video memory and allows higher display resolutions.

IPv6 is taking off

I’m running a IPv6-enabled network, for a few years already. Internet connectivity goes through a SixXS tunnel. DNS services are provided by a bind instance. The machines on the local network have both IPv4 and IPv6 DNS entries there. Bind does also act as DNS cache, and of course it is reachable with both IPv4 and IPv6. dhcpd hands out IPv4 addresses to everybody, radvd does the same for IPv6, both point to to the bind instance for DNS service.

Recently I’ve purchased a cubietruck so I can play around with arm virtualization. Installed & configured the box, created fedora arm guests. Logged in, updated the virtual machines (loading updates from the fedora mirror network), all worked just fine. Then, after a few days, I’ve figured that dhcp was not turned on in the guests. They ran without IPv4 connectivity, and I didn’t even notice!

Simliar thing happened with my jenkins build host. It somehow lost IPv4 connectivity. Not fully sure why, but there was a power failure and the dhcp server wasn’t available for a while, guess this was the reason. Nevertheless almost everything continued to work just fine. Jenkins performed builds, I could see the results in the web gui. What didn’t work was pulling github repos, because github can’t be reached using IPv6. Needless to say this took me a few days too to figure.

Is amazing how well things are working meanwhile with IPv6 only.


colorful crash

Most of the time the crashes are boring, the machine just hangs or reboots. This one is different 😉

Looks like someone dumps random crap into the vga text mode memory, root cause not figured yet.