Canonical Voices

Posts tagged with 'mir'

Alan Griffiths

mircade-snap

mircade, miral-kiosk and snapcraft.io

mircade is a proof-of-concept game launcher for use with miral-kiosk. It looks for installed games, works out if they use a toolkit supported by Mir and allows the user to play them.

miral-kiosk is a proof-of-concept Mir server for kiosk style use. It has very basic window management designed to support a single fullscreen application.

snapcraft.io is a packaging system that allows you to package applications (as “snaps”) in a way that runs on multiple linux distributions. You first need to have snapcraft installed on your target system (I used a dragonboard with Ubuntu Core as described in my previous article).

The mircade snap takes mircade and a few open games from the Ubuntu archive to create an “arcade style” snap for playing these games.

Setting up the Mir snaps

The mircade snap is based on the “Mir Kiosk Snaps” described here.

Mir support on Ubuntu Core is currently work in progress so the exact incantations for installing the mir-libs and mir-kiosk snaps to work with mircade varies slightly from the referenced articles (to work around bugs) and will (hopefully) change in the near future. Here’s what I found works at the time of writing:

$ snap install mir-libs --channel edge
$ snap install mir-kiosk --channel edge --devmode
$ snap connect mir-kiosk:mir-libs mir-libs:mir-libs
$ sudo reboot

Installing the mircade-snap

I found that installing the mircade snap sometimes ran out of space on the dragonboard /tmp filesystem. So…

$ TMPDIR=/writable/ snap install mircade --devmode --channel=edge
$ snap connect mircade:mir-libs mir-libs:mir-libs
$ snap disconnect mircade:mir;snap connect mircade:mir mir-kiosk:mir
$ snap disable mircade;sudo /usr/lib/snapd/snap-discard-ns mircade;snap enable mircade

Using mircade on the dragonboard

At this point you should see an orange screen with the name of a game. You can change the game by touching/clicking the top or bottom of the screen (or using the arrow keys). Start the current game by touching/clicking the middle of the screen or pressing enter.

Read more
kevin gunn

1)Put the latest ubuntu-core image for dragonboard on boot (you’ll want a screen and keyboard at least)

You can find the image here http://releases.ubuntu.com/ubuntu-core/16/

Make sure you’re on the latest with the following


ssh$ snap refresh core

 

2)Then install the mir-libs and mir-kiosk

 

ssh$ snap install mir-libs --channel=edge
ssh$ snap install mir-kiosk --channel=edge
ssh$ snap install ubuntu-app-platform

 

 

3)Using the snap built from this branch

https://code.launchpad.net/~osomon/webbrowser-app/mirkiosk-snap  

This particular snap

https://code.launchpad.net/~osomon/+snap/webbrowser-mirkiosk/+build/16501

Seemed to work find, download copy over and install


ssh$ snap install webbrowser-app*.snap --devmode --dangerous

 

4) NOTE: because of bug  you have to do the following, hopefully the pull request will get merged soon and this step we can remove

 

ssh$ snap disconnect webbrowser-app:mir
ssh$ snap disconnect webbrowser-app:platform
ssh$ snap connect webbrowser-app:mir mir-kiosk:mir
ssh$ snap connect webbrowser-app:platform ubuntu-app-platform:platform
ssh$ snap disable webbrowser-app
ssh$ snap enable webbrowser-app

 

5) Now launch and use


$ webbrowser-app

 

If you should experience a crash of the web browser, just restart with the same command. Also, you will see some spew at the console you may ignore from the browser launching related to audio and Qt stuff.

 

Debugging: if you should find things aren’t working as expected, as in you do not see the web browser. Try rebooting first, which should auto launch mir-kiosk, then repeat the connection process and launching the browser. If that still doesn’t work, inspect all the connections via ssh$ snap interfaces and make sure mir-kiosk:mir-libs, webbrowser-app:mir-kiosk, webbrowser-app:ubuntu-app-platform, webbrowser-app:mir-libs are all connected as expected. Feel free to ping me or others on freenode at #snappy or #ubuntu-unity or #ubuntu-mir

Read more
Alan Griffiths

MirAL 1.0

There’s a new MirAL release (1.0.0) available in ‘Zesty Zapus’ (Ubuntu 17.04) and the so-called “stable phone overlay” ppa for ‘Xenial Xerus’ (Ubuntu 16.04LTS). MirAL is a project aimed at simplifying the development of Mir servers and particularly providing a stable ABI and sensible default behaviors.

Surprisingly, given the project’s original goal, the ABI is changed. This allowed us to address a couple of minor issues and the timing seemed good as downstreams are faced with Mir-0.25 moving some necessary APIs from libmircommon to the more ABI stable libmircore.

The changes in 1.0.0 are:

  1. The default movement of child windows can be overridden by the window management policy;
  2. A new “miral-app” script that runs the miral example servers as an application on an existing desktop;
  3. Bug fix LP: #1646431 “Examples fail to start under Unity8”;
  4. Bug fix LP: #1646735 “[miral-shell –window-manager tiling] windows are not correctly constrained to tiles”; and
  5. A couple of deprecated APIs have been removed.

Read more
Alan Griffiths

Testing for Mir

Need a Mir server?

A couple of times in the last week I’ve been asked about a Mir server for testing. These requests have been from folks wanting to test their client-side work against Mir.

Most application developers will be using a toolkit or other graphics development library and not care if they are running on X11, Mir or even Windows. But the developers of those libraries will want to test with Mir.

For this purpose, the simplest Mir server to use is miral-shell. If you’re on Ubuntu Zesty Zapus then this is readily available:

$ sudo apt install miral-examples mir-graphics-drivers-desktop qtubuntu-desktop

If you’re on an earlier version of Ubuntu then you either need a ppa (such as the “stable phone overlay”) or, less risky to your system, just build and install it yourself. (If you’re not on Ubuntu this is still possible: there are some pointers here.)

What does miral-server provide?

Currently miral-server is the only Mir server to offer libmiral’s “basic window management”. That unique status is due to change real soon as this implementation is being merged into Unity8.

The simplest way way to run miral-shell is using Mir’s “Mir on X” support. From a terminal window just type:

$ miral-shell

Then you can connect your application from another terminal:

$ miral-run <application>

You should see your application appear in the “Mir on X” window.

A lot of the current work is focused on the placement of windows (menues, popup, etc.) and to help with this there’s a facility to trace the window management calls. Start miral-shell like this:

$ miral-shell --window-management-trace

And all the window management events and decisions are logged.

Another interesting option is to use a “tiling” window manager:

$ miral-shell --window-manager tiling

Which has a completely different approach to laying out the application windows.

For a full list of the option:

$ miral-shell --help

Documentation of the Mir “toolkit”API

A related question I’ve been asked is for documentation of the libmirclient API. You can find the documentation like this:

$ sudo apt install mir-doc
$ xdg-open /usr/share/doc/mir-doc/html/group__mir__toolkit.htm

This will open the default browser on the relevant page.

Read more
kevin gunn

more sample client updates

I can’t even remotely take credit for this. Alberto from the Mir team took the mir-client snap and updated to utilize the mir-libs snap through the content interface. This is helpful as a guide for others who want to avoid making useless copies of libraries in a mir-client app snap. He also added some additional example client applications to run on the mir-kiosk, along with using the snap set command to dynamically change those from the command line. I’ve updated the mir-snaps wiki on how to utilize this. enjoy! If you wanna discuss or have issues, find me (kgunn) on freenode #ubuntu-mir

Read more
Alan Griffiths

Miral on Dragonboard

Miral on Dragonboard

Having seen Kevin Gunn’s post on the mir-kiosk I thought I’d give it a try.  This is what I found.

Preliminaries

First, I dug out the Dragonboard I’d been testing Mir on a few months ago from a drawer and borrowed the Logitech k400r keyboard I use in my “den”. (This is convenient there because it provides both keyboard and mouse input from one device I can use from my lap.)

There was still a micro SD card in the board, so I extracted that and inserted it via a micro SD adapter into my desktop. Then I downloaded the dragonboard image from the link in Kevin’s article, verified the checksum and discovered that “Disk Image Writer” was the default app for opening the file. That sounded promising, so I opened the file with it, selected the SD card and started writing the image.

While that was going on I found an HDMI/VGA adapter and an old monitor to connect to the Dragonboard and started checking how to install from the SD. The main thing was to get the boot switches on the back of the board into 0110 positions (which means it will boot from the micro SD card).

With all that ready I went to brew tea while “Disk Image Writer” continued reporting progress.

It boots!

Returning with my tea I found that the image was written, so I disconnected the card and put it into the Dragonboard and connected the power.

I was greeted by a blank screen.

After a bit of experimentation I found that I need to connect to a real HDMI monitor during boot, but can switch to the adapter+VGA monitor after that. Annoying, but it works. (As I’m not yet sure where the fault lies I won’t mention the brands.)

Setup is a few simple questions, the only annoyance I had was that the keyboard layout defaults to US which makes typing my network password “interesting” on a UK keyboard.

One thing that could be a trap is that it asked for my email address to connect to the snap store. I’ve never been there before but I took a punt and used my canonical email address. That seemed to work and pick up the credentials on Launchpad.

That meant I could ssh into the Dragonboard from my desktop.

Installing miral-kiosk

Installing the mir libraries and the miral kiosk looks really easy using the commands Kevin provides (in the ssh session):

$ sudo snap install mir-libs --channel=edge --devmode
error: cannot perform the following tasks:
- Download snap "mir-libs" (3) from channel "edge" (unexpected EOF)

Actually, before that message was shown I got one estimating the download time at over an hour! Try again…

$ sudo snap install mir-libs --channel=edge --devmode
mir-libs (edge) 0.1 from 'albaguirre' installed

That’s better! (And only took a few seconds.)

$ sudo snap install mir-kiosk --channel=edge --devmode
mir-kiosk (edge) 0.1 from 'albaguirre' installed

Wow, I see the orange miral-kiosk startup “splash” and a mouse pointer! miral-kiosk is running on the Dragonboard.

A Client Application

To get a client application kg’s blog continues with the instructions:

“Download the appropriate architecture of the mir-client snap and then copy that over to your running ubuntu-core image. “

In this case we’re arm64, so I followed the link and picked out mir-client_0.24.1_arm64.snap. And then copied it to the dragonboard (back to a desktop terminal session):

$ scp Downloads/mir-client_0.24.1_arm64.snap alan-griffiths@192.168.1.159:~
mir-client_0.24.1_arm64.snap 100% 103MB 173.7KB/s 10:05

That took an unexpectedly long time. Back to ssh session:

$ sudo snap install mir-client_0.24.1_arm64.snap --channel=edge --devmode 
mir-client 0.24.1 installed

And there it is – kg’s sample client application running on miral-kiosk.

Read more
kevin gunn

If you’ve been following along, you’ll know that we’ve put some snap work in to show how you might use Mir as a framework to build a kiosk style product. This post touches on a couple of recent evolutions.

First, there’s been recent work in improving Mir’s API stability at the server level, to be a true toolkit for shells through Miral which you can read about here. And you can read about the latest Miral 0.3 release here. Part of Miral provides 2 default shell implementations. One is miral-shell and the other is miral-kiosk. Miral-kiosk, as the name suggests, is a very minimal shell, keeping the footprint and complexity low. Hence it’s perfect for targeting products requiring simple, single application user interfaces. So we’ve created a snap utilizing this, named “mir-kiosk”.

Eventually Miral will become part of Mir itself, we just need to work through supported trusted prompts in more complex shell use cases (which is happening as I type). But the point of this post, is demonstrating miral-kiosk in a snap. If anyone reading this is considering using Mir snaps for production in a kiosk style product, I would recommend miral-kiosk as the preferred method. The same confinement achieve before still exists and you can run the same example applications.

Second, with the advent of the content interface available in the latest snapd release we are moving out the Mir libraries into their own snap that can be leveraged by the shell and mir-clients. This will make sure the Mir libraries stay in sync with one another and there’s a little deduplication gain so there’s not a lot of snaps with copies of Mir libraries as stage packages. This snap’s name is “mir-libs”.

Both the mir-kiosk & mir-libs snaps are available in the snap store. It can be demonstrated using the same mir-client snap that’s been used before in other posts.

Now, to experience this you need to download the latest ubuntu-core image, which is Release Candidate 2 (RC2). Download the appropriate architecture of the mir-client snap and then copy that over to your running ubuntu-core image. You can then ssh into your device/VM and install in this particular order.

$ snap install mir-libs --channel=edge --devmode
$ snap install mir-kiosk --channel=edge --devmode
$ snap install mir-client_0.24.1_amd64.snap --devmode --dangerous

 

At this point you should witness PhotoViewer running on mir-kiosk using mir-libs via content interface on your device or VM.

One last note, you might notice I’ve added –devmode to the installation steps here, that is due to a small regression in the RC2 image, it’s a bug that’s actively being worked. Confinement is still maintained with the the mir-kiosk snap.

 

Read more
Alan Griffiths

There are a few “gotchas” in running X11 applications (via Xmir) on Mir servers so I’m sharing a short script to make it easier.

The following script will work with the example servers from the “mir-demos” package, the miral-shell (from “miral-examples”) and my own egmde project. (With Unity8 there’s a little more to it but as there is existing “magic” in place for launching X11 applications I won’t bother to discuss it further.)

The principle issue is that each Xmir session is seen as a single application by the Mir server, so we need to create an Xmir server for each application for everything to make sense. And that means each application needs a separate port to connect to its Xmir server.

For this to work you need to have a Mir server running, and have Xmir installed.

Here’s the script:

$ cat ~/bin/Xmir-run
#!/bin/bash
port=0
while [ -e "/tmp/.X11-unix/X${port}" ]; do
    let port+=1
done

Xmir -rootless :${port} & pid=$!
DISPLAY=:${port} $*
kill ${pid}

The first part of this script finds an available port to run Xmir on.

The next part starts an Xmir server in “rootless” mode and remembers the pid.

Then we run the command passed to the script until it exits.

Finally, we kill the Xmir server.

Simple!

Read more
kevin gunn

a better kiosk demo

Hey just having more fun snapping on dragonboard. I’ve updated the mir-client snap to use a Qt demo that is probably a bit more like what a kiosk style application might be. It’s the photoviewer on dragonboard as an example. Which improved not only the demo experience but provides developers a better guide since it actually uses the qmake plugin of snapcraft to build the demo from source. I can’t emphasize how easy it was to modify my snapcraft project to add this to the demo. Again, good ‘ol mir-snaps wiki can be used as a guide. And if you don’t want to build, you can grab my personal builds of these snaps for arm64 for mir-server snap and mir-client-snap respectively.

Read more
kevin gunn

hey just a very quick update. Had some more time to play around today and touch is working after all (only difference is I left my usb keyboard disconnected today so maybe it was getting confused)

Anyhow, here’s videos of Qt clocks with touch and Qt samegame with touch

Read more
kevin gunn

OK, I’m really overdue on posting something about this as I’ve had _something_ running on the dragonboard 410c for a while. If you don’t know about dragonboard you can check out dragonboard from 96boards .

So dragonboard is targeted to be a supported reference board by our Snappy team and they’re in the process of pushing out beta images to play with in the 16 series. I had been concerned that we I was going to have to go and build the graphics drivers into our Ubuntu core snap. When I started I wasn’t even sure of the state of the freedreno drivers vs closed source vendor drivers. But as luck would have it, someone quite recently had turned on the gallium drivers to be built and package as part of the Ubuntu distro, which means I got the freedreno drivers with no effort! Lots of love to Rob Clark for all the work he’s done on freedreno (if your interested in learning more  check out freedreno on github ).

So getting a devmode mir snap demo up and running was relatively painless. However, I do want to say I found a little difference in my runs amd64 VM vs the native arm64. This resulted in some tweaks to the mir interface in snapd (which had already landed and should be in the next snapd release). Also, never use setterm when developing with Mir, that create all sorts of chaos for me 🙂 I had used setterm for convenience to prevent the screen from blanking, ended up causing failures when I was working on making sure the mir snaps could run confined.

If you follow the good ol’ mir snaps wiki, you can easily duplicate this – running the mir snaps fully confined on dragonboard core snap. Also, I wanted to point out again there are other Qt demos you can try besides the clock app – simply modify the helper file in the client example (client-start) to be something besides “clock”, for example “maroon” or “samegame”. You can do this with an HDMI monitor and mouse attached  like in this  video of various Qt apps as mir-client snaps running on dragonboard . Still need to investigate some mouse oddities that seem to only occur with apps other than clock.

And lastly, I got new toy over the weekend. I ordered a 7″ touch screen from adafruit. Here’s a quick video of the 7″ display attached. I need to tinker with it to see about getting the touch to work, but it was nice to just hook it together and the display come up.

Read more
Alan Griffiths

–window-management-trace

A feature added to the lp:miral trunk yesterday is making life a lot easier for developers working on MirAL based servers, and the toolkit extensions that support Mir. The feature is:

miral-shell --window-management-trace

Actually, the –window-management-trace switch will work with any MirAL base server (so miral-kiosk and egmde support it too).

What this does is cause the server to log every interaction with the window management policy – all the notifications it receives and all the calls it makes to the tools as a result.

This means that it is easy to find out that, for example, a “modal” gtk based dialog window is being created without specifying a parent. (Which is why the behaviour isn’t quite as expected – Mir servers will treat it as non-modal.)

To use this feature before the next MirAL release you do need build it yourself, but this only takes a minute (depending on your kit).It really is the easiest way to see exactly what is going on and why.

Read more
kevin gunn

So first, if you didn’t catch it, series 16 Ubuntu Core beta images are available here http://cdimage.ubuntu.com/ubuntu-snappy/16.04/current/

I just verified the Mir snaps are functioning and made some small updates to match here https://developer.ubuntu.com/en/snappy/build-apps/mir-snaps/

One of which being a command line switch for installing the snaps locally… –dangerous, what a great flag name 🙂

Also, I made some updates to the snaps themselves to check for the architecture from $SNAP_ARCH and then set up all the correct paths. So this means the scripts will work properly on the various archs without having to tinker. I’ve also changed the mir-client snap specifically to pull the demos from the archive, this way you’ll also get the right binary for the right arch as well.

Enjoy!

 

Read more
Alan Griffiths

An Example Mir Desktop Environment

The world of Mir

Mir is a set of libraries supporting the development of display servers, desktop environments, shells and implementing toolkit support for applications running on them.

As part of their “convergence” of computing devices Canonical are committed to providing Mir on a range of platforms including desktops, phones, tablets and “the internet of thing”. You can read about the work to provide it as a “snap” in the “internet of things” on Kevin Gunn’s blog [kg’s blog].

Toolkits and other “client” platforms that have support for Mir include GTK, Qt and SDL – which mean that applications using these can work on Mir. There’s a separate Xmir project to support X11 applications on Mir.

On the driver side of things Mir works with Mesa on desktop-like systems and, using libhybris, on an android based stack on a number of devices. The Mesa support is a work in progress and Ubuntu carries a set of patches to enable it. There are some notes about how to incorporate these on other platforms here: [Enabling Mir EGL]. Work is in progress to upstream Mir support.

The “Mir Abstraction Layer” (MirAL) packages the server-side functionality in a way that makes it easy to use. My previous post[MirAL] introduced MirAL in more detail.

Before we begin

If you want to experiment with this code I suggest using Ubuntu 16.04 or later on desktop for the reasons mentioned above. On an Ubuntu phone you can use the current stable version with this code, but doing development on a phone is a whole other article. On other distributions you likely need to patch Mesa and rebuild toolkits with Mir support enabled.

Having got an Ubuntu desktop you also need to install MirAL, at present this is not available from the Ubuntu archives, so you have to build and install it yourself:

$ sudo apt-get install devscripts equivs bzr
$ bzr branch lp:miral
$ sudo mk-build-deps -i --build-dep miral/debian/control
$ mkdir miral/build
$ cd miral/build
$ cmake ..
$ make
$ sudo make install

egmde [Example Mir Desktop Environment]

To illustrate MirAL I’m going to show what is involved in writing a (very simple) window manager. It runs on desktops, tablets and phones and supports keyboard, mouse and touch input. It will support applications using the GTK and Qt toolkits, SDL applications and (using Xmir) X11 applications.

The full code for this example is available on github:

$ git clone https://github.com/AlanGriffiths/egmde.git

Naturally, the code is likely to evolve, especially if I write a follow-up article but the version presented here is tagged v1.0. Assuming that you’ve MirAL installed as described above you can build it as follows:

$ mkdir egmde/build
$ cd egmde/build
$ cmake ..
$ make

The example code

A lot of the functionality (default placement of windows, menus etc.) comes with the MirAL library. For this exercise we’ll implement one class and write a main function that injects it into MirAL. The main program looks like this:

int main(int argc, char const* argv[])
{
    miral::MirRunner runner{argc, argv};

    return runner.run_with(
        {
            set_window_managment_policy<ExampleWindowManagerPolicy>()
        });
}

Yes, you’ve guessed it: the class we’ll be implementing is ExampleWindowManagerPolicy. It looks like this:

class ExampleWindowManagerPolicy : public CanonicalWindowManagerPolicy
{
public:
    using CanonicalWindowManagerPolicy::CanonicalWindowManagerPolicy;

    // Switch apps  : Alt+Tab
    // Switch window: Alt+`
    // Close window : Alt-F4
    bool handle_keyboard_event(MirKeyboardEvent const* event) override;

    // Switch apps  : click on the corresponding window
    // Switch window: click on the corresponding window
    // Move window  : Alt-leftmousebutton drag
    // Resize window: Alt-middle_button drag
    bool handle_pointer_event(MirPointerEvent const* event) override;

    // Switch apps  : tap on the corresponding window
    // Switch window: tap on the corresponding window
    // Move window  : three finger drag
    // Resize window: three finger pinch
    bool handle_touch_event(MirTouchEvent const* event) override;

private:
    void pointer_resize(Window const& window, Point cursor, Point old_cursor);
    void resize(WindowInfo& window_info, Point new_pos, Size new_size);

    // State held for move/resize by pointer
    Point old_cursor{};
    bool resizing = false;
    bool is_left_resize = false;
    bool is_top_resize = false;

    // State held for move/resize by touch
    int old_touch_pinch_top = 0;
    int old_touch_pinch_left = 0;
    int old_touch_pinch_width = 0;
    int old_touch_pinch_height = 0;
};

For this simple example the only functionality we’ll be providing is the handling of input events to allow the user to switch between applications and windows and to resize and move windows.

We only need to provide three handle_XXX_event() functions as we’re relying on the default behaviour for everything else. The first of these functions is the simplest:

bool ExampleWindowManagerPolicy::handle_keyboard_event(MirKeyboardEvent const* event)
{
    auto const action = mir_keyboard_event_action(event);
    auto const shift_state = mir_keyboard_event_modifiers(event) & shift_states;

    if (action == mir_keyboard_action_down &&
        shift_state == mir_input_event_modifier_alt)
    {
        switch (mir_keyboard_event_scan_code(event))
        {
        case KEY_F4:
            tools.ask_client_to_close(tools.active_window());
            return true;

        case KEY_TAB:
            tools.focus_next_application();
            return true;

        case KEY_GRAVE:
            tools.focus_next_within_application();
            return true;

        }
    }

    return false;
}

I don’t think this needs a lot of explanation: we select key presses while only the Alt button is pressed and act according to the button pressed.

The other two functions are a bit longer as they need to remember some state in order to interpret mouse or touchpad gestures. There’s in them nothing I consider worth pointing out here and the full code is available on github. There’s not much:

$ wc *
15 28 446 CMakeLists.txt
357 957 10528 egmde.cpp
372 985 10974 total

I have much to compare it with, bit I don’t think 372 lines is bad for a functional window manager.

Running egmde

Once egdme has been built you can run it in two modes: the simplest is the “Mir-on-X” mode:

$ ./egmde

This will produce a rather boring black “Mir-on-X” window. You can attach Mir clients to this:

$ miral-run gnome-system-monitor
$ miral-run gnome-terminal

These will run in the window and make it less boring. You can also have egmde launch them when it starts as follows:

$ ./egmde --startup gnome-terminal:gnome-system-monitor

If you don’t want to run in a window under X11 you can also give egmde access to your hardware directly by running it as root and specifying a “virtual terminal”. But I think it is probably better to use mir_demo_server in “system-compositor” mode to handle the hardware and launch egdme a normal user.

$ sudo apt install mir-demos
$ sudo mir_demo_server --vt 3 --window-manager system-compositor

You’ll find this switches you to vt3, switch back with Ctrl-Alt-F7 and, from another terminal, you can connect egdme to this. (We need the VT switch here as the host mir_demo_server is paused while X11 is in control.)

$ sudo chvt 3&&./egmde --host /tmp/mir_socket --startup gnome-terminal

You don’t have to start the gnome-terminal, but it is convenient as you can launch other applications. For example:

$ mir_demo_client_eglplasma

Conclusion

I hope that the egmde example is enough to spark someone’s interest in MirAL and to provide a starting point for developing something more ambitious.

Much of what has been discussed is a work-in-progress, especially MirAL which I hope to get added to the Ubuntu archives soon. I am optimistic this will happen as it is in the interests of both Canonical and the wider community to have a stable server ABI against which to write desktop environments (including Unity8). Indeed I’ve been working with the main developer of the QtMir abstraction layer to migrate it to MirAL.

Similarly, a significant portion of the work needed on Mesa to support Mir is similar to the work to support Wayland/Weston and everyone would benefit from cleaning this up (and, for example, removing hard dependencies on X11).

Read more
Alan Griffiths

MirAL: Mir is not all about Unity8

Introducing The Mir Abstraction Layer

The principle Open Source project I’ve been working on for the last few years is Mir. Mir is a library for writing Linux display servers and shells that are independent of the underlying graphics stack. It fits into a similar role as an X server or Weston (a Wayland server) but was initially motivated by Canonical’s vision of “convergent” computing.

The Mir project has had some success in meeting Canonical’s immediate needs – it is running in the Ubuntu Touch phones and tablets, and as an experimental option for running the Unity8 shell on the desktop. But because of the concentration of effort on delivering the features needed for this internal use it hasn’t really addressed the needs of potential users outside of Canonical.

Mir provides two APIs for users: the “client” API is for applications that run on Mir and that is largely used by toolkits. There is support for Mir in the GTK and Qt toolkits, and in SDL. This works pretty well and the Mir client API has remained backwards compatible for a couple of years and can do so for the foreseeable future.

The problem is that the server-side ABI compatibility is broken by almost every release of Mir. This isn’t a big problem for Canonical, as the API is fairly stable and both Mir and Unity8 are in rapid development: rebuilding Unity8 every time Mir is released is a small overhead. But for independent developers the lack of a stable ABI is problematic as they cannot easily synchronize their releases to updates of Mir.

My answer to this is to provide a stable “abstraction layer” written over the top of the current Mir server API that will provide a stable ABI. There are a number of other goals that can be addressed at the same time:

  • The API can be considerably narrowed as a lot of things can be customized that are of no interest to shell development;
  • A more declarative design style can be followed than the implementation focused approach that the Mir server API follows; and,
  • Common facilities can be provided that don’t belong in the Mir libraries.

At the time of writing the Mir Abstraction Layer (miral) is a proof-of-concept, but work is in progress to package an initial version that can support the functionality needed by some examples, and by qtmir (the Qt support used by Unity8).

Building and using MirAL

These instructions assume that you’re using Ubuntu 16.04LTS or later, I’ve not tested earlier Ubuntu versions or other distributions.

You’ll need a few development and utility packages installed, along with the Mir development packages:

$ sudo apt-get install cmake g++ make bzr python-pil uuid-dev libglib2.0-dev
$ sudo apt-get install libmirserver-dev libmirclient-dev mirtest-dev
$ sudo apt-get install mir-graphics-drivers-desktop libgles2-mesa-dev

(If you’re working on a phone or tablet use mir-graphics-drivers-android in place of mir-graphics-drivers-desktop.)

With these installed you can checkout and build miral:

$ bzr branch lp:miral
$ mkdir miral/build
$ cd  miral/build
$ cmake ..
$ make

This creates libmiral.so in the lib directory and an example shell (miral-shell) in the bin directory. This can be run directly:

 $ bin/miral-shell

With the default options this runs in a window on X (which is convenient for development).

The miral-shell example is simple, don’t expect to see a sophisticated launcher by default. You can start mir apps from the command-line. For example:

 $ bin/miral-run gnome-terminal

That’s right, a lot of standard GTK+ applications will “just work” (the GDK toolkit has a Mir backend). Any that assume the existence of an X11 and bypass the toolkit by making X11 protocol calls will have problems though.

To exit from miral-shell press Ctrl-Alt-BkSp.

To run independently of X11 you need to grant access to the graphics hardware (by running as root) and specify a VT to run in. For example:

$ sudo bin/miral-shell --vt 4 --arw-file --file $XDG_RUNTIME_DIR/mir_socket

For convenient testing there’s a “testrun” script that wraps this command to start the server (as root) and then launches the gnome-terminal (as the current user):

$ ../scripts/testrun

Running applications on MirAL

If you have a terminal session running in the MirAL desktop (as described above) you can start programs from it. GTK, Qt and SDL applications will “just work” provided that they don’t bypass the toolkit and attempt to make X11 protocol calls that are not available.

$ gedit
$ 7kaa

From outside the MirAL session the “miral-run” script sets a few environment variables to configure the Mir support in the various toolkits. (There’s some special treatment for gnome-terminal as starting that can conflict with the desktop default.)


$ bin/miral-run gnome-calculator
$ bin/miral-run 7kaa

There are also some examples of native Mir client applications in the mir-demos package. These are typically basic graphics demos:

$ sudo apt-get install mir-demos
$ mir_demo_client_egltriangle

Support for X11 applications

If you want to run X11 applications that do not have native Mir support in the toolkit they use then the answer is Xmir: an X11 server that runs on Mir. First you need Xmir installed:

$ sudo apt install xmir

Then you can use the testrun script to start miral-shell with Xmir:

$ ../scripts/testrun -Xmir

This starts an X11 server on DISPLAY=:1. This is set in the terminal the script starts so that applications launched from it will automatically connect to miral through Xmir.

Running Qt applications

To run Qt applications under Mir you may need to install qtubuntu-desktop:

$ sudo apt-get install qtubuntu-desktop

Read more
kevin gunn

Mir interface landed!

So the Mir interface has landed in snapd. This means that the mir-server snap can be downloaded and taken into use in a fully confined mode, along with matching mir-client snap. I’ve been keeping updated instructions here on the best way to operate and develop your mir-client snap. The one caveat is there’s still a snappy bug (launchpad bug# 1577897)  with snap auto-connections.  This means that you have to manually make the plug-slot connection of the mir-server and mir-client, and then restart the mir-client service. I’ve just uploaded a new mir-server snap which is on Mir version 0.23.5, it should be reflected in the store soon.

At any rate, I’d love to hear back from folks trying this out. In the coming weeks I’m hoping to spend some time on dragonboard and getting this working there.

Read more
Christopher Halse Rogers

XMir Performance

Or: Why XMir is slower than X, and how we'll fix it

We've had a bunch of testing of XMir now; plenty of bugs, and plenty of missing functionality.

One of the bugs that people have noticed is a 10-20% performance drop over raw X. This is really several bits of missing functionality - we're doing a lot more work than we need to be. Oddly enough, people have also been mentioning that it feels "smoother" - which might be placebo, or unrelated updates, or might be to do with something in the Mir/XMir stack. It's hard to tell; it's hard to measure "smoother". We're not faster, but faster is not the same as smoother.

Currently we do a lot of work in submitting rendering from an X client to the screen, most of which we can make unnecessary.

The simple bit

The simple part is composite bypass support for Mir - most of the time unity-system-compositor does not need to do any compositing - there's just a single full-screen XMir window, and Mir just needs to flip that to the display. This is in progress. This cuts out an unnecessary fullscreen blit.

The complicated part is in XMir itself

The fundamental problem is the mismatch between rendering models - X wants the contents of buffers to be persistent; Mir has a GLish new-buffer-each-frame. This means each time XMir gets a new buffer from Mir it needs to blit the previous frame on first, and can't simply render straight to Mir's buffer. Now, we can (but don't yet) reduce the size of this blit by tracking what's changed since XMir last saw the buffer - and a lot of the time that's going to be a lot smaller than fullscreen - but there's still some overhead¹.

Fortunately, there's an way around this. GLX matches Mir's buffer semantics nicely - each time a client SwapBuffers it gets a shiny new backbuffer to render into. So, rather like Compiz's unredirect-fullscreen-windows option, if we've got a fullscreen² GLX window we can hand the buffer received from Mir directly to the client and avoid the copy.

Even better, this doesn't apply only to fullscreen games - GNOME Shell, KWin, and Unity are all fullscreen GLX applications.

As always, there are interesting complications - applications can draw on their GL window with X calls, and applications can try to be fancy and only update a part of their frontbuffer rather than calling SwapBuffers; in either case we can't bypass. Unity does neither, but Shell and KWin might.

Enter the cursor

In addition to the two unnecessary fullscreen blits - X root window to Mir buffer, Mir buffer to framebuffer - XMir currently uses X's software cursor code. This causes two problems. Firstly, it means we're doing X11 drawing on top of whatever's underneath, so we can't do the SwapBuffers trick. Secondly, it causes a software fallback whenever you move the cursor, making the driver download the root window into CPU accessible memory, do some CPU twiddling, and then upload again to GPU memory. This is bad, but not terrible, for Intel chips where the GPU and CPU share the same memory but with different caches and layouts. It's terrible for cards with discrete memory. Both these problems go away once we support setting the HW cursor image in Mir.

Once those three pieces land there shouldn't be a meaningful performance difference between XMir-on-Mir and X-on-the-hardware.

¹: If we implemented a single-buffer scheme in Mir we could get rid of this entirely at the cost of either losing vsync or blocking X rendering until vsync. That's probably not a good tradeoff.

²: Technically, if we've got a GLX client whose size matches that of the underlying Mir buffer. For the moment, that means "fullscreen", but when we do rootless XMir for 14.04 all windows will be backed by a Mir buffer of the same size.

Read more
Daniel Holbach

In many Ubuntu conversation I’ve been part of many of the participants agreed  that we need “more transparency”. It’s very easy to agree on as transparency is a good thing, it feels good and it makes things better. Achieving it in a meaningful way is a hard problem to solve though. Meaningful to me means not just “all information is available”, but also “relevant information is easy to find”. In Ubuntu development where hundreds of people put of lots of hard work into Ubuntu, we depend on thousands of other open source projects, where there’s discussions on IRC, on mailing lists, hangouts, in specifications and elsewhere, it’s incredibly easy to lose track of what’s important or relevant.

A lot of teams forming the core of Ubuntu send out weekly summaries of their work, which is great. Among them the Mir and Unity 8 team, the kernel team, Unity APIs team, the Ubuntu Touch team and there’s bits of information everywhere. While this is a great start in being able to get a more complete picture, it also takes some time to read, digest, understand and probably talk to people. To help with this we came up with an idea we already discussed at UDS.

The plan is to read and digest the news and have regular hangouts to which invite engineers to talk about what they’ve been doing, show what’s new and answer questions from the audience. To make this even a bit more interesting, we’d like to invite people from tech blogs and tech news sites. The idea being that they know what their readers would like to hear about and what’s interesting. This would bring together the best of many worlds: what’s new in Ubuntu, the new devices, apps, great stuff from the tech press and live interviews with engineers.

What I’d need now is a bit of help with organising this and setting this up. Please leave a comment or drop me a mail, if you think this is a great idea too and would like to help. :-D

Read more
Christopher Halse Rogers

Artistic differences

The latest entry in my critically acclaimed series on Mir and Wayland!

Wayland, Mir, and X - different projects

Apart from the architectural differences between them, which I've covered previously, Mir and Wayland also have quite different project goals. Since a number of people seem to be confused as to what Wayland actually is - and that's not unreasonable, because it's a bit complicated - I'll give a run-down as to what all these various projects are and aim to do, throwing in X11 as a reference point.

X11, and X.org

Everyone's familiar with their friendly neighbourhood X server. This is what we've currently got as the default desktop Linux display server. For the purposes of this blog post, X consists of:

The X11 protocol

You're all familiar with the X11 protocol, right? This gentle beast specifies how to talk to an X server - both the binary format of the messages you'll send and receive, and what you can expect the server to do with any given message (the semantics). There are also plenty of protocol extensions; new messages to make the server do new things, like handle more than one monitor in a non-stupid way.

The X11 client libraries

No-one actually fiddles with X by sending raw binary data down a socket; they use the client libraries - the modern XCB, or the boring old Xlib (also known as libX11.so.6). They do the boring work of throwing binary data at the server and present the client with a more civilised view of the X server, one where you can just XOpenDisplay(NULL) and start doing stuff.
Actually, the above is a bit of a lie. Almost all the time people don't even use XCB or Xlib, they use toolkits such as GTK+ or Qt, and they use XLib or XCB.

The Xorg server

This would be the bit most obviously associated with X - the one, the only, the X.org X server! This is the actual /usr/bin/X display server we all know and love. Although there are other implementations of X11, this is all you'll ever see on the free desktop. Or on OS X, for that matter.

So that's our baseline stack - a protocol, one or more client libraries, a display server implementation. How about Wayland and Mir?

The Wayland protocol

The Wayland protocol is, like the X11 protocol, a definition of the binary data you can expect to send and receive over a Wayland socket, and the semantics associated with those binary bits. This is handled a bit differently to X11; the protocol is specified in XML, which is processed by a scanner and turned into C code. There is a binary protocol, and you can technically¹ implement that protocol without using the wayland-scanner-generated code, but it's not what you're expected to do.

Also different from X11 is that everything's treated as an extension - you deal with all the interfaces in the core protocol the same way as you deal with any extensions you create. And you create a lot of extensions - for example, the core protocol doesn't have any buffer-passing mechanism other than SHM, so there's an extension for drm buffers in Mesa. The Weston reference compositor also has a bunch of extensions, both for ad-hoc things like the compositor<->desktop-shell interface, and for things like XWayland.

The Wayland client library

Or libwayland. A bit like XCB and Xlib, this is basically just an IPC library. Unlike XCB and Xlib it can also used by a Wayland server for server→client communication. Also unlike XCB and Xlib, it's programmatically generated from the protocol definition. It's quite a nice IPC library, really. Like XCB and Xlib, you're not really expected to use this, anyway; you're expected to use a toolkit like Qt or GTK+, and EGL + your choice of Khronos drawing API if you want to be funky.
There's also a library for reading X cursor themes in there.

The Wayland server?

This is where it diverges; there is no Wayland server in the sense that there's an Xorg server. There's Weston, the reference compositor, but that's strictly intended as a testbed to ensure the protocol works.

Desktop environments are expected to write their own Wayland server, using the protocol and client libraries.

The Mir protocol?

We kinda have an explicit IPC protocol, but not really. We don't intend to support re-implementations of the Mir client libraries, and will make no effort to not break them if someone tries. We're using Google Protobuf for our IPC format, by the way.

The Mir client libraries

What toolkit writers use; it's even called mir_toolkit. Again, you're probably not going to use this directly; you're going to use a toolkit like GTK+ or Qt, and like Wayland if you want to draw directly you'll be using EGL + GL/GLES/OpenVG.

The Mir server?

Kind of. Where the Wayland libraries are all about IPC, Mir is about producing a library to do the drudge work of a display-server-compositor-thing, so in this way it's more like Xorg than Wayland. In fact, it's a bit more specific - Mir is about creating a library to make the most awesome Unity display-server-compositor-thingy. We're not aiming to satisfy anyone's requirements but our own. That said, our requirements aren't likely to be hugely specific, so Mir will likely be generally useful.

To some extent this is why GNOME and KDE aren't amazingly enthused about Mir. They already have a fair bit of work invested in their own Wayland compositors, so a library to build display-server-compositor-thingies on isn't hugely valuable to them. Right now.

Perhaps we'll become so awesome that it'll make sense for GNOME or KDE to rebase their compositors on Mir, but that's a long way away.

¹: Last time I saw anyone try this on #wayland there were problems around the interaction with the Mesa EGL platform which meant you couldn't really implement the protocol without using the C existing library. I'm not sure if that got resolved.

Read more
Christopher Halse Rogers

Server Allocated Buffers in Mir

…Or possibly server owned buffers

One of the significant differences in design between Mir and Wayland compositors¹ is the buffer allocation strategy.

Wayland favours a client-allocated strategy. In this, the client code asks the graphics driver for a buffer to render to, and then renders to it. Once it's done rendering it gives a handle² to this buffer to the Wayland compositor, which merrily goes about its job of actually displaying it to the screen.

In Mir we use a server-allocated strategy. Here, when a client creates a surface, Mir asks the graphics driver for a set of buffers to render to. Mir then sends the client a reference to one of the buffers. The client renders away, and when it's done it asks Mir for the next buffer to render to. At this point, Mir sends the client a reference to another buffer, and displays the first buffer.

You can see this in action - in the software-rendered case - in demo_client_unaccelerated.c (I know the API here is a bit awkward; I'm agitating for a better one).
The meat of it is:


mir_wait_for(mir_surface_next_buffer(surface, surface_next_callback, 0));
which asks the server for the next buffer and blocks until the server has replied. This also tells the server that the current buffer is available to display.
Then

mir_surface_get_graphics_region(surface, &graphics_region);
gets a CPU-writeable block of memory from the current buffer. You'll note that we don't have a mir_wait_for around this; that's because we've already got the buffer from the server above; this just accesses it.

The code then writes an interesting or, in fact, highly boring pattern to the buffer and loops back to next_buffer to display that rendering and get a new buffer, ad infinitum.

Great. So why are you doing it, again?

The need for server-allocated buffers is primarily driven by the ARM world and Android graphics stack, an area in which I'm blissfully unaware of the details. So, as with my second-hand why Mir post, you, gentle reader, get my own understanding.

The overall theme is: server-allocated buffers give us more control over resource consumption.

ARM devices tend to be RAM constrained, while at the same time having higher resolution displays than many laptops. Applications also tend to be full-screen. Window buffers are on the order of 4 MB for mid-range screens, 8 MB for the fancy 1080p displays on high end phones, and 16 MB for the Nexus 10s. Realistically you need to at least double-buffer, and it's often worth triple-buffering, so you're eating 8-12MB on the low end and 32-48MB on the high end, per window. Have a couple of applications open and this starts to add up on devices with between 512MB and 2GB of RAM and no dedicated VRAM.

In addition to their lack of dedicated video memory, ARM GPUs also tend to be much less flexible. On a desktop GPU you can generally ask for as many scanout-capable³ buffers as you like. ARM GPUs tend to have several restrictions on scanout buffers. Often they need to be physically contiguous, which in practice means allocated out of a limited block of memory reserved at boot. Sometimes they can only scanout of particular physical addresses. I'm sure there are other oddities out there.

So scanout buffers are a particularly precious resource. However, you really want clients to be able to use scanout-capable buffers - in the common case where you've got a single, opaque, fullscreen window, you don't have to do any compositing, so having the application draw directly to the scanout buffer saves you a copy; a significant performance win. Desktop aficionados might recognise this as the same effect as Compiz's unredirect fullscreen windows option.

So, there's the problem domain. What do server-allocated buffers buy us?

The server can control access to the limited scanout buffers.
This one's pretty self-explanatory.

The server can steal buffers from inactive clients.
When applications aren't visible, they don't really need their window buffers, do they? The server can either destroy the inactive client's buffers, or clear them and hand them off to a different application.

When applications are idle for an extended period we'll want to clear up more resources - maybe destroy their EGL state, and eventually kill them entirely. However, applications will be able to resume faster if we've just killed their window buffers than if we've killed their whole EGL state.

The server can easily throttle clients.
There's an obvious way for the server to prevent a client from rendering faster than it needs to; delay handing out the next buffer.

Finally:

Having a single allocation model is cleaner.
It's cleaner to have just one allocation model. We need to do at least some server-allocation, so that's the one allocation model. Plus, even though desktops have lots more memory than phones, people still care about memory usage ☺.

But doesn't X do server-allocated buffers, and it is terrible?

Yes. But we're not doing most of the things that make server-allocated buffers terrible. In X's case
  • X allocates all sorts of buffers, not just window surfaces. We leave the client to allocate pixmap-like things - that is, GL textures, framebuffer objects, etc - itself. Notably, we're not allocating ancillary buffers in the server, just the colour buffers. We shouldn't need to change protocol to support new types of buffers (HiZ springs to mind), like has happened with DRI2.
  • X isn't particularly integrated. When you resize an X window, the client gets an DRI2 Invalidate event, indicating that it needs to request new buffers. It also gets a separate ConfigureNotify event about the actual resize. We won't have this duplication; clients will automatically get buffers of the new size on the next buffer advance, and these buffers know their size.
  • X has more round-trips than necessary. After submitting rendering with SwapBuffers, clients receive an Invalidate event. They then need to request new, valid buffers. We won't have as many roundtrips; the server responds to a request to submit the current rendering with the new buffer.

A possibly significant disadvantage we can't mitigate is that the server no longer knows how big the client thinks its window is, so we may display garbage in the window if the client thinks its window is smaller than the server's window. I'm not sure how much of a problem that will be in practise.

Summary

Many of the benefits of server-allocation could also be gained with sufficiently smart clients and some extra protocol messages - although that falls down once you start suspending idle clients, as it requires the client to actually be running. We think that doing it in the server will be easier, cleaner, and more reliable.

I'm less convinced that server-allocation is the right choice than the rest of Mir's architecture. I'm not convinced that it's a mistake, either, and some of the benefits are attractive. We'll see how it goes!


¹: Yes, I know you can write a server-allocated buffer model with a Wayland protocol. All the Wayland compositors for which we have source, however, use a client-allocated model, and that's what existing Wayland client code expects. Feel free to substitute “existing Wayland compositors” whenever I say “Wayland”.

²: Actually, in the common (GPU-accelerated) case the client itself basically only gets a handle to the buffer; the GEM handle, which it gets from the kernel.

³: scanout is the process of reading from video memory and sending to an output encoder, like HDMI or VGA. A scanout-capable buffer is the only thing the GPU is capable of putting on a physical display.

Read more