How to run X11 applications on simple Mir servers

There are a few “gotchas” in running X11 applications (via Xmir) on Mir servers so I’m sharing a short script to make it easier.

The following script will work with the example servers from the “mir-demos” package, the miral-shell (from “miral-examples”) and my own egmde project. (With Unity8 there’s a little more to it but as there is existing “magic” in place for launching X11 applications I won’t bother to discuss it further.)

The principle issue is that each Xmir session is seen as a single application by the Mir server, so we need to create an Xmir server for each application for everything to make sense. And that means each application needs a separate port to connect to its Xmir server.

For this to work you need to have a Mir server running, and have Xmir installed.

Here’s the script:

$ cat ~/bin/Xmir-run
#!/bin/bash
port=0
while [ -e "/tmp/.X11-unix/X${port}" ]; do
    let port+=1
done

Xmir -rootless :${port} & pid=$!
DISPLAY=:${port} $*
kill ${pid}

The first part of this script finds an available port to run Xmir on.

The next part starts an Xmir server in “rootless” mode and remembers the pid.

Then we run the command passed to the script until it exits.

Finally, we kill the Xmir server.

Simple!

MirAL-0.2

There’s a new MirAL release (0.2.0) available in Yakkety. MirAL is a project aimed at simplifying the development of Mir servers and particularly providing a stable ABI and sensible default behaviors.

Unsurprisingly, given the project’s original goal, the ABI is unchanged. The changes in 0.2.0 fall into four categories:

  1. additional features for shell developers to use;
  2. enabling features available in newer Mir versions,
  3. improvements to the example server; and,
  4. bugfixes.

Additional Features for Shell Developers to use

There is a new “–window-management-trace” option for debugging window management. This provides detailed logging of the calls to and from the window management policy.

There is a new miral::CursorTheme class to load cursor themes.

Features available in newer Mir versions

Pointer confinement is now available (where underlying Mir version >= 0.24)
Enhanced “surface placement” handling and the results of “surface placement” requests are notified to clients (where supported by underlying Mir version >= 0.25).

Improvements to the Example Server

Improved resizing of size-constrained windows in miral-shell example.

Bugfixes

#1621917 tests fail when built against lp:mir

#1626659 Dialogs with parents should be modal i.e. they receive focus when the parent is clicked

#1628482 Deadlock in default window manager when Ctrl+Alt+Backspace with a client connected

#1590099 Need to support pointer confinement in Mir and toolkits using Mir

#1616818 User can drag menus (and other inappropriate) surfaces

#1628033 Starting qtcreator on miral-shell gives an orphan “titlebar”

#1628594 advise_focus_lost() isn’t called when the active window is minimised/hidden

#1628630 miral should not change surface geometry because it got maximized

#1628981 TitlebarWindowManager: sometime the initial display of titlebars doesn’t happen

#1625853 Mouse cursor disappears (or just never changes) when entering the windows of Qt apps

Debugging mir server/toolkit interactions

–window-management-trace

A feature added to the lp:miral trunk yesterday is making life a lot easier for developers working on MirAL based servers, and the toolkit extensions that support Mir. The feature is:

miral-shell --window-management-trace

Actually, the –window-management-trace switch will work with any MirAL base server (so miral-kiosk and egmde support it too).

What this does is cause the server to log every interaction with the window management policy – all the notifications it receives and all the calls it makes to the tools as a result.

This means that it is easy to find out that, for example, a “modal” gtk based dialog window is being created without specifying a parent. (Which is why the behaviour isn’t quite as expected – Mir servers will treat it as non-modal.)

To use this feature before the next MirAL release you do need build it yourself, but this only takes a minute (depending on your kit).It really is the easiest way to see exactly what is going on and why.

MirAL hits the Yakkety archives

Just a quick note: thanks to the efforts of my colleague Larry Price, libmiral is now available from the yakkety archive.

This means that if you’re running Ubuntu 16.10 you can install MirAL using apt and don’t need to build it yourself. Here’s what’s available:

  • libmiral-dev Developer files for the Mir ABI-stable abstraction layer
  • libmiral1 Display server for Ubuntu – ABI preserving abstraction layer
  • miral-doc API documentation for MirAL
  • miral-examples Display server for Ubuntu – demonstration programs

For example, before building egmde you just need to do this:

$ sudo apt install libmiral-dev

You can also install and run the miral-shell program with:

$ sudo apt install miral-examples
$ miral-shell

If you want to run X11 applications under miral-shell on X11 then you can do that too:

$ sudo apt install xmir
$ miral-shell&
$ Xmir -rootless :1&
$ DISPLAY=:1 gedit

An Example Mir Desktop Environment

The world of Mir

Mir is a set of libraries supporting the development of display servers, desktop environments, shells and implementing toolkit support for applications running on them.

As part of their “convergence” of computing devices Canonical are committed to providing Mir on a range of platforms including desktops, phones, tablets and “the internet of thing”. You can read about the work to provide it as a “snap” in the “internet of things” on Kevin Gunn’s blog [kg’s blog].

Toolkits and other “client” platforms that have support for Mir include GTK, Qt and SDL – which mean that applications using these can work on Mir. There’s a separate Xmir project to support X11 applications on Mir.

On the driver side of things Mir works with Mesa on desktop-like systems and, using libhybris, on an android based stack on a number of devices. The Mesa support is a work in progress and Ubuntu carries a set of patches to enable it. There are some notes about how to incorporate these on other platforms here: [Enabling Mir EGL]. Work is in progress to upstream Mir support.

The “Mir Abstraction Layer” (MirAL) packages the server-side functionality in a way that makes it easy to use. My previous post[MirAL] introduced MirAL in more detail.

Before we begin

If you want to experiment with this code I suggest using Ubuntu 16.04 or later on desktop for the reasons mentioned above. On an Ubuntu phone you can use the current stable version with this code, but doing development on a phone is a whole other article. On other distributions you likely need to patch Mesa and rebuild toolkits with Mir support enabled.

Having got an Ubuntu desktop you also need to install MirAL, at present this is not available from the Ubuntu archives, so you have to build and install it yourself:

$ sudo apt-get install devscripts equivs bzr
$ bzr branch lp:miral
$ sudo mk-build-deps -i --build-dep miral/debian/control
$ mkdir miral/build
$ cd miral/build
$ cmake ..
$ make
$ sudo make install

egmde [Example Mir Desktop Environment]

To illustrate MirAL I’m going to show what is involved in writing a (very simple) window manager. It runs on desktops, tablets and phones and supports keyboard, mouse and touch input. It will support applications using the GTK and Qt toolkits, SDL applications and (using Xmir) X11 applications.

The full code for this example is available on github:

$ git clone https://github.com/AlanGriffiths/egmde.git

Naturally, the code is likely to evolve, especially if I write a follow-up article but the version presented here is tagged v1.0. Assuming that you’ve MirAL installed as described above you can build it as follows:

$ mkdir egmde/build
$ cd egmde/build
$ cmake ..
$ make

The example code

A lot of the functionality (default placement of windows, menus etc.) comes with the MirAL library. For this exercise we’ll implement one class and write a main function that injects it into MirAL. The main program looks like this:

int main(int argc, char const* argv[])
{
    miral::MirRunner runner{argc, argv};

    return runner.run_with(
        {
            set_window_managment_policy<ExampleWindowManagerPolicy>()
        });
}

Yes, you’ve guessed it: the class we’ll be implementing is ExampleWindowManagerPolicy. It looks like this:

class ExampleWindowManagerPolicy : public CanonicalWindowManagerPolicy
{
public:
    using CanonicalWindowManagerPolicy::CanonicalWindowManagerPolicy;

    // Switch apps  : Alt+Tab
    // Switch window: Alt+`
    // Close window : Alt-F4
    bool handle_keyboard_event(MirKeyboardEvent const* event) override;

    // Switch apps  : click on the corresponding window
    // Switch window: click on the corresponding window
    // Move window  : Alt-leftmousebutton drag
    // Resize window: Alt-middle_button drag
    bool handle_pointer_event(MirPointerEvent const* event) override;

    // Switch apps  : tap on the corresponding window
    // Switch window: tap on the corresponding window
    // Move window  : three finger drag
    // Resize window: three finger pinch
    bool handle_touch_event(MirTouchEvent const* event) override;

private:
    void pointer_resize(Window const& window, Point cursor, Point old_cursor);
    void resize(WindowInfo& window_info, Point new_pos, Size new_size);

    // State held for move/resize by pointer
    Point old_cursor{};
    bool resizing = false;
    bool is_left_resize = false;
    bool is_top_resize = false;

    // State held for move/resize by touch
    int old_touch_pinch_top = 0;
    int old_touch_pinch_left = 0;
    int old_touch_pinch_width = 0;
    int old_touch_pinch_height = 0;
};

For this simple example the only functionality we’ll be providing is the handling of input events to allow the user to switch between applications and windows and to resize and move windows.

We only need to provide three handle_XXX_event() functions as we’re relying on the default behaviour for everything else. The first of these functions is the simplest:

bool ExampleWindowManagerPolicy::handle_keyboard_event(MirKeyboardEvent const* event)
{
    auto const action = mir_keyboard_event_action(event);
    auto const shift_state = mir_keyboard_event_modifiers(event) & shift_states;

    if (action == mir_keyboard_action_down &&
        shift_state == mir_input_event_modifier_alt)
    {
        switch (mir_keyboard_event_scan_code(event))
        {
        case KEY_F4:
            tools.ask_client_to_close(tools.active_window());
            return true;

        case KEY_TAB:
            tools.focus_next_application();
            return true;

        case KEY_GRAVE:
            tools.focus_next_within_application();
            return true;

        }
    }

    return false;
}

I don’t think this needs a lot of explanation: we select key presses while only the Alt button is pressed and act according to the button pressed.

The other two functions are a bit longer as they need to remember some state in order to interpret mouse or touchpad gestures. There’s in them nothing I consider worth pointing out here and the full code is available on github. There’s not much:

$ wc *
15 28 446 CMakeLists.txt
357 957 10528 egmde.cpp
372 985 10974 total

I have much to compare it with, bit I don’t think 372 lines is bad for a functional window manager.

Running egmde

Once egdme has been built you can run it in two modes: the simplest is the “Mir-on-X” mode:

$ ./egmde

This will produce a rather boring black “Mir-on-X” window. You can attach Mir clients to this:

$ miral-run gnome-system-monitor
$ miral-run gnome-terminal

These will run in the window and make it less boring. You can also have egmde launch them when it starts as follows:

$ ./egmde --startup gnome-terminal:gnome-system-monitor

If you don’t want to run in a window under X11 you can also give egmde access to your hardware directly by running it as root and specifying a “virtual terminal”. But I think it is probably better to use mir_demo_server in “system-compositor” mode to handle the hardware and launch egdme a normal user.

$ sudo apt install mir-demos
$ sudo mir_demo_server --vt 3 --window-manager system-compositor

You’ll find this switches you to vt3, switch back with Ctrl-Alt-F7 and, from another terminal, you can connect egdme to this. (We need the VT switch here as the host mir_demo_server is paused while X11 is in control.)

$ sudo chvt 3&&./egmde --host /tmp/mir_socket --startup gnome-terminal

You don’t have to start the gnome-terminal, but it is convenient as you can launch other applications. For example:

$ mir_demo_client_eglplasma

Conclusion

I hope that the egmde example is enough to spark someone’s interest in MirAL and to provide a starting point for developing something more ambitious.

Much of what has been discussed is a work-in-progress, especially MirAL which I hope to get added to the Ubuntu archives soon. I am optimistic this will happen as it is in the interests of both Canonical and the wider community to have a stable server ABI against which to write desktop environments (including Unity8). Indeed I’ve been working with the main developer of the QtMir abstraction layer to migrate it to MirAL.

Similarly, a significant portion of the work needed on Mesa to support Mir is similar to the work to support Wayland/Weston and everyone would benefit from cleaning this up (and, for example, removing hard dependencies on X11).

MirAL: Mir is not all about Unity8

Introducing The Mir Abstraction Layer

The principle Open Source project I’ve been working on for the last few years is Mir. Mir is a library for writing Linux display servers and shells that are independent of the underlying graphics stack. It fits into a similar role as an X server or Weston (a Wayland server) but was initially motivated by Canonical’s vision of “convergent” computing.

The Mir project has had some success in meeting Canonical’s immediate needs – it is running in the Ubuntu Touch phones and tablets, and as an experimental option for running the Unity8 shell on the desktop. But because of the concentration of effort on delivering the features needed for this internal use it hasn’t really addressed the needs of potential users outside of Canonical.

Mir provides two APIs for users: the “client” API is for applications that run on Mir and that is largely used by toolkits. There is support for Mir in the GTK and Qt toolkits, and in SDL. This works pretty well and the Mir client API has remained backwards compatible for a couple of years and can do so for the foreseeable future.

The problem is that the server-side ABI compatibility is broken by almost every release of Mir. This isn’t a big problem for Canonical, as the API is fairly stable and both Mir and Unity8 are in rapid development: rebuilding Unity8 every time Mir is released is a small overhead. But for independent developers the lack of a stable ABI is problematic as they cannot easily synchronize their releases to updates of Mir.

My answer to this is to provide a stable “abstraction layer” written over the top of the current Mir server API that will provide a stable ABI. There are a number of other goals that can be addressed at the same time:

  • The API can be considerably narrowed as a lot of things can be customized that are of no interest to shell development;
  • A more declarative design style can be followed than the implementation focused approach that the Mir server API follows; and,
  • Common facilities can be provided that don’t belong in the Mir libraries.

At the time of writing the Mir Abstraction Layer (miral) is a proof-of-concept, but work is in progress to package an initial version that can support the functionality needed by some examples, and by qtmir (the Qt support used by Unity8).

Building and using MirAL

These instructions assume that you’re using Ubuntu 16.04LTS or later, I’ve not tested earlier Ubuntu versions or other distributions.

You’ll need a few development and utility packages installed, along with the Mir development packages:

$ sudo apt-get install cmake g++ make bzr python-pil uuid-dev libglib2.0-dev
$ sudo apt-get install libmirserver-dev libmirclient-dev mirtest-dev
$ sudo apt-get install mir-graphics-drivers-desktop libgles2-mesa-dev

(If you’re working on a phone or tablet use mir-graphics-drivers-android in place of mir-graphics-drivers-desktop.)

With these installed you can checkout and build miral:

$ bzr branch lp:miral
$ mkdir miral/build
$ cd  miral/build
$ cmake ..
$ make

This creates libmiral.so in the lib directory and an example shell (miral-shell) in the bin directory. This can be run directly:

 $ bin/miral-shell

With the default options this runs in a window on X (which is convenient for development).

The miral-shell example is simple, don’t expect to see a sophisticated launcher by default. You can start mir apps from the command-line. For example:

 $ bin/miral-run gnome-terminal

That’s right, a lot of standard GTK+ applications will “just work” (the GDK toolkit has a Mir backend). Any that assume the existence of an X11 and bypass the toolkit by making X11 protocol calls will have problems though.

To exit from miral-shell press Ctrl-Alt-BkSp.

To run independently of X11 you need to grant access to the graphics hardware (by running as root) and specify a VT to run in. For example:

$ sudo bin/miral-shell --vt 4 --arw-file --file $XDG_RUNTIME_DIR/mir_socket

For convenient testing there’s a “testrun” script that wraps this command to start the server (as root) and then launches the gnome-terminal (as the current user):

$ ../scripts/testrun

Running applications on MirAL

If you have a terminal session running in the MirAL desktop (as described above) you can start programs from it. GTK, Qt and SDL applications will “just work” provided that they don’t bypass the toolkit and attempt to make X11 protocol calls that are not available.

$ gedit
$ 7kaa

From outside the MirAL session the “miral-run” script sets a few environment variables to configure the Mir support in the various toolkits. (There’s some special treatment for gnome-terminal as starting that can conflict with the desktop default.)


$ bin/miral-run gnome-calculator
$ bin/miral-run 7kaa

There are also some examples of native Mir client applications in the mir-demos package. These are typically basic graphics demos:

$ sudo apt-get install mir-demos
$ mir_demo_client_egltriangle

Support for X11 applications

If you want to run X11 applications that do not have native Mir support in the toolkit they use then the answer is Xmir: an X11 server that runs on Mir. First you need Xmir installed:

$ sudo apt install xmir

Then you can use the testrun script to start miral-shell with Xmir:

$ ../scripts/testrun -Xmir

This starts an X11 server on DISPLAY=:1. This is set in the terminal the script starts so that applications launched from it will automatically connect to miral through Xmir.

Running Qt applications

To run Qt applications under Mir you may need to install qtubuntu-desktop:

$ sudo apt-get install qtubuntu-desktop