Some time ago, Canonical started internal discussions about our convergence strategy, clearly spelling out the distant target of shaping and developing a single computing platform and operating system that is able to power the cloud, classic desktop machines, laptops, TV sets, phones and tablets (fridges anyone?). More to this, we stated that we want a single mobile device, a bundle of portable computing power together with the respective operating system, that seamlessly adapts to different form-factors and use-cases. Or to put it a little more catchy:

Your phone is your TV, is your desktop, is your … you name it.

And yes, we were aiming for the moon. We worked hard to draft a system that would allow us to reach the moon, while at the same time catering towards our other central goal of providing a beautiful and lean user experience. It became apparent in conversations and discussions that one of the cornerstones of the future system we were designing will be the graphics stack, including the Unity shell. After much discussions about existing solutions (X & Weston), and how we could leverage them, we took a step back and distilled down our (technical) requirements for a future graphics stack:

• Tailored towards an EGL/GL(ES) world.
• Minimal assumptions regarding the underlying driver model.
• Ability to leverage existing drivers implementing the Android driver model.
• Ability to leverage existing hardware compositors.
• Efficient, in terms of memory-, CPU- and GPU-consumption/usage.
• Tightly integrated with the Unity shell, fulfilling the shell’s requirements while at the same time not dictating any sort of semantics up the stack.
• An efficient and secure input subsystem supporting demanding mobile use-cases.
• Fully tested from the ground up.
• Adaptable to future requirements.

With these priorities in mind, we revisited and carefully evaluated existing solutions again and found that neither of them satisfies our requirements. In particular, X and its driver model is way too complicated and feature-laden, resulting in a less efficient system and a driver model that is unlikely to be widely supported on mobile platforms. In the case of Weston, the lack of a clearly defined driver model as well as the lack of a rigorous development process in terms of testing driven by a set of well-defined requirements gave us doubts whether it would help us to reach the “moon”. We looked further and found Google’s SurfaceFlinger, a standalone compositor that fulfilled some but not all of our requirements. It benefits from its consistent driver model that is widely adopted and supported within the industry and it fulfills a clear set of requirements. It’s rock-solid and stable, but we did not think that it would empower us to fulfill our mission of a tightly integrated user experience that scales across form-factors. However, SurfaceFlinger was chosen as our initial solution for getting started with the overall Ubuntu Touch project, planning to replace SurfaceFlinger with Mir as soon as possible.

In summary, our evaluation of existing solutions led us to the conclusion that neither of them fits our requirements and that adjusting/adapting them would require substantial efforts, too. For this, we decided to go for our own solution, catering directly towards our goals and our vision, effectively saying: Yes, we are going to do our own display server.

The project was born and we decided to name it Mir (as in the space station), as it would be our outpost that finally enables us to reach our goal of arriving at the moon, providing a seamless and beautiful user experience spanning multiple form-factors. The team set off to work on Mir while large parts of Canonical started working on the Ubuntu Touch project (relying on SurfaceFlinger). A lot of knowledge and experience was and is transferred from the Ubuntu Touch project to the Mir team, with the work on the phone and tablet revealing more and more technical requirements and subtleties that heavily influenced the Mir architecture and implementation. Thus, one of the earliest technical decisions that has been taken by the team concerned the protocol that applications rely upon to communicate to Mir. We evaluated Wayland and compared it to the SurfaceFlinger approach, realizing that neither of them fits our needs. Wayland is a very promising and sensible attempt at standardizing the way that applications talk to a display server, however, it exposes privileged sections like the shell integration that we planned to handle differently, both for security reasons and as we wanted to decouple the way the shell works on top of the display server from the application-facing protocol. In the end, we decided to implement Mir’s core in a protocol-agnostic way, considering the communication protocol a very important part of the overall story, but not the driving one. We have chosen Google’s protocol buffers as our data and interface description language (DDL & IDL) and implemented a lean RPC layer that abstract away transport-specific details (relying on an ordinary socket by default). It is well tested and the protobuf IDL and DDL provide us with forward- as well as backward-versioning capabilities. However, the communication component of the overall system is adaptable and can be adjusted to account for future requirements and developments.

A final word on timelines: At the time of this writing, we are preparing to deploy the next version of the Unity shell tightly integrated with Mir in the not-too-distant future on the phone and the tablet. However, regarding the desktop use-case: We have integrated X, leveraging the prior XWayland work, to run on top of Mir (XMir) and started initial porting efforts for toolkits with a focus on Qt5. With our X-integration in place, you can run Mir on your desktop machine if your system runs a GPU that supports the free driver stack. For the closed-source desktop drivers: We are in active conversations with GPU vendors to enable Mir on those drivers/GPUs, too. [Updated] More to this, we are working together with NVIDIA towards a more unified driver model sitting on top of EGL.

As we are moving along with the development of Mir, we will work on integrating existing toolkits with the new display stack to allow application developers a seamless transition between different form-factors and prevent them from the burden of supporting multiple different display servers. As noted before, Qt5 is our first target here, with GTK3 and XUL being in the pipeline. To support legacy X applications that cannot be adjusted to work against Mir, we will provide a translation-layer that leverages the existing XMir implementation and allows legacy X apps to use Mir transparently.

And that’s pretty much it. Thanks for reading. I hope I could give you some insight into the history of Mir, its motivation and its trajectory. If you want to dive even further into the Mir world, head over to wiki.ubuntu.com/MirSpec. A lot of work has already been done, but more of it lies ahead of us. So if you are curious, have questions, want to help us or simply want to get to know the team, join the Mir sessions at the upcoming virtual UDS. You might also want to read Olli’s description of the bigger picture, including our overall convergence strategy for Unity.

To the moon,

Thomas

These ancient entries were taken from my old Advogato diary, written in my early twenties, a year after I joined the development team of Conectiva Linux. I’m copying them for historic purposes, with the content untouched. It’s curious to look back and have such details of what was going on at the time, things that feel good, and things that feel awkward such as the “Dear Diary, …” style of writing, and the amount of exclamations!!

6 Feb 2002

Wow!! It has been a long time since my last diary entry.

I’ve left Linuxconf development team coordination in favor of the Conectiva Linux port to the S390 platform coordination (ok, I’m mostly coordinating myself now . Most of the work is done. I have developed an acceptable installer (in Python!) and most of the packages are ported. We had some problems with IBM OCO modules (ick!), but we are already workarounding them (we gave up on some of our kernel patches, and I patched insmod to recognize OCO modules). Anyway, more information about this later (if I don’t disappear for another year.. …

In the process of porting Conectiva Linux to S390 and PPC (Harald Welve started the PPC port, and I’m keeping it up to date and working on missing stuff) we are learning some lessons. We are trying to use those lessons to build a defacto package building system using the Python language. Unfortunately, we don’t have enough people here to develop it quickly, so we are trying a more realistic and evolutive approach this time. The first part of it is almost done. While devloping it, I’ve studied a little bit about process groups and extended python with a missing killpg() system call. I’ve also discovered that when python spawns a new thread, it blocks all signals. With this information in mind, I have also extended it with a new execv() syscall, that besides doing the usual work, unblocks every signal before the real call to execv(). I hope this project becomes real someday.

I’ve also been playing with Python optimization lately. There’s a big opportunity for somebody wanting to study and implement some concepts there. I’ve read some documentation about Stack Machine Optimization and made some tries (basically, optimizations around the inner loop and the Big Switch, stack caching, and other flavors of this joy). Today I found a paper from Skip Montanaro documenting some of the tries I’ve made (reading it first would save me a lot of time, but this knowledge will be useful anyway). You should have a look at his paper if you have any interest in the topic. Oh, don’t forget to get yourself a copy of Lemburg’s pybench to have a general idea of what you’re doing (don’t trust too much on it, it’s just a benchmark). I’ve written Skip a mail to discuss a little bit about what could be integrated into the interpreter. Let’s see where we get.

Oh… I must not forget to update the people I’ve certified in the past to reflect what they’ve been doing.

27 Jan 2001

I’ve just tested the patch floppy_cs on kernel 2.2.17 and it works just fine!!! I had no problems applying it. Now my Libretto 50CT has a floppy drive. Maybe Conectiva Linux can ship with this patch. The only drawback is that floppy support must be modular.

In the few last days, I’ve implemented support for Inputgrid into gnome-linuxconf. It allows one to define sensitive areas into a drawing area. Gurus are already working with it.

I’ve also created two new commands for Linuxconf’s gui protocol: Splash and Hidesplash. Linuxconf will send them while it is starting, and the graphic frontend is suposed to show a nice splash screen. Support in gnome-linuxconf has also been implemented with an image designed by Everaldo (thanks!!!).

Btw, yesterday I’ve fixed a bug in Python’s bsddb module. It was handling DB_RECNO databases with string keys. As the documentation says, these databases must use, in key’s “data” field, a pointer to a memory location holding a recno_t type.

My parents have arrived yesterday!! They’ll stay here until monday and then will go to Minas Gerais, visiting Raul and Lulude.

22 Jan 2001

Yesterday I’ve implemented a message signing module for Mailman. Darian (aka dmalloc), from openprojects.net has asked if I could do such module to use on lists.openprojects.net. (I said yes… . I’m going to ask the people here at Conectiva if they want to use this module in some of our lists.

Pybot has a few new features: CTCP handling/sending, timer module, unhandled messages hook, and something else I probably forgot.

About Linuxconf, I have spent the last days fixing a few simple bugs introduced in 1.24r2 and in the last modules developed by Conectiva. I hope to release a Linuxconf update to Conectiva 6.0 tomorrow.

gnome-linuxconf has won a home with screenshots and everything else at http://distro.conectiva.com/projetos/45. I’ve also published it at freshmeat and put files to download at SourceForge.

Oh, good news I forgot to tell: for those of you that are using wxxt-linuxconf, Jack (Jacques Gelinas) has implemented the Treemenu icons into this frontend as well.

16 Jan 2001

Today I’ve fixed a bug in the Pythonmod module of Linuxconf. Linuxconf has a default handler for the SIGCHLD signal that controls all of its child termination. This method has a few disadvantages. Before calling any external processes without using default Linuxconf methods, you must block this handler, otherwise Linuxconf will get on your way. Because of this, If a Pythonmod module tried to fork external processes, they were failing. Now Pythonmod is setting the SIGCHLD signal to SIG_DFL (POSIX doesn’t allow us to SIG_IGN it) before calling python code, and after returning from a few Linuxconf API functions that set the handler back. When the python code returns, popen_initsignal() is called, putting the Linuxconf handler back in place.

On the gnome-linuxconf side, I’ve implemented the drawing context command Defpen. Now we have colored lines and primitives!! (ok… not that good…

I’ve also spent a few hours in the last two days backing up and restoring data in my colocated machine. Now my personal emails are back online and the server has an updated kernel. I hope it doesn’t bother me for a long time.

Unfortunately, the server stuff didn’t let me work on Pybot, but I had time to implement dynamically loading, unloading and reloading of modules, before I started on the server. This will help a lot in the development, since I don’t have to reboot the bot everytime something is wrong. Anyway, now that the server is ok (I hope so), I’m planning to spend some of my spare time on the bot (yes, I still have some… .

11 Jan 2001

Today I’ve added the ability of using icons while in the Treemenu mode of Linuxconf. I have just changed some functions to pass the icon name around until it got into the treemenu module and then sent it to the GUI front-end. A little hack on gnome-linuxconf did the work at the front-end side. Following this line of improvements, I’m planning to add a splash screen or something like that soon. Icons would also be welcome in the web interface.

Besides that, I’m also playing with a Python IRC bot. It’s not meant to be a war or a channel control bot. I’m planning to implement useful modules to help making IRC even more useful as an information media (no it won’t be just another infobot clone). The core and a few modules are ready. I’ll post more information later… for now, I’ll just tell that it is a multi-channel, multi-server bot, and that I’m trying to make its commands with natural language (eg. forward messages from #blah on servername to #bloh on servername).

Happy birthday Diogo!!

16 Aug 2000

Created advogato account.

Meeting Minutes

IRC Log of the meeting.

Meeting minutes.

Agenda

20130226 Meeting Agenda

ARM Status

R/master: upstream landed a patch for the ipv6 stack alignment problem – waiting
for it to enter linux-next or stable before picking it up.
Trying to make the binary pvr omap4 module to work with the
upstream kernel (but so far no success).

Release Metrics and Incoming Bugs

Release metrics and incoming bug data can be reviewed at the following link:

• http://people.canonical.com/~kernel/reports/kt-meeting.txt

Milestone Targeted Work Items

The above summarizes the remaining work items owned by individuals on
our team for the rest of the cycle.

Status: Raring Development Kernel

We have recently uploaded the 3.8.0-7.16 Raring kernel. This upload
contains a fix for CVE-2013-1763 and config updates following a full
v3.8 config review.
Important upcoming dates:

• Raring:
  • Thurs Mar 07 – 13.04 Feature Freeze (~1 week)
  • Thurs Mar 21 – 13.04 Beta Freeze (~3 weeks)
  • Thurs Mar 28 – 13.04 Beta Release (~4 weeks)

Status: CVE’s

== 2013-02-26 (weekly) ==
Currently we have 36 CVEs on our radar, with 6 CVEs added and 5 CVEs retired this week.
See the CVE matrix for the current list:

• http://people.canonical.com/~kernel/cve/pkg/ALL-linux.html

  Overall the backlog has decreased slightly this week:

• http://people.canonical.com/~kernel/status/cve-metrics.txt

• http://people.canonical.com/~kernel/cve/pkg/CVE-linux.txt

Status: Stable, Security, and Bugfix Kernel Updates – Quantal/Precise/Oneiric/Lucid/Hardy

Status for the main kernels, until tiday (Feb. 26):

• Hardy – Nothing this cycle
• Lucid – In Prep; (2 commits)
• Oneiric – In Prep; 4 upstream releases; (68 commits)
• Precise – In Prep; 2 upstream releases; (197 commits)
• Quantal – In Prep; 2 upstream releases; (175 commits)

  Due to an high priority CVE, all the stable kernels are being respined this week.
  Current opened tracking bugs details:

• http://people.canonical.com/~kernel/reports/kernel-sru-workflow.html

  For SRUs, SRU report is a good source of information:

• http://people.canonical.com/~kernel/reports/sru-report.html

  Future stable cadence cycles:

• https://wiki.ubuntu.com/RaringRingtail/ReleaseInterlock

  Note: The week of March 28 is the week the last Hardy and Oneiric kernels may be built.

Open Discussion or Questions? Raise your hand to be recognized

No open discussions.

Ubuntu Phone OS integrates with Orange and Deutsche Telekom in GSMA OneAPI initiative

Mobile World Congress kicks off today and we’re gearing up to show off Ubuntu running on multiple devices. We’ll be demonstrating phones, tablets and desktops at the stand, have Ubuntu developers flashing spare hardware, as well as be showing integration and interoperability with Orange and Deutsche Telekom through the GSMA’s One API initiative.

GSMA’s OneAPI initiative aims to provide application programming interfaces (APIs) that enable applications to exploit mobile network capabilities, such as messaging, authentication, payments and location-finding with a cross-operator reach. For example, a payment network API could be used to add an in-app purchase directly to the user’s mobile phone bill.

Ubuntu is the first smartphone operating system to be able to demonstrate integration and interoperability with a carrier’s authentication and billing systems. Working with Deutsche Telekom and Orange, we’ll show how a single API can be used to instantly log users in with their operator identity and seamlessly link that with Ubuntu One, Ubuntu’s identity and payments services, and provide carrier billing options upon purchase of music and eventually, apps.

This is a massive step forward for the industry as the GSMA and partners such as Canonical, are spearheading an initiative to standardise access to operator facilities via network APIs across all operators. The initiative will benefit operators, developers and consumers:

• It puts operators in a position to forge stronger relationships with their customers.
• For developers, OneAPI reduces the time and effort needed to create applications for and content that is portable across mobile operators, increasing reach and ultimately enhancing the consumer experience.
• For consumers, it makes it really quick and easy to make application purchases directly from their phone. It’s also more secure because it’s not necessary to input credit card details for each purchase.

Also at Mobile World Congress:

• Mark Shuttleworth, founder of Ubuntu, will participate in a keynote panel discussion alongside Mozilla and Tizen on Tuesday 26th Feb at 18.00 at the MWC Conference Auditorium and broadcast live on Mobile World Live
• We’ll be taking part in the App Developer Day on Tuesday 26th Feb. Stuart Langridge, technical architect at Canonical will be presenting the Ubuntu phone, SDK, HTML5 and native apps as well as discussing app development for Ubuntu on phones and tablets. We’ll also have engineers available at the event to flash spare handsets with Touch Developer Preview of Ubuntu. This will take place from 9.00-9.30 and 11.40-11.55, and 13.30-14.00 in Hall 8.0, Theatre A.
• The GSMA Seminar on “Unlocking Value with Network APIs” will run on Thursday 28th from 9am to 10.30 am in Room CC1.1. Canonical’s Stuart Langridge will present and demo the Ubuntu Phone during the session. We’ll also be demonstrating Ubuntu’s OneAPI solution at the GSMA stand daily.
• Look out for Ubuntu engineers who will flash spare hardware with developer images for phone and tablet throughout the show close to the Ubuntu stand.

Meeting Minutes

IRC Log of the meeting.

Meeting minutes.

Agenda

20130219 Meeting Agenda

ARM Status

nothing new to report this week

Release Metrics and Incoming Bugs

Release metrics and incoming bug data can be reviewed at the following link:

• http://people.canonical.com/~kernel/reports/kt-meeting.txt

Milestone Targeted Work Items

The above summarizes the remaining work items owned by individuals on
our team for the rest of the 13.04 cycle.

Status: Raring Development Kernel

We have rebased the Raring kernel to the final upstream v3.8 release and
uploaded. We also made a few misc config changes and added some i915
fixes.
Important upcoming dates:

• Raring:
  • Thurs Mar 07 – 13.04 Feature Freeze
  • Thurs Mar 21 – 13.04 Beta Freeze
  • Thurs Mar 28 – 13.04 Beta Release

Status: CVE’s

Currently we have 30 CVEs on our radar, with 2 CVEs added and 3 CVEs retired this week.
See the CVE matrix for the current list:

• http://people.canonical.com/~kernel/cve/pkg/ALL-linux.html

• http://people.canonical.com/~kernel/status/cve-metrics.txt

• http://people.canonical.com/~kernel/cve/pkg/CVE-linux.txt

Status: Stable, Security, and Bugfix Kernel Updates – Quantal/Precise/Oneiric/Lucid/Hardy

Due to an high priority CVE, all the stable kernels are being respined this week.
Current opened tracking bugs details:

• http://people.canonical.com/~kernel/reports/kernel-sru-workflow.html

  For SRUs, SRU report is a good source of information:

• http://people.canonical.com/~kernel/reports/sru-report.html

  Future stable cadence cycles:

• https://wiki.ubuntu.com/RaringRingtail/ReleaseInterlock

  Note: The week of March 28 is the week the last Hardy and Oneiric kernels will be built.

Open Discussion or Questions? Raise your hand to be recognized

No open discussions.

Dell announced today an updated XPS 13, preloaded with Ubuntu, which has a full high-definition 1080p display. It will be available for sale in the USA  and Canada, but as part of this update Dell will also be making it available in parts of Europe, the Middle East and Africa.

As we reported in November, the Dell XPS 13 is a high-end ultramobile laptop, offering developers a complete client-to-cloud experience. It is the result of Dell’s bold Sputnik initiative which embraced the community and received terrific response from developers around the world.  With Ubuntu 12.04 LTS preloaded, the machine is perfect for developers and anyone who wants high speed, brilliant graphics and smart design.

If you’re keen to get your hands on a new Dell XPS 13 Developer Edition with Ubuntu pre-loaded, check-out our web page for more details and links:

  http://www.ubuntu.com/partners/dell/dellxps

We’ll post more links allowing you to buy in additional countries as soon as we can.

Inside Canonical, we have projects going on and testers testing them all over the world. This means transferring daily builds of projects that can be gigabytes in size, to people who might not have a great deal of bandwidth.

We had been using rsync to do this, using a previous daily build as a seed so that rsync only transferred data when it found a diff. However, that still meant transferring a lot of data, as rsync’s capability of calculating a delta between ISOs could be better. Still, it did have a speedup of 25-30%.

Zsync is a project that does it better. The deltas are significantly smaller for our use cases, and it goes over http which means we don’t have to set up shell access for users.

We tested both rsync and zsync against two of our projects that get a lot of activity, and have large ISOs to transfer to testers. Here’s the results of the delta size that each tool transferred.

Target ISO Size for Project X: 1976.3mb

Age of seed ISO: 7 days
rsync transfer : 1486.7mb
zsync transfer : 531mb

Age of seed ISO: 2 days
rsync transfer : 1479.9mb
zsync transfer : 375.9mb

Target ISO size for Project Y: 1104mb

Age of seed ISO: 7 days
rsync transfer : 758mb
zsync transfer : 66.1mb

So for our use case, zsync seemed the obvious choice. There were a couple of barriers to using it though.

Our image archives are HTTPS and protected by OpenID authentication. The zsync client does neither of these things, as it has its own internal HTTP client. The project itself has not seen any activity since 2010, so the chances of a new version using libcurl getting released are pretty much zero.

There were a couple of projects attempting to update zsync, at various stages of completion, but they were either incomplete for our needs (i.e. missing authentication methods) or at the early stages of development.

So, we spun our own solution.

Zsync-curl is a fork of the zsync client that uses libcurl. To solve our OpenID problem, it allows you to set arbritary cookies, so we have a script that authenticates against our OpenID provider, and sets the authenticated session cookie when calling our libcurl backed zsync binary.

The zsync-curl packages install a new zsync_curl binary which sits alongside zsync and zsyncmake from the official zsync distribution.

All of our benchmarks show this will save a significant amount of data transfer, so we’re looking forward to getting it out to our testers around the world and seeing how much it saves in real world usage.

Meeting Minutes

IRC Log of the meeting.

Meeting minutes.

Agenda

20130212 Meeting Agenda

ARM Status

R/master: still working on last week’s “BUG: scheduling while atomic” bug in
the ipv6 stack – nothing new to report.

Release Metrics and Incoming Bugs

Release metrics and incoming bug data can be reviewed at the following link:

• http://people.canonical.com/~kernel/reports/kt-meeting.txt

Milestone Targeted Work Items

Status: Raring Development Kernel

We have rebased the Raring kernel to the latest v3.8-rc7 upstream
kernel and uploaded. Perf has also been re-enabled with the latest
upload.
Important upcoming dates:

• Raring:
  • Mon Feb 18 – 13.04 Month 4 Milestone (~1 week)
• Precise:
  • Thu Feb 14 – 12.04.2 Release (2 days)

Status: CVE’s

Currently we have 29 CVEs on our radar, with 1 CVE added and 3 CVEs retired this week.
See the CVE matrix for the current list:

• http://people.canonical.com/~kernel/cve/pkg/ALL-linux.html

  Overall the backlog has decreased slightly this week:

• http://people.canonical.com/~kernel/status/cve-metrics.txt

• http://people.canonical.com/~kernel/cve/pkg/CVE-linux.txt

Status: Stable, Security, and Bugfix Kernel Updates – Quantal/Precise/Oneiric/Lucid/Hardy

Here is the status for the main kernels, until today (February 12):

• Hardy – Nothing in this cycle
• Lucid – In Testing; 1 CVE; (2 commits)
• Oneiric – In Verification; 3 CVEs; 4 upstream stable release(s); (154 commits)
• Precise – In Verification; 0 CVEs; 2 upstream stable release(s); (222 commits)
• Quantal – In Verification; 0 CVEs; 2 upstream stable release(s); (375 commits)
  Current opened tracking bugs details:

• http://people.canonical.com/~kernel/reports/kernel-sru-workflow.html

  For SRUs, SRU report is a good source of information:

• http://people.canonical.com/~kernel/reports/sru-report.html

  Future stable cadence cycles:

• https://wiki.ubuntu.com/RaringRingtail/ReleaseInterlock

Open Discussion or Questions? Raise your hand to be recognized

No open discussions.

Meeting Minutes

IRC Log of the meeting.

Meeting minutes.

Agenda

20130205 Meeting Agenda

ARM Status

R/master: work continues on the arm multiplatform kernel – right now i’m
tracking down a “BUG: scheduling while atomic” (actually an alignment problem -
how conform
http://paste.ubuntu.com/1613275/) that only happens on omap4, seems to be ipv6
related and seems to be triggered by our articulated kernel configuration.

Release Metrics and Incoming Bugs

Release metrics and incoming bug data can be reviewed at the following link:

• http://people.canonical.com/~kernel/reports/kt-meeting.txt

Milestone Targeted Work Items

We were able to clean up quite a few work items last week and are now
well below the trend line for our overall burn down chart. Thanks to
everyone who closed out items.

Status: Raring Development Kernel

We have rebased the Raring kernel to the latest v3.8-rc6 upstream
kernel and uploaded. We’ve also pulled our first set of patches for arm
multiplatform support \o/.
Also just a small reminder, the 12.04.2 point release is nearing. If
anyone has free cycles and is able to test, please do so.
Important upcoming dates:

• Raring:
  • Mon Feb 18 – 13.04 Month 4 Milestone (~2 weeks)
• Precise:
  • Thu Feb 14 – 12.04.2 Release (~1 week)

Status: CVE’s

Currently we have 32 CVEs on our radar, with 3 CVEs added and 4 CVE retired this week.
See the CVE matrix for the current list:

• http://people.canonical.com/~kernel/cve/pkg/ALL-linux.html

  Overall the backlog has decreased slightly this week:

• http://people.canonical.com/~kernel/status/cve-metrics.txt

• http://people.canonical.com/~kernel/cve/pkg/CVE-linux.txt

Status: Stable, Security, and Bugfix Kernel Updates – Quantal/Precise/Oneiric/Lucid/Hardy

Here is the status for the main kernels, until today (February 05):

• Hardy – Nothing in this cycle
• Lucid – Nothing in this cycle
• Oneiric – In Preparation; 3 CVEs; 4 upstream stable release(s); (154 commits)
• Precise – In Preparation; 0 CVEs; 2 upstream stable release(s); (222 commits)
• Quantal – In Preparation; 0 CVEs; 2 upstream stable release(s); (368 commits)
  Current opened tracking bugs details:

• http://people.canonical.com/~kernel/reports/kernel-sru-workflow.html

  For SRUs, SRU report is a good source of information:

• http://people.canonical.com/~kernel/reports/sru-report.html

  Future stable cadence cycles:

• https://wiki.ubuntu.com/RaringRingtail/ReleaseInterlock

Open Discussion or Questions? Raise your hand to be recognized

No open discussions.

Welcome to Canonical Voices. This is your first post. Edit or delete it, then start blogging!

Part of my duties as a Hardware Certification Engineer is to develop and maintain our test and bug reports. Previously these have been Python scripts containing lots of inline HTML (yuck :/), which I’m ashamed to admit it because I didn’t know any better when I initially wrote them. I’m currently in the middle of refactoring (hopefully not ref**ktoring) the scripts to take advantage of a new API that’s available for querying data from the Certification website and have decided to use Jinja (precisely Jinja2) as the template engine.

So I have a list of systems which may be in different locations and I want to sort them by location. Initially I did this in the Python code which gathered the data, using a simple .sort(key = lambda obj: obj.datacentre). Then the template code used the sorted list and all was well. Later on when reading the Jinja template documentation I discovered a way to sort a list in the template itself. I considered this to be a cleaner, more transparent way to modify the list (somebody looking at the template asking ‘why does it come out in this order’ has the answer right in front of them). So I gave it a try by changing

for hw in {%reported_hardware %}

to

{% for hw in reported_hardware|sort(attribute='datacentre') %}

and was unfortunately confronted with a traceback from the template engine:

Traceback (most recent call last):
...
AttributeError: Hardware instance has no attribute '__getitem__'

This looks like it’s saying that the ‘Hardware’ object (which I’ll introduce in a second) isn’t the type that the template engine was expecting.

Initially I defined the Hardware object like this:

class Hardware:

def __init__(self..., datacentre):
...
self.datacentre = datacentre

My first attempt to fix this involved making datacentre a property:

@property

return self._datacentre


But this didn’t seem to do the trick. Eventually I read in the Python documentation about the property built in working only on new-style classes and this set off the lightbulb

class Hardware(object):

This seemed to be the magic ingredient, and the sort in the template engine started working. I also realised that making datacentre a property wasn’t really necessary, so reverted that change.

Update: I found out that this whole problem is avoided by using Python3.x because all classes there are new-style classes. So the advice above only applies if you’re stuck with Python2.x

Last week I wrote about my first impressions on working with QML vs Android as I tried to translate a SimpleTodo application.

This week, I managed to find a few more spare hours to finish it. Not only that, but I have been able to go beyond what I had for Android. Specially on the UI part, QML makes it really simple to build in transitions and animations. I also found that defining States for the application had simplified the complexity of the program.

I don’t think that today there are official menus published for Ubuntu qml components, so I have implemented my own menu “a la” Ubuntu for phones. Here is a small video of the app:

or you can access the link here: http://www.youtube.com/watch?v=T-gDv-vDMhg

Overall, I highly recommend everyone to play with QML. Here are two very useful links if you are new to it:

• QML Elements
• Qt Quick intro

And of course, lets not forget the awesome developer.ubuntu.com pages!

The original Half-Life and Counter-Strike games were quietly released for Linux and Mac OS X last week, and as the maintainer of SteamLink, a repackaging of Half-Life: Uplink for Steam, I went out to see if the mod's files could be installed on these platforms.

Turns out there is a bug in Steam for these platforms, where it tries to launch the Windows version of Half-Life for GoldSrc mods from within Steam. However, Half-Life can be manually launched and pointed at the mod.

I have released a new version of SteamLink as a zip file. If you would like to run Half-Life: Uplink on Linux or OS X, simply download and extract the zip, and run the installer shell script. It will determine the Half-Life installation directory, install the mod, and give you a symlink to a script to launch it.

Django-group-access is an app for django that controls row-level access to data based on group membership. This was perfect for a few projects we were developing in Canonical. So perfect, in fact, that we wrote django-group-access, and open sourced it.

It does a few nifty things. And these nifty things got us out of a tight spot where we had to redefine access control rules at short notice.

1) Declare access control rules, instead of having to code them in your views.

This means that if you want to control access to instances of MyModel, you just have to declare that MyModel is access controlled. MyModel will automatically pick up all the attributes needed to share it with groups, and have a user take ownership of it. It’s as simple as:

from django_group_access import register
from myapp.models import MyModel

register(MyModel)

2) Optional middleware will automatically filter your querysets so your views do not have to be aware of access control.

In some access control apps (admittedly, ones that are more focussed on providing row-level permissions, rather than row-level access) you have to modify your code and models to be aware of the access control. For example, you might have to do something like this:

def my_view(request):
    records = MyModel.objects.accessible_by_user(request.user).all()

That could be a lot of work if you’re integrating it into an existing app, and prohibitive if you’re integrating into a third party app.

With the automatic filtering in django-group-access, it remains:

def my_view(request):
    records = MyModel.objects.all()

That would give you all the records that the currently logged in user has access to. Plus, at no extra charge, it includes the .unrestricted method to give you unfiltered access to a queryset, like this:

def my_view(request):
    # all the visible records for the current user
    records = MyModel.objects.all()
    # all the records that exist
    all_records = records.unrestricted()

3) It allows you to create a hierarchy of models that control access.

Say you had a Room model, and a House model. If you had access to a Room, you want access to the House too.

With django-group-access, that’s easy.

class House(models.Model):
    address = models.TextField()

class Room(models.Model):
    description = models.TextField()
    house = models.ForeignKey(House)

register(Room)
register(House, control_relation='room')

Now, because django-group-access allows us to declare access rules instead of coding them into the models, and because it does automatic filtering, changing how the visibility of records is worked out becomes as easy as changing how your models are registered, and migrating your schema. (And, of course, changing the test data that your extensive unit and integration tests, set up. That’ll teach you to use test data factories…)

Development on django-group-access is ongoing, with plans for django-access-tng which will include row-level permissions and finer grained access control that doesn’t need to share records with groups to grant access.

But pretend you didn’t read that last paragraph. Our current product is the best. The new one might never come. Buy now to avoid disappointment.

Meeting Minutes

IRC Log of the meeting.

Meeting minutes.

Agenda

20130129 Meeting Agenda

ARM Status

R/master: first batch of arm multiplatfom patches with support for omap3/4 and
imx6 was sent to the ml, awaiting comments.
In the mean time, i’m tracking down the SATA imx6 regression in 3.8.

Release Metrics and Incoming Bugs

Release metrics and incoming bug data can be reviewed at the following link:

• http://people.canonical.com/~kernel/reports/kt-meeting.txt

Milestone Targeted Work Items

I’ve been trying to clean house with our work items and drop us blow the
trend line. If anyone has items which they can close or postpone,
please do so. Thanks.

Status: Raring Development Kernel

We have rebased the Raring kernel to the latest v3.8-rc5 upstream
kernel. I’ll plan to upload before EOD tomorrow.
Important upcoming dates:

• Raring:
  • Mon Feb 18 – 13.04 Month 4 Milestone (~3 weeks)
• Precise:
  • Thu Feb 14 – 12.04.2 Release (~2 weeks)

Status: CVE’s

Currently we have 34 CVEs on our radar, with 0 CVEs added and 2 CVE retired this week.
See the CVE matrix for the current list:

• http://people.canonical.com/~kernel/cve/pkg/ALL-linux.html

  Overall the backlog has decreased slightly this week:

• http://people.canonical.com/~kernel/status/cve-metrics.txt

• http://people.canonical.com/~kernel/cve/pkg/CVE-linux.txt

Status: Stable, Security, and Bugfix Kernel Updates – Quantal/Precise/Oneiric/Lucid/Hardy

We’re skipping a kernel SRU cadence to allow for an additional 3 weeks of testing for the kernel that will ship with the 12.04.2 point release.
However, due to regressions, we currently have new Precise, Quantal and lts-Quantal kernels in Testing.
Next week the regular SRU cycles will restart.
Future stable cadence cycles:

• https://wiki.ubuntu.com/RaringRingtail/ReleaseInterlock

Open Discussion or Questions? Raise your hand to be recognized

Thanks everyone

A few months back, I decided to write a Simple ToDo app for Android, then I hooked it up to a cloud backend, using Juju. That was my first Android application, so I got to experience first hand the latest developer documentation and development environment.

Last month, Canonical launched Ubuntu for Phones, that gave me the idea to re-write the same application on QML using the Ubuntu Components.

Clearly comparing a new SDK-Alpha with a stable platform like Android will seem hardly fair, however, keep reading as you might be surprised of the results.

QML vs Dalvik Java

Lets start with QT/QML vs Dalvik/Java – I found QML really easy to get to grips with and be productive. I had the UI (see picture below) running in no time and I would say much faster than with Android.  QML is a very flexible declarative environment that allows you to embedded quick logic into the layout. This is a blessing and  a curse.

While with Android, it was very easy to keep a nice MVC  separation, I struggled to stop the leaks in QML. So while it is very easy to quickly write a functional application, it does not impose what you would consider as good development practices.

In summary, they are both very powerful development environments.

IDE: Eclipse vs QtCreator

Part of the development experience is the IDE. I must say that I simply love the QTCreator. Possibly not as polish as Eclipse but you don’t need to read a manual to use it. Also, with a quick integration with the HUD, it is just very simple to use.

So what is QTCreator missing? A good emulator. The Android Development Kit (ADK) provides a really good user experience to develop mobile solutions. QMLScene gives you similar functionality but does not simulate a phone environment. However, all the technology is there, and  I am sure that will be included in the v1.0 version of the SDK.

Documentation

I can’t fault Android developer documentation, but taking into account its popularity, you  wouldn’t expect anything else.

I was very surprise of the quality of information on http://developer.ubuntu.com/ and specially with the component showcase.

The only thing to watch out for is that in Android you can get all the info you need from a single website. With QML you quickly end up pinging between Digia, Nokia and Ubuntu pages.

The Code

The code is on my launchpad repos. The actual source functionality is not finished as I am still trying to figure out how to add menu options to access Done items. Anyway, the whole thing is pretty compact compare to the Dalvik code. The actual logic is almost identical in both. A ListView that is populate from an List model. All the data is persisted in SQLite db.

Conclusion

Both environments have been equally painless to work with, the difference is that the Ubuntu environment has *just* been released as an Alpha. I think this is the start of a very vibrant App development ecosystem.

Meeting Minutes

IRC Log of the meeting.

Meeting minutes.

Agenda

20130122 Meeting Agenda

ARM Status

R/master: still working on multiplatform, USB finally works for omap and imx, i’m getting close to a first release

Release Metrics and Incoming Bugs

Release metrics and incoming bug data can be reviewed at the following link:

• http://people.canonical.com/~kernel/reports/kt-meeting.txt

Milestone Targeted Work Items

Status: Raring Development Kernel

We have rebased the Raring kernel to the latest v3.8-rc4 upstream
kernel and uploaded last week. Please test and let us know your
results.
Important upcoming dates:

• Raring:
  • Mon Feb 18 – 13.04 Month 4 Milestone (~4 weeks)
• Precise:
  • Thu Feb 14 – 12.04.2 Release (~3 weeks)

Status: CVE’s

Currently we have 35 CVEs on our radar, with 1 CVE added and 1 CVE retired this week.
See the CVE matrix for the current list:

• http://people.canonical.com/~kernel/cve/pkg/ALL-linux.html

  The backlog hasn’t change this week:

• http://people.canonical.com/~kernel/status/cve-metrics.txt

• http://people.canonical.com/~kernel/cve/pkg/CVE-linux.txt

Status: Stable, Security, and Bugfix Kernel Updates – Quantal/Precise/Oneiric/Lucid/Hardy

As noted last week, we’re skipping a kernel SRU cadence to allow for an additional 3 weeks of testing for the kernel that will ship with the 12.04.2 point release.
Future stable cadence cycles:

• https://wiki.ubuntu.com/RaringRingtail/ReleaseInterlock

Open Discussion or Questions? Raise your hand to be recognized

trijntje requested a review of bug 1075876.

Today’s inauguration of Barack Obama to his second term provides a good opportunity to look back at last year’s campaign and appreciate it in a bit more detail. We’ll skip discussion of the adverts, polls, photo ops, sound bites, political theatre and even the much appreciated informed debate on the issues, and focus instead on the interesting stuff – the IT infrastructure that powers something as dynamic as a presidential campaign. You can imagine the demands placed on such an infrastructure – scalability, reliability, cost effectiveness, manageability, openness, cloud. Once you have those requirements in mind, the clear choice for meeting those demands is Ubuntu. And so it’s no surprise that the Obama campaign reached the same conclusion.  We recently spoke with Harper Reed, the CTO of the Obama campaign, about the challenges he faced and solutions he and his team put in place during the campaign. We’ve published that piece in honour of today’s inauguration; you can find it on our new Insights blog.

Takashi Iwai, the Linux sound maintainer, is about to merge a patch set of about 150 patches into linux-next. These changes, in short, unify the different HDA codec drivers.

Introduction and background

First, the basics: HDA Intel is the current standard protocol for accessing your built-in soundcard, as well as HDMI/DisplayPort audio, and is used in almost all desktop and laptop computers since about 2005. In the HDA Intel world, there are controllers and codecs. The codecs also have a configuration, which tells which pins of the codecs that are connected to what input or output (this is set by BIOS/UEFI).

Hardware is very diverse: The HDA Intel driver supports at least 50 different controllers, and about 300 different codecs. On top of that, every codec usually support many different configurations.
The codecs come from 10-12 different vendors, and within the same vendor, it is more likely that the codec layout is somewhat like other codecs from the same vendor. As a result, the HDA codec driver is split up in 10-12 codec driver files, one per vendor. These files are to some degree copies of each other, but also contains every vendor’s specials.

What changes?

Takashi’s patches are solving a long term maintenance problem: as we want to add new features to the kernel drivers, we would previously have to do this once per codec – whereas with the unification of codec drivers, we would just have to add this code once. Or possibly twice, as the HDMI codecs will still have its own codec driver. In addition, new codec hardware is more likely to “just work”, or at least partially work, without explicit support in the kernel. The potential downside, of course, is that as you improve the driver to solve some edge case, you’re more likely to screw some other edge case up.

There’s not much of new features added in this new, generic driver at this point. However, if you have an “unusual” codec chip vendor, you might see minor improvements as these are brought up to feature parity with the more common codecs.

When does it change?

As usual, you can’t know until it’s in there. Takashi’s latest plan is to make the move for 3.9 for at least some of the codecs, and 3.10 for the rest of them.

Update: Some blog, referencing this one, said this feature would come to Ubuntu 13.04. Ubuntu 13.04 uses kernel 3.8, so this feature won’t be in Ubuntu until 13.10.

Update 2: As of 2013-01-23, the new code has been merged, for all codecs, for linux-next (kernel 3.9)!

Regressions and testing

Judging from the database you might have contributed to by yourself by submitting your alsa-info, there are about 6000 different machines out there, and there is not enough manpower to test them all. So we need a different approach. Conveniently, Takashi has written an emulator called hda-emu to test the codec driver code, and I’ve improved that emulator with some scripting, so that hda-emu effectively becomes an automated test suite. The test suite is still very incomplete, but it at least runs a few different tests such as faked playback, S3, and manipulating of volume controls, and checks if hda-emu crashes or reports an error. And sure enough, when running this test suite over all the alsa-infos in the database, a few regressions were discovered that I was able to fix.

So far, so good. If it weren’t for the fact that hardware often does not work exactly as advertised. The parser algorithm for reading the codec layout and creating a working kernel driver out of it, must now take all codecs from all vendors into account. The old vendor-specific parser might have done things in one way and the new parser might do things a different way, causing the audio to be routed differently.

As an example, assume the codec is broken in such a way that it advertises two audio paths, but in practice only one of the paths actually works. The new parser might then route the audio differently from the old one – and as a result it will look like audio should really work, in theory. In practice, there is nothing but silence. Another example could be that maybe the new driver will power down different parts of the codec in different order than the old driver did, causing your speakers to click.

How can I help?

Right now, what’s needed is more testing on real hardware. Takashi has called for testing of his hda-migrate branch of the sound-unstable tree.

If you’re running Ubuntu, I have made packages for 12.04, 12.10 and 13.04 – just download, install it (you might need to install dkms and kernel headers too). Then reboot and test. Simply uninstall the package and reboot when you’ve finished testing.

Update: I will probably close the comments soon due to the amount of spam coming in that way. Please report back – especially if you did see a regression – to the alsa-devel mailing list. Thanks!

Update 2: As the code is now merged into linux-next, please use these instructions to try it out on Ubuntu. Thanks!

Meeting Minutes

IRC Log of the meeting.

Meeting minutes.

Agenda

20130115 Meeting Agenda

ARM Status

R/master: still working on multiplatform support: omap3/4 boots, i’m working on adding imx6 support now

Release Metrics and Incoming Bugs

Release metrics and incoming bug data can be reviewed at the following link:

• http://people.canonical.com/~kernel/reports/kt-meeting.txt

Milestone Targeted Work Items

Status: Raring Development Kernel

We have rebased the Raring kernel to the latest v3.8-rc3 upstream
kernel and uploaded last week. Please test and let us know your
results.
I also want to note that the 12.04.2 point release date has been moved
out by 2 weeks to Thurs Feb 14:

https://lists.ubuntu.com/archives/ubuntu-devel-announce/2013-January/001003.html

Important upcoming dates:

• Raring:
  • Fri Jan 18 – 13.04 Month 3 Milestone (3 days)
  • Mon Feb 18 – 13.04 Month 4 Milestone (~5 weeks)
• Precise:
  • Thu Feb 14 – 12.04.2 Release (~4 weeks)

Status: CVE’s

Currently we have 34 CVEs on our radar, with 0 CVEs added and 3 CVEs retired this week.
See the CVE matrix for the current list:

• http://people.canonical.com/~kernel/cve/pkg/ALL-linux.html

  Overall the backlog has decreased slightly this week:

• http://people.canonical.com/~kernel/status/cve-metrics.txt

• http://people.canonical.com/~kernel/cve/pkg/CVE-linux.txt

Status: Stable, Security, and Bugfix Kernel Updates – Quantal/Precise/Oneiric/Lucid/Hardy

Here is the status for the main kernels, until today (January 08):

• Hardy – Nothing in this cycle
• Lucid – In -updates; 1 CVE; (2 commits)
• Oneiric – In -updates; 2 CVEs; 4 upstream stable release(s); (78 commits)
• Precise – In Testing; 3 CVEs; 1 upstream stable release(s); (104 commits)
• Quantal – In Testing; 3 CVEs; 1 upstream stable release(s); (254 commits)
  Current opened tracking bugs details:

• http://people.canonical.com/~kernel/reports/kernel-sru-workflow.html

  For SRUs, SRU report is a good source of information:

• http://people.canonical.com/~kernel/reports/sru-report.html

  Future stable cadence cycles:

• https://wiki.ubuntu.com/RaringRingtail/ReleaseInterlock

  The kernel SRU cadence cycle that was to start this week has been canceled. The kernels in -proposed will be used for the 12.04.2 release.

Open Discussion or Questions? Raise your hand to be recognized

No open discussion.