Canonical Voices


12.04: Testing FTW

I arrived back home in Augsburg, from last week’s Ubuntu Developer Summit in Orlando, FL. As this is a quality/LTS cycle, we pretty much already knew in advance what to do (bug fixing, bug fixing, some boot speed, and did I mention bug fixing?), but still we had many highly interesting and exciting sessions this time, not so much about what we are going to do, but how we are going to build 12.04.

So far our common practice has been to toss everything new into the development release until Feature Freeze and then try and clean up most of the fallout. Me and many other developers have always cried for having more time for fixing long-standing bugs and not introducing breakage in the first place. It seems that now with 12.04, Ubuntu/Canonical are actually getting serious about it.

(Any resemblance to that postcard from the Kennedy Space Center which I went to last Sunday is of course absolutely unintended and purely coincidental :-) ).

The mission statement is now to have working ISOs, stable ? development, and daily intra-development upgrades every day, quick and regular cleanup of uninstallable packages, component-mismatches, NBS etc., backed by a new “stable +1″ team backed by three people on a rotational shift.

QA team is now setting up daily automatic smoketesting of the installer and other packages which have tests. For the latter we’ll convert some packages to the DEP-8, the proposed format for running autopkgtest on (I’ll do udisks, postgresql-common, pygobject, apport, and jockey soon).

We’ll try do put uploads which might break something (like new libraries) to a staging area first, against which we can run test suites of reverse dependencies before it lands in the new release. As doing this on a large scale still requires infrastructure to be created, we’ll only exercise it for a few packages by uploading to precise-proposed first, but this has a high potential for extension.

We want to commit to fixing major breakage within 3 hours of development time, or otherwise revert the faulty package to the previous version (unless that aggravates problems, such as file conflicts).

Finally, for Canonical upstreams we are introducing “acceptance criteria”, which will hopefully significantly raise the quality and lower the regressions of each Unity etc. release.

So, the mission is clear. In practice we’ll probably have to make some real-life concessions, and Murphy’s law dictates that there still will be some breakage, but we can learn from that as we go.

Let’s build 12.04 LTS!

Read more

Just took the plunge, using the excellent bandwidth and local mirror at UDS:

$ lsb_release -irc
Distributor ID: Ubuntu
Release: 12.04
Codename: precise

Nothing blew up in my face, so it seems today is a good day to die^Wupgrade.

Read more

On a rather calm ten-hour flight to Orlando I once again did some pygobject, udisks, and Apport hacking (It’s scary how productive one can be when not constantly being interrupted by IRC, email, etc). One more visible change amongst these was finally fixing a five year old five-digit bug to integrate apport-retrace into the GUI, now that it does not potentially wreck your installation any more.

If the apport-retrace package is installed, the crash detail dialog will show a new “Examine locally” button:

Apport crash detail dialog

After clicking this, you can choose what do do exactly:

Retrace action dialog

I know this dialog is not a beauty, as it’s implemented using the ui_question_choice() API which is used by package hooks. That makes it work for all available UIs (GTK, KDE, CLI), though, and can easily be extended to have more actions. And if you get this far and want to stack traces, you are used to looking at eye-bleeding gibberish anyway..

Presumably the most useful (and default) action is to download all the debug symbols, open a Terminal, and put you into a GDB session with all these, and the core dump loaded, so that you can poke around the crashed program state with all symbols available.

But you can also run gdb without downloading debug symbols, or just update the .crash report file with a fully symbolic stack trace.

This works just as well in apport-cli, but not yet in the KDE version: Someone needs to implement the equivalent of the apport-gtk implementation to apport-kde and kde/bugreport.ui, i. e. show an “Examine locally” button if self.can_examine_locally() is true, and add an appropriate ui_run_terminal() method (which should be fairly similar to the GTK one, just with Qt/KDEish terminal emulators). But as Kubuntu does not currently use Apport (and also because I didn’t have all the dependencies installed on my laptop) I did not yet do this. Please catch me on IRC/mail/merge proposal if you want to work on this. If you look at above commit, the changes to the GtkBuilder file look huge, but that’s only because I haven’t touched it for ages and the current Glade shuffled the elements quite a bit; it just adds the button to the dialog.

For now this is all sitting in trunk, I’ll do a new upstream release and Ubuntu precise upload soon.

Happy debugging!

Read more

7 years ago, The Ubuntu 4.10 “The Warty Warthog” was announced. A huge congrats to the community, Canonical, and especially Mark for getting so far from “there” to “here”.

This brings back old memories of my first conference in Oxford in August, the great-great-grandfather to what is UDS these days. Back then, there was no company, no Launchpad, no Blueprints, no work items, no detailled plans, just a bunch of ideas, BoFs, and this was a third of the entire crowd:

Warty Hack Room

Back then we worked on the famous TRLS technology (“Totally Rad Laptop Support”) and were proud when we got the ThinkPads to suspend once. During that conference I wrote pmount to provide automatic mounting of USB sticks in a safe manner. Those were the days… :-)

But I can also safely say that there are some things that haven’t changed. Even though both the community and the company (which changed away from recently) grew by two magnitudes since then, we still have the same serious attitude, stern look, and formal attire as we had back then:

We are professionals, really!

We are professionals, really!

Read more

Ich habe gerade Gestatten, Elite zu Ende gelesen (ging schnell, hab erst gestern angefangen). War im Grunde genommen nichts wirklich Neues, was man nicht irgendwie schon gewusst oder geahnt hätte. Aber die gut recherchierte und bewiesene Vehemenz, mit der sich die Oberschicht abschottet und sich selbst als eine Art neuer Adel erhält und das vielbeschworene Leistungsprinzip untergräbt war dann doch schon recht schockierend für mich.

Eine der “Elite”-Schulen die dort unter die Lupe genommen wird — Schloss Neubeuern — haben wir auf unserer Sommerradtour gesehen. Ich war schon beeindruckt von dem Haus, und damals habe ich auch gedacht “Mensch, auf so einer Schule hätte ich mich vielleicht wohlgefühlt”. Aber nach dieser Lektüre bin ich heilfroh dass es mich da nicht hinverschlagen hat.

Read more

Hot on the heels of the PostgreSQL 9.1.0 release I am happy to announce that the final version is now packaged for Debian unstable, the current Ubuntu development version “Oneiric”, and also in my Ubuntu backports PPA for Ubuntu 10.04 LTS, 10.10, and 11.04.

Enjoy trying out all the cool new features like builtin synchronous replication or per-column collation settings for correctly handling international strings, or an even finer-grained access control for large environments. Please see the detailled explanation of the new features.

As already announced a few days ago, 9.0 is gone from Ubuntu 11.10, as it is still only a development version and not an LTS. 9.1 will be the version which the next 12.04 LTS will support, so this slightly reduces the number of major upgrades Ubuntu users will need to do. However, 9.0 will still be available in Debian unstable and backports, and the Ubuntu backports PPA for a couple of months to give DB administrators some time to migrate.

Read more

PostgreSQL 9.1 has had its first release candidate out for some two weeks without major problem reports, so it’s time to promote this more heavily. If you use PostgreSQL, now is the time to try it out and report problems.

We always strive to minimize the number of major versions which we have to support. They not only mean more maintenance for developers, but also more upgrade cycles for the users.

9.0 has not been in any stable Debian or Ubuntu release, and 9.1 final will be released soon. So we recently updated the current Ubuntu development release for 11.10 (“oneiric”) to 9.1. In Debian, the migration from 8.4/9.0 to 9.1 is making good progress, and there is not much which is left until postgresql-9.0 can be removed.

Consequently, I also removed 9.0 from my PostgreSQL backports PPA, as there is nothing any more to backport it from. However, that mostly means that people will now set up installations with 9.1 instead of 9.0, and won’t magically make your already installed 9.0 packages go away. They will just be marked as obsolete in the postgresql-common debconf note.

If you want to build future 9.0 packages yourself, you can do this based on the current branch: bzr branch lp:~pitti/postgresql/debian-9.0, get a the new upstream tarball, name it accordingly, add a new changelog with a new upstream version number, and run bzr bd to build the package (you need to install the bzr-builddeb package for this).

Update 2011-09-09: As I got a ton of pleas to continue the 9.0 backports for a couple of months, and to keep it in Debian unstable for a while longer, I put them back now. I also updated the removal request in Debian to point out that I’m mainly interested in getting 9.0 out of testing. I don’t mind much maintaining it for a couple of more months in unstable. My dear, I had no idea that my backports PPA was that popular!

Read more

The tool to reprocess an Apport crash report to produce a symbolic stack trace, apport-retrace, has been pretty hard to use on a developer system so far: It either installed the packages from the crash report, plus its debug symbol packages (“ddebs”) into the running system (which frequently caused problems like broken dependencies), or it required setting up a chroot and using apport-chroot with fakechroot and fakeroot.

I’m happy to announce that with Apport 1.22, which landed in Oneiric yesterday, this has now become much easier: In the default mode it just calls gdb on the report’s coredump, i. e. expects that all the necessary packages are already installed and will complain about the missing ones. But with the new --sandbox/-Smode, it will just create a temporary directory, download and unpack packages there, and run gdb with some magic options to consider that directory a “virtual root”. These options haven’t been available back when this stuff was written the first time, which is why it used to be so complicated with fakechroots, etc. Now this does not need any root privileges, chroot() calls, etc.

As it only downloads and installs the bare minimum, and does not involve any of the dpkg/apt overhead (maintainer scripts, etc.), it has also become quite a lot faster. That’s how the apport retracers were able to dig through a backlog of about a thousand bugs in just a couple of hours.

So now, if you locally want to retrace or investigate a crash, you can do

   $ apport-retrace -s -S system /var/crash/_usr_bin_gedit.1000.crash

to get the stack traces on stdout, or

   $ apport-retrace -g -S system /var/crash/_usr_bin_gedit.1000.crash

to be put into a gdb session.

If you do this regularly, it’s highly recommended to use a permanent cache dir, where apt can store its indexes and downloaded packages: Use -C ~/.cache/apport-retrace for this (or the long version --cache).

You can also use this to reprocess crashes for a different release than the one you are currently running, by creating a config directory with an appropriate apt sources.list.

The manpage has all the details. (Note that at the time of this writing, still has the old version — use the local one instead.)

Enjoy, and let me know how this works for you!

Read more

Wir sind wieder da, und diesmal sogar völlig ohne menschliche oder radtechnische Schäden! Im letzten Jahr ging unsere Radtour an der Donau entlang von Passau nach Wien, was eher gemütlich war. Diesmal hat es uns an den Inn verschlagen, wo wir die Kultur vom letzten Jahr eingetauscht haben gegen das Hochkeuchen und Runterrasen eines alpin geprägten Berg-Wegprofils.

Das war unsere erste Radtour mit “richtigen” Bergen, wo man auch schon mal 200 Höhenmeter in der Stunde hoch und wieder runter schafft. Mit unseren Tourenrädern und viel Gepäck (Zeltausruestung) mussten wir auch stellenweise schieben, aber so oder so fand ich es eine gute sportliche Herausforderung. Und der fantastische Blick ins Tal mit den Bergen drumherum entschädigt auf jeden Fall für die Mühe!

Der Inn fliesst aus dem Lej da San Murezzan

Blick hinauf nach Guarda

Blick hinauf nach Guarda

Der alpine Teil startete in St. Moritz in der Schweiz, wo der Inn aus dem See “Lej da San Murezzan” quasi entspringt (die eigentliche Quelle ist noch höher in Majola, aber da mit dem Zug/Bus hinzukommen war dann zu anstrengend). Der Inn führte uns dann über Zernez, Scuol, Pfunds (dann schon in Österreich), Landeck, Imst, Innsbruck zunächst bis Kufstein.

Dort, an der österreich-deutschen Grenze, hörten dann auch die Alpen fast schlagartig auf, und von da an ging es dann eher gemächlich fast ohne größere Steigungen weiter über Rosenheim, Altötting, Braunau, und Bad Füssing bis nach Passau, wo der Inn dann in die Donau fließt. Wir sind bis zum “bitteren Ende” gefahren zum Dreiflüsse-Eck:

Inn-Brücke in Passau

Dreiflüsse-Eck in Passau

Zwischendurch gab es auch noch ein paar Höhepunkte:

In Imst sind wir mit dem “Alpine Coaster” 3,5 km den Berg heruntergedonnert, eine Art Sommerrodelbahn im Achterbahnformat. Den wollten wir ja schon vor ein paar Wochen bei unserer Klettersteig-Tour ausprobieren, aber leider brauchte der Auf- und Abstieg viel mehr Zeit als geplant, so dass Bergbahn und Coaster schon geschlossen hatten. Auch diesmal taten Wetter und nicht fahrende Busse ihr möglichstes, uns davon abzuhalten, aber diesmal haben wir gewonnen! Hat sich auch wirklich gelohnt.

Kristall-Modell des Lenin-Mausoleums

In Wattens haben wir uns die Glitzerwelt der Swarovski-Kristallwelten angeschaut. Es war im großen und ganzen eher eine Kunst-Ausstellung, ich hatte etwas anderes erwartet, aber der riesige Kristall-Dom (kuriose Akustik da drin!) und die Kristallmodelle berühmter Gebäude waren schon faszinierend.

Auf dem Weg gab es auch noch viele andere Eindrücke, wie das sehr schöne Dorf Neubeuren, das Geburtshaus von Papst Benedikt in Marktl oder das Vogelschutzgebiet Europareservat unterer Inn. Aber die meiste Zeit haben wir dann doch mit Radeln, und nachmittags mit faulenzen und lesen verbracht, so dass es ein wirklich entspannender Urlaub wurde.

Alle Fotos der Tour

Read more

Hot on the heels of the Announcement of the second 9.1 Beta release there are now packages for it in Debian experimental and backports for Ubuntu 10.04 LTS, 10.10. and 11.04 in my PostgreSQL backports for stable Ubuntu releases PPA.

Warning for upgrades from Beta 1: The on-disk database format changed since Beta-1. So if you already have the beta-1 packages installed, you need to pg_dumpall your 9.1 clusters (if you still need them), and pg_dropcluster all 9.1 clusters before the upgrade. I added a check to the pre-install script to make the postgresql-9.1 package fail early to upgrade if you still have existing 9.1 clusters to avoid data loss.

Read more

Two weeks ago, PostgreSQL announced the first beta version of the new major 9.1 version, with a lot of anticipated new features like synchronous replication or better support for multilingual databases. Please see the release announcement for details.

Due to my recent moving and the Ubuntu Developer Summit it took me a bit to package them for Debian and Ubuntu, but here they are at last. I uploaded postgresql-9.1 to Debian experimental; currently they are sitting in the NEW queue, but I’m sure our restless Debian archive admins will get to it in a few days. I also provided builds for Ubuntu 10.04 LTS, 10.10. and 11.04 in my PostgreSQL backports for stable Ubuntu releases PPA.

I provided full postgresql-common integration, i. e. you can use all the usual tools like pg_createcluster, pg_upgradecluster etc. to install 9.1 side by side with your 8.4/9.0 instances, attempt an upgrade of your existing instances to 9.1 without endangering the running clusters, etc. Fortunately this time there were no deprecated configuration options, so pg_upgradecluster does not actually have to touch your postgresql.conf for the 9.0 ?9.1 upgrade.

They pass upstream’s and postgresql-common’s integration test suite, so should be reasonably working. But please let me know about everything that doesn’t, so that we can get them in perfect shape in time for the final release.

I anticipate that 9.1 will be the default (and only supported) version in the next Debian release (wheezy), and will most likely be the one shipped in the next Ubuntu LTS (in 12.04). It might be that the next Ubuntu release 11.10 will still ship with 9.0, but that pretty much depends on how many extensions get ported to 9.1 by feature freeze.

Read more

Update at 13:06 UTC: Corrected NetworkManager description, thanks Mathieu for pointing out.

A few months ago, Matt Zimmerman kicked offa new tradition of a quarterly review of the most popular Ubuntu Brainstorm ideas. He did the December review, now it was my turn to coordinate the March review.

7zip desktop support (#26504)

The 7zip compression format becomes increasingly more popular these days; Ubuntu releases up to 10.10 did not support it on the desktop support as well as older formats like zip or bzip2.

Ubuntu developer Sebastien Bacher responds:

The 7z format has in fact been supported by file-roller for quite some time but it does require the installation of the command lines utilities to work. The issue is pretty much addressed in Ubuntu 11.04 (Natty) though since file-roller [...] will ask you if you want to install “p7zip” when you try open an archive using that format.

The other part of the brainstorm request is to also add support for it to gvfs, i. e. that you can browse a 7zip archive as a virtual storage device.This can’t be supported, as the library which is used for this (libarchive) only supports streamable format for efficiency. 7zip is not streamable, and thus would provide a very poor performance.

Empty directories in the Nautilus file manager (#26335)

In tree view mode, nautilus currently displays an expander symbol even if a directory is empty. This looks slightly confusing and makes it harder to see which directories actually have content.

This is indeed a long-standing known problem (the upstream bug is almost ten years old!). Rodrigo Moya, one of the GNOME maintainers in the Ubuntu desktop team, explains why fixing this is actually a lot harder than it might seem initially: Checking each folder to see if it’s got children or not might be time and CPU consuming when displaying lots of subfolders; it gets worse if you are browsing a directory on a remote or slow virtual file system like gphoto cameras or compressed tarballs.

One possible improvement would be to do the test asynchronously and display/hide the expander arrow as the subfolders are checked, and possibly restrict this to local file systems with a maximum number of directory entries. This would create an inconsistency, though.

So unfortunately it is not very realistic to see this being addressed soon.

Login screen (gdm) improvements (#26482)

This item suggests adding features to gdm which make it more useful, such as adding a clock, widgets, or a guest session without requiring an already existing running user session.

Ubuntu and GNOME developer Robert Ancell has a lot of experience with both gdm as well as his own LightDM project.

He points out that in GNOME 3 a clock was added to the login screen and looks similar to the proposed design. So we will get that in Ubuntu 11.10. Other changes to gdm should be discussed and proposed in the upstream bug tracker.

For 11.10 there is an existing proposal to use LightDM by default. LightDM offers a a lot more and easier possibilities for customization and theming, so any contributions for writing widgets or other improvements will be welcome.

Easy side-by-side window arrangement (#26152)

With nowaday’s modern big screens it often is too wasteful or even impractical to run applications fullscreen. A common case is to arrange two applications (such as a web browser and a document editor) side by side. This hasn’t had any particular support up to Ubuntu 10.10, aside from moving and resizing windows manually to fit.

John Lea of the Canonical Design Team explains how the main use case has been implemented in Ubuntu 11.04:

Windows can be opened into semi-maximised state where they occupy 50% of the screen width simply by dragging the window to the left or right border of the screen. A preview shadow informs the user that if they drop the window in this location the window will be resized. This interaction provides a simple, clean solution to the problem without introducing any additional window chrome.

Note that the remaining part of the request, resizing two adjacent windows at the same time, is not currently provided. It is quite a complex interaction which can also trigger false positives, and probably also requires some deeper design studies to get the user experience and definition of “adjacent” right. There are currently no plans to implement this.

man usability (#25975)

First-time users of the man utility often wonder how to quit the program again after they are done reading. Neither the manpage itself nor –help explain that, or other keys for navigation.

Colin Watson is one of the man-db upstream developers. He responds:

I’ve made a change upstream for man-db 2.6.0 which will address this, by adding “(press h for help or q to quit)” to the default prompt string which is displayed on the bottom line of the screen when reading manual pages. I think this is a reasonable balance between providing guidance and taking up too much screen space, and people who get fed up of seeing it can always follow the documentation in man(1) for customising the prompt.

[...] It will definitely be in Ubuntu 11.10.

Naming of Ethernet connections in the UI (#27250)

When connecting to a wired network, it automatically gets assigned a name like “Auto eth0″. Many people will not know what this is, or even if they do, distinguishing between one or another is difficult.

Our NetworkManager maintainer Mathieu Trudel-Lapierre adopted this problem, and wrote a detailled blog entry about how connection naming will be done in Ubuntu 11.10. In particular, network-manager will make the meaning of the default profiles clearer, and notifications will contain “Wired network” in addition to “eth0″. We still need to keep the actual interface name for more experienced users who want to customize their network configuration.

For the case of telling apart multiple ethernet adapters, Ubuntu 11.04 already layed the foundation for integrating biosdevname, which will provide more meaningful names to Ethernet ports than just enumerating them in an arbitrary order, provided that the BIOS provides names for these. It is not enabled by default yet, but might be in 11.10.

Save dialogs should have the three most recently used folders (#26471)

When saving files you often choose the same couple of folders to store your data. Sadly, the drop down menu for the save-as-dialog box only shows the last folder where you have saved a file. Another common use case is to save a document in e. g. Firefox somewhere, and wanting to open it in another application again.

The desktop world is moving towards better tracking of what the user did most recently, so we asked the Zeitgeist developers about the feasibility of this. Manish Sinha discussed the idea within the project and also with the GTK developers, and summarized the possible options in an email to the technical board list.

We don’t currently know about any developer who wants to work on this. GTK developer Federico Mena Quintero said that it is not too difficult to do, and that he would be happy to guide someone who wants to pick this up. So if this interests you, please give him a ping.

Configure auto-mounting of internal drives (#26946)

Ubuntu (and GNOME in general) does not automount internal hard drive partitions in general, as this might cause unwanted data disruption on e. g. Windows system partitions, and also has a performance impact. However, in some use cases it would actually be practical to do so for selected partitions.

David Zeuten and Martin Pitt, the current udisks upstream maintainers, discussed options how this should be integrated and found an agreement (see the response in brainstorm for details). In short, the gnome-disk-utility program will grow some options which allow you to configure individual partitions similar to this:

Automatically mount this drive:
( ) Never
(X) When I log in
( ) On computer startup

(note that this is in no way a finished design or even user fiendly strings).

The current timeline for this is to implement this for GNOME 3.4, which would be in time for Ubuntu 12.04.

Read more

As a followup action to my recent Talk about PyGI I now re-used my notes to provide some real wiki documentation.

It would be great if you could add package name info for Fedora/SUSE/etc., and perhaps add more example links for porting different kinds of software! Please also let me know if you have suggestions how to improve the structure of the page.

Read more

On next Monday this cycle’s Ubuntu Application Developer Week classes will start.

The topic that kept me busy most in this cycle was Python gobject-introspection, and porting pygtk2 apps to PyGI (see my initial steps and my report from the PyGI hackfest.)

To spread the love, there will be two talks about this next week: On Monday 17:00 UTC the very Tomeu Vizoso himself will explain what gobject-introspection (“GI”) is, why we need it, and how library developers use it to ship a good and useful GI binding (“typelib”) for application developers. I will then follow up on Tuesday 16:00 UTC about the app developer side, in particular how to use the GI typelibs in Python, and how to port PyGTK2 applications to PyGI.

For the most part these sessions are distribution neutral (we don’t have any special sauce for this in Debian/Ubuntu, it all happened right upstream :-) ); only a very small fraction of it (where I explain package names, etc.) will be specific to Debian/Ubuntu, but shouldn’t be hard to apply to other distributions as well.

So please feel invited to join, and bombard us with questions!

Read more

GNOME 3.0 and Ubuntu Natty are currently undergoing a major architectural shift from GTK 2.0 to 3.0. Part of this is that the previous set of manually maintained language bindings, such as PyGTK, are being deprecated in favor of GObject Introspection, a really cool technology!

For us this means that we have to port all our PyGTK applications from PyGTK 2 to gobject-introspection and GTK 3.0 at the same time. I started with that for my own projects (Apport and Jockey) a few days ago, and along the way encountered a number of problems. They are being fixed (particular thanks to the quick responsiveness of John Palmieri!), so I guess after a few of those iterations, porting should actually become straight forward and solid.

So now I’m proud to announce Apport 1.16 which is now fully working with GTK 3.0 and pygobject-introspection. I just uploaded it to Ubuntu Natty, where it can get some wider testing. I also have a pygi/GTK3.0 branch for Jockey, which is also mostly working now, but it’s blocked on the availability of a GIR for AppIndicator. Once that lands, I’ll release and upload the Jockey as well.

For other people working on porting, these are the bugs and problems I’ve encountered:

  • [pygobject] GtkMessageDialog constructors did not work at all. This was fixed in pygobject 2.27, so it works fine in Natty, but there is little to no chance of making these work in Ubuntu 10.10.
  • [pygobject] Gtk.init_check() crashes (upstream bug). It’s not strictly necessary, though, just avoids crashes (and crash reports) when calling it without a $DISPLAY. I’ll put that back once that gets fixed.
  • [pygobject] You can’t pass unicode objects as method arguments or property values (upstream bug). Method arguments were recently fixed in upstream git head, and I backported it to the Natty package, so these work now. Property values are still outstanding. The workaround for both is to convert unicode values to bytes with .encode('UTF-8') everywhere.
  • [pygobject] is a great help which automates most of the mechanical rewriting work and thus gets you 90% of the porting done automatically. It’s currently missing MessageDialog constants, and is a bit inconvenient to call. I sent two patches upstream (upstream bug) which improve this.
  • [GIR availability] libnotify did not build a GIR yet (upstream bug). This was fixed in upstream git head with some contributions of mine, and I uploaded it to Natty. This works quite well now, although add_action() crashes on passing the callback. This is still to be investigated.
  • [GIR availability] There is no Application Indicator GIR, as already mentioned. Our DX team and Ken VanDine are working on this, so this should get fixed soon.
  • [GTK] Many dialogs now scale in a very ugly manner: Instead of resizing the contents, the dialog just grows huge outer padding. This is due to a change of the default “fill” property, and apparently is not fully understood yet (upstream bug). As a workaround I explicitly added the fill property to the top-level GTKBox in my GTKBuilder .ui files. (Note that this happens in C as well, it’s not a GI specific bug.)
  • [GTK] Building GtkRadioButton groups is currently a bit inconvenient and requires special-casing for the first group member, due to some missing annotations. I sent a patch upstream which fixes that (upstream bug).
  • [GTK] After initial porting, a lot of my dialogs got way too narrow with wrapped labels and text, because GTK 3 changed the behaviour and handling of widget and window sizes. This is intended, not a bug, but it does require some adaptions in the GtkBuilder files and also in the code. The GTK 2 ? 3 migration guide has a section about it.

Despite those, I’m impressed how well gobject-introspection already works. It was relatively painless for me to generate a GIR from a libnotify (thanks to the annotations already being mostly correct for building the documentation) and use it from Python. In another couple of months this will be rock solid, and porting our bigger pygtk projects like ubiquity and software-center will hopefully become feasible.

Read more

Apport has provided built-in support for automatically identifying and marking duplicate bug reports for normal signal as well as Python crashes. However, we have more kinds of bug reports submitted through Apport which could benefit from automatic duplication: GPU freezes, package installation failures, kernel oopses, or gcc internal compiler errors, i. e. pretty much everything that gets reported automatically these days.

The latest Apport 1.20 (which also just hit current Ubuntu Natty) now allows package hooks to set a special field DuplicateSignature, which abstracts the concept for other kinds of bug reports where Apport doesn’t do automatic duplication. This field should both uniquely identify the problem class (e. g. “XorgGPUFreeze”) as well as the particular problem, i. e. variables which tell this instance apart from different problems. Aside from these requirements, the value can be any free-form string, Apport only treats it as an opaque value. It doesn’t even need to be ASCII only or only be one line, but for better human inspection I recommend this.

So your report could do something like

   report['DuplicateSignature'] = 'XorgGPUFreeze: instruction %s regs:%s:%s:%s' % (
                     current_instruction, regs[0], regs[1], regs[2])


    report['DuplicateSignature'] = 'PackageFailure: ' + log.splitlines()[-1]

This is integrated into Apport’s already existing CrashDatabase class, which maintains a signature ?master bug mapping in a SQLite database. So far these contained the crash signatures (built from executable name, signal number, and the topmost 5 stack trace names). As usual, if an incoming report defines a duplicate signature (from the crash stack trace or from DuplicateSignature), the first one will become the master bug, and all subsequent reports will automatically get closed as a duplicate in Launchpad.

Thanks to Bryce Harrington, who already came up with using this in the latest Intel graphics driver for GPU hangs!

Read more

(Update: Link to Tomeu’s blog post, repost for

Last week I was in Prague to attend the GNOME/Python 2011 Hackfest for gobject-introspection, to which Tomeu Vizoso kindly invited me after I started working with PyGI some months ago. It happened at a place called brmlab which was quite the right environment for a bunch of 9 hackers: Some comfy couches and chairs, soldering irons, lots of old TV tubes, chips, and other electronics, a big Pirate flag, really good Wifi, plenty of Club Mate and Coke supplies, and not putting unnecessary effort into mundane things like wallpapers.

It was really nice to get to know the upstream experts John (J5) Palmieri and Tomeu Vizoso (check out Tomeu’s blog post for his summary and some really nice photos). When sitting together in a room, fully focussing on this area for a full week, it’s so much easier to just ask them about something and getting things done and into upstream than on IRC or bugzilla, where you don’t know each other personally. I certainly learned a lot this week (and not only how great Czech beer tastes :-) )!

So what did I do?

Application porting

After already having ported four Ubuntu PyGTK applications to GI before (apport, jockey, aptdaemon, and language-selector),
my main goal and occupation during this week was to start porting a bigger PyGTK application. I picked system-config-printer, as it’s two magnitudes bigger than the previous projects, exercises quite a lot more of the GTK GI bindings, and thus also exposes a lot more GTK annotation and pygobject bugs. This resulted in a new pygi s-c-p branch which has the first 100 rounds of “test, break, fix” iterations. It now at least starts, and you can do a number of things with it, but a lot of functionality is still broken.

As a kind of “finger exercise” and also to check for how well pygi-convert works for small projects now, I also ported computer-janitor. This went really well (I had it working after about 30 minutes), and also led me to finally fixing the unicode vs. str mess for GtkTreeView that you got so far with Python 2.x.

pygobject and GTK fixes

Porting system-config-printer and computer-janitor uncovered a lot of opportunities to improve, a big “perl -e” kind of script to do the mechanical grunt work of the porting process. It doesn’t fix up changed signatures (such as adding missing arguments which were default arguments in PyGTK, or the ubiquitous “user_data” argument for signal handlers), but at least it gets a lot of namespaces, method, and constant names right.

I also fixed three annotation fixes in GTK+. We also collaboratively reviewed and tested Pavel’s annotation branch which helped to fix tons of problems, especially after Steve Frécinaux’s excellent reference leak fix, so if you play around with current pygobject git head, you really also have to use the current GTK+ git head.

Speaking of which, if you want to port applications and always stay on top of the pygobject/GTK development without having to clutter your package system with “make install”s of those, it works very well to have this in your ~/.bashrc:

export GI_TYPELIB_PATH=$HOME/projects/gtk/gtk:$HOME/projects/gtk/gdk
export PYTHONPATH=$HOME/projects/pygobject

Better GVariant/GDBus support

The GNOME world is moving from the old dbus-glib python bindings to GDBus, which is integrated into GLib. However, dbus-python exposed a really nice and convenient way of doing D-Bus calls, while using GDBus from Python was hideously complicated, especially for nontrivial arguments with empty or nested arrays:

from gi.repository import Gio, GLib
from gi._gi import variant_type_from_string

d = Gio.bus_get_sync(Gio.BusType.SESSION, None)
notify = Gio.DBusProxy.new_sync(d, 0, None, 'org.freedesktop.Notifications',
    '/org/freedesktop/Notifications', 'org.freedesktop.Notifications', None)

vb = GLib.VariantBuilder()
vb.add_value(GLib.Variant('s', 'test'))
vb.add_value(GLib.Variant('u', 1))
vb.add_value(GLib.Variant('s', 'gtk-ok'))
vb.add_value(GLib.Variant('s', 'Hello World!'))
vb.add_value(GLib.Variant('s', 'Subtext'))
# add an empty array
eavb = GLib.VariantBuilder()
# add an empty dict
eavb = GLib.VariantBuilder()
vb.add_value(GLib.Variant('i', 10000))
args = vb.end()

result = notify.call_sync('Notify', args, 0, -1, None)
id = result.get_child_value(0).get_uint32()
print id

So I went to making the GLib.Variant constructor work properly with nested types and boxed variants, adding Pythonic GVariant iterators and indexing (so that you can treat GVariant dictionaries/arrays/tuples just like their Python equivalents), and finally a Variant.unpack() method for converting the return value of a D-Bus call back into a native Python data type. This looks a lot friendlier now:

from gi.repository import Gio, GLib

d = Gio.bus_get_sync(Gio.BusType.SESSION, None)
notify = Gio.DBusProxy.new_sync(d, 0, None, 'org.freedesktop.Notifications',
    '/org/freedesktop/Notifications', 'org.freedesktop.Notifications', None)

args = GLib.Variant('(susssasa{sv}i)', ('test', 1, 'gtk-ok', 'Hello World!',
    'Subtext', [], {}, 10000))
result = notify.call_sync('Notify', args, 0, -1, None)
id = result.unpack()[0]
print id

I also prepared another patch in GNOME#640181 which will provide the icing on the cake, i. e. handle the variant building/unpacking transparently and make the explicit call_sync() unnecessary:

from gi.repository import Gio, GLib

d = Gio.bus_get_sync(Gio.BusType.SESSION, None)
notify = Gio.DBusProxy.new_sync(d, 0, None, 'org.freedesktop.Notifications',
    '/org/freedesktop/Notifications', 'org.freedesktop.Notifications', None)

result = notify.Notify('(susssasa{sv}i)', 'test', 1, 'gtk-ok', 'Hello World!',
            'Subtext', [], {}, 10000)
print result[0]

I hope that I can get this reviewed and land this soon.

Thanks to our sponsors!

Many thanks to the GNOME Foundation and Collabora for sponsoring this event!

Read more

A common source of unnoticed depwaits or uninstallability are main packages which introduce new build or binary dependencies from universe. These either require fixing, or filing a main inclusion report.

To help with this, I added a new check-mir script into ubuntu-dev-tools version 0.110, which walks through all build and binary dependencies, checks if they are in main/restricted, also considers alternative dependencies, and create a report with a few hints.

For a main package where everything is alright, it looks like this:

$ check-mir
Checking support status of build dependencies...

Checking support status of binary dependencies...
All dependencies are supported in main or restricted.

Example output for a totally synthetic situation with universe dependencies, non-preferred alternatives in main, and other special cases:

Checking support status of build dependencies...
 * pmount binary and source package is in universe
  ... alternative libexif-dev is already in main; consider preferring it
 * weechat binary and source package is in universe
 * sendmail-bin is in universe, but its source sendmail is already in main; file an ubuntu-archive bug for promoting the current preferred alternative

Checking support status of binary dependencies...
 * wesnot does not exist (pure virtual?)
 * wesnoth binary and source package is in universe

Please check if this source package needs to get into in main/restricted, or reconsider if the package really needs above dependencies.

Read more

For a test suite I need to create a local SSL-enabled HTTPS server in my Python project. I googled around and found various recipes using pyOpenSSL, but all of those are quite complicated, and I didn’t even get the referenced one to work.

Also, Python has shipped its own built-in SSL module for quite a while. After reading some docs and playing around, I eventually got it to work with a remarkably simple piece of code using the builtin ssl module:

import BaseHTTPServer, SimpleHTTPServer
import ssl

httpd = BaseHTTPServer.HTTPServer(('localhost', 4443), SimpleHTTPServer.SimpleHTTPRequestHandler)
httpd.socket = ssl.wrap_socket (httpd.socket, certfile='path/to/localhost.pem', server_side=True)

(I use port 4443 so that I can run the tests as normal user; the usual port 443 requires root privileges).

Way to go, Python!

Read more

Since last Tuesday, packages built in natty don’t come with a Debian changelog included any more. Due to the continuous demand for downsizing both our installation media, as well as the install footprint, we looked for packages which we should eliminate (duplicate libraries, unnecessary runtimes like our current effort to eliminate perl (-modules, not -base), but also for stuff that users generally don’t need and won’t miss. IMO package changelogs very much fall into the latter category, so they were very high on the “first against the wall” list.

Changelogs are of course a valuable developer tool, but for those it is usually less important to have them available locally, as long as there is a convenient method to access them. For that I wrote a helper tool “apt-changelog” which retrieves it from


$ apt-changelog gnome-panel

now replaces

$ zless /usr/share/doc/gnome-panel/changelog.Debian.gz

apt-changelog is now shipped in apt-utils in Natty.

So far this was discussed pre-UDS on and at UDS in a blueprint and various hallway conversations.

However, there were some concerns, so we got a new compromise to just ship the top 10 changelog records, and add a comment about apt-changelog at the bottom. That way, the most interesting entries are still shipped, and developers and users will get used to “apt-changelog”; this will provide a smoother transition, but still get rid of about 90% of the changelog size. This is implemented by pkgbinarymangler version 81 (just uploaded).

We can re-evaluate this after the next LTS, and eventually drop them completely.

I’ll make sure that by the end of the release all packages that got built between last Tuesday and now will be rebuilt at least once and thus get their changelogs back.

Read more