Canonical Voices

Posts tagged with 'debian'


It’s been a decade ago when I did my first steps with contributing to Free Software, about seven years when I joined Debian, and about 6 with Canonical and Ubuntu. Time for some reflection what I have done over these years!

Distribution Packaging and Maintenance

My first sponsored Debian upload ever was cracklib2, which seriously needed some love and was looking for a new maintainer. So in that upload I managed to close all outstanding bugs. Thanks to my mentor Martin Godisch about providing a lot of guidance for this!

Since then I’ve maintained various packages, where the most popular ones are certainly the free database server “PostgreSQL” (see next section) and the e-book management software “Calibre”.

“Maintaining” by and large means “making it really easy to get and use this software”. This decomposes to:

  • Packaging it in a way that a simple apt-get install makes the software work out of the box (as far as possible)
  • Provide a default configuration/customizations so that it integrates and plays well with the rest of the system; this includes the paths and permissions for log files, log rotation, debconf, configuration file standards, etc.
  • Be the front line for bug reports from users, sort, answer, and de-duplicate them, and either fix them myself, or forward useful bugs to upstream.
  • Providing security updates for stable releases
  • To some extent, help with the development of the software; this gets mostly driven by user demand, and of course my personal interests.

In August 2004 I got employed by Canonical to work full time on Ubuntu, which pretty much turned a hobby into a profession. I never regretted this in the past years, it’s an awesome job to do!

In principle I’m doing the same thing in Ubuntu as well: Bring stuff from developers to the people out there. Except with a different focus, in Ubuntu my daily bread and butter is the GNOME desktop and stuff around (and immediately below) it. And even though after a long day of bug triaging and debugging I feel a bit low-hearted (“50.000 bugs away from perfection”), when I take a step back and see how much the usage of Free Software in the world has grown since 2004, I am very proud of being part of Ubuntu, which certainly has its fair contribution and share in this growth. So what seemed like a crazy idea from Mark back in 2004 actually has made a remarkable progress.

In the beginning of Ubuntu I mostly sent back patches to the Debian bug tracker, but this evolved quite a bit on both sides: These days I try to keep “my” packages in sync and commit stuff directly to Debian, which works very well with e. g. the pkg-utopia team, which is responsible for HAL, udisks, upower, PolicyKit, and related packages. At this point I want to thank Michael Biebl for being such an awesome guy on the Debian side! Also, it seems that Debian has moved a fair bit away from the strong “Big Maintainer Lock” towards team based maintenance, so these days it is easier than ever to commit stuff directly to Debian for a lot of packages, without much fuss.


I have done a handful of changes to PostgreSQL, but these mostly concerned easy packaging and crash fixes, nothing out of the ordinary. I’m not really a PostgreSQL upstream developer.

The thing I am really proud of is the postgresql-common package, which is a very nice example what a distro can provide on top of upstream: If you install the upstream tarball, you have to manually care for creating clusters, providing a sensible configuration for them, set up SSL, set up log rotation, etc. With postgresql-common, this is all done automatically. The biggest feature it provides is a robust and automatic way of upgrading between major releases with the pg_upgradecluster tool, which takes care of a dozen corner cases and the nontrivial process of dumping the old cluster and reloading the new one. Also, you can effortlessy run several instances of the same version in parallel, so that you can e. g. have a production and a development instance, or try the new 9.0 RC1 while still running the 8.4 production one. (more details)

Crash Reporting with Apport

This has been a pet peeve of mine pretty much from day 1. Back in the old days, crashes in software were a pain to track down: many crashes are hard to reproduce, it takes ages to get useful information from bug reporters, and a lot of data cannot be recovered any more when you try to reproduce and analyze a crash after the fact.

With the growing demand for QA from both Canonical and Ubuntu, in 2006 I finally got some time to start Apport, which would make all this a lot easier: It intercepts crashes as they happen, collects the data that we as a developer need, and makes it very easy for the user to submit them as a bug report. This is accompanied by a backend service (called “retracers”) which would reprocess the bug reports by taking the core dump, reproducing a chroot with the packages and versions that the reporter had, installing the debugging symbols, and re-running gdb, to produce a fully symbolic stack trace.

See this bug report for how this looks like. Since then, tons of crashes were fixed, way more than we could ever have done “the old way” with asking users to rebuild with “-g -O0″, running gdb, etc.

By today, Apport has grown quite a bit: rich bug reports, per-package hooks, automatic duplication of crashes, interactive GUI elements in hooks, etc.

Plumbing Development

Handling hotpluggable hardware has always interested me, since the day when I got my first USB stick and it was ridiculously hard (from an user perspective) to use it:

    $ su -
    # mount -t vfat -o uid=1000 /dev/sda1 /mnt

My first go at this was to write pmount which would allow normal users to mount hotpluggable storage without root privileges and worrying about mount paths and options, and then integrate it into GNOME and HAL. Personally I abandoned it years ago, but it seems other people still use it, so I’m glad that Vincent Fourmond took over the maintenance now.

Since then, the entire stack evolved quite a bit: HAL grew to something useful and rather secure, and finally into some monstrous unmaintainable beast, which is why it was declared dead in 2008, and replaced with the “U” stack: udev, udisks, upower, etc. I enjoy hacking on that a lot, and since it’s part of my Desktop Team Tech Lead/Developer role in Canonical, I can spend some company time on it. so far I worked on bug fixes, small new features, and writing a rather comprehensive test suite for udisks (see my udisks commits so far). I also did a fair share of porting stuff away from HAL to the new stack, including some permanent commitments like maintaining the keymaps in udev.


Before I started with Canonical I was never much of a GUI person: I was fully content with fvwm and a few xterms around it. But as an Ubuntu developer I do dogfooding, and thus I switched to GNOME for my day to day work. It didn’t take long before I really fell in love with it!

Similar to my Debian packages, my upstream involvement with GNOME is mostly integration and bug fixing. As already explained, we get a looooot of bug reports, so my focus is mostly on bug fixing. To date, I sent 93 patches to bugzilla. Since January 2010 I became a committer, so that it’s easier for me to get patches upstream.

Debugging problems and fixing bugs is a pretty tedious task, but I still enjoy the rewarding nice feeling when you finally tracked down something and can close a bug with 50 duplicates, and you have made people’s life a little bit easier from now on.

Read more

For the first year and a half in Canonical I worked with the amazing Launchpad team, with the ambitious goal of building a new user interface, introducing AJAX in an established code base and rolling it all out on time. While all of that was overwhelming in itself, what was more important to me was making sure the UI remained consistent across time.
Long story short, it was a success and it's been 8 months since I've left the team and the established process is still on-going.

I wrote a paper on the whole experience and presented it at the agile conference XP2010 in Norway.

Here's the introduction:

When I started working with the Launchpad team I was tasked with designing and rolling out a new interface using cutting-edge technology on a well established product and team. The existing processes and team structure made it very hard to roll out big changes while also ensuring consistency as time went by.
While the general designs and work ow changes were being eshed out, I started to drive some change to the existing processes, enabling me to be successful at an objective that would take a year to accomplish, and unexpectedly, beyond that.
The project was 4 years old and had over 500 dynamic pages with different templates and layouts that had been left untouched at different points in time. The goal for the next year was to make the application easier to use, even enjoyable. I also had to make the UI consistent across the board, take the project from static HTML pages into the wonderful world of in-line editing, streamlined work-flows and predictable interactions. In parallel, fundamental features that had been developed were going completely unused and we needed to turn that around. A re-usable AJAX infrastructure had to be developed from the ground up, new features needed to be designed and delivered, and general navigation issues needed to be addressed.
However, this story isn't about the success of the roll out of a new interface, but rather the success in the process changes that evolved during that year and how the project went from nobody feeling ownership over the user interface, to the developers taking strong ownership.

I feel very passionate about this subject, and hope this experience can help other projects and teams.

Here's the paper for download: xp2010_paper.pdf

Read more

PostgreSQL 9.0 with a whole lot of new features and improvements is nearing completion. The first release candidate was just announced.

As with the beta versions, I uploaded RC1 to Debian experimental again. If you want to test/use them on Ubuntu 10.04 (Lucid Lynx), you can get packages from my “PostgreSQL backports for stable Ubuntu releases” PPA. Please let me know if you need them for other releases.

Just for the records, both Debian 6.0 “Squeeze” and Ubuntu 10.10 “Maverick Meerkat” will release and officially support 8.4 only, as 9.0 is too late for the feature freezes of both. Also, it will take quite some time to update all the packaged extensions to 9.0. As usual, 9.0 will be provided as official backports for both Debian and Ubuntu.

Happy testing!

Read more

During our Debconf we were wondering how many people shared the common link of being both Ubuntu Developers and working in Debian. The initial list is 62! (Make that 59, see below) Thanks to Lucas Nussbaum and Michel Bienia for this first cut.

Ond?ej Surý, Fabio Tranchitella, Kees Cook, LI Daobing, Fathi Boudra, Steve Kowalik, Benjamin Mako Hill, Scott Kitterman, Alexander Sack, Colin Watson, Sebastien Bacher, Martin Meredith, Andrew Mitchell, Daniel Silverstone, LaMont Jones, Fabio Massimo Di Nitto, Loïc Minier, ???? ???????? (Ahmed El-Mahmoudy), Luca Falavigna, Laurent Bigonville, Philipp Kern, Martin Pitt, Matthias Urlichs, Sebastian Dröge, Luke Faraone, Steve Langasek, Thom May, Matthias Klose, Alessio Treglia, Iulian Udrea, Timo Jyrinki, Emilio Pozuelo Monfort, Albin Tonnerre, Stefan Potyra, Andrea Veri, Scott Howard, Nicolas Valcárcel, Raphaël Hertzog, Reinhard Tartler, Julian Andres Klode, Barry deFreese, Benjamin Drung, Michael Banck,  Riccardo Setti, Lucas Nussbaum, Adam Conrad, Michael Vogt, Jelmer Vernooij, Adrian Perez, Robert Collins, Tollef Fog Heen, Gerfried Fuchs, Mark Shuttleworth, Andrew Pollock, Sylvestre Ledru, Michael Casadevall, Ben Collins, Jo Shields, and Chris Cheney

This is a list of people who are in ubuntu-dev or ubuntu-core-dev AND have their key in the Debian keyring, it’s not an indicator of how active that person may or may not be. Here are the scripts they used if you’re interested in working on this sort of thing.

UPDATE: mdz, keybuk, and Kyle McMartin are emeritus, I’ve removed them from the list.

Read more

We have very exciting and challenging plans for the future of the new web+mobile Ubuntu One team (more on this soon), and we’re looking for an exceptional web engineer to join us.

The summary for this position is:

We are looking for an exceptional engineer to work on Ubuntu One’s web infrastructure with a proven track record for exceptional problem solving and integration into third-party systems. This person should help the team design, build, and deploy web and mobile applications with a high degree of quality and passion. If you’re the type of person who gets excited about delivering cutting-edge technology to hundreds of thousands of users, in a lean and friendly environment, we are looking for you!

If this sounds like you, check out the full job description and send us your CV!

Read more
Barry Warsaw

My friend Tim is working on a very cool Bazaar-backed wiki project and he asked me to package it up for Ubuntu. I'm getting pretty good at packaging Python projects, but I always like the practice because each time it gets a little smoother. This one I managed to package in about 10 minutes so I thought I'd outline the very easy process.

First of all, you want to have a good, and if you like to cargo cult, you can start with this one. I highly recommend using Distribute instead of setuptools, and in fact the former is what Ubuntu gives you by default. I really like adding the which gives you nice features like being able to do python test and many other things. See lines 18 and 19 in the above referenced file.

The next thing you'll want is Andrew Straw's fine stdeb package, which you can get on Ubuntu with sudo apt-get install python-stdeb. This package is going to bootstrap your debian/ directory from your file. It's not perfectly suited to the task (yet, Andrew assures me :), but we can make it work!

These days, I host all of my packages in Bazaar on Launchpad, which is going to make some of the following steps really easy. If you use a different hosting site or a different version control system, you will have to build your Ubuntu package using more traditional means. That's okay, once you have your debian/ directory, it'll be fairly easy (but not as easy as described here ). If you do use Bazaar, you'll just want to make sure you have the bzr-builddeb. Just do sudo apt-get install bzr-builddeb on Ubuntu and you should get everything you need.

Okay, so now you have the requisite packages, and a, let's build us a deb and upload it to our personal package archive so everyone on Debian and Ubuntu can easily try it out.

First, let's create the debian directory. Here's the first little icky bit:

% python --command-packages=stdeb.command sdist_dsc

Notice that this leaves us with a deb_dist/ directory, not the debian/ directory we want. The latter is in there, just buried a bit. Let's dig it out:

% mv deb_dist/wikkid-0.1/debian .
% rm -rf deb_dist
% bzr add debian
% bzr commit -m'Debianize'

Note that "wikkid-0.1" will be replaced by the name of your package. In order to build the .deb package, you need an "orig.tar.gz" file. Packaging sort of assumes that you've got an original upstream tarball somewhere and you're just adding the necessary Debian goo to package the thing. In this case, we don't have an upstream tarball, although we could easily create one, and upload it to the Cheeseshop or Launchpad or wherever. However, that just slows us down so let's skip that for now! (Aside: if you do have an upstream tarball somewhere, you'll want to add a debian/watch which points to it; that'll eliminate the need to do the next step, by downloading the tarball instead).

Let's create the tarball right now and copy it to where the following step will expect it:

% python sdist
% mv dist/Wikkid-0.1.tar.gz ../wikkid_0.1.orig.tar.gz

Here's the second icky bit. Building a Debian source package imposes a very specific naming convention on the tarball. Wikkid's happens to build a tarball with an incompatible name, while the sdist command leaves it in a place where the next step can't find it. The rename just gets everything into the proper place. YMMV.

Now we can build the Debian source package. It's the source package that we'll upload to our Launchpad PPA. Launchpad will then automatically (if we've done everything right) build the binary package from the uploaded source package, from which Ubuntu and Debian users can easily install.

Oops! Before we do this, please edit your debian/changelog file and change unstable to lucid. You should also change the version number by adding a ~ppa1 to the end of it. Yeah, more ickiness.

Alright now we're ready to build our source package:

% bzr bd -S

Now let's upload it (assuming you've enabled a PPA):

% cd ..
% dput ppa:barry/python wikkid_0.1-1~ppa1_source.changes

That's it! If you've done everything successfully, you'll have the package in your PPA in 5 minutes or so. Then anybody who's added your PPA can just apt-get install wikkid (or whatever your package is called).

I do hope to work with the appropriate developers to make some of the ickiness go away. Please do contact me if you want to help!

Addendum (2010-06-10)

Let's say you publish your tarball on the Cheeseshop or Launchpad, and you don't want to have to build a different tarball locally in order to package it. Here's what I think works:

Create a debian/watch file that points to the download location you publish to. If your package is not yet available in Debian or Ubuntu, then use this command to build your source package:

bzr bd -S -- -sa

The bit at the end tells the Debian packaging primitives to include your tarball when your source package is uploaded. The debian/watch file is used to download your published tarball and automatically renamed to the required .orig.tar.gz name. When you dput your package, your tarball will be uploaded too, and everything should build properly.

Oh, and don't forget to look carefully at the lintian output. Try to make this as clean as possible. The Debian and Ubuntu packaging guides can help here.

Addendum 2 (2010-06-10)

Andrew Straw has added a debianize command to his stdeb package, which makes things much nicer. With this you can create the debian/ directory right next to your AFAIK, this version of stdeb isn't released yet, so you need to install his git head in a virtualenv, and it has a few minor buglets, but it does seem like the best-of-breed solution. I'll post another article with a more detailed follow up later.

Read more

I just did the 1000th commit of postgresql-common, the Debian/Ubuntu PostgreSQL management utilities. Wow, what started as a small hack in December 2004 to be able to install several major PostgreSQL versions in parallel has turned out to be a > 600 kB project providing a comprehensive tool set for uniformly setting up, upgrading, and maintaining PostgreSQL database instances from version 7.4 up to the just announced 9.0 beta-1, with a comprehensive test suite that I’m really proud of (it tests just about every aspect, option, and corner case of the installation, integration, upgrade, locale support, and error handling, and takes about half an hour on my system).

The actual commit is rather dull though, it’s just the release/upload tag for version 107 which I just uploaded to Debian unstable (it will hit Ubuntu maverick and backports soon). 107 introduces support for PostgreSQL 9.0, and I fixed up the scripts and tests enough so that all the tests pass now, and thus it’s good for public release.

I also uploaded the 9.0 beta 1 server itself now. It’ll be in Debian’s NEW queue for a bit, and hit experimental in a few days (or hours; recently the ftpmasters have been awesome!) It has a few cool new features (see the announcement), and upstream really appreciates testing and feedback. So, bug reports appreciated!

In particular, if you have existing 8.4 clusters you can just try to pg_upgradecluster them to 9.0 beta 1. Remember, if anything goes wrong, the cluster of the previous version is still intact and untouched, so you can run upgrades as many times as you like and only pg_dropcluster the old one when you’re completely satisfied with the upgrade.

Read more

PostgreSQL did microrelease updates three weeks ago: 8.4.3, 8.3.10, and 8.1.20 are the ones relevant for Debian/Ubuntu. There haven’t been reports about regressions in Debian or the upstream lists so far, so it’s time to push these into stable releases.

The new releases are in Lucid Beta-2, and hardy/jaunty/karmic-proposed. If you are running PostgreSQL, please upgrade to the proposed versions and give feedback to LP #557408.

Updates for Debian Lenny are prepared as well, and await release team ack.

On a related note, I recently fixed quite a major problem in pg_upgradecluster in postgresql-common 106: It did not copy database-level ACLs and configuration settings (Debian #543506). Fixing this required some reenginering of the upgrade process. It’s all thoroughly test case’d, but practical feedback would be very welcome! Remember, if anything goes wrong, the cluster of the previous version is still intact and untouched, so you can run upgrades as many times as you like and only pg_dropcluster the old one when you’re completely satisfied with the upgrade.



Read more
Timo Jyrinki

Thanks to Dr. H. Nikolaus Schaller's efforts, a new "A7+" version of the world's only 100% free software (and even free hardware design, leading to further community development) phone, Neo FreeRunner, is available for sale at for 299€! New in this hardware version is prolonged battery life, due to a fix applied to the famous "#1024" bug. Now you should have theoretically about 5 days time suspended, but that's of course only if you don't actually do anything with this phone-computer.

In other news, despite the fact or because Openmoko Inc. has ceased its development efforts for now at least, concentrating on the WikiReader to recover from the economic problems, community finally questioned the reasoning behind some of the Linux kernel debug configuration in the official Openmoko kernel branch. Results? Speedup of certain kernel operations in the range of 2x to 5x! In practice that means Neo isn't actually anymore the sluggish device you used to get to know with. Of course it's not top of the line by any means, but being the only Free phone available on the market still, more free than most full-size computers in fact, it's a quite nice improvement to eg. boot time, application start up time et cetera. I merely was a messenger of these news from the kernel mailing list to the community, but I also provided a readily compiled kernel which I use in Debian and which seems to works for others as well (until their distributions package it up).

Over 1,5 years after launch of the FreeRunner, and even more since the original Neo 1973, the software is getting better all the time. The pace is slow, as is the case with any free/open project with limited community-only resources, but the best thing is that it never has to stop. A lot of the middleware, applications and so on will make it to future phones as well. Things like Intone music player, TangoGPS and literki keyboard might be nice little finger-usable applications in the future as well.

So, if you can manage without 3G and want to still have an unique mobile computer experience with basic phone functionality, running for example Debian for the "familiar experience" if you use Debian or Ubuntu on your other computers, it's still not too late to catch it. It seems we're still a couple of years away from any next effort of such level of freedom. I'm making through it by having bought a 59€ 3G modem for the more serious data needs. I'm still also thinking about a privoxy setup on my home server that would clean up and compress pages even via Neo's GPRS connection.

Read more

I finally listened to Sebastien Bacher and applied for GNOME commit rights yesterday, after hassling Seb once more about committing an approved patch for me. Surprisingly, it only took some 4 hours until my application was approved and my account created, wow! Apparently 71 patches are enough. :-)

With my new powers, I fixed a crash in gdm, and applied two stragglers into gvfs’ build system today.

More to come!

Read more

We’re trying to improve the icons we have in Launchpad so they’re more usable across different cultures and types of users, and our first step is to do some user testing on our current icons.

The Canonical User Experience team has set up a survey to gather information on how users see our icons, so if you have a few spare minutes (it’s very quick!), please take the survey and pass it on to other people, especially if they don’t use Launchpad, as they will be less biased.

Survey is available at:

Read more

As promised, Launchpad has been fully open sourced (as opposed to the initial idea, nothing has been held back). Get it now, fix your favorite pet bug, and improve tens thousands of people’s experience.

Mark Shuttleworth really deserves a lot of praise for this bold and brave move, open sourcing not only the code, but all  it’s history. It’s a fantastic day today.

Update: yes, fully means including soyuz and codehosting, Mark has decided to release everything. The whole history is there.

See the loggerhead page:


Read more

About a month ago, I went to Canonical’s office in London for a sprint, and made good use of my Sunday by visiting the National Gallery. One fantastic thing about London, is the fact that all museums are free, not just because otherwise a few years back I couldn’t of afforded going, but because the fact that they are free gives you the freedom of going to the same ones over and over again, and just calmly visit the bits you’re interested in.

As I was walking by, I saw a painting that really struck me. It was a terrible and dark dragon eating two men, one of them is in agony while it’s face is being eaten off. Quite shocking:

Two Followers of Cadmus devoured by a Dragon

After looking at it for a little while, I went closer to read the description of it, which unexpectedly shocked me ten times as much:

“This gruesome episode comes from the story of Cadmus which is told in Ovid’s ‘Metamorphoses’ (III: 1-151). Cadmus was sent by the Delphic oracle to follow a cow and build a town where it sank from exhaustion. The cow stopped on the future site of Thebes, and Cadmus, intending to sacrifice it, sent his followers to get water from the neighbouring well of Ares. They were killed by the guardian of the well, a dragon who was the son of Ares. Cadmus then killed the dragon and on the advice of Athena sowed its teeth in the ground, from which sprang up armed men who slew each other, with the exception of five who became the ancestors of the Thebans.”

This got me thinking on how much first impressions are important in the user experience, but really hit me how much more important the actual content is. We tend to relay the content creation and management to “the marketing folks”, when I feel it’s a crucial part that should be worked on together to balance off the amount of text, with the tone in which it’s written, and to ensure that we’re adding value to the users’ experience.

Yes, I’m starting to see UI everywhere.

Read more