On next Monday this cycle’s Ubuntu Application Developer Week classes will start.

The topic that kept me busy most in this cycle was Python gobject-introspection, and porting pygtk2 apps to PyGI (see my initial steps and my report from the PyGI hackfest.)

To spread the love, there will be two talks about this next week: On Monday 17:00 UTC the very Tomeu Vizoso himself will explain what gobject-introspection (“GI”) is, why we need it, and how library developers use it to ship a good and useful GI binding (“typelib”) for application developers. I will then follow up on Tuesday 16:00 UTC about the app developer side, in particular how to use the GI typelibs in Python, and how to port PyGTK2 applications to PyGI.

For the most part these sessions are distribution neutral (we don’t have any special sauce for this in Debian/Ubuntu, it all happened right upstream ); only a very small fraction of it (where I explain package names, etc.) will be specific to Debian/Ubuntu, but shouldn’t be hard to apply to other distributions as well.

So please feel invited to join, and bombard us with questions!

Apport has provided built-in support for automatically identifying and marking duplicate bug reports for normal signal as well as Python crashes. However, we have more kinds of bug reports submitted through Apport which could benefit from automatic duplication: X.org GPU freezes, package installation failures, kernel oopses, or gcc internal compiler errors, i. e. pretty much everything that gets reported automatically these days.

The latest Apport 1.20 (which also just hit current Ubuntu Natty) now allows package hooks to set a special field DuplicateSignature, which abstracts the concept for other kinds of bug reports where Apport doesn’t do automatic duplication. This field should both uniquely identify the problem class (e. g. “XorgGPUFreeze”) as well as the particular problem, i. e. variables which tell this instance apart from different problems. Aside from these requirements, the value can be any free-form string, Apport only treats it as an opaque value. It doesn’t even need to be ASCII only or only be one line, but for better human inspection I recommend this.

So your report could do something like

   report['DuplicateSignature'] = 'XorgGPUFreeze: instruction %s regs:%s:%s:%s' % (
                     current_instruction, regs[0], regs[1], regs[2])

or

    report['DuplicateSignature'] = 'PackageFailure: ' + log.splitlines()[-1]

This is integrated into Apport’s already existing CrashDatabase class, which maintains a signature ?master bug mapping in a SQLite database. So far these contained the crash signatures (built from executable name, signal number, and the topmost 5 stack trace names). As usual, if an incoming report defines a duplicate signature (from the crash stack trace or from DuplicateSignature), the first one will become the master bug, and all subsequent reports will automatically get closed as a duplicate in Launchpad.

Thanks to Bryce Harrington, who already came up with using this in the latest Intel X.org graphics driver for GPU hangs!

A new plugin bzr-wincrypt stores passwords encrypted in the Windows CryptoAPI keyring.

A common source of unnoticed depwaits or uninstallability are main packages which introduce new build or binary dependencies from universe. These either require fixing, or filing a main inclusion report.

To help with this, I added a new check-mir script into ubuntu-dev-tools version 0.110, which walks through all build and binary dependencies, checks if they are in main/restricted, also considers alternative dependencies, and create a report with a few hints.

For a main package where everything is alright, it looks like this:

$ check-mir
Checking support status of build dependencies...

Checking support status of binary dependencies...
All dependencies are supported in main or restricted.

Example output for a totally synthetic situation with universe dependencies, non-preferred alternatives in main, and other special cases:

Checking support status of build dependencies...
 * pmount binary and source package is in universe
  ... alternative libexif-dev is already in main; consider preferring it
 * weechat binary and source package is in universe
 * sendmail-bin is in universe, but its source sendmail is already in main; file an ubuntu-archive bug for promoting the current preferred alternative

Checking support status of binary dependencies...
 * wesnot does not exist (pure virtual?)
 * wesnoth binary and source package is in universe

Please check https://wiki.ubuntu.com/MainInclusionProcess if this source package needs to get into in main/restricted, or reconsider if the package really needs above dependencies.

I’m enormously pleased to announce that Jelmer Vernooij will be joining the bzr team at Canonical on a full-time basis from January next year, replacing Robert Collins (who recently became Launchpad’s technical architect). Jelmer has contributed to Bazaar over many years including driving the svn and git foreign branch plugins and the bzr-rewrite plugin.

Jelmer already works at Canonical on the Launchpad team, and has recently been involved with the recipe builds feature that automatically assembles deb packages from the contents of upstream branches. We will very likely look to hire someone to replace Jelmer on the Launchpad team, through the Canonical jobs page.

I got a lot of very good applications from the bzr community and beyond, and I just wish I could work full time with more of you. Thanks to everyone who did apply.

Since last Tuesday, packages built in natty don’t come with a Debian changelog included any more. Due to the continuous demand for downsizing both our installation media, as well as the install footprint, we looked for packages which we should eliminate (duplicate libraries, unnecessary runtimes like our current effort to eliminate perl (-modules, not -base), but also for stuff that users generally don’t need and won’t miss. IMO package changelogs very much fall into the latter category, so they were very high on the “first against the wall” list.

Changelogs are of course a valuable developer tool, but for those it is usually less important to have them available locally, as long as there is a convenient method to access them. For that I wrote a helper tool “apt-changelog” which retrieves it from changelogs.ubuntu.com.

So

$ apt-changelog gnome-panel

now replaces

$ zless /usr/share/doc/gnome-panel/changelog.Debian.gz

apt-changelog is now shipped in apt-utils in Natty.

So far this was discussed pre-UDS on and at UDS in a blueprint and various hallway conversations.

However, there were some concerns, so we got a new compromise to just ship the top 10 changelog records, and add a comment about apt-changelog at the bottom. That way, the most interesting entries are still shipped, and developers and users will get used to “apt-changelog”; this will provide a smoother transition, but still get rid of about 90% of the changelog size. This is implemented by pkgbinarymangler version 81 (just uploaded).

We can re-evaluate this after the next LTS, and eventually drop them completely.

I’ll make sure that by the end of the release all packages that got built between last Tuesday and now will be rebuilt at least once and thus get their changelogs back.

After 20 days of final polishing and maturing since the release candidate, the PostgreSQL team released the final 9.0 version today.

Hot off the press, I uploaded postgresql-9.0 final into Debian unstable; they will not go into Debian Squeeze, because Squeeze is frozen and it will take a long time to port all the packaged server side extensions to 9.0.

If you are on Ubuntu 10.04 LTS or Ubuntu 10.10, you can add my PostgreSQL backports for stable Ubuntu releases PPA, which will carry 9.0 until it can be moved to the official Ubuntu backports (i. e. when 9.0 goes into Ubuntu Natty).

Enjoy, and kudos to the PostgreSQL team!

For an embedded/thin client project without GNOME, KDE, or even full XFCE I needed a small daemon to automount USB sticks. Using the full gvfs/gdu/nautilus or Thunar stack is too heavyweight for my purposes, but a simple udev rule just doesn’t cut it — I need to mount these USB sticks for a particular user (permissions), and also want to do an action like pop up a window with the contents.

This finally provided me the opportunity to write something bigger than just a 10 line demo in Vala (well, it’s not that much bigger admittedly ). Since that is my first real Vala project, it took quite a lot longer than anticipated; some areas of Vala are still a bit underdocumented, e. g. I spent some half an hour trying to find out how to set a result callback for an asynchronous function invocation. Mikkel Kamstrup Erlandsen suggested to just use a wrapper instead which uses yield, which works fine indeed. Mikkel, thanks for bearing with me!

Anyway, here it is: https://launchpad.net/udisks-automounter, complete with a first release, bzr branch (lp:udisks-automounter), and a package for Ubuntu 10.04 LTS in my PPA.

For avoidance of doubt, this won’t ever make sense on a GNOME/KDE/XFCE desktop, which already have their (much better developed) automounting services. But perhaps it’s useful for someone else with similar constraints.

This morning I woke up with this song in my head. It reminds me quite a bit of the Rolling Stones, and it’s very appropriate for the mood I’m in today.

I’m sitting here at Canonical’s office in Montreal, where we are having another Landscape sprint. There’s a couple things that make this sprint special, one of them being that Frank Wierzbicki has joined us! Welcome to the team, Frank, and I hope you enjoy!

It has also coincided with the end of our 1.5.5 milestone, which rolled out yesterday, with some interesting highlights:

Speed!

Contrary to what other people might tell you, YES speed does matter. We’ve been slowly (ha!) rolling out improvements to page load speed, and I’m happy to say that the improvements are very noticeable. Things we’ve done to improve this range from setting proper caching headers for static resources to heavily abusing the YUI Loader. I should write more about that on a more appropriate occasion. There’s always things to improve on, so keep an eye on it!

Faster Trials

Over the last four years, the trial registration for Landscape hasn’t been… enjoyable to say the least. This week we’ve finally rolled out the first iteration of our simplified registration process, which removes a manual approval step which used to take several weeks to be done. That means you can now register for a trial account in Landscape and start using it right away! Quite a novel concept, isn’t it? :) We’ll planning some more updates to it, so that when starting the registration process you have a better sense of where you are in the process and what are steps involved.

We have more exciting updates coming up before the end of the year, so keep an eye on it. But in the meantime, hit that ‘Registration’ link in Landscape, get it like you like it, and enjoy!

PostgreSQL 9.0 with a whole lot of new features and improvements is nearing completion. The first release candidate was just announced.

As with the beta versions, I uploaded RC1 to Debian experimental again. If you want to test/use them on Ubuntu 10.04 (Lucid Lynx), you can get packages from my “PostgreSQL backports for stable Ubuntu releases” PPA. Please let me know if you need them for other releases.

Just for the records, both Debian 6.0 “Squeeze” and Ubuntu 10.10 “Maverick Meerkat” will release and officially support 8.4 only, as 9.0 is too late for the feature freezes of both. Also, it will take quite some time to update all the packaged extensions to 9.0. As usual, 9.0 will be provided as official backports for both Debian and Ubuntu.

Happy testing!

?John Barlow’s new Bazaar TFS plugin adds support for Microsoft Team Foundation Server repositories, allowing one to use Bazaar to branch, merge, and commit code to remote TFS repositories.

The Debian import freeze is settled, the first rush of major changes went into Maverick, and the dust now has settled a bit. Thus it’s time to turn back some attention to crashes and quality in general.

This morning I created maverick chroots for the Apport retracers, and they are currently processing the backlog. I also uploaded a new Apport package which now enables crash reporting by default again.

Happy segfaulting!

PostgreSQL did microrelease updates three weeks ago: 8.4.3, 8.3.10, and 8.1.20 are the ones relevant for Debian/Ubuntu. There haven’t been reports about regressions in Debian or the upstream lists so far, so it’s time to push these into stable releases.

The new releases are in Lucid Beta-2, and hardy/jaunty/karmic-proposed. If you are running PostgreSQL, please upgrade to the proposed versions and give feedback to LP #557408.

Updates for Debian Lenny are prepared as well, and await release team ack.

On a related note, I recently fixed quite a major problem in pg_upgradecluster in postgresql-common 106: It did not copy database-level ACLs and configuration settings (Debian #543506). Fixing this required some reenginering of the upgrade process. It’s all thoroughly test case’d, but practical feedback would be very welcome! Remember, if anything goes wrong, the cluster of the previous version is still intact and untouched, so you can run upgrades as many times as you like and only pg_dropcluster the old one when you’re completely satisfied with the upgrade.

Thanks,

Martin

Parth announced bzr-grep 0.2.0.  Amongst other things there are performance enhancements such that Eli says:

Thanks, this looks great. I just tried it on Windows XP searching for a fixed string in the Emacs repository — took 28 seconds with a cold cache and only 7.5 with a warm cache, which is impressive.

By contrast, “grep -F -R” (with suitable –exclude patterns, to prevent it from searching binary files and inside .bzr) took about 12.5 seconds with a warm cache.

Yesterday PostgreSQL released new security/bug fix microreleases 8.4.2, 8.3.9, and 8.1.19, which fix two security issues and a whole bunch of bugs.

Updates for all supported Ubuntu releases are built in the ubuntu-security-proposed PPA. They pass the upstream and postgresql-common test suites, but more testing is heavily appreciated! Please give feedback in bug LP#496923.

Thanks!

This is a really short announcement, since it’s almost 4am here in Brazil and I should actually be sleeping, instead of building installers for Plone (ha!). But hey, this is exciting enough to keep some people awake all night.

Today, Canonical announces the availability of Landscape Dedicated Server!

So what is it?

One of the many things the Landscape team at Canonical has been working on since early this year is a version of Landscape that can be run on a local network, as opposed to the hosted, Software-as-a-Service version of Landscape that is available to the general public at the moment.

Many people have left us feedback saying that this would be desirable for them, and would actually make Landscape an option in environments where data cannot leave the local network boundaries due to strict policies. So if you’re one of this people or you have evaluated Landscape in the past but decided it was not for you due to this specific reason, this is the time to give Landscape a second look!

So thanks to everyone that has submitted ideas and requests for new features. We’re listening! Even more feedback-driven features are being added monthly, free of charge for existing customers, and the user interface is being polished and fine-tuned for managing large installations. Stay tuned for more announcements!

No, this is not a blog post about the kind of landscape you’re accustomed to, though it might trigger a few ‘I want to be a Landscape Architect’ thoughts from a person or two.

The news this time is that I’m going through a landscape change myself, and to me it’s still a bit scary just to think about it.

The last such change in my life happened roughly ten years ago when I left my job as PHP Programmer and Systems Administrator at a small ISP to start my own company with a few colleagues from university. At that time, leaving PHP behind to learn this new (to me) fancy things called Zope and Python felt really weird, not to mention the fact that I was about 20 and knew no-one of my age that had successfully started a web development company (I mean, I live in a very small town, this is not Silicon Valley).

So cutting to the chase, I would like to let everybody know I will be joining the Landscape team at Canonical starting January 5th. I will not be leaving Enfold Systems though. We keep working together part time at least until April, where we expect to make at least one big release of a fully eggified Enfold Server based on Plone 3.2 in that timeframe. After April, I will still be doing work for Enfold Systems, but the time available for that will much more constrained.

Joining the Landscape team is very exciting to me, not only because I will get to work with some really smart people, but also because I will be working from home and being supervised by one of the top minds in the Brazilian Python scene: Gustavo Niemeyer, responsible for pearls like `Mocker`, `Storm` and `Smart`. And on top of that, I will get to learn new ways of managing and collaborating within a distributed software development team, a subject that was theme for my graduation thesis.

Last, but not least important, there’s a long story of back and forth between me and Canonical that dates back as far as 2004. On that year, DebConf 4 was happening in Porto Alegre which is about one hour drive from where I live.

Hanging out on the #zope3-dev irc channel I’ve noticed that one guy, Steve Alexander, connected from an IP range in Brazil. That struck me as odd because I didn’t know Steve very well, but maybe well enough to guess that something hot was going on. So Steve told me he was here for DebConf, and I self-invited me to stop by and say `hi`.

Arriving at DebConf, I met Steve personally for the first time, and again had an odd sensation there: Steve and about 30-40 folks where separated from the rest of the DebConf crowd on their own room, hacking away. I also met Mark Shuttleworth there and saw him talk about his space flight. That was one of the most exciting things and very heart-warming to me since my dream as a kid was to be an astronaut.

Later on that same day I came to know they were creating a new distro: they were taking name suggestions and voting. It was not surprise at least to me when I first saw Ubuntu being mentioned in the news, though I don’t recall clearly if that was one of the suggestions being voted.

Having just finished a big project using Zope 3 and relational databases, Steve quickly asked for my input on the preliminary design of a Zope 3-based web application for managing translations, which I promptly gave. The code I was looking at right there was the beginning of Launchpad. Up until very recently, Launchpad was powered by a creation of mine named `sqlos`, which has now been replaced by `storm`.

Steve asked me if I was interested in joining the team, but I had to decline given that I had just signed up with Enfold Systems, which was only about a month old at that time. I did put them in touch with a good friend I met only a couple years before: Christian Reis (kiko). In retrospect, that might have been one of the best contributions I made to Canonical, and to Open Source in general.

So, all in all, very exciting news for me. Canonical has a special place in my heart since I basically saw when Ubuntu and Launchpad were born and have some really good friends working there. The situation is no different with Enfold Systems. I have been working with Alan Runyan since before Enfold was Enfold, and really since before Plone was Plone: when Plone 1.0 was announced I was with Alan Runyan, Paul Everitt and Alexander Limi in Paris, at SolutionsLinux 2003, participating on a Zope 3 sprint, mind you.

To summarize, I will be splitting my time between Enfold Systems and Canonical between January 5th and April 1st 2009. After that I will be working full time at Canonical but still expect to contribute significantly at Enfold Systems. One of the responsibilities I have right now and which I don’t want to drop is building the Plone installers for Windows. Hopefully I will keep doing that. Unless some Ubuntu dude sneaks in by night and erases my Windows partition, that is. :)