Canonical Voices

Posts tagged with 'conference'

Robin Winslow

Last weekend I went to my first Pycon, my second conference in a fortnight.

The conference runs from Friday to Monday, with 3 days of talks followed by one day of “sprints”, which is basically a hack day.

PyCon has a code of conduct to discourage any form of othering:

Happily, PyCon UK is a diverse community who maintain a reputation as a friendly, welcoming and dynamic group.

We trust that attendees will treat each other in a way that reflects the widely held view that diversity and friendliness are strengths of our community to be celebrated and fostered.

And for me, the conference lived up to this, with a very friendly feel, and a lot of diversity in its attendants. The friendly and informal atmosphere was impressive for such a large event with more than 450 people.

Unfortunately, the Monday sprint day was cut short by the discovery of an unexploded bomb.

Many keynotes, without much Python

There were a lot of “keynote” talks, with 2 on Friday, and one each on Saturday and Sunday. And interestingly none of them were really about Python, instead covering future technology, space travel and the psychology of power and impostor syndrome.

But of course there were plenty of Python talks throughout the rest of the day – you can read about them on my other post. And I think it was a good decision to have more abstract keynotes. It shows that the Python community really is more of a general community than just a special interest group.

Van Lindberg on data economics, Marx and the Internet of Things

In the opening keynote on Friday morning, the PSF chairman showed that total computing power is almost doubling every year, and that by 2020, the total processing power in portable devices will exceed that in PCs and servers.

He then used the fact that data can’t travel faster than 11.8 inches per nanosecond to argue that we will see a fundamental shift in the economics of data processing.

The big-data models of today’s tech giants will be challenged as it starts to be quicker and make more economic sense to process data at source, rather than transfer it to distant servers to be processed. Centralised servers will be relegated to mere aggregators of pre-processed data.

He likened this to Marx seizing the means of production in a movement which will empower users, as our portable Things start to hold the real information, and choose who to share it with.

I really hope he’s right, and that the centralised data companies are doomed to fail to be replaced by the Internet of Autonomous Things, because the world of centralised data is not an equal world.

Does Python have a future on small processors? Isn’t it too inefficient?

In a world where all the interesting software is running on light-weight portable devices, processing efficiency becomes important once again. Van used this to argue that efforts to run Python effectively on low-powered devices, like MicroPython, will be essential for Python as a language to survive.

Daniele Procida: All I really want is power

The second keynote was just after lunch on Friday, Daniele Procida, organiser of DjangoCon Europe openly admitted that what he really wanted out of life was power. He put forward the somewhat controversial idea that power and usefulness are the same thing, and that ideas without power are useless.

He made the very good point that power only comes to those who ask for it, or fight for it. And that if we want power not to be abused, we really need to talk about it a whole lot more, even though it makes people uncomfortable (try asking someone their salary). We should acknowledge who has the power, and what power we have, and watch where the power goes.

He suggested that, while in politics or industry, power is very much a rivalled good, in open source it is entirely an unrivalled good. The way you grab power in the open source community is by doing good for the community, by helping out. And so by weilding power you are actually increasing power for those around you.

I don’t agree with him on this final point. I think power can be and is hoarded and abused in the open source community as well. A lot of people use their power in the community to edge out others, or make others feel small, or to soak up influence through talks and presentations and then exert their will over the will of others. I am certainly somewhat guilty of this. Which is why we should definitely watch the power, especially our own power, to see what effect it’s having.

The takeaway maxim from this for me is that we should always make every effort to share power, as opposed to jealously guarding it. It’s not that sharing power in the open source community is inevitable or necessarily comes naturally, but at least in the open source community sharing power genuinely can help you gain respect, where I fear the same isn’t so true of politics or industry.

Dr Simon Sheridan: Landing on a comet: From planning to reality

Simon Sheridan was an incredibly most humble and unassuming man, given his towering achievements. He is a world-class space scientist who was part of the European Space Agency team who helped to land Rosetta on comet 67P.

Most of what he mentioned was basically covered in the news, but it was wonderful to hear it from his perspective.

Naomi Ceder: Confessions of a True Impostor

When, a short way into her Sunday morning keynote, Naomi Ceder asked the room:

How many of you would say that you have in some way or another suffered from imposter syndrome along with me?

Almost everybody put their hands up. This is why I think this was such an important talk.

She didn’t talk about this per se, but contributing to the open source community is hard. No-one talks about it much, but I certainly feel there’s a lot of pressure. Because of its very nature, your contributions will be open, to be seen by anyone, to be criticised by anyone. And let’s face it, your contributions are never going to be perfect. And the rules of the game aren’t written down anywhere, so the chance of being ridiculed seem pretty high. Open source may be a benevolent idea, but it’s damned scary to take part in.

I believe this is why less than 2% of open source contributors are female, compared with more like 25-30% women in software development in general. And, as with impostor syndrome, the same trend is true of other marginalised groups. It’s not surprising to me that people who are used to being criticised and discriminated against wouldn’t subject themselves to that willingly.

And, as Naomi’s question showed, it is not just marginalised people who feel this pressure, it’s all of us. And it’s a problem. As we know, confidence is no indicator of actual ability, meaning that many many talented people may be too scared to contribute to open source.

As Naomi pointed out, impostor syndrome is a socially created condition – when people are expected to do badly, they do badly. In fact I completely agree with her suggestion that the existing Wikipedia definition of impostor syndrome (at the time of writing) could be more sensitively phrased to define it as a “social condition” rather than a “psychological phenomenon”, as well as avoiding singling out women.

While Naomi chose to focus in her talk on how we personally can try to mitigate feelings of being an impostor, I think the really important message here is one for the community. It’s not our fault that open source is scary, that’s just the nature of openness. But we have to make it more welcoming. The success of the open source movement really does depend on it being diverse and accepting.

What I think is really interesting is that stereotype threat can be mitigated by reminding people of their values, of what’s important to them. And this is what I hope will save open source. The more we express our principles and passion for open source, the more we express our values, the easier it is to counter negative feelings, to be welcoming, to stop feeling like impostors.

A great conference

Overall, the conference was exhausting, but I’m very grateful that I got to attend. It was inspiring and informative, and a great example of how to maintain a great community.

If you want you can now go and read about the other talks.

(Also published on

Read more
Robin Winslow

The weekend before last, I went to PyCon UK 2015.

I already wrote about the keynotes, which were more abstract. Here I’m going to talk about the other talks I saw, which were generally more technical or at least had more to do with Python.


The talks I saw covered a whole range of topics – from testing through documentation and ways to achieve simplicity to leadership. Here are some key take-aways:

The talks

Following are slightly more in-depth summaries of the talks I thought were interesting.


Leadership of Technical Teams – Owen Campbell

There were two key points I took away from this talk. The first was Owen’s suggestion that leaders should take every opportunity to practice leading. Find opportunities in your personal life to lead teams of all sorts.

The second point was more complex. He suggested that all leaders exist on two spectra:

  • Amount of control: hand-off to dictatorial
  • Knowledge of the field: novice to expert

The less you know about a field the more hands-off you should be. And conversely, if you’re the only one who knows what you’re talking about, you should probably be more of a dictator.

Although he cautioned that people tend to mis-estimate their ability, and particularly when it comes to process (e.g. agile), people think they know more than they do. No-one is really an expert on process.

He suggested that leading technical teams is particularly challenging because you slide up and down the knowledge scale on a minute-to-minute basis sometimes, so you have to learn to be authoritative one moment and then permissive the next, as appropriate.

Document all the things – Kristian Glass

Kristian spoke about the importance, and difficulty, of good documentation.
Here are some particular points he made:

  • Document why a step is necessary, as well as what it is
  • Remember that error messages are documentation
  • Try pair documentation – novice sitting with expert
  • Checklists are great
  • Stop answering questions face-to-face. Always write it down instead.
  • Github pages are better than wikis (PRs, better tracking)

One of Kristian’s main points was that it goes against the grain to write documentation, ‘cos the person with the knowledge can’t see why it’s important, and the novice can’t write the documentation.

He suggested pair documentation as a solution, which sounds like a good idea, but I was also wondering if a StackOverflow model might work, where users submit questions, and the team treat them like bugs – need to stay on top of answering them. This answer base would then become the documentation.


Asking About Gender – the Whats, Whys and Hows – Claire Gowler

Claire spoke about how so many online forms expect people to be either simply “male” or “female”, when the truth can be much more complicated.

My main takeaway from this was the basic point that forms very often ask for much more information than they need, and make too many assumptions about their users. When it comes to asking someone’s name, try radically reducing the complexity by just having one text field called “name”. Or better yet, don’t even ask their name if you don’t need it.

I think this feeds into the whole field of simplicity very nicely. A very many apps try to do much more than they need to, and ask for much more information than they need. Thinking about how little you know about your user can help you realise what you actually don’t need to know about your user.

Finding more bugs with less work – David R. MacIver

David MacIver is the author of the Hypothesis testing library.

Hypothesis is a Python library for creating unit tests which are simpler to write and more powerful when run, finding edge cases in your code you wouldn’t have thought to look for. It is stable, powerful and easy to add to any existing test suite.

When we write tests normally, we choose the input cases, and we normally do this and we often end up being really kind to our tests. E.g.:

What Hypothesis does it help us test with a much wider and more challenging range of values. E.g.:

There are many cases where Hypothesis won’t be much use, but it’s certainly good to have in your toolkit.


Simplicity Is A Feature – Cory Benfield

Cory presented simplicity as the opposite of complexity – that is, the fewer options something gives you, the more simple and straightforward it is.

“Simplicity is about defaults”

To present as simple an interface as possible, the important thing is to have many sensible defaults as possible, so the user has to make hardly any choices.

Cory was heavily involved in the Python Requests library, and presented it as an example of how to achieve apparent simplicity in a complex tool.

“Simple things should be simple, complex things should be possible”

He suggested thinking of an “onion model”, where your application has layers, so everything is customisable at one of the layers, but the outermost layer is as simple as possible. He suggested that 3 layers is a good number:

  • Layer 1: Low-level – everything is customisable, even things that are just for weird edge-cases.
  • Layer 2: Features – a nicer, but still customisable interface for all the core features.
  • Layer 3: Simplicity – hardly any mandatory options, sensible defaults
    • People should always find this first
    • Support 80% of users 80% of the time
    • In the face of ambiguity do the right thing

He also mentioned that he likes README driven development, which seems like is an interesting approach.

How (not) to argue – a recipe for more productive tech conversations – Harry Percival

I think this one could be particularly useful for me.

Harry spoke about how many people (including him) have a very strong need to be right. Especially men. Especially those who went to boarding school. And software development tends to be full of these people.

Collaboration is particularly important in open source, and strongly disagreeing with people rarely leads to consensus, in fact it’s more likely to achieve the opposite. So it’s important that we learn how to get along.

He suggests various strategies to try out, for getting along with people better:

  • Try simply giving in, do it someone else’s way once in a while (hard to do graciously)
  • Socratic dialogue: Ask someone to explain their solution to you in simple terms
  • Dogfooding – try out your idea before arguing for its strength
  • Bide your time: Wait for the moment to see how it goes
  • Expose yourself to other social situations, where arguments are less acceptable

All of this comes down to stepping back, waiting and exercising humility. All of which are easier said than done, but all of which are very valuable if I could only manage it.

FIDO – The dog ate my password – Alex Willmer

After covering fairly common ground of how and why passwords suck, Alex introduced the FIDO alliance.

The FIDO alliance’s goal is to standardise authentication methods and hopefully replace passwords. They have created two standards for device-based authentication to try to replace passwords:

  • UAF: First-factor passwordless biometric authentication
  • U2F: Second-factor device authentication

Browsers are just starting to support U2F, whereas support for UAF is farther off. Keep an eye out.

Data Visualisation with Python and Javascript – crafting a data-visualisation for the web – Kyran Dale

Kyran spoke about visualising data, and demoed using Scrapy and Pandas to retrieve the Nobel laureatte data from Wikipedia, using Flask to serve it as a RESTful API, and then using D3 to create an interactive browser-based visualisation.

(Also published on

Read more

I’m on my way home from Düsseldorf where I attended the LinuxCon Europe and Linux Plumber conferences. I was quite surprised how huge LinuxCon was, there were about 1.500 people there! Certainly much more than last year in New Orleans.

Containers (in both LXC and docker flavors) are the Big Thing everybody talks about and works with these days; there was hardly a presentation where these weren’t mentioned at all, and (what felt like) half of the presentations were either how to improve these, or how to use these technologies to solve problems. For example, some people/companies really take LXC to the max and try to do everything in them including tasks which in the past you had only considered full VMs for, like untrusted third-party tenants. For example there was an interesting talk how to secure networking for containers, and pretty much everyone uses docker or LXC now to deploy workloads, run CI tests. There are projects like “fleet” which manage systemd jobs across an entire cluster of containers (distributed task scheduler) or like which auto-build packages from each commit of projects.

Another common topic is the trend towards building/shipping complete (r/o) system images, atomic updates and all that goodness. The central thing here was certainly “Stateless systems, factory reset, and golden images” which analyzed the common requirements and proposed how to implement this with various package systems and scenarios. In my opinion this is certainly the way to go, as our current solution on Ubuntu Touch (i. e. Ubuntu’s system-image) is far too limited and static yet, it doesn’t extend to desktops/servers/cloud workloads at all. It’s also a lot of work to implement this properly, so it’s certainly understandable that we took that shortcut for prototyping and the relatively limited Touch phone environment.

On Plumbers my main occupations were mostly the highly interesting LXC track to see what’s coming in the container world, and the systemd hackfest. On the latter I was again mostly listening (after all, I’m still learning most of the internals there..) and was able to work on some cleanups and improvements like getting rid of some of Debian’s patches and properly run the test suite. It was also great to sync up again with David Zeuthen about the future of udisks and some particular proposed new features. Looks like I’m the de-facto maintainer now, so I’ll need to spend some time soon to review/include/clean up some much requested little features and some fixes.

All in all a great week to meet some fellows of the FOSS world a gain, getting to know a lot of new interesting people and projects, and re-learning to drink beer in the evening (I hardly drink any at home :-P).

If you are interested you can also see my raw notes, but beware that there are mostly just scribbling.

Now, off to next week’s Canonical meeting in Washington, DC!

Read more

Yesterday at GopherCon I had the chance to sit together with Dave Cheney and Jamu Kakar to judge the entries received for the Go QML Contest. The result was already announced today, live at the second day of GopherCon, including a short demo on stage of the three most relevant entries received. This blog post provides more details for the winner and also for a few of these additional entries.


The winner of the contest is Robert Nieren with the entry gofusion, a tight and polished clone of the game 2048:


The game code is worth looking into. It’s clean, has all the logic written in Go while making good use of the declarative features of QML, and leverages the OpenGL support of the qml package for drawing the tiles. It’s also fun to play!

Runner ups

Two other contest entries were demonstrated during GopherCon. The first one was Mark Saward’s multi-player game King’s Epic:

King's Epic

King's Epic

Deciding between Mark’s entry and Robert’s was not easy. King’s Epic is a much more ambitious project, and Mark did an impressive job on short notice. It’s also intended to be a real multi-platform game, which is something we really hope to see happening. For the contest, the fact that there were a few issues while running the code and that the interesting interactions are still in development have weighted against it.

The source code for the King’s Epic client is available at GitHub, and there’s an introduction that explains how to get started on it.

The second runner up demonstrated during the conference was Maxim Kouprianov’s teg-workshop:


Maxim’s entry is a Timed Event Graphs editor, and is being developed as part of his bachelor’s thesis. Although the topic is unknown to us, the interactions with the tool looked very attractive, as demonstrated in the video provided with the entry. What weighted against it was mainly that the rendering of these interactions was implemented in Javascript, while they might have been easily implemented in Go instead.

Honorable mention

Although it was not demonstrated during GopherCon, Zev Goldstein’s submission not only deserves being mentioned, but we also hope to see it being improved and made into a complete application. Zev submitted a very early version of Fallback Messenger:


The goal of Fallback Messenger is to be a multi-protocol messenger for mobile phones, and at the moment it is able to communicate with Google Talk over XMPP. It’s a promising idea and a great start, but still in very early stages as Zev pointed out during the submission.


It was a pleasure to interact with some of the participants over these two months in which the contest was run. It was also inspiring to see both the level of the entries, and how most of the entries were or became long term projects.

In terms of Go QML, these entries provide a clear message: although Go was designed with server-side systems in mind, it is by no means limited to that.

Thanks to everyone that submitted entries for the contest, and to Dave Cheney and Jamu Kakar for their time as judges during the tight schedule of GopherCon.

Read more

Go QML Contest

UPDATE: The contest result is out.

A couple of weeks ago a probe message was sent to a few places questioning whether there would be enough interest on a development contest involving Go QML applications. Since the result was quite positive, we’re moving the idea forward!

OpenGL Gopher

This blog post provides further information on how to participate. If you have any other questions not covered here, or want technical help with your application, please get in touch via the mailing list or twitter.

Eligible applications

Participating applications must be developed in Go with a graphic interface that leverages the qml package, and must be made publicly available under an Open Source license approved by the OSI.

The application may be developed on Linux, Mac OS, or Windows.

Review criteria

Applications will be judged under three lenses:

  • Quality
  • Features
  • Innovation

We realize the time is short, so please submit even if you haven’t managed to get everything you wanted in place.


All submissions should be made before the daylight of the Monday on April 21st.

Sending the application

Applications must be submitted via email including:

  • URL for the source code
  • Build instructions

If necessary for judging, a screencast may be explicitly requested, and must be provided before the daylight of Wednesday 23rd.


There are two prizes for the winner:

Judging and announcement

Judging will be done by well known developers from the Go community, and will take place during GopherCon. The result will be announced during the conference as well, but the winner doesn’t have to be in the conference to participate in the contest.

Learning more about Go QML

The best places to start are the package documentation and the mailing list.

Read more

The Ubuntu Developer Summit was held in Copenhagen last week, to discuss plans for the next six-month cycle of Ubuntu. This was the most productive UDS that I've been to — maybe it was the shorter four-day schedule, or the overlap with Linaro Connect, but it sure felt like a whirlwind of activity.

I thought I'd share some details about some of the sessions that cover areas I'm working on at the moment. In no particular order:

Improving cross-compilation

Blueprint: foundations-r-improve-cross-compilation

This plan is a part of a mutli-cycle effort to improve cross-compilation support in Ubuntu. Progress is generally going well — the consensus from the session was that the components are fairly close to complete, but we still need some work to pull those parts together into something usable.

So, this cycle we'll be working on getting that done. While we have a few bugfixes and infrastructure updates to do, one significant part of this cycle’s work will be to document the “best-practices” for cross builds in Ubuntu, on This process will be heavily based on existing pages on the Linaro wiki. Because most of the support for cross-building is already done, the actual process for cross-building should be fairly straightforward, but needs to be defined somewhere.

I'll post an update when we have a working draft on the Ubuntu wiki, stay tuned for details.

Rapid archive bringup for new hardware

Blueprint: foundations-r-rapid-archive-bringup

I'd really like for there to be a way to get an Ubuntu archive built “from scratch”, to enable custom toolchain/libc/other system components to be built and tested. This is typically useful when bringing up new hardware, or testing rebuilds with new compiler settings. Because we may be dealing with new hardware, doing this bootstrap through cross-compilation is something we'd like too.

Eventually, it would be great to have something as straightforward as the OpenEmbedded or OpenWRT build process to construct a repository with a core set of Ubuntu packages (say, minbase), for previously-unsupported hardware.

The archive bootstrap process isn't done often, and can require a large amount of manual intervention. At present, there's only a couple of folks who know how to get it working. The plan here is to document the bootstrap process in this cycle, so that others can replicate the process, and possibly improve the bits that are a little too janky for general consumption.

ARM64 / ARMv8 / aarch64 support

Blueprint: foundations-r-aarch64

This session is an update for progress on the support for ARMv8 processors in Ubuntu. While no general-purpose hardware exists at the moment, we want to have all the pieces ready for when we start seeing initial implementations. Because we don't have hardware yet, this work has to be done in a cross-build environment; another reason to keep on with the foundations-r-improve-cross-compilation plan!

So far, toolchain progress is going well, with initial cross toolchains available for quantal.

Although kernel support isn’t urgent at the moment, we’ll be building an initial kernel-headers package for aarch64. There's also a plan to get a page listing the aarch64-cross build status of core packages, so we'll know what is blocked for 64-bit ARM enablement.

We’ve also got a bunch of workitems for volunteers to fix cross-build issues as they arise. If you're interested, add a workitem in the blueprint, and keep an eye on it for updates.

Secure boot support in Ubuntu

Blueprint: foundations-r-secure-boot

This session covered progress of secure boot support as at the 12.10 Quantal Quetzal release, items that are planned for 13.04, and backports for 12.04.2.

As for 12.10, we’ve got the significant components of secure boot support into the release — the signed boot chain. The one part that hasn't hit 12.10 yet is the certificate management & update infrastructure, but that is planned to reach 12.10 by way of a not-too-distant-future update.

The foundations team also mentioned that they were starting the 12.04.2 backport right after UDS, which will bring secure boot support to our current “Long Term Support” (LTS) release. Since the LTS release is often preferred Ubuntu preinstall situations, this may be used as a base for hardware enablement on secure boot machines. Combined with the certificate management tools (described at sbkeysync & maintaining uefi key databases), and the requirement for “custom mode” in general-purpose hardware, this will allow for user-defined trust configuration in an LTS release.

As for 13.04, we're planning to update the shim package to a more recent version, which will have Matthew Garrett's work on the Machine Owner Key plan from SuSE.

We're also planning to figure out support for signed kernel modules, for users who wish to verify all kernel-level code. Of course, this will mean some changes to things like DKMS, which run custom module builds outside of the normal Ubuntu packages.

Netboot with secure boot is still in progress, and will require some fixes to GRUB2.

And finally, the sbsigntools codebase could do with some new testcases, particularly for the PE/COFF parsing code. If you're interested in contributing, please contact me at

Read more

Yesterday, GUADEC hosted a PyGObject hackfest. I was really happy to see so many participants, and a lot of whom who are rather new to the project. I originally feared that it would just be the core crew of four people, as this is not exactly the shiniest part of GNOME development.

So I did not work on the stuff I was planning for, but instead walked around and provided mentoring, help, and patch review. Unfortunately I do not know all the results from the participants, hopefully they will blog some details themselves. But this is what I was involved in:

  • Manuel Quiñones added an gtk_tree_view_column_set_attributes() override (the original C function uses varargs and thus is not introspectable). Most time was spent figuring out an appropriate test case.
  • I showed Didier Roche some tricks about porting a pygtk application to PyGI/GTK3. He gave a shot to porting Meld, but unfortunately it uses a lot of pygtk hacks/tricks, most of which are obsolete now. So this proved too big a project for one day eventually :-(
  • Paolo and I guided Marta Maria Casetti, one of this year’s GNOME GSoC students, through her first pygobject patch. The test case still needs some love (again, nothing regarding GtkTreeView is easy), but the actual patch is good. Thanks Marta for participating, and not getting intimidated by all the new stuff!
  • While working on above patch, Marta encountered a rather curious TypeError: Expected Gtk.TreeViewColumn, but got GObjectMeta when writing the override. What seemed to be a trivial problem at first quickly turned into an one-hour debugging session involving grandmaster John Palmieri and me, with others chipping in as well. In the end it (of course!) turned out to be a trivial four-character change in Marta’s patch, but it was fun to get to understand the problem (a loong-forgotten special case of overrides resolution in overrides code). Now pygobject gives a proper error message which is actually helpful, i. e. which argument causes the problem and which module/class/method is provided, which should prevent us from being misguided into the totally wrong direction the next time this happens.
  • John Stowers got the Windows build working again, and showed off the gtk-demo under Windows. This is really amazing, I hope we can get that into trunk soon and not let it bitrot again for so long. Thanks!
  • Simon and Manuel worked on porting some Sugar extensions. Together with Paolo we also discussed the GStreamer 1.0 API a bit, which parts can become API additions and which need to become overrides.
  • Michal Hruby debugged a leak in the handling of GVariant arrays when using libdee.

Thanks everyone for participating! I hope everyone enjoyed it and got to learn a new thing or two. See you at the next one!

PyGObject hackfest at GUADEC 2012

PyGObject hackfest at GUADEC 2012

Read more

I have had the pleasure of attending GUADEC in full this year. TL;DR: A lot of great presentations + lots of hall conversations about QA stuff + the obligatory be{er,ach} = ?.

Last year I just went to the hackfest, and I never made it to any previous one, so GUADEC 2012 was a kind of first-time experience for me. It was great to put some faces and personal impressions to a lot of the people I have worked with for many years, as well as catching up with others that I did meet before.

I discussed various hardware/software testing stuff with Colin Walters, Matthias Clasen, Lennart Poettering, Bertrand Lorentz, and others, so I have a better idea how to proceed with automated testing in plumbing and GNOME now. It was also really nice to meet my fellow PyGObject co-maintainer Paolo Borelli, as well as seeing Simon Schampier and Ignacio Casal Quinteiro again.

No need to replicate the whole schedule here (see for yourself on the interwebs), so I just want to point out some personal highlights in the presentations:

  • Jacob Appelbaum’s keynote about Tor brought up some surprising facts about how the project has outgrown its past performance problems and how useful it was during e. g. the Arab revolution
  • .

  • Philip Whitnall’s presented Bendy Bus, a tool to mock D-Bus services for both unit and fuzz testing. He successfully used it to find and replicate bugs in Evolution (by mocking evolution-data-server) as well as libfolks (by mocking the telepathy daemons). It should work just as well to mock system services like upower or NetworkManager to test the UI bits that use it. This is a topic which has been on my wishlist for a long time already, so I’m happy that there is already an existing solution out there. We might have to add some small features to it, but it’s by and large what I had in mind, and in the discussion afterwards Philip said he’d appreciate patches against it.
  • Christophe Fergeau showed how to easily do Windows builds and installers from GNOME tarballs with MinGW-w64, without having to actually touch/use Windows (using cross-building and running tests etc. under wine). I found it surprising how easy that actually is, and it should not be hard to integrate that in a jhbuild-like setup, so that it does not keep breaking every time.
  • Colin Walters gave an introduction to OSTree, a project to build bootable images from kernel/plumbing/desktop upstream git heads on a daily basis. This is mostly to avoid the long delays that we otherwise have with doing upstream releases, packaging them, and getting them into a form that can safely be tested by users. In an afterwards discussion we threw some ideas around how we can integrate existing and future tests into this (something in spirit like our autopkgtest). This will be the area where I’ll put most focus on in the next time.
  • Adam Dingle of yorba fame shared his thoughts about how we can crowdfunding of Free Software Projects work in practice, comparing efforts like codefoundry and kickstarter. Of course he does not have a solution for this yet, but he raised some interesting concerns and it spun off lots of good discussions over lunchtime.
  • Last but not least, the sport event on Saturday evening was awesome! In hindsight I was happy to not have signed up for soccer, as people like Bastian or Jordi played this really seriously. I participated in the Basketball competition instead, which was the right mix of fun and competition without seriously trying to hurt each other. :-)

There were a lot of other good ones, some of them technical and some amusing and enlightening, such as Frederico’s review of the history of GNOME.

On Monday I prepared and participated in a PyGObject hackfest, but I’ll blog about this separately.

I want to thank all presenters for the excellent program, as well as the tireless GUADEC organizer team for making everything so smooth and working flawlessly! Great Job, and see you again in Strasbourg or Brno!

Read more

I just received confirmation that my request for a PyGObject hackfest has been approved by the GUADEC organizers.

If you are developing GObject-introspection based Python applications and have some problems with PyGObject, this is the time and place to get to know each other, getting bugs fixed, learn about pygobject’s innards, or update libraries to become introspectable. I will prepare a list of easy things to look into if you are interested in learning about and getting involved in PyGObject’s development.

See you on July 30th in A Coruña!


Read more

Phew, tiring trip to PyCon this year. This was my second year after hitting up my first last year. The conference definitely felt larger than last year as they crossed 2,200 attendees. It’s unbelievable to see how large the Python community has gotten. I can’t stress what great job the people that put this together.

Last year I hardly knew anyone. This year, however, I got to put faces to people I’ve interacted with over the last year, welcome back those I met last year, and get some face to face time with new co-workers from Canonical. The social aspect was a larger chunk of my time this year for sure.

Side note, I listen to The Changelog podcast from time to time, and I love their question on who you’d love to pair up/hack with as a programming hero type question. I got to meet and greet mine at this PyCon by meeting up with Mike Bayer. He’s behind some great tools like SqlAlchemy and Mako. What I love is that, not only does he rock the code part, but the community part as well. I’m always amazed to see the time he puts into his responses to questions and support avenues. Highlight of my PyCon for sure.

I’ll post a seperate blog post on my sprint notes. I feel that if you’re going to go, you might as well stay for sprints. I get as much out of that as the conference parts itself. I think I made some good progress on things for Bookie this year. The big thing is that an invite system is in place, so if you’d like an account on let me know and I’ll toss an invite your way.


  • Introduction to Metaclasses
    • Basic but reminded me how the bits worked and had some good examples. I like this because I often write ‘the code I want to be writing’ and then write my modules/etc to fit and metaclasses help with this sometimes.
  • Fast Test, Slow Test
    • Just a reminder that fast tests are true unit tests and run during dev which helps make things easier/faster as you go vs the whole ‘mad code’ then wait for feedback on how wrong you are.
  • Practical Machine Learning in Python
    • – check out for lots of notes/etc on ML in OSS
    • – teach me some ML please
    • sluggerml – app he built as a ML demo
    • scikit-learn : lots of potential, very active right now
  • Introduction to PDB
    • whoa…where have you been all my life ‘until’ command?
    • use ‘where’ more to move up stack vs adding more debug lines
  • Flexing SQLAlchemy’s Relational Power
  • Hand Coded Applications with SQLAlchemy
    • <3"><3 SqlAchemy. Some really good examples of writing less code by automating the biolerplate with conventions.
  • Web Server Bottlenecks And Performance Tuning
    • lesson: if you think it’s apache’s fault think again. You’re probably doing it wrong.
  • Advanced Celery
    • check out cyme, possible way to more easily run/distribute celery work?
    • cool to see implementations of map/reduce using celery
    • chords and groups are good, check them out more
  • Building A Python-Based Search Engine
    • Good talk for into into terms and such for fulltext search
  • Lighting talks of note
    • py3 porting docs:
    • bpython rewind feature is full of win over ipython
    • ‘new virtualenv’ trying to get into stdlib for py3.3, cool!
    • asyncdynamo cool example of async boto requests for high performance working with AWS api (uses tornado)
    • I WANT the concurrent.features library…but it’s Python 3 :(

Read more
Gustavo Niemeyer

Back at the Ubuntu Platform Rally last week, I’ve pestered some of the Bazaar team with questions about co-location of branches in the same directory with Bazaar. The great news is that this seems to be really coming for the next release, with first-class integration of the feature in the command set. Unfortunately, though, it’s not quite yet ready for prime time, or even for I’m-crazy-and-want-this-feature time.

Some background on why this feature turns out to be quite important right now may be interesting, since life with Bazaar in the past years hasn’t really brought that up as a blocker. The cause for the new interest lies in some recent changes in the toolset of the Go language. The new go tool not only makes building and interacting with Go packages a breeze, but it also solves a class of problems previously existent. For the go tool to work, though, it requires the use of $GOPATH consistently, and this means that the package has to live in a well defined directory. The traditional way that Bazaar manages branches into their own directories becomes a deal breaker then.

So, last week I had the chance to exchange some ideas with Jelmer Vernooij and Vincent Ladeuil (both Bazaar hackers) on these problems, and they introduced me to the approach of using lightweight checkouts to workaround some of the limitations. Lightweight checkouts in Bazaar makes the working tree resemble a little bit the old-style VCS tools, with the working tree being bound to another location that actually has the core content. The idea is great, and given how well lightweight checkouts work with Bazaar, building a full fledged solution shouldn’t be a lot of work really.

After that conversation, I’ve put a trivial hack together that would make bzr look like git from the outside, by wrapping the command line, and did a lightning talk demo. This got a few more people interested on the concept, which was enough motivation for me to move the idea forward onto a working implementation. Now I just needed the time to do it, but it wasn’t too hard to find it either.

I happen to be part of the unlucky group that too often takes more than 24 hours to get back home from these events. This is not entirely bad, though.. I also happen to be part of the lucky group that can code while flying and riding buses as means to relieve the boredom (reading helps too). This time around, cobzr became the implementation of choice, and given ~10 hours of coding, we have a very neat and over-engineered wrapper for the bzr command.

The core of the implementation is the same as the original hack: wrap bzr and call it from outside to restructure the tree. That said, rather than being entirely lazy and hackish line parsing, it actually parses bzr’s –help output for commands to build a base of supported options, and parses the command line exactly like Bazaar itself would, validating options as it goes and distinguishing between flags with arguments from positional parameters. That enables the proxying to do much more interesting work on the intercepted arguments.

Here is a quick session that shows a branch being created with the tool. It should look fairly familiar for someone used to git:

[~]% bzr branch lp:juju
Branched 443 revisions.

[~]% cd juju
[~/juju]% bzr branch
* master

[~/juju]% bzr checkout -b new-feature
Shared repository with trees (format: 2a)
shared repository: .bzr/cobzr
Branched 443 revisions.
Branched 443 revisions.
Tree is up to date at revision 443.
Switched to branch: /home/niemeyer/juju/.bzr/cobzr/new-feature/

[~/juju]% bzr branch other-feature
Branched 443 revisions.

[~/juju]% bzr branch
* new-feature

Note that cobzr will not reorganize the tree layout before the multiple branch support is required.

Even though the wrapping is taking place and bzr’s –help output is parsed, there’s pretty much no noticeable overhead given the use of Go for the implementation and also that the processed output of –help is cached (I said it was overengineered).

As an example, the first is the real bzr, while the second is a link to cobzr:

[~/juju]% time /usr/bin/bzr status
/usr/bin/bzr status 0.24s user 0.03s system 88% cpu 0.304 total

[~/juju]% time bzr status
bzr status 0.19s user 0.08s system 88% cpu 0.307 total

This should be more than enough for surviving comfortably until bzr itself comes along with first class support for co-located branches in the next release.

In case you’re interested in using it or are just curious about the command set or other details, please check out the web page for the project:

Read more

7 years ago, The Ubuntu 4.10 “The Warty Warthog” was announced. A huge congrats to the community, Canonical, and especially Mark for getting so far from “there” to “here”.

This brings back old memories of my first conference in Oxford in August, the great-great-grandfather to what is UDS these days. Back then, there was no company, no Launchpad, no Blueprints, no work items, no detailled plans, just a bunch of ideas, BoFs, and this was a third of the entire crowd:

Warty Hack Room

Back then we worked on the famous TRLS technology (“Totally Rad Laptop Support”) and were proud when we got the ThinkPads to suspend once. During that conference I wrote pmount to provide automatic mounting of USB sticks in a safe manner. Those were the days… :-)

But I can also safely say that there are some things that haven’t changed. Even though both the community and the company (which changed away from recently) grew by two magnitudes since then, we still have the same serious attitude, stern look, and formal attire as we had back then:

We are professionals, really!

We are professionals, really!

Read more

(Update: Link to Tomeu’s blog post, repost for

Last week I was in Prague to attend the GNOME/Python 2011 Hackfest for gobject-introspection, to which Tomeu Vizoso kindly invited me after I started working with PyGI some months ago. It happened at a place called brmlab which was quite the right environment for a bunch of 9 hackers: Some comfy couches and chairs, soldering irons, lots of old TV tubes, chips, and other electronics, a big Pirate flag, really good Wifi, plenty of Club Mate and Coke supplies, and not putting unnecessary effort into mundane things like wallpapers.

It was really nice to get to know the upstream experts John (J5) Palmieri and Tomeu Vizoso (check out Tomeu’s blog post for his summary and some really nice photos). When sitting together in a room, fully focussing on this area for a full week, it’s so much easier to just ask them about something and getting things done and into upstream than on IRC or bugzilla, where you don’t know each other personally. I certainly learned a lot this week (and not only how great Czech beer tastes :-) )!

So what did I do?

Application porting

After already having ported four Ubuntu PyGTK applications to GI before (apport, jockey, aptdaemon, and language-selector),
my main goal and occupation during this week was to start porting a bigger PyGTK application. I picked system-config-printer, as it’s two magnitudes bigger than the previous projects, exercises quite a lot more of the GTK GI bindings, and thus also exposes a lot more GTK annotation and pygobject bugs. This resulted in a new pygi s-c-p branch which has the first 100 rounds of “test, break, fix” iterations. It now at least starts, and you can do a number of things with it, but a lot of functionality is still broken.

As a kind of “finger exercise” and also to check for how well pygi-convert works for small projects now, I also ported computer-janitor. This went really well (I had it working after about 30 minutes), and also led me to finally fixing the unicode vs. str mess for GtkTreeView that you got so far with Python 2.x.

pygobject and GTK fixes

Porting system-config-printer and computer-janitor uncovered a lot of opportunities to improve, a big “perl -e” kind of script to do the mechanical grunt work of the porting process. It doesn’t fix up changed signatures (such as adding missing arguments which were default arguments in PyGTK, or the ubiquitous “user_data” argument for signal handlers), but at least it gets a lot of namespaces, method, and constant names right.

I also fixed three annotation fixes in GTK+. We also collaboratively reviewed and tested Pavel’s annotation branch which helped to fix tons of problems, especially after Steve Frécinaux’s excellent reference leak fix, so if you play around with current pygobject git head, you really also have to use the current GTK+ git head.

Speaking of which, if you want to port applications and always stay on top of the pygobject/GTK development without having to clutter your package system with “make install”s of those, it works very well to have this in your ~/.bashrc:

export GI_TYPELIB_PATH=$HOME/projects/gtk/gtk:$HOME/projects/gtk/gdk
export PYTHONPATH=$HOME/projects/pygobject

Better GVariant/GDBus support

The GNOME world is moving from the old dbus-glib python bindings to GDBus, which is integrated into GLib. However, dbus-python exposed a really nice and convenient way of doing D-Bus calls, while using GDBus from Python was hideously complicated, especially for nontrivial arguments with empty or nested arrays:

from gi.repository import Gio, GLib
from gi._gi import variant_type_from_string

d = Gio.bus_get_sync(Gio.BusType.SESSION, None)
notify = Gio.DBusProxy.new_sync(d, 0, None, 'org.freedesktop.Notifications',
    '/org/freedesktop/Notifications', 'org.freedesktop.Notifications', None)

vb = GLib.VariantBuilder()
vb.add_value(GLib.Variant('s', 'test'))
vb.add_value(GLib.Variant('u', 1))
vb.add_value(GLib.Variant('s', 'gtk-ok'))
vb.add_value(GLib.Variant('s', 'Hello World!'))
vb.add_value(GLib.Variant('s', 'Subtext'))
# add an empty array
eavb = GLib.VariantBuilder()
# add an empty dict
eavb = GLib.VariantBuilder()
vb.add_value(GLib.Variant('i', 10000))
args = vb.end()

result = notify.call_sync('Notify', args, 0, -1, None)
id = result.get_child_value(0).get_uint32()
print id

So I went to making the GLib.Variant constructor work properly with nested types and boxed variants, adding Pythonic GVariant iterators and indexing (so that you can treat GVariant dictionaries/arrays/tuples just like their Python equivalents), and finally a Variant.unpack() method for converting the return value of a D-Bus call back into a native Python data type. This looks a lot friendlier now:

from gi.repository import Gio, GLib

d = Gio.bus_get_sync(Gio.BusType.SESSION, None)
notify = Gio.DBusProxy.new_sync(d, 0, None, 'org.freedesktop.Notifications',
    '/org/freedesktop/Notifications', 'org.freedesktop.Notifications', None)

args = GLib.Variant('(susssasa{sv}i)', ('test', 1, 'gtk-ok', 'Hello World!',
    'Subtext', [], {}, 10000))
result = notify.call_sync('Notify', args, 0, -1, None)
id = result.unpack()[0]
print id

I also prepared another patch in GNOME#640181 which will provide the icing on the cake, i. e. handle the variant building/unpacking transparently and make the explicit call_sync() unnecessary:

from gi.repository import Gio, GLib

d = Gio.bus_get_sync(Gio.BusType.SESSION, None)
notify = Gio.DBusProxy.new_sync(d, 0, None, 'org.freedesktop.Notifications',
    '/org/freedesktop/Notifications', 'org.freedesktop.Notifications', None)

result = notify.Notify('(susssasa{sv}i)', 'test', 1, 'gtk-ok', 'Hello World!',
            'Subtext', [], {}, 10000)
print result[0]

I hope that I can get this reviewed and land this soon.

Thanks to our sponsors!

Many thanks to the GNOME Foundation and Collabora for sponsoring this event!

Read more