Canonical Voices

Posts tagged with 'launchpad'


I often do upstream releases of my upstream projects that I do on Launchpad, mostly for Apport and jockey. But doing this has been quite tedious until now: You have to go to the project page, pick the series (usually “trunk”), create a new release, create a new milestone along the way, then go to “add download file”, and upload your .tar.gz and .tar.gz.asc.

Because this is rather inconvenient, I don’t do as many upstream releases as I should. But thanks to our tireless launchpadlib developers it is now possible to automate all that, so I wrote a new script lp-project-upload which does all that in a simple command:

  $ lp-project-upload apport 1.8.2 apport-1.8.2.tar.gz
  Release 1.8.2 could not be found for project. Create it? (Y/n) y

The script is based on Brad Crittenden’s recipe for uploading project files, and I added the creation of milestones and releases.

The script is contained in current Karmic’s ubuntu-dev-tools package now. Enjoy, and of course feel free to extend it for changelogs, release notes, etc.

Read more
James Henstridge

Today Ohloh finished importing the Launchpad source code and produced the first source code analysis report.  There seems to be something fishy about the reported line counts (e.g. -3,291 lines of SQL), but the commit counts and contributor list look about right.  If you’re interested in what sort of effort goes into producing an application like Launchpad, then it is worth a look.

Read more

Following up on our survey on icons in Launchpad, Charline Poirier provides us with the outcome:

Edit edit:  High level of understanding, but a strong association with “attention”, “warning”, and “danger”.  Might be worth modifying colour or shape to distance the icon from that interpretation.

Merge merge-proposal-icon:  Reasonable understanding of “merge”.  However, participants were not entirely sure if the icon referred to the state ‘merged’ or the branches themselves.

Remove remove:  Icon strongly associated with “do not enter” and “delete”.  The interpretation “remove” comes only in third place.  The icon is strongly evocative and might be better used to designate a more consequential or prohibitive action.

Remote bug bug-remote:  A reasonable percentage of  respondents understood the “remote bug” icon.  Many, however, did not.  It appears that the key for interpreting this icon is the representation of the bug itself.  Various potential states of a bug were suggested as interpretations.  This icon could be made more explicit.

External link link:  Relatively well understood.  It is worth noting that the icon has powerful suggestion of globality and reach (associated with translation, languages, internationalization, etc).   It is a very evocative icon that could be more fully exploited perhaps in another context.

What next?  We’ll attempt to create new versions of the icons, run another session of user-testing, and if understanding improves, Launchpad gets new icons  \0/

Read more

Following up on my last post about user testing icons, it has been incredibly successful!  We’ve had over 100 responses, and are now going through the data to put together a summary. I will post information on our findings as soon as we finish the work.

In the mean time, Charline Poirier, who is in charge of user testing in our team, has created another survey with 5 more icons to help us get more data. If everyone could give this survey another spin, and create some networking effects to help spread the survey to non-Launchpad users, it would be tremendously helpful to us. Here’s the link:

Read more

We’re trying to improve the icons we have in Launchpad so they’re more usable across different cultures and types of users, and our first step is to do some user testing on our current icons.

The Canonical User Experience team has set up a survey to gather information on how users see our icons, so if you have a few spare minutes (it’s very quick!), please take the survey and pass it on to other people, especially if they don’t use Launchpad, as they will be less biased.

Survey is available at:

Read more

Soyuz 3.0 review

The Soyuz team was very, very busy over the last year fixing lots of bugs and adding plenty of new features.  These are the highlights that I’d like to mention!

New features

  • Multiple PPAs per person – split your packages into different repositories without the hassle of creating new LP users
  • Karma for uploads! Distro and PPA uploaders (and the package creator if different) will be recognised for their work and get karma.
  • Massively improved webservice APIs to control various operations, such as copying packages, manipulating builds, inspecting PPAs etc.  Allows script-based control of many soyuz-related operations.
  • Hugely faster build farm scanner, builds are dispatched a LOT quicker now.  That means there’s less waiting around for your packages to get built.
  • Private PPA subscriptions / tokens.  People can now control who is able to download their PPA software.
  • Package sets and their upload ACLs implemented for Ubuntu.  Karmic and onwards will be using package sets for upload permissioning.
  • Security in Soyuz. Complete support for the Canonical security team to use a private PPA and directly unembargo packages from it

3.0 UI

We also did a lot of complete page redesigns for 3.0:

  • PPA page split into two pages; one user-focused, one developer-focused
  • New build page, with a cleaner look
  • New global /builders page; sortable table data and a time-based estimate of the queue sizes
  • New distribution source package release page; much better presentation of data
  • New distribution source package page; makes more use of upstream package description/logo etc.
  • Distro series source package release page gets a new layout
  • Builder +admin and builder +edit collapsed into a single page
  • New builder page

But the work doesn’t stop here.  We’re already thinking about 4.0!

Read more

Thanks to the hard work from everyone in the Soyuz team, the Soyuz effort to convert all our templates to the new 3.0 UI standard is all done!  If you’re a beta tester, the new pages can be seen on on the edge service already.

Many of the changes were complete page re-designs based on better knowledge of use-cases that has been gathered over the years.  There was certainly a lot of dust on some of them!

There are still some minor tweaks to be made, but all feedback is most welcome.

Read more

I came out of the dark after so long just to say: Launchpad is now open source!

For those who don’t know Launchpad, you should check out the Launchpad tour, here. It’s full of awesomeness. </p>
            <a href=Read more

Joey Stanford

Launchpad is now Open Source!  Congratulations Launchpad Team!

Read the announcement:

Join the fun: #launchpad-dev on freenode

Read more

As promised, Launchpad has been fully open sourced (as opposed to the initial idea, nothing has been held back). Get it now, fix your favorite pet bug, and improve tens thousands of people’s experience.

Mark Shuttleworth really deserves a lot of praise for this bold and brave move, open sourcing not only the code, but all  it’s history. It’s a fantastic day today.

Update: yes, fully means including soyuz and codehosting, Mark has decided to release everything. The whole history is there.

See the loggerhead page:


Read more
Tim Penhey (thumper)

Breaking up work for review

It was Friday morning after three days of working on one feature. Last thing Thurday I counted the size of the change and it was over 1100 lines and I wasn't quite finished. I found myself in the situation that turns up periodically that I wanted to break up my work into cohesive reviewable chunks. Now it isn't a matter of taking commits x through y as chunk one, and so on, as the size grew organically as I changed what needed to be changed, and wrote what needed to be written without really stopping to think about the size of the change until it was done. However now it was done, I wanted to break it up.

Last time I did this, I used looms, but Aaron told me we could do it easily using his new Bazaar pipeline plugin. So I spent some time talking through with Aaron on how to do it, promising to write it up if it worked well. I must say that it was good. During the process we identified a number of enhancements to the plug in to make it even easier.

I'm going to show the progression we made, along with our thoughts. I have trimmed some of the output when I've decided that it doesn't add value.

The first thing I had to do was to get the pipeline plugin.

$ bzr branch lp:bzr-pipeline ~/.bazaar/plugins/pipeline

Unfortunately this seemed to clash slightly with the QBzr plugin. The were both trying to redefine merge. Personally I don't use QBzr and had probably just installed it to take a look, so I removed that plugin.

Caution: the pipeline plugin relies on switch so works with lightweight checkouts. This is how I work anyway, so I didn't have anything to do here, but if you work differently, YMMV.

The pipeline plugin is designed around having a set of branches one after (individual pipes) the other that perform a pipeline, clever eh? When you have the pipeline plugin, any branch is also considered a pipeline of one.

$ bzr show-pipeline
* nice-distribution-code-listing

What I was wanting to do was to break up this work into a number of distinct change sets, each that could be reviewed independently. We decided that the way to do this was to create a pipe before the current one, and bring changes in. This is done with the command add-pipe.

$ bzr add-pipe factory-tweaks --before nice-distribution-code-listing
$ bzr show-pipeline
* nice-distribution-code-listing

Right here we decided that there should be an easier way to add a pipe before the current pipe, as right now it needs a pipe name. A bug was filed to track this.

You can see from the show-pipeline command that the new pipe is before the current one. The pipeline plugin addes a number of branch aliases:

  • :first - the first pipe in the pipeline

  • :prev - the pipe before the current pipe

  • :next - the pipe after the current pipe

  • :last - the last pipe in the pipeline

Now to make the switch to the first pipe. Both :prev and :first refer to the same branch here, and I could have used either.

$ bzr switch :prev
... changed files shown
All changes applied successfully.
Now on revision 8747.

Now this pipe was added from the pipe after it, so it starts off with the same head revision. Not exactly the starting point I wanted, so we replaced the head of this branch with the last revision of the trunk branch that we had merged in.

$ bzr pull --overwrite -r submit:

The submit: alias refers to the submit branch. This is often trunk, and is in my project layout (specified using submit_branch in .bazaar/locations.conf).

Now the lower pipe was a copy of trunk. A good place to start adding changes I think. The next problem was how to get the changes from the following pipe into this one. Our first attempt was to merge in the following branch, shelve what we didn't want, throw away the actual merge, but keep the changed text, and commit.

$ bzr merge :next
$ bzr shelve
$ bzr stat
pending merge tips: (use -v to see all merge revisions)
Tim Penhey 2009-07-02 New view added.
$ bzr revert --forget-merges
$ bzr stat
$ bzr commit -m "More default args to factory methods and whitespace cleanup."

Now this seemed very convoluted. Why merge and then forget the merge? I seemed kinda icky, but it worked. The next thing to do is to merge these changes down the pipeline. This is done through another command pump.

$ bzr pump

This merges and commits the changes down the pipeline. If there are conflicts, it stops and leaves you in the conflicted pipe. This didn't occur here, nor did it occur for any of my other ones. Here you can see the commit message that pump used:

$ bzr switch :next
$ bzr log --line -r -1
8714: Tim Penhey 2009-07-03 [merge] Merged factory-tweaks into nice-distribution-code-listing.
$ bzr switch :prev

Now it was time to add the next pipe.

$ bzr add-pipe code-test-helpers
$ bzr show-pipeline
* factory-tweaks
$ bzr switch :next

This time, instead of merging in the changes, we shelved them in. The shelve command. The shelve command can apply changes from arbitrary revisions, and it also knows about files. The change that I wanted in this branch was a single added file, so I could tell shelve about that file.

$ bzr shelve -r branch::next lib/lp/code/tests/
Selected changes:
+ lib/lp/code/tests/
Changes shelved with id "2".

However the big problem with this is it all looks backwards. We are shelving from the future not the past. This really did my head in. Shelve would say "remove this file?" and by shelving it, it would add it in. It worked but made my head fuzzy. We filed a bug about this too. By adding a better way to take the changes, the command could do the reversal for you and provide you with a nicer question.

More of the same happened for the next few pipes, and I won't bore you with repeated commands.

On the whole, the pipeline plugin worked really well. I was able to break my work up into five hunks which could be reviewed easily. In the end I kept working on the branch that was my original, so all my original history remained intact. It would have been just as easy to add another pipe and take the remaining changes. This would have left me with five branches, each with one commit. This works well for the way we work as we have reviews based on branches. Each pipe could be pushed to Launchpad and a review initiated for it. With some more UI polish, I think pipelines will be even more awesome than I think they are now.

Read more

One of the things that’s frustrating about being a Launchpad Developer is waiting three hours for the test suite to complete.  It’s always irked me that my quad-core sat there with three idle cores, and parallelising the tests would be tough, so I thought I’d try and partition the tests across some virtual machines.

I know pretty much nothing about VMs so I was pretty pleased to come across this JeOS VMBuilder resource!  I got my first VM built pretty quick with it, but then, what next?  The instructions don’t tell me how to actually run up my new VM.

So, I had a little chuckle to myself when I saw this blog post from Dustin Kirkland just appear!  I’m going to try his suggestions out now and see how it works out, but it looks like just what I need.

Read more

Check out what happened when AJAX bug tags editing landed in April:


Note that since Launchpad has the edge/production split, so the changes in the graphs are less drastic since a set of users start interacting with new code before others.

Read more
James Henstridge

I promised Zeeshan that I’d have a look at his Rygel UPnP Media Server a few months back, and finally got around to doing so.  For anyone else who wants to give it a shot, I’ve put together some Ubuntu packages for Jaunty and Karmic in a PPA here:

Most of the packages there are just rebuilds or version updates of existing packages, but the Rygel ones were done from scratch.  It is the first Debian package I’ve put together from scratch and it wasn’t as difficult as I thought it might be.  The tips from the “Teach me packaging” workshop at the Canonical All Hands meeting last month were quite helpful.

After installing the package, you can configure it by running the “rygel-preferences” program.  The first notebook page lets you configure the transcoding support, and the second page lets you configure the various media source plugins.

I wasn’t able to get the Tracker plugin working on my system, which I think is due to Rygel expecting the older Tracker D-Bus API.  I was able to get the folder plugin working pretty easily though.

Once things were configured, I ran Rygel itself and an extra icon showed up on my PlayStation 3.  Getting folder listings was quite slow, but apparently this is limited to the folder back end and is currently being worked on.  It’s a shame I wasn’t able to test the more mature Tracker back end.

With LPCM transcoding enabled, I was able to successfully play a Vorbis file on the PS3.  With transcoding disabled, I wasn’t able to play any music — even files in formats the PS3 could handle natively.  This was apparently due to the folder backend not providing the necessary metadata.  I didn’t have any luck with MPEG2 transcoding for video.

It looks like Rygel has promise, but is not yet at a stage where it could replace something like MediaTomb.  The external D-Bus media source support looks particularly interesting.  I look forward to trying out version 0.4 when it is released.

Read more

I recently added functionality to Launchpad so that you can retry and re-score builds via the webservice API.  This was the number one requested API feature from PPA users and Ubuntu packagers alike!

It’s trivial to use:

>>> ubuntu = launchpad.distributions['ubuntu']
>>> main_archive = ubuntu.main_archive
>>> sources = main_archive.getPublishedSources(source_name='my_package_name')
>>> builds = sources[0].getBuilds()
>>> for build in builds:
...     build.rescore(score=1000)
...     build.retry()

`rescore` requires that you have buildd admin privileges.   `retry` simply requires that you have permission to upload the source.

A more advanced example is to fetch all builds that are failed in a series:

>>> karmic = ubuntu.getSeries(name_or_version="karmic")
>>> failed_states = (
...     'Failed to build',
...     'Dependency wait',
...     'Chroot problem',
...     'Failed to upload',
...     )
>>> builds = []
>>> for build_state in failed_states:
...     builds.extend(
...         karmic.getBuildRecords(build_state=build_state))

This will potentially return builds that are superseded so they need to be filtered to see if the build’s source is published.  This is something we’re going to fix soon, so that you can specify that you only want builds with published sources in the getBuildRecords() call.

Read more
Tim Penhey (thumper)

launchpadlib updates for branches

Just a quick note to yet you know of a few changes to launchpadlib for branches. Mainly because I've removed a method that I know someone is using.

You used to be able to get to the branches for a project by saying my_project.branches, but I've removed this. It would have been nicer to deprecate it but we don't have a nice deprecation method right now for launchpadlib, and since it is still in beta, I didn't feel too bad.

The branches of a project was an attribute, now we have a getBranches method. The old attribute would give you all the branches of the project, including the merged and abandoned ones. The method defaults to give you the active branches, and allows you to pass in the statuses that you'd like to get.

Also with this change you can now get the branches for a project group, or the branches owned by a person using the same getBranches method call.

Project groups also grew the method getMergeProposals in the same way that the method was already available for people and projects.

Please file any bugs on the launchpad-code project on Launchpad.

Read more
Joey Stanford

After attending the latest Canonical employee gathering (called All Hands) the behind the scenes secret of the company was plainly obvious, even in several of the brand new hires.

When you work for a traditional company where many of the employees are co-located, you often have a power structure at play which dictates to a large extent the culture. This culture usually involves some sort of dress code (often informal, peer driven) and at times the installation of utter fear and unapproachability of executives.  There is often an overlay of formalness, and some times rigidness, as well. You will often see bitter internal competition between managers and teams.

In Canonical we don’t have this. We replace all of that with a simple (unwritten) concept (or really, a culture): Brotherhood (or Fraternity if you prefer).  Everyone is your brother or sister. Everyone is approachable.  This feeling is so strong that we often hug each other in greeting and parting or at the very least give each other a two arm handshake, high five,  or a strong slap on the back.  If you thought this was the exclusive realm of Daniel Holbach, or something tied to romantic interests, think again. This is the only company I’ve worked for where I can meet the COO in the hallway and a spontaneous hug ensues, and likewise get bear hugged by one of my employees.  Even Mark, who is normally reserved, will walk up to you and give you a slap on your back and ask you have you have been.

You want proof? Scour the Internet for pictures from Canonical company events and Ubuntu UDS events. Or better yet, go to one of these yourself. Here are some pictures I took in passing last week:

Once you’ve worked in such a supportive and close-knit group it’s hard to imagine working anywhere else. The good news is that you don’t have to work for Canonical to gain access to this spirit. You can practice this at UDS and in your local Ubuntu teams. Some people may start refering to this concept as the “Church of Canonical” or some other weirdness but in fact it’s not. It’s Ubuntu. Remember, Ubuntu is an African concept of ‘humanity towards others’. It is ‘the belief in a universal bond of sharing that connects all humanity’.  Canonical simply is “drinking the Ubuntu Kool-aid” and I for one am damn proud of it and to work here.

If you have photographic evidence of this culture, please post links in the comments to this post!  Maybe some cultural anthropologist Ph.D. candidate will want to examine this further. :-)

Read more
Joey Stanford

Leading with Kindness

I did some analysis work for a group in Canonical last November and December.  It was really interesting for me and a rewarding experience but as I was writing and revising my final report I kept feeling more and more uneasy about the way I articulated my findings. I realized a bit too late that I still had some New York attitude left in me.  It’s good to be engaged and excited about what you are doing, especially when it helps others, but to do so with an aggressive posture is not.  I’ve been trying to cultivate “champa”, “Loving Kindness” in Tibetan, and “Sheshin”, “Awareness” in Tibetan,  and after all was said and done this report showed me I had more room for improvement.

One of the nice things about practicing and leading with kindness at work is that it makes your workplace a better, more enjoyable, and more productive place. One study suggests a 30% improvement in productivity.

Last night I saw a wonderful show on PBS called “Leading from Kindness”.  It’s based on the book “Leading with Kindness: How Good People Consistently Get Superior Results”. The show really resonated with me and what I’ve slowly been learning over my career.  I do believe that cultivating this style of leadership is essential in engaging and retaining superior employees as well as transforming organizations into very high performance teams.

I’ve had good luck practicing this style on my direct reports in Canonical.

The only caveat seems to be that folks who have never been in an organization that respects them as individuals can sometimes confuse kindess with weakness.  When you see this mistake being made, a gentle nudge seems to resolve it.

My hope is that others can benefit from this approach.

Read more
James Henstridge

Last week, we released the source code to django-openid-auth.  This is a small library that can add OpenID based authentication to Django applications.  It has been used for a number of internal Canonical projects, including the sprint scheduler Scott wrote for the last Ubuntu Developer Summit, so it is possible you’ve already used the code.

Rather than trying to cover all possible use cases of OpenID, it focuses on providing OpenID Relying Party support to applications using Django’s django.contrib.auth authentication system.  As such, it is usually enough to edit just two files in an existing application to enable OpenID login.

The library has a number of useful features:

  • As well as the standard method of prompting the user for an identity URL, you can configure a fixed OpenID server URL.  This is useful for deployments where OpenID is being used for single sign on, and you always want users to log in using a particular OpenID provider.  Rather than asking the user for their identity URL, they are sent directly to the provider.
  • It can be configured to automatically create accounts when new identity URLs are seen.
  • User names, full names and email addresses can be set on accounts based on data sent via the OpenID Simple Registration extension.
  • Support for Launchpad‘s Teams OpenID extension, which lets you query membership of Launchpad teams when authenticating against Launchpad’s OpenID provider.  Team memberships are mapped to Django group membership.

While the code can be used for generic OpenID login, we’ve mostly been using it for single sign on.  The hope is that it will help members of the Ubuntu and Launchpad communities reuse our authentication system in a secure fashion.

The source code can be downloaded using the following Bazaar command:

bzr branch lp:django-openid-auth

Documentation on how to integrate the library is available in the README.txt file.  The library includes some code written by Simon Willison for django-openid, and uses the same licensing terms (2 clause BSD) as that project.

Read more