Canonical Voices

Posts tagged with 'front-page'

Matthew Revell

We’re looking for a couple of smart, motivated and experienced people to join us on the Launchpad team at Canonical.

First up is a Software Engineer, to join one of the Launchpad development squads working on both new Launchpad features and maintenance of existing functionality.

There’s also an opening for a Usability and Communications Specialist. This is to join Launchpad’s Product Team, where we’re looking for someone who can run a usability research programme and produce documentation, blog posts and so on.

If you’ve got any questions about either role, feel free to grab me (mrevell) on FreeNode.

Read more
Robert Collins

I’m thrilled to be writing this blog post just over a year after starting as Launchpad’s technical architect. During that year we have been steadily improving our ability to deploy changes to Launchpad without causing downtime (of any or all services). Our ability to do this directly impacts our ability to deliver bug fixes and new functionality – our users are very sensitive to downtime.

There has been one particularly tricky holdout though – our monthly 90 minute downtime window where we apply schema changes, do DB server maintenance and so forth.

Starting very soon we will instead have very short windows – approximately 60 seconds long – where we perform schema changes, database server failover (in order to permit DB maintenance on the master server) and so forth.

We expect to do these about 6 times a month based on our historical rate of schema patches, and we are – for now – planning on doing these at 0800 UTC consistently.

This will deliver much less total downtime – 6 minutes a month rather than 90 – at the cost of more frequent interruptions.

If you have API scripts running against Launchpad, you may want to build in a retry mechanism to deal with up to a few minutes of downtime.

We cannot remove downtime entirely for purely technical reasons: Our primary database (postgresql) blocks new readers (or writers) when a schema change is being executed, and the schema change blocks on existing readers (or writers) to complete – it needs an exclusive lock on each relation being altered.

What we can do is automate the process of disconnecting and interrupting existing database connections to let the schema change execute rapidly, and make our schema changes as minimal as possible. Previously, we shut down all the application servers (via a script, but shutting down gracefully takes time), and then ran schema changes which did data migration and so forth. In this new process we will leave the appservers running and just interrupt their connections for the time it take to apply the schema change. That, combined with moving data migration to a background job rather than doing it during the schema change, gives us the short downtimes we’re about to start doing.

More information is available in the LEP and my mailing list post about the project starting.

Read more
Curtis Hovey

Users of Launchpad Answers will see that asking a question, editing it, or posting a comment to it is faster. Email about question changes is sent a few minutes latter. Many bugs relating to question emails were fixed as we moved the work of sending emails to the new process.

Users and answer contacts saw slow pages or time out errors when working with questions in large projects. Simple actions like asking a question or providing an answer would fail. It was common to see errors converting bugs into questions. A few weeks ago, we saw that 8 of the top 10 kinds of time outs involved questions; though this ratio was caused in part by the tremendous work of making other parts of Launchpad faster.

The root cause of the slow question pages was sending email to all the subscribers before showing the next page. The solution was to queue the the event to notify subscribers, and send the emails later. While updating the code, there were many opportunities to fix related Answers bugs. I am particularly pleased with the changes to the rules to create a question. There were four lines of code, and while I intended to fix one line, I realised there was a bug related to each line of code. In a matter of minutes I had fixed four bugs. The most obvious change you will see is that question emails will now state that you received the email because you asked the question, where previously you were merely described as a subscriber.

Read more
Matthew Revell

Launchpad is Go!

Go!There’s a new Launchpad client library, called lpad, for the Go programming language.

Gustavo writes:

lpad is based on a two-layered design. The top layer offers a static API which allows a more comfortable interaction with the API with static checks, better documentation, and more. The bottom layer is fully dynamic and enables the developer to access all the features of Launchpad, even those not supported by the top static layer.

There’s still work to do but the library is pretty much complete and it’s well tested, including integration tests which communicate with the real production servers.

You can get hold of lpad with a simple:

bzr branch lp:lpad

Check out the full API documentation.

Photo by Iain Farrell. Licence: CC BY-ND 2.0.

Read more
Matthew Revell

French lessons on floppy diskIt’s incredible to think that more than 57,000 people have used Launchpad to translate software from English into their own language.

Many of them have worked directly on upstream projects, such as the OpenShot video editor. Others have helped translate Ubuntu packages of software. And then there’s a whole group of people who translate upstream software outside of Launchpad.

Today we’ve taken another step in bringing those efforts closer together by making it far easier to get upstream translations directly into Ubuntu.

We want the strings produced by translators working directly on software projects, whether in Launchpad or elsewhere, to be easily available to the Ubuntu translators and we believe it should be just as easy for software projects to take advantage of the work of Ubuntu translators.

How it works

Translation sharing between different releases of a project, or Ubuntu, has been available in Launchpad for some time now. Also, sharing translations between an upstream project translated in Launchpad and its Ubuntu package has been helping to bring those two communities of translators closer together.

What’s changed today is that strings from upstream projects who make their translations outside Launchpad are now just as easily imported and ready for use by Ubuntu.

Now, so long as the upstream project is set up in Launchpad to do this, a change made in an upstream project’s source code — whether hosted directly in Launchpad or elsewhere in Bazaar, Git, Subversion of CVS — will be available to Ubuntu translators just a few hours later.

Previously, Ubuntu took translation templates and files from the source packages as they were uploaded. There was no automated route for those new upstream translations to get into Ubuntu after that initial import. In effect, this allowed Ubuntu translations to diverge from upstream during the six month Ubuntu cycle.

This change has a nice side benefit of making it easier for upstream projects to make use of translation work done for Ubuntu, because the English strings will diverge far less and it will be easier to spot where the Ubuntu community has done new translation work, rather than their being a divergence due to the two efforts drifting apart.

To start with, this is available for projects that use intltools, which includes all of GNOME. To get your project’s translations automatically imported into Launchpad, see our help guide.

Photo by Kino Praxis. Licence: CC BY 2.0

Read more
Diogo Matsubara

Let’s explore

Recently, Ursula and I have been improving the way that we test new Launchpad features.

Already, Launchpad has an extensive test suite but there are some things that automated tests can’t look out for. Rather than just testing the quality of our code, we also want to test the quality of the experience.

To do that, we’ve been doing more exploratory testing. Now, when a feature is getting close to deployment we will try out every part of the feature and make notes of anything in the experience that needs to be fixed before we release it. In particular, we’re interested in the feature’s:

  • ease of use and discoverability
  • completeness
  • quality of implementation
  • suitability to the problem it is solving
  • conformity to Launchpad’s principles and UI guidelines.

We’re also aiming to timebox the testing; something that takes too long to explore is likely too complex. For now, we’re using a 25 minute limit, as borrowed from The Pomodoro Technique, as it seems like a good starting point.

If you’re interested in what we’re doing, you can follow our progress both on the launchpad-dev mailing list and on the Launchpad dev wiki. Also, I’m hoping that we can get your help in testing beta features. I’ll write more about that soon.

Read more
Martin Pool

Launchpad has a web UI, an email interface, and a ReST API that exposes every object in the database.

There are also a bunch of client programs, command line and graphical, that talk to Launchpad to do various things.

What we don’t yet have, and what I think would be great, is a systematic client that lets you manipulate
from the command line. There’s some code that starts towards this in Hydrazine, lptools and others, but I think having just a single tool that eventually does everything would be more discoverable and avoid unnecessary fragmentation or duplication.

(That’s not to say there’s not room for others that are guis, that are specialized to particular projects or that encapsulate a lot of policy or opinion about what they’re doing.)

So dobey and I have agreed to gradually merge hydrazine into lptools, and with other people to work towards making lptools cover everything you can do through the web UI or the API. If you have scripts you’ve written yourself, perhaps you’d like to merge them in.

Read more
Martin Pool

Short story: takes you to bug 12345, and describes more abbreviations.


Sometimes you’d like to point people to an interesting bug in a project that uses Launchpad, like bug 685380 (that ‘1′ and ‘l’ may need to be more distinct in the new Ubuntu Font).

Typing out is a bit tedious, and it uses up a fair bit of space in a microblog entry. You can use any of innumerable URL-shortening services, but then the URL’s opaque; which is a shame since it really just wants to represent a 6-digit number.

Therefore: (pad love), transparent short URLs for bugs, and other things including projects, people, bug-filing forms, packages, and more.

Maybe someone would like to make bookmarklets that generate these links, or add them into the Launchpad UI?

Thanks to Latvia for letting us use a fraction of their domain name space!

Read more
Robert Collins

As part of improving performance we have disabled the substring matching of source package names. This fixes bug 268508 and bug 607960. However its a slightly contentious issue – opinions vary about whether bug 268508 is a valid bug or not.

So we have only disabled it – the code is still present and when we have more leeway on the performance of bug searching we’ll revisit this and look into some design and UI analysis to decide whether substring matching of this sort should be done or not.

For now though, there should be less timeouts in bug searches.

Read more
Brad Crittenden

Launchpad’s bug mail can be a bit chatty sometimes as I’m sure you’ve noticed.  This cycle the Yellow Squad is working to give you more control about the bug mail Launchpad sends to you.

One problem that we’ve known about for a really long time was reported as bug 548 and has recently been closed thanks to the effort of  ?????? ????? and Gary Poster.  Their fix allows you to globally specify whether you want to receive email about actions you did.

Most people probably do not need to be reminded of something they did a few minutes ago and will want to turn off those emails.  But, since this is new functionality we’ve preserved the old behavior unless a user changes the setting.  If you do nothing you’ll continue to get email about actions you instigate on bugs.

Opting out of those messages is easy.  Simply go to and uncheck the box as shown below by the big red arrow.

opt out of bug mail

As mentioned, this is just the first of many features and refinements that we’re working on to help you customize the stream of bug mail coming from Launchpad to suit your needs.

Read more
Robert Collins

We have a small quandry on the Launchpad development team at the moment. As bug 268508 discusses, when one searches for a bug on Launchpad we do a substring search on the names of bug targets.

For instance, searching in Ubuntu for ‘gcc’ will return all bugs on the packages ‘gcc’, ‘gcc-4.4′, ‘gcc-4.3′, ‘gcc-3.3′ and so forth. Likewise search for bugs in a project group will do a similar substring search on each of the individual projects in the project group.

It turns out that doing this search is itself expensive. I asked on the Ubuntu devel list about turning it off. We would close bug 268508 and also significantly improve search performance.

However this is a possibly contentious change – there was one mail strongly in favour of the current behaviour – so I’d like to get this change proposed to a wider community.

If you’ve got a strong opinion – that the current behaviour is good, or like bug  268508 describes, that its a poor behaviour and we would be better off without it, then I’d love to hear from you. Just leave a comment on this post, drop me an email – robert at – or post to the launchpad-users mailing list.

Rob (LP technical architect)

Read more
Brad Crittenden

Removing team polls

Polls were introduced to Launchpad as a way for teams to conduct internal surveys.  Unfortunately the user interface and feature set were always problematic.  The feature never really caught on and wasn’t used much (a little over 500 polls since 2006).

A discussion[1] on the Launchpad Users mailing list saw a consensus agree that polls in their current state were not a viable feature and they should be removed rather than being fixed.  As part of the December Launchpad Bug Jam I’ve taken on the task of removing them from the site.

Currently there are only four polls that are underway and the owners of the teams responsible for those polls will be notified that the feature is being removed.  The data from all existing polls will be saved and made available to the teams at PollFeatureRemoved.


Read more
Francis J. Lacoste

From today, all Launchpad bugs, code, questions and blueprints are tracked under the one launchpad project.

We’ve already moved everything from the individual projects over to the parent launchpad project. All you need do differently is search/file bugs, questions and blueprints under that parent Launchpad project, rather than Rosetta, for example.

Don’t worry, though, there are redirects in place so that old links will still work.

There are also a couple of one-time steps you may need to take:

  • Update your bug subscriptions: if you’re subscribed to individual bugs, you need do nothing. If you’re subscribed to all bugs for a particular project, Malone for example, you’ll now need to subscribe to all Launchpad bugs.
  • Check your answer contact status: if you’re an answer contact for one particular application in Launchpad, and want to continue as such, you’ll need to become an answer contact for all of Launchpad.

To start with, bugs that we’ve merged in from one of the old sub-projects will have a tag that shows which project it came from. However, we’re planning to drop those tags once everyone’s settled into using just the one project.

Our code hosting won’t change at all as we’ve always hosted code under the parent Launchpad project.

This new approach will better reflect that Launchpad is one codebase but will also have a big practical benefit: it’ll be easier to find bugs and dupes because everything will be under the same project.

Why we’re doing this

For almost four years now, Canonical’s Launchpad team has been divided along application lines: i.e. we have sub-teams who each look after a particular part of Launchpad. So, Deryck, Abel, Gavin and Graham are currently the Launchpad Bugs team and work on nothing other than Launchpad’s bug tracker.

Reflecting this team structure in our Launchpad projects has made it easier for those sub-teams to plan their work.

It has worked pretty well but we’re about to change the structure of Canonical’s Launchpad team for a couple of reasons:

  • we want to focus on releasing features, and fixing problems, wherever they are
  • we want all Canonical Launchpad developers to be familiar with the full Launchpad codebase, rather than focusing only on one part.

So, as of February 17th the Canonical Launchpad team will have five squads. At any one time, three of those squads will each be working on a particular feature and the other two will be working on maintenance. Once a feature squad finishes its feature, it’ll switch places with one of the maintenance squads.

This will mean that there’ll always be ten Canonical Launchpad developers dedicated to fixing bugs, dealing with critical issues and generally making sure that Launchpad is serving you well. And of course there’ll be fifteen developers working on new features.

Rather than make this post even longer, I’ll write more soon and in the meantime point you to Rob Collins’ rousing launchpad-developers post in favour of the new structure.

As ever, if you have questions then please join us on the launchpad-developers mailing list or feel free to contact me directly.

Read more

I’m running a short survey — four questions — to find out how people want to get system status updates about Launchpad.

If you’ve got an opinion on how you’d prefer to get info about pending and unplanned Launchpad service interruptions, take the survey :)

Read more
Robert Collins

Launchpad edge site deprecated

I previously posted about our continuous deployment efforts in Launchpad. Since then the project has come a long way. We can deploy to nearly all our services without downtime. The remaining services are a bit trickier – but we are working on them.

As part of the project we are consolidating the ’edge’ domain –, and other similar domains – into the main launchpad UI. These domains are now deprecated.

The most important thing this means for you is that for members of our beta test program, we will no longer redirect you to – instead we are serving our beta UI directly from the main website. The edge site is now running exactly the same code as the main Launchpad cluster and is updated at exactly the same time.

We have done this to deliver new features to our users more efficiently and at the same time simplify our production environment. So far the project has been very successful from our perspective – as I write this we have 5 days of inventory – code we’ve written but not deployed. This is down from an average of 2 weeks prior to this initiative starting, and we often sit lower – 1 to 2 days worth.

In the coming months as we refine this process and project we want to remove the edge cluster. As part of this we will start redirecting browser requests to ‘edge’ domains to the main Launchpad domain.

API clients cannot be redirected in this way, so we also ask that anyone writing or using Launchpad API scripts update them to use the primary cluster. We will slowly decrease the cluster size and disable it completely once we see no traffic on it. The main cluster is currently 3 times the size and should perform better for nearly any API script. To do this, use LPNET_SERVICE_ROOT rather than EDGE_SERVICE_ROOT. To get the LPNET_SERVICE_ROOT symbol, import it from launchpadlib.uris:

from launchpadlib.uris import LPNET_SERVICE_ROOT

If you have any questions about any of this we’d be delighted to hear from you – here, on IRC or the launchpad-user mailing list.

Rob Collins
Technical Architect

Read more
Matthew Revell

Some bugs get reported more than once. That’s why we’ve got the dupe finder.

Some duplicate bugs slip through the dupe finder. Really common issues get quite a few dupes and someone from the relevant project usually goes through and marks them as duplicates of the master bug where the actual discussion and tracking is taking place.

There has been a really annoying bug in the way Launchpad has handled all this, though, and Deryck’s just fixed it :)

Let’s say you’ve got a bug report that has a few duplicates attached to it. You then discover that, actually, there’s an older bug with a more mature discussion and that, really, that’s the master bug. Until now, before you marked your bug as a duplicate of the master, you’d first have to take all the dupes of your bug and manually make them dupes of the master.

Still with me? :)

For a busy bug with many dupes, some of which have their own dupes, that’s a real disincentive to clearing up the multiple duplicates and gathering everything together on the one true master bug.

Now, though, simply mark your bug as a duplicate and Launchpad will automatically transfer your bug’s dupes to the new master bug.

Simple :)

Read more
Matthew Revell

Some bugs attract many, many comments.

For a while now, Launchpad has displayed only the first 80 comments on any bug report, with the option of viewing the full comment history. That’s been good for speeding up page loads but not so great at offering an accurate view of the current state of discussion about the bug.

Bryce has fixed that. Now, a bug report page still shows only 80 comments, by default. However, to give a better overview of the state of discussion, it now shows the first 40 and the last 40 comments.

So, half way down the comments you’d see something like:

Here's what you see in the middle of the bug comment history

Here's what you see in the middle of the bug comment history

Then, at the bottom of the summarised comment history there’s this:

Comment history summary

Comment history summary

With any luck, this should result in new bug comments that take into account the most recent discussion.

Read more
Matthew Revell

Better dupe finding

One of my favourite things about Launchpad’s bug tracker is the dupe finder: when you report a new bug, it’ll search to see if there’s already a similar bug report. It’s the same for questions in Launchpad Answers, too.

Getting to see possible dupes before you file a bug or question is a great time saver for you and the people on the other end. However, the dupe finder has been timing out a lot lately.

Rob Collins, Launchpad’s new Technical Architect, has introduced some changes that should make the dupe finder more reliable.

Other than fewer timeouts, here’s what you might notice:

  • the dupe finder now returns fewer matches — three or four rather than ten or more
  • the results should be more relevant.

We want to know how this works in practice. Let us know how you get on with the new dupe finder. Either leave a comment here, mail or join us on the launchpad-users mailing list.

How Rob did it

The previous dupe finder had a number of problems, not least that the search engine it’s built on is less efficient than we need. We’re planning to replace the search engine but not straight away, so Rob looked for a temporary solution that would work for the next five or six months.

I’ll hand over to Rob to explain what he actually did:

The old search did a pre-pass over every possible hit, which is 400,000 items for Ubuntu bugs and very slow to do. It then did a search matching any document that had a rare search term in it.

So, by rare we mean that the term showed up in less than half of the possible hits.

For example, if you searched for “firefox crashes on <website> in flash” on /ubuntu/+filebug it would search for any bug with any of “firefox” (< 50% of bugs are on firefox), "crash" (<50% of bugs say "crash"), "<<50%...), "flash" (< 50%...)

However, many, many bugs mention "firefox" and many, many bugs mention "crash" and many, many mention "flash".

So, the total return from the search could be 10,000 or 100,000 quite easily and — unlike other search engines — the more terms you typed in, to make it more precise, the less precise it became.

That sounds odd but here's why: it started bring back bugs from anywhere that happened to mention any search term and, adding in the relevance weighting we had just added confusion to it.

What we do now is: if you sesarch for "firefox crashes on <website> in flash" we search for any bug continaing three of the four non stopwords, i.e."firefox, crashes, <website>,flash". If a bug mentions any three, it will be returned.

We can switch this off easily if we have to, so we do want feedback about how people find this.

Read more
Julian Edwards

Improvements, you say?

Very recently we saw the beta release of a new feature on Launchpad: building packages from recipes. Recipes bring Bazaar branches for the upstream source code and a packaging branch together to generate a Debian source package. We informally referred to this internally as “the Wellington project” because we first embarked on this long road of development back in November 2009 with a coding sprint in Wellington, New Zealand.

So, why do we need to use the build farm for this?

Very early on we realised that this would be a challenging project because packaging recipes allow untrusted arbitrary code to run as part of building the package, so we decided that the only place we could do this operation was in the same place that we do one of the other untrusted operations – the PPA build machines.

The PPA build machines work for untrusted code because they run as a virtual machine. After the build job has finished, the machine is simply ripped out and restarted so if any code did something nasty, we throw all the nastiness away!

Why can’t the build farm do this already?

The Launchpad build farm has been around for a while now, but it’s of course only used for building source packages into binary debs. We realised that we’d need a whole new data model on the Launchpad side, changes to the master/slave protocol and changes in the slave code to make it actually run this new job type – a lot of work! It was also about this time I became aware of a further untrusted job type, generation of translation templates for Launchpad Translations, so we began work on making the build farm able to process any kind of job.

What had to change?

By far the biggest change for this was to fix how we store the build/job information in the database. The canonical way to refer to a “job” on the build farm was via “build” record, and this is what all of the history pages in the UI were using. We’ve now re-architected this to use a generic “build farm job” and a bunch of new database tables that re-factor all the information needed for various jobs on the build farm, in fact we found out that the recipe build jobs and the source package build jobs had quite a lot in common which resulted in some merciless code re-factoring indeed.

The other major change was the slave builder code itself. This runs as a Twisted executive that we talk to using XMLRPC from the build manager, which kicks off external processes to run the job and monitors them. The only job it knew about was to run sbuild to build source packages. We’ve changed this to also run bzr builder and a custom script to generate the translations templates.

So, will I see anything different?

A recipe build in action

A recipe build in action

You won’t see much visually that’s changed in the Build Farm. If you’re observant, you’ll see the occasional job appear in the list that announces it’s a recipe build, but they’re generally pretty quick to complete!

We’ll post a more detailed explanation of how to use recipe builds in the future, but for now it’s remaining in beta until all the problems are ironed out. If you have any feedback, please file a bug or send us an email.

Read more