Canonical Voices

Posts tagged with 'general'

Diogo Matsubara

How to do Juju – Charming oops-tools

Recently the Launchpad Red Squad and Product Team started working on a new cloud project. As part of that project we’ll be using Juju, a tool that helps you easily deploy services on the cloud.

As an opportunity to learn more about how Juju works, I wrote a charm to deploy oops-tools, an open source Django application that helps visualize and aggregate error reports from Launchpad, on Amazon’s EC2 and Canonical’s private Openstack cloud.

You might be asking, what’s a charm? Charms are basically instructions on how to deploy services, that can be shared and re-used.

Assuming you already have a bootstrapped juju environment, deploying oops-tools using this charm is as easy as:

$ juju deploy --repository=. local:oops-tools
$ juju deploy --repository=. local:postgresql
$ juju add-relation postgresql:db oops-tools
$ juju expose oops-tools

That’s it! With just a few commands, I have an instance of oops-tools up and running in a few minutes.

Under the hood, the oops-tools charm:

  • starts two Ubuntu instances in the chosen cloud provider, one for the webserver and another for the database server
  • downloads the latest trunk version of oops-tools and its dependencies from Launchpad
  • configures oops-tools to run under Apache’s mod_wsgi
  • configures oops-tools to use the database server

There’s still work to do, like add support for RabbitMQ (oops-tools uses rabbit to provide real-time error reports), but this initial iteration proved useful to learn about Juju and how to write a charm. As it is, it can be used by developers who want to hack on oops-tools and can be easily changed to deploy oops-tools in a production environment.

If you’d like to give it a try, you can get the charm here: https://code.launchpad.net/~charmers/charms/oneiric/oops-tools/trunk

Enjoy!

 

 

(“Harry Potter’s The Standard Book of Spells” by Craig Grobler, licensed under a CC BY-NC-ND license)

Read more
Dan Harrop-Griffiths

Fighting fire with fire – Changes to bug heat


 bug heat storm trooper candle

We’re making changes to the way that bug heat is calculated and displayed in Launchpad.  From 6th February, bug heat will no longer age/degrade, and the active flame counter symbol will be replaced by a number, next to a static flame.  Here’s why.

Bug heat ageing is the cause of a wide range of timeouts in Launchpad. Every bug’s heat has to be recalculated and updated every day using a series of complex calculations, and when there are around 1 million bugs reports to track, that’s a lot of pressure on the system, consuming a significant chunk of resources.  Turning off bug aging is the simplest way to solve this issue.

 

new bug heat image

 Display

The flame counter symbol, although adding some visual flair (and flare), also needs to update every time the bug age recalculations are made.  The continual stream of updates to the bug rows also results in poor search index performance.

We’ll still have a flame symbol, however it’ll be static, with the bug heat number next to it. Although not as visually dynamic, it’ll be easier to work out bug heat scores more exactly, at a glance.

Although I’m sure some of us will miss this little Launchpad feature, less timeouts is good news for everyone.

 

 

(“Happy and safe birthday” by Stéfan, licensed under a CC:BY-NC-SA license)

Read more
Dan Harrop-Griffiths

New feature – Customise your bug listings

Custom Bug Listings

Over the past few months the Launchpad Orange Squad has been working to make it easier to get the information that matters to you from bug listings.

A lot of you have said in the past that you’d like to be able to filter bugs in a way that works best for you. Hopefully this new feature, with its customisable functionality should help with this goal, filling your screen with more of what you want to see.

Custom bug listings green bug

Features

You can now sort bugs by criteria such as name, importance, status and age. You can switch on the criteria that you use most and turn off criteria that you don’t use. So if you always like to see bug age, but aren’t interested in heat, you can switch on age and switch off heat, and so on.

bug column screen shot

Display

We’ve also redesigned how bug listings are displayed – fitting more information into each bug listing, and adding sort options such as bug age, assignee, reporter, and tags.

You can put your results into ascending or descending order without having to reload the page, and you’ll be able to save your preferred layout, so your settings will be saved for the next time you need to look over your bugs.

User research

This was my first main project since joining the Launchpad team back in November as the new Usability & Communications Specialist. User research has played an important part in how we’ve defined the feature and the decisions the team has made to improve the display, wording and functionality.

A number of you took part in one to one interviews, at group sessions at UDS-P and by taking part in an online survey. Thanks to everyone involved – what you told us has really helped to make this feature a more user-friendly experience. Some of our user research results (link) are already available online, with more being added soon. We’ll be carrying out some further tests in the weeks ahead, so please get in touch if you’d like to get involved.

Bugs

Every new feature has teething problems, and custom bug listings is no different. We still have a number of bugs that need tweaking, so please bear with us, and file any bugs if you spot anything that’s still out there.

Read more
Aaron Bentley

New approaches to new bug listings

The new bug listings listings were the first time my squad, the Orange Squad, had a chance to work on some really nice in-depth client-side UI since our squad was formed. Not only were we implementing the feature, we wanted to lay groundwork for future features.  Here are some of the new things we’ve done.

Synchronized client-side and server-side rendering

Early on, we decided to try out the Mustache template language, because it has client and server implementations. Although we wanted to make a really responsive client-side UI, we also wanted to have server-side rendering, so that we’re not broken for web crawlers and those with JavaScript disabled. Being able to use the same template on the server and the client seemed ideal, since it would ensure identical rendering, regardless what environment the rendering was done in.

It’s been a mixed bag. We did accomplish the goal of using a single template across client and server, but there are significant bugs on both sides.

The JavaScript implementation, mustache.js, is slow on Firefox. Rendering 75 rows of data takes a noticeable length of time. If you’re a member of our beta team, you can see what I mean. Go to the bugs page for Launchpad itself in Firefox. Click Next. Now click Previous. This will load the data from cache, but it still takes a visible length of time before the listings are updated (and the Previous link goes grey).

mustache.js also has bugs that cause it to eat newlines and whitespace, but those can be worked around by using the appropriate entity references, e.g. replacing “\n” with “
”

The Python implementation, Pystache, does not implement scoping correctly. It is supposed to be possible to access outer variables from within a loop. On the client, we use this to control the visibility of fields while looping over rows. On the server, we have to inject these control variables into every row in order for them to be in scope.

We needed a way to load new batches of data. Mustache can use JSON data as its input. Launchpad’s web pages have long had the ability to provide JSON data to JavaScript, but Brad Crittenden and I recently added support for retrieving the same data without the page, via a special ++model++ URL. This seemed like the perfect fit to me, and it’s turned out pretty well. Using the ++model++ URL rather than a the Launchpad web service means the server-side rendering can tightly parallel the client-side rendering.  Each uses the same data to render the same template.  It also means we don’t have to develop a new API, which would probably be too page-specific.

Client-side Feature Flags

While in development, the feature was hidden behind a Feature Flag. But at one point, we found we wanted access to feature flags on the client side, so we’ve now implemented that.

History as Model

We wanted users to be able to use their browser’s Next and Back buttons in a sensible way, especially if they wanted to return to previous settings. We also wanted all our widgets to have a consistent understanding of the page state.

We were able to address both of these desires by using YUI’s History object as a common model object for all widgets.  History provides a key/value mapping, so it can be used as a model.  That mapping gets updated when the user clicks their browser next and back buttons.  And when we update History programmatically, we can update the URL to reflect the page state, so that the user can bookmark the page (or reload it) and get the same results.  Any update to History, whether from the user or from code, causes an event to be fired.

We’re able to boil the page state down to a batch identifier and a list of which fields are currently visible. The actual batches are stored elsewhere, because History isn’t a great place to store large amounts of data.  For one thing, there are limits on the amount of data that can be stored.  For another, the implementation that works with old browsers, HistoryHash, can’t store anything more complex than a string as a value.

All our widgets then listen for events indicating History has changed, and update themselves according to the new values in the model.

Summing up

It’s been an interesting feature to work on, both because of the new techniques we’ve been able to try out, and because we’ve been closely involved with the Product team, trying to bring their designs to life.  We haven’t quite finalized it yet, but I’m going on leave today, so I wanted to let you know what we’ve all been up to.  Happy holidays!

Read more
Francis J. Lacoste

Bugs

The Launchpad maintenance teams have been working since the beginning of the year at reducing our Critical bugs count to 0. Without much success this far. The long term trend keeps the backlog at around 300.  And it’s not because we haven’t been fixing these. Since the beginning of the year, more than 800 Critical bugs were fixed, but more than 900 were reported :-(

So I investigated what was the source of all these new critical bugs we were finding. A random sample of 50 critical bugs filed were analyzed to see where and why they were introduced. The full analysis is available as a published Google document.

Here are the most interesting findings from the report:

  • Most of the new bugs (68%) are actually legacy issues lurking in our code base.
  • Performance and spotty test coverage represents together more than 50% of the cause of our new bugs. We should refocus maintenance on tackling performance problems, that’s what is going to bring us the most bang for the bucks (even though it’s not cheap).
  • As a team, we should increase our awareness of testing techniques and testing coverage. Always do TDD, maybe investigate ATDD to increase the coverage and documentation our the business rules we should be supporting.
  • We also need to pay more attention to how code is deployed, it’s now very usual for scripts to be interrupted, and for the new and ancient version of the code to operate in parallel.

Another way of looking at this is that Launchpad represents a very deep mine of technical debt. We don’t know how exactly deep the mine is, but we are going to find existing Critical issues until we hit the bottom of the mine. (Or until we finish refactoring enough of Launchpad for better testing and performance. That’s what the SOA project is all about.)

In the mean time, we should pay attention to regressions and fallouts, (those are really the new criticals) to  make sure that we aren’t extending the mine!

Photo by Brian W. Tobin. Licence: CC BY-NC-ND 2.0.

Read more
Dan Harrop-Griffiths

Custom bug listings – have your say

Our custom bug listings beta has been up and running for just over a week now – thanks to everyone in the Launchpad beta testers group that have tried it out, and thank you for all your valuable feedback and comments. If you haven’t tried it yet, you can get access by joining our beta team here:  https://launchpad.net/~launchpad-beta-testers

We want to improve how the default information is displayed to make this tool work better, so we’ve put together a super-quick survey to find out:

- What information about a bug you most want to see in bug listings

- What the default ‘order by’ options should be

- If you’d like to see any other ‘order by’ options.

These three questions should only take a few minutes to complete, but they’ll add real value to our work redesigning how bug listings appear and function. Here’s the link if you’d like to take part

Read more
Raphaël Badin

Edit 2011-11-15 08:18 UTC: The problem is now fixed and we’ve re-enabled the new menu.

Edit 2011-11-11 13:42 UTC: We’ve temporarily disabled the new menu while we fix some unfortunate side effect.

We’ve just deployed a new, simplified version of the branch menu displayed on the right hand side of personal code pages (e.g. personal page for the Launchpad team). It looks like this:

Old menu

New menu

Calculating the number of branches took way too much time for people/teams with a huge number of branches (e.g. https://code.launchpad.net/~ubuntu-branches), up to the point that they were getting timeouts.

The new design, along with optimisations we’ve made to the database queries, should improve performance for everyone.

Read more
Martin Pool

We’ve just upgraded Launchpad’s builder machines to Bazaar 2.4. Most importantly, this means that recipe builds of very large trees will work reliably, such as the daily builds of the Linaro ARM-optimized gcc. (This was bug 746822 in Launchpad).

We are going to do some further rollouts over the next week to improve supportability of recipe builds, support building non-native packages, handle muiltiarch package dependencies, improve the buildd deployment story etc.

Read more
Matthew Revell

Welcome to BerliOS projects

It’s sad to read that BerliOS will close in December, after nearly twelve years of serving open source projects. One fewer project hosting site means that the open source world is that bit poorer.

If you’ve been hosting your project on the BerliOS Developer platform and you’re looking for a new home, you’ve got plenty of choice.

We’d love to welcome you to Launchpad and here are a few reasons why you should consider Launchpad:

If you have questions, you’re very welcome to join us in #launchpad on FreeNode and the launchpad-users mailing list.

Read more
Martin Pool

We’ve recently deployed two features that make it easier to find bugs that you’re previously said affect you:

1: On your personal bugs page, there’s now an Affecting bugs that shows all these bugs.

2: On a project, distribution or source package bug listing page, there’s now a “Bugs affecting me” filter on the right (for example, bugs affecting you in the Launchpad product).

Counts of the number of affected users already help developers know which bugs are most urgent to fix, both directly and by feeding into Launchpad’s bug heat heuristic. With these changes, the “affects me” feature will also make it easier for you to keep an eye on these bugs, without having to subscribe to all mail from them.

screenshot of

Read more
Martin Pool

If you use gmail, you should now be able to send commands to Launchpad without gpg-signing.

gmail puts a DKIM cryptographic signature on outgoing mail, which is a cryptographic signature that proves that the mail was sent by gmail and that it was sent by the purported user. We verify the signature on Launchpad and treat that mail as trusted which means, for example, that you can triage bugs over mail or vote on merge proposals. Previously you needed to GPG-sign the mail which is a bit of a hassle for gmail.

(DKIM is signed by the sending domain, not by the user, so it doesn’t inherently prove that the purported sender is the actual one. People could intentionally or unintentionally set up a server that allows intra-domain impersonation, and it’s reported to be easy to misconfigure DKIM signers so that this happens. (Consider a simple SMTP server that accepts, signs and forwards everything from 192.168/16 with no authentication.) However, in cases like gmail we can reasonably assume Google don’t allow one user to impersonate another. We can add other trusted domains on request.)

If you have gmail configured to use some other address as your From address it will still work, as long as you verify both your gmail address and your other address.

You can use email commands to interact with both bugs and code merge proposals. For instance when Launchpad sends you mail about a new bug, you can just reply

  status confirmed
  importance medium

Thanks for letting us know!

We do this using the pydkim library.

Note that you do need at least one leading space before the commands.

If you hit any bugs, let us know.

Read more
Matthew Revell

Deployment reports are now public

Steve Kowalik writes:

For a while now Launchpad has been using deployment reports that tell us what state our QA is in, which revisions are safe to deploy to production, or which revisions are not safe to deploy since they failed QA.

Some time ago, I started the process to make these reports public, and I’m proud to announce that today, they are!

If you’re waiting to see when your code in Launchpad is likely to go live, take a look at the new public deployment report.

Read more
Francis J. Lacoste

Speeding up development

SpeedometerToday we reached a significant milestone, we completed our first fast down-time deployment. Two obvious reasons for doing this were already mentioned in the announcement and our technical architect”s post describing the change:

  1. We’ll have less downtime per month (at the cost of more frequent but short interruption).
  2. We’ll be able to deploy fixes and changes involving DB schema more frequently.

But from my perspective, the most important benefit I think we’ll get from this is a speed up in our rate of development, particularly, in terms of completing feature projects. It’s not a secret, our feature squads spends a lot of time to complete their projects. There are multiple reasons for this, but in the end, there usually fall under two broad cateries: the time it takes to actually make the change, and the delays in getting feedback on the change itself.

To help with the first category, you’ll want better and more powerful libraries, better architecture, developers’ training, etc. Think the time difference between developping a database web application in Django vs as a CGI C application using only the standard C library. Launchpad isn’t using the most modern libraries and toolkits, and we could still make a lot of improvements there.  But the costs of making changes in this space are compounded by the problems of the other category.

Once you wrote the change, you are far from done. There are lots of hoops you still have to jump through before saying “done-done“: you’ll need to make sure the tests pass, to get your changes reviewed, merged, QAed and then deployed. And finally, you’ll probably want to make sure that it matches the user’s expectations, but until it’s in production, this is hard to assess reliably. All of these steps takes time and introduce delays, some bigger than others. The Launchpad team is always on the look-out to cut in these delays and the new “fast down-time deployments” cut on of the biggest one we had.

Cycle time distribution

Since a picture is worth at least a thousand words, have a look at the chart above to have a better idea of what I’m talking about. It shows the distribution of the time it takes to complete a “change”. (What this plots is the cycle time from coding to deployment of our Kanban cards which roughly map to one logical change.) You’ll see that 50% of our changes are deployed to production in about a week. And the next 45% takes between 1 and 5 weeks. Now, our feature projects are composed of many many of these smaller changes. If those are all  relatively small changes, why do they take so long?

One of the big bottleneck was the batch size of our DB deployment. If a change required a DB schema it waited until the next downtime deployment which happened once every month. In theory, that means that on average a change involving the database would wait 2 weeks in the queue before deployment. In practice, it’s more complex than that, because squad leads would often plan around these. So a database change would be hold off onto because it was deemed that it couldn’t be safely completely to be part of the next downtime deployment. So it might be put on hold in favour of other work, and delayed to the next downtime deployment. It’s also frequent to have other changes building on the first also queued up waiting for the next deployment window. Add on top of that, that it’s common for the completion of a feature to require several iterations of DB change based on feedback and you quickly understand how you can be working months on a feature project!

But this major bottleneck is now gone! We’ll be able to land and deploy DB changes reliably within days, giving us much more rapid feedback. I’m looking forward to the change in the cycle time distribution in the coming months. The whole distribution should move toward the left. I’ll write a follow-up in two months to see if this prediction comes true.
Photo by Nathan E Photography. Licence: CC BY 2.0.

 

 

Read more
Francis J. Lacoste

Matt RevellIt’s my pleasure to announce that as of today, Matthew Revell is the new
Launchpad Product Manager replacing Jonathan Lange.

We were seduced by his bold vision for Launchpad along with his data-centric approach that he intends to bring to the role. He also has an already extensive experience interacting with Launchpad developers and users. If you read this blog, you probably read something written by him! Or you might have interacted with him in one of the many user-research sessions he ran. The introduction of user-research helped us release better designed feature. Building on this experience, we hope that his leadership will bring Launchpad to the next level.

Matt will communicate more about his plans for Launchpad shortly. In the
mean time, let’s give him our warmest congratulations!

 

 

Read more
Curtis Hovey

Users can use the affects form on the bug page to change which project or distribution the bug affects. You can also select the affected package. Lp API users can assign a project, distribution, or package to the BugTask target property to change the affected bug target. The behaviour is similar to the way questions can be retargeted between projects and distributions. Affected series cannot be changed, though the affected series package can be.

Retargeting a bug to a distribution, package, or project

Retargeting a bug to a distribution, package, or project

Previously, users had to mark a bug affecting a project or distribution as invalid, then add a new affected project or distribution. This cluttered the UI, caused excessive emails, and made pages slower.

Read more
Matthew Revell

Not long back, Sumana Harihareswara, who is Volunteer Development Coordinator at the Wikimedia Foundation, contacted some members of the Launchpad community to ask how we handle code review.

She’s written a summary of her research for the Wikimedia community.

It makes interesting reading for someone, such as me, who’s close to the process. If you’re interested in contributing to Launchpad, Sumana’s report is a really useful overview of how we review and deploy code.

Read more
Francis J. Lacoste

Echoes from the Dublin Thunderdome

ThunderdomeI can’t believe it’s been two weeks already! From June 25th to July 1st, the whole Launchpad team was gathered in Dublin for our semi-annual all-hands event. Like usual, we also invited the Bazaar team and  one of our friendly sysadmin­.

The theme this time around was “UI, UI, UI”. Since the January squad reorganisation, it became evident that a big stumbling block for squads working on features was UI work. Our team is still inexperienced with JavaScript and the YUI3 framework. So each squad has encountered similar problems and sometime found different solutions. We wanted to take advantage of the face-time to come to a common understanding of the best patterns to use.

In the end, it was a week of intense hacking on UI infrastructure, with part of the mornings dedicated on presentation where people shared what they learnt in the past 6 months.

Projects that were worked on during the week:

  • An asynchronous notification system (extracted from Landscape) which will allow us to update pages as soon as tasks complete on the backend.
  • We ditched our buggy Windmill-based Javascript integration tests, in favour of a yuitest-based suite. We already use yuitest for JS unit tests, but now we’ll be able to use it  for integration tests where  we need to make XHR requests to a live server. We’ll even be able to set server-side fixtures from within the test!
  • Investigate the use of selenium2 for acceptance testing. These are tests ensuring a complete workflow works and that ideally will be able to run against both the staging server as well as within tree for regression testing.
  • Integrating loggerhead-backed branch data on some Launchpad pages. You should soon be able to get the diff of a particular revision directly on the merge proposal page.
  • Infrastructure to refresh all the client-side representation of objects related to a page in one request.
  • Many UI bugs were fixed.

This was also the week we said farewell to old friend: Jonathan our Product Strategist  and Ursula Junque who is now working as QA analyst for the Ubuntu Server team. In the end, Jono didn’t get pied as we still had a huge number of Critical bugs left open. We did rejoice in achieving our performance improvements target. Given the trend, I estimated that we have still 6 months before emptying the Critical bugs list. The new target is to reduce to a 100 for October. Let’s see if I was more realistic this time around.

And we also took the traditional team picture.

Launchpad Team Photo

Great fun it was, and I can’t wait for the next one in January.

Photo by Miss_Colleen. Licence: CC BY-NC-SA 2.0.

Read more
Matthew Revell

Squiggle

Bags of Tilda riceVisit this URL:

https://launchpad.net/~

Cool, eh?

Same works for https://bugs.launchpad.net/~, as well as code, translations, answers and blueprint.

Actually, it’ll work anywhere that you’d normally put /people/+me or your own Launchpad id — so long as you’re logged in.

For example: visit https://launchpad.net/~/+participation to see a list of all your team memberships.

Thanks to Martin Pool.

Photo based on an original by Michael Francis McCarthy. Licence: CC BY 2.0

Read more
Matthew Revell

How to tell who has what role

So, you’re about to select someone in Launchpad — maybe you’re assigning a bug — and you’re not exactly sure you’ve got the right person. One way to confirm would be for Launchpad to tell you what relation that person has to the project you’re currently dealing with.

Sound like something that might help you?

If you’ve got 25 minutes today or tomorrow, you can help us choose the best way to show those affiliations. Sign up to let us know if you’re interested. There may even be an Amazon voucher in it for you :)

Read more
Julian Edwards

In for the long poll

A long pollLaunchpad has quite a few background tasks that update things outside of your browser request, and the only way you know that they finished is to reload a page to see if it changes.

Updating the preview diff on a merge proposal is one example.

At our Launchpad sprint in Dublin a couple of weeks ago, we added a so-called “long poll” where the browser opens a long-running connection using XHR (a background Javascript connection) to a new piece of infrastructure called the “long poll server”

This server is something that the Landscape team originally wrote and we’ve now open-sourced it so we can use it in Launchpad. The server connects to a RabbitMQ instance and relays information back to the browser when the background task completes so that the user doesn’t need to keep reloading the page – the page will automatically update thanks to the magic of Javascript.

The proof-of-concept is done but we now need to make it work in our production environment, which means making it more reliable, configuring the deployment and add some stats collection so we can see how it behaves. We’ll start off by only letting a small subset of people use it and grow it over time until we consider it stable enough for all users.

Photo by Jay Gooby. Licence: CC BY 2.0

Read more