Canonical Voices

Ara Pulido


For those unfamiliar with the title of this blog post, let me introduce you to one of the most important tags in Launchpad: regression-potential.

What bugs tagged as regression-potential mean? Basically, they mean that a regression has been found in the development release of Ubuntu (Lucid Lynx, at the moment of writing).

Why are they so important? Because it means that a regression has been found but, good news, we still can do something about it.

These bugs are specially important in the kernel. Nobody likes to see hardware, working in previous versions of Ubuntu, failing once upgraded to the new one.
Jeremy Foshee, a QA member of the Kernel team, is trying to avoid as many regressions in the kernel as possible. As announced in several mailing lists, he is going to be organizing a weekly bug day of kernel bugs marked as regression-potential from today and until the release of Ubuntu 10.04 (Lucid Lynx). If you want to help avoiding regressions in Lucid, every Tuesday, you can check the Kernel Bug Day page, which includes a list of bugs that need some love. If you have doubts on how to help, please, join the kernel team on #ubuntu-kernel at freenode IRC, and feel free to ask any question.

The more people helping triaging regression-potential bugs, the fewer regression-release bugs Lucid will have.

Read more
Ara Pulido

The Software Testing Club Magazine

Yesterday, the Software Testing Club, a community of software testers, released the first issue of their online magazine.

I am happy to announce that it features an article written by me. I am doubly happy. First, it is the first article that I have published in a software testing magazine; second, I am happy to see a FLOSS related article in a software testing magazine, which it is quite unusual. Formal testing seems to be a field where there is much more documentation in the proprietary software world.

PDF can be downloaded from their site. My article starts in page 32. I hope you like it.

Read more
Ara Pulido

One of the most amazing things about Ubuntu is the commitment to release free security updates for the next 18 months after a release. This commitment grows up to three years for the desktop and five years for the server when it comes to LTS releases. Mark gives a wonderful overview about Ubuntu release process in his post “The Art of Release”.

For LTS releases, on top of the updates, every six months, a new point release gets out, until the next LTS is released. This week Ubuntu 8.04.4 is going to be released. This is going to be Hardy’s last point release, as Lucid, arriving on April, is an LTS release as well.

This is wonderful and very helpful for people willing to run very stable releases in their production machines. But, to make this happen, things need to be tested. SRUs need to be verified, the security team need to keep up with the vulnerabilities database, ISOs needs to be tested for point releases as well, etc.

The problem about these tasks is that they lack of hype. It is always more entertaining to test the latest feature, this new cool theme or install the next release Beta ISO and see how good it is going to look.

But Ubuntu is built to last. We need these people to keep this commitment every release. This post is a big thank you for all those people who work (frequently behind the curtains) to keep Ubuntu releases in shape during their life-cycles.

Thank you guys, you rock. LTS releases rock.
LTS releases are the new hype.

Read more
Ara Pulido

If you have ever participated in Ubuntu ISO testing you may know what this title is about. To coordinate testing and to avoid duplicating efforts, every time one of us starts a new testcase, we enter a line like the one in the title in the #ubuntu-testing Freenode IRC channel.

In this example it means that I have started the Full Disk testcase for the Ubuntu Live CD i386 image. Others willing to help will know that I am already working on that one and will be able to concentrate their efforts in other testcases.

This system is far from perfect, as not everybody is in IRC and, even if you are, you can lose the messages sent before you logged in.

To improve the system I have added a new “Started” status to the test reports. Now, when you start a testcase, instead of having to communicate it in the IRC channel, add a “Started” result to that testcase and others will know that you are working on it (it will show up in the list of results with an icon of a clock.

Testcase started

Hopefully this will improve the coordination of the ISO testing activities.

Read more
Ara Pulido

Mago introduction in GNOME Journal

This is old news, but I have been busy lately and I just haven’t had the time for a quick post about it.

I wrote an introductory article about Mago for the GNOME Journal and it was published in its November issue.

If you want to learn about Mago and how can it can help you testing your desktop application, the article is a good starting point.

GNOME Journal: GNOME desktop testing automation and how to use Mago

Read more
Ara Pulido

Lucid Alpha 1 is coming to town!

Next Thursday, December 10th, Lucid Lynx Alpha 1 is going to be released.

We will be spending next week testing the ISOs and coordinating efforts in #ubuntu-testing Freenode IRC channel.

If you are planning to help with the testing, start syncing your Lucid images today, to avoid network bottlenecks and last minute hurries.
To sync your images, you can use the rsync URLS at, or use Steves Beattie’s script that do all the work for you.

Lucid knows when you’ve been bad or good, so be good for goodness sake! (and help us testing the candidate images…)

Updated: Steve’s script URL is now fixed.

Read more
Ara Pulido

Enabling line wrapping in Thunderbird

I am a happy user of the daily build of Thunderbird 3. Although it is still under development, is the best email client for Linux I’ve tried so far, at least, the one that better works for me. Email search & filtering is fast and reliable.

The only thing I don’t like is that wrapping your emails to 72 characters when sending plain text is not straightforward. Once you have selected to send your emails as plain text and have set the line wrap option to 72, there is still one more option to tweak.

Thunderbird has a “feature” that sends by default all plain text with the option format=flowed, which unwraps the email in the client receiver. To turn this feature off, you have to toggle one of the options in the chrome editor:

To disable flowed paragraphs, enforcing line breaks as formatted in the message, set the preference:
mailnews.display.disable_format_flowed_support true


To disable paragraph flow when you send plain text messages, and in the plain text part of multipart messages, set the preference:
mailnews.send_plaintext_flowed false


I put it here, in case is helpful to someone else.

Read more
Ara Pulido

As you may already know, next Ubuntu release, Lucid Lynx (10.04) is an LTS release.

For testers this means one important thing: upgrades should be smooth from either Ubuntu 9.10 (Karmic Koala) or latest Ubuntu LTS release (8.04, Hardy Heron).

As we all know, nowadays, computer storage is very very cheap, but bandwidth is not. Later in the cycle we are going to need to test as many upgrades from Hardy and Karmic as possible. So, why not planning ahead and start downloading today Hardy and Karmic images? The unstoppable Shane Fagan has started doing so already! You rock!

Later in the cycle you will be able to easily install Hardy or Karmic in a spare machine or a virtual machine and upgrade from there. You will have part of the work done. And you can start contributing to your beloved distribution just now :-)

Other releases from Ubuntu derivatives can be found at:

Read more
Ara Pulido

Well, I think the title of this post is a bit strong, but let me explain you my point.

I work for Canonical, a 100% distributed company where the majority of employees work remotely. We meet every once in a while to work together for a week, but, normally, we don’t see each other.

I joined the company a year and a half ago and, even back then, the people who had been there for more than 3 years stated “It is big now!! Do you remember when we knew the name of everybody?”. Well, Canonical is not THAT big, but it is getting harder and harder to remember people’s name and, because it is a distributed environment, people’s faces. Even harder, we rely a lot on IRC communication so you have to match a real name, with a face, and with an IRC nickname. For me this is almost impossible to achieve.

When I wrote an email I relied too much on email addresses auto-completion and this was making the problem bigger. “I think his name started with an M, and it might continue with an E, no, wait, an I. Here he is!”. I don’t do that any more. I write a good amount of emails, but not too many to not be able to spend one minute writing down the addresses it goes to.

Canonical has an internal web tool called “Directory” where you can quickly search a person (by team, name, IRC nickname, town, manager, email address, etc.) and you will soon find the record with a picture, real name, email address and IRC nickname. Then I copy and paste the email address to the “To:” field. It does the trick for me. It helps me visualizing who I am writing to and helps me matching the face with an IRC nickname, so the next time I get a ping from someone I can see the face behind.

Does your company has such a web tool? I really recommend it.

Read more
Ara Pulido

Ubuntu Testing Team

If you think that testing software is an unskilled activity that “even a two-year-old can perform”, keep reading, I’ll try to change your mind. If you do not agree with that sentence, keep reading, you may be interested in joining us.

Software testing is generally seen as the poor cousin of programming. While the bad reputation of testers happens in all software environments, this is more common in free software communities, probably because the “show me the code” motto is too deeply attached to the open source communities. This, unfortunately, is too often translated in unreliable software released with a lot of bugs (some of them critical).

Testing software, as any human activity, is a task that almost everybody can perform to some sort of proficiency. However, that does not mean that it is an unskilled activity. You have to know what to do. You need to have (or to develop), among others, excellent communication skills, technical writing skills, software architecture knowledge, technical research expertise, a critical mindset, etc.

We cannot leave quality to good luck. We cannot rely in having millions of users who will find bugs as they use the applications. Our users want to use the software, not to find bugs and report them. FOSS projects in general and Ubuntu in particular need a new way of rethinking testing as a skilled activity and an opportunity to contribute to the project.

We want to build a Testing Team in Ubuntu to try to minimize the impact of bugs in the released versions. This team would have a mailing list and regular meetings on IRC. Activities will be diverse and will include things like: formal manual testing, exploratory testing, writing new test cases, organize and conduct community testing days, automated testing and developing new tools (yes, if you like to code, there’s also a place for you).

We would love you to join us and make it happen.

We are having a session at UDS Lucid to discuss this topic (scheduled for Wednesday). You can subscribe to the blueprint as well.

Read more
Ara Pulido

This Thursday Karmic reaches the last milestone before the final release. As for every milestone, we need to test all the ISO images we produce, with every possible installation.

All of these test cases will appear, with instructions to follow, in the ISO tracker. If you don’t know how to use the tracker, this blog post will serve as starting guide.

One of the complains of the new comers is that they don’t know which test case needs testing. The coordination is done at #ubuntu-testing at Freenode and not everybody can access IRC. This time, Dave Morley and I, will try something new. As the RC images start appearing and testing begins, we are going to update in Twitter, using #ubuntutesting as tag.

If you want to help us testing RC images, please, follow us in Twitter and make sure to search for #ubuntutesting for updates. And if you’re helping testing, please, tweet about it!

Of course, this is an extra way to get informed. Coordination will happen, as usual, at #ubuntu-testing IRC channel.

Read more
Ara Pulido

Ubuntu 9.10 (Karmic Koala) Beta was released last Thursday. I am so glad to announced that we 98.9% coverage of the test cases in the ISO tracker. I would like to thank the community members that helped testing the ISOs, specially those who joined recently. Thanks! I am discussing with the Community team about the possibility of including this participation in the Ubuntu Hall Of Fame, just as the bug triagers or sponsors are.

I will blog about Release Candidate ISO testing as we approach the milestone week ;-)

Also, and because we are getting new contributions, I would like to comment some of the reports we got, so we can improve every milestone.

Not really a failure

We got this comment, in a test case marked as failure:

I have a tablet fujitsu p1630 and the stylus works in the life cd! great, congratulations!
(missing is the calibration tool which should be loaded. The stylus is not properly calibrated and cannot reach the top line (where the application menus sit!).[...]

In the ISO test tracker we mark as failures those experiences that prevented us to do what we want to achieve in that test case. I.e. If we want to install, and the partition manager fails, that’s a failure. If we do install (or can access to the Live environement, as in this case), the test didn’t fail as such. We would mark that as success, but will link the non-critical bugs that we find.

Usability bugs are bugs

The lack of colour in the default options during installation could cause problems for new users.
The default setting of Mute, for sound could cause problems for new users.

These are great examples of usability bugs. Thanks for noticing them! Usability bugs are bugs, so do not only put them as comments in your report, also go and file bugs in Launchpad for them. They will help a lot to new users to understand how Ubuntu works.

Read more
Ara Pulido

Karmic Beta Testing

Love Ubuntu? Want to help?
Karmic Beta candidate images have started to appear in the ISO tracker.

You don’t know how to use it? It is pretty easy!

Read more
Ara Pulido

Mago Tutorial

As part as the Ubuntu Developer Week, I gave a Mago tutorial on how to add new test cases to available applications in the Mago library.

I have reformatted a bit the logs to make the reading easier and it is now part of the Mago documentation. Enjoy!

Mago tutorial

Read more
Ara Pulido

Old friend ISO Testing Tracker

This post is going to be all about the ISO Testing Tracker. You might know it already, it’s been with us for a while. But, like old friends, sometimes you need a reminder that they are there and give them a hug. But before we continue with the tracker, let’s start with the basics:

What’s ISO Testing?

If you are reading this, you would probably know that Ubuntu has a 6-month release cycle, which means that every 6 months we have a new Ubuntu release, ready to install.

The good thing about software fixed cycles is that you can schedule anything you want. In fact, for Ubuntu, we schedule just at the beginning of the cycle up to 8 development releases. You can check at the Karmic schedule that there was an Alpha 1, Alpha 2, Alpha 3, Alpha 4, Alpha 5 and Alpha 6. And that there’s going to be a Beta and a Release Candidate before the “real” Ubuntu 9.10.

Those milestones need to be tested properly, because we release CDs for people to test on their hardware, and we have to make sure that everything is more or less OK. (with “more or less” I mean that these milestones are still software in development, so you have to be careful, anyway).

How do we track milestone testing?

So, here it is how the tracker appears in the show. Milestone releases happen normally on Thursdays. On Monday, in scheduled milestone weeks, testers start looking at the tracker looking for the candidate images: the ISOs that might be announced as the “Karmic Alpha something”. But these images need to be tested before they get announced.

How is the test tracker structured?

At Ubuntu, we produce builds for many different *buntu flavours. In the ISO Tracker we have a category for each of them. You can filter by category using the “Filter” menu in the bottom left of the page. That would allow you to concentrate in the flavour you want to test.

Filter by category

Filter by category

Once you select a category, you will be able to select an image to download for that category. A category like “Kubuntu” may have different images: i386 Live CD, amd64 alternate, etc., and all of them need to be tested before releasing a milestone.

Kubuntu Karmic Alpha 6 images

Kubuntu Karmic Alpha 6 images

If you click now in any of those images, you will have access to a direct link of the ISO to download and a selection of tests that you may run for that image. For every test, there is a link to the test case description in the test case wiki. The test cases are easy to follow when running any of them.

So, to sum up, every category contains a set of ISO images and every image contains at least one test case.

Iso Tracker structure

Iso Tracker structure

How do I help?

As I wrote before, the week of the release of a milestone we need a lot of help testing the candidate images. If we want full coverage we need to run more than 150 test cases. If you have a spare machine or a virtual machine installed, you can help making Ubuntu better by running the untested test cases during the week of the release of a milestone.

First thing, you will need an account in the ISO tracker. Unfortunately, your Launchpad Open ID won’t work for this site (indeed something that needs to be improved). Click on “Log in” and fill the requested data. As the page explains, if you already have an account in Ubuntu Brainstorm, you can use it for the ISO Testing Tracker.

Create a QA account

Create a QA account

Once you have an account just select the image you want to test, choose a test case for that image, start running the installation in a virtual machine or a spare machine and report any bugs you may find in Launchpad.

Once finished, you have to report back in the ISO Testing Tracker. Click on the test case you have run and report Pass/Fail depending on your success. You can also add the bug numbers of the bugs you found. They will be linked to the Launchpad bug and it will help the release team to track them.

Please, report back!

Please, report back!

The time is NOW!

Next week, on Thursday 1st of October, we are releasing Ubuntu Karmic Beta, which eventually will become Ubuntu 9.10. This is a very important milestone, because many people upgrade during the beta, so it is very important to release something very stable at this point. From Monday, 27th of September, we will be testing the candidate images at #ubuntu-testing. I will send reminders to the different mailing lists later this week and at the beginning of next week, but you can start right now creating your account and making yourself familiar with the ISO Testing Tracker.

Help us releasing Ubuntu better and better every 6 months! And remember that we are always at #ubuntu-testing if you need to ask any question!

Read more
Ara Pulido

Testing automation meetings

If you search for software testing automation in your favourite search engine you will find a lot of matches about testing automation of web applications, lots of proprietary software to test proprietary desktop software, lots of FOSS to test web applications and some blogs about testing proprietary desktop software.

What about testing desktop FOSS? How do I find people interested in these topics? Is there anybody out there? Hello?

Fortunately, there is a growing community in the FOSS world interested in testing desktop applications automatically. To try to keep things going, we are having weekly meetings at #ubuntu-testing ( every Wednesday at 16:30UTC, running for half an hour.

If you are interested in these topics, please, join us and feel free to add any items you may want to discuss to the agenda.

Read more
Ara Pulido

Mago Roadmap

It is great to see more and more people interested in Mago.

When I started the project about a year ago (by that time it was called Ubuntu Desktop Testing), it was an in-house project I created to solve the problems I was having on the desktop testing I had to do as part of my daily job.

Up to three sessions at the Ubuntu Developer Week, a talk at Gran Canaria Desktop Summit, some UDS and two changes of name later, Mago seems to have found its path and we are seeing a lot of new contributors adding fantastic ideas to it.

To avoid losing any of these ideas, I have created a Roadmap in the Mago site to track the status of the new Mago features. Please, check it out to see what’s happening on the Free Desktop testing automation world.

Read more
Ara Pulido

Mago tutorial

In about a couple of hours (16UTC) I will be giving a Mago tutorial as part of the Ubuntu Developer Week.

Please, join us on #ubuntu-classroom for a quick hands-on Mago training session!

Read more
Ara Pulido

Perfect timing

Today everything went smoothly:

Teamwork rocks!

Read more
Ara Pulido

Mago is in Karmic!!

Yes, if you updated your Karmic repositories lately, you can install Mago just typing “sudo apt-get install mago”. What does that mean? Not much, for the moment.

We have packaged the library and harness, but not the tests. Once you have installed Mago in your machine, you can start writing tests using the library, you will be able to run the tests with the Mago harness and get nice reports in XML and HTML. You won’t have the already written tests, though. Tests change a lot, so it does not make sense to keep them in the repositories, which are quite static. PPAs are a much better place for tests, which I am planning to maintain.

I still have a lot of work to be done regarding this: I have to set up the PPA for the tests and update the documentation at the Mago site, to start. Why did I push Mago into Karmic, then? There are two main reason: Feature Freeze and Holidays. Ubuntu development schedule has an important date called Feature Freeze, after which no new packages are generally accepted and only bug fixes are supposed to get uploaded. Karmic FF is happening August 27th, but I will be on holidays since August 14th, so it was now or never (or Karmic+1).

The main advantage of having Mago in the repositories is that, if a project wants to use it as part of the testing of their daily builds, they can set Mago as a Build-Depends of the project and forget about whether Mago is installed or not in the build machine. And, don’t worry, I will be updating the documentation and setting up the tests PPAs after my holidays, but, in the mean time, Mago is there, ready before Feature Freeze. Happy testing!

Read more