Canonical Voices

Posts tagged with 'ubuntu'

Nicholas Skaggs

It's finally here! We've been working on a way to allow those who have a ubuntu phone to participate more directly in testing the software that runs on their device. This includes things like helping test OTA updates before they are shipped and to verify and look for bugs in applications like the core apps and system services.

Introducing Pilot, a new application you can find today in the ubuntu store. The application utilizes checkbox as a way of distributing tests to you on the phone. This first round of testing includes tests from 4 of your favorite core applications including dekko, clock, music, and weather.

To help test, search for Pilot in the store and install it.

Start the app, and click the Start Testing button once it's loaded.

Select a test plan to run. Right now you can choose to test specific features of the different core apps.

Select the tests to run. You can choose to run all of tests for that feature, or just one if you wish.

Run through the test, following each step. If everything works as listed in the test, press the Pass button. Otherwise press Fail.

You can also add comments about the test or skip the test using the buttons at the top of this page.

Finally, submit your results back to the QA team by pressing the Submit Results to Community Practitest button. You'll need to supply your ubuntu SSO information to do so. You may also view your submitted results on this screen by pressing the corresponding button.

It's that easy. Over time, we'll push new tests via application updates, so you can help test new things as they are developed. As the number of devices grows, we want to ensure every device has the same level of quality. With your help, we can make sure ubuntu gets better with each update. Thanks for your help!

Read more
Nicholas Skaggs

Some of you may remember the birth of the ubuntu font family during the 10.10 cycle. The time has come to finish that work as well as fix a few issues with the current font set. To start with, the design team has been working on Arabic, and is ready for some feedback on how the font looks and interacts.

To help gather your feedback, we've made a simple survey. It contains the information you need to get the font, as well as the opportunity to leave feedback.

We would love to hear from you! If you encounter any issues trying to test or use the survey, feel free to get in touch, but otherwise leave your feedback on the font in the survey. Thanks again for your help!

For those of you who don't happen to speak Arabic or a related language, an opportunity to test the full ubuntu font family is coming up soon. Get ready!

Read more
Nicholas Skaggs

Final Beta Testing for Wily

Another cycle draws to a close, and it's time to test our images and make sure Wily is in good shape. We're entering crunch time.

How can I help? 
To help test, visit the iso tracker milestone page for final beta.  The goal is to verify the images in preparation for the release. Find those bugs! The information at the top of the page will help you if you need help reporting a bug or understanding how to test. 

There's a first time for everything! Check out the handy links on top of the isotracker page detailing how to perform an image test, as well as a little about how the qatracker itself works. If you still aren't sure or get stuck, feel free to contact the qa community or myself for help.

How long is this going on?
The testing runs through tomorrow, Thursday September 24th, when the the images for final beta will be released. If you miss the deadline we still love getting result Test against the daily image milestone instead.

Thanks and happy testing everyone!

Read more
Nicholas Skaggs

Ubuntu SDK Autopilot Plugin

Those of you who have developed an application using the Ubuntu SDK understand how nice it is to have a tool to support your workflow for writing an application. You can code, build, run and iterate on your code easily right from inside the SDK. However, to test your application, it was necessary to open a terminal and execute some commands. Leaving the Ubuntu SDK is an interruption to your workflow! It's even enough to throw you off your coding zen! It certainly may have dissuaded you from running tests. Seeing as testing should be a positive experience, this certainly won't do!

Thankfully Akiva thought the same thing. Thus he created a new plugin for the SDK. I'd like to celebrate and thank him for making all of our lives easier. Thanks Akiva! A big thank you to Benjamin from the SDK team as well for reviewing and helping get the plugin in shape.

The plugin scans your project for autopilot tests, and then creates a run configuration for them. From there, it's as easy as hitting the run button to run the application. See for yourself!

To learn more about how to install the plugin, or how it works, checkout the documentation on running autopilot tests found on 

Go forth and test all the things! Try out using the plugin in your existing workflow. I'd love to hear feedback. If you are interested in making the plugin better, or expanding it to include other things, get in touch. As always, code is welcome!

Read more
Nicholas Skaggs

It's never been easier to write tests for your application! I wanted to share some details on the new documentation and other tidbits that will help you ensure your application has a nice testsuite. If you've used the SDK in the past, you understand how nice it can make your development workflow. Writing code and running it on your desktop, device, or emulator is a snap.

Fortunately, having a nice testsuite for your application can also be just as easy. First, you will notice that now all of the wizards inside the SDK now come with nice testsuites already in place. They are ready for you to simply add-on more tests. The setup and heavy lifting is done. See for yourself!

Secondly, has a great section on every level of testing; no matter which language you use with the SDK. You'll find API references for the tools and technology used, along with helpful guides to get you in the proper mindset.

For autopilot itself, there's also API documentation for the various 'helpers' that will make writing tests much easier for you. In addition, there's a guide to running autopilot tests. This has been made even easier by the addition of Akiva's Autopilot plugin inside the SDK. I'll be sharing details on this as soon as it's packaged, but you can see a sneak peek in this video.

Finally, you will find a guide on how to structure your functional tests. These are the most demanding to write, and it's important to ensure you write your tests in a maintainable way. Don't forget about the guide on writing good functional tests either.

No matter what language or level you write tests for, the guides are there to help you. Why not trying adding a test or two to your project? If you are new, check out one of the wizards and try adding a simple testcase. Then apply the same knowledge (and templated code!) to your own project. Happy test writing!

Read more
Nicholas Skaggs

It's Show and Tell Time!

Err, show and tell?
Who remembers their first years of schooling? At least for me growing up in the US, those first years invovled an activity called 'Show and Tell'. We were instructed to bring something in from home and talk about it. This could be a picture or souvenir from a trip or unique life event, something we made, another person who does interesting things, or just something we found really interesting. It was a way for us to learn more about each other in the classroom, as well as share cool things with each other.

Online Summit
Ok, snapping you back to reality, it's nearing time for UOS 15.05. UOS is the Ubuntu Online Summit we hold each cycle to talk about what's happening in ubuntu. UOS 15.05 will be on May 5th - May 7th.

So what does the childhood version of me reminiscing about show and tell have to do with UOS? Well, I'm glad you asked! There is a 'Show and Tell' track available to everyone as a platform for sharing interesting and unique things with the rest of the community. These sessions can be very short (5 or 10 minutes) and are a great way to share about your work within ubuntu.

With that in mind, it's a perfect opportunity for you to participate in 'Show and Tell' with the rest of the community. I encourage you to propose a session on the 'Show and Tell' track. This track exists for things like demos, quick talks, and 'show and tell' type things. It's perfect to spend 5 or 10 minutes talking about something you made or work on. Or perhaps something you find interesting. Or just a way to share a little about the team you work with or a project you've done. For those of you who may have been a part of the 'lightning talks' during the days of the physical UDS, anything that would have been considered a lightning talk is more than welcome in this track.

Cool, where do I signup?
Proposing a session is simple to do, and there's even a webpage to help! If you really get stuck, feel free to contact me, Svetlana Belkin, Marco Ceppi, or Allan Lesage who are your friendly track leads for this track. Once it's proposed the session will be assigned a date and time. Myself or another track lead will follow-up with you before UOS to ensure you are ready and the date and time is suitable for you.

Is there another way to participate?
Yes! Remember to checkout the show and tell sessions and participate by asking questions and enjoying the presentations. I guarantee you will learn something new. Maybe even useful!

Thanks for helping make UOS a success. I'll see you there!

Read more
Nicholas Skaggs

Whoosh, Spring is in the air, Winter is over (at least for us Northern Hemisphere folks). With that, it's time for polishing the final beta image for vivid.

How can I help? 
To help test, visit the iso tracker milestone page for final beta.  The goal is to verify the images in preparation for the release. Find those bugs! The information at the top of the page will help you if you need help reporting a bug or understanding how to test. 

There's a first time for everything! Check out the handy links on top of the isotracker page detailing how to perform an image test, as well as a little about how the qatracker itself works. If you still aren't sure or get stuck, feel free to contact the qa community or myself for help.

What if I'm late?
The testing runs through this Thursday March 26th, when the the images for final beta will be released. If you miss the deadline we still love getting results! Test against the daily image milestone instead.

Thanks and happy testing everyone!

Read more
Nicholas Skaggs

Unity 8 Desktop Testing

While much of the excitement around unity8 and the next generation of ubuntu has revolved around mobile, again I'd like to point your attention to the desktop. The unity8 desktop is starting to evolve and gain more "desktopy" features. This includes things like window management and keyboard shortcuts for unity8, and MIR enhancements with things like native library support for rendering and support for X11 applications.

I hosted a session with Stephen Webb at UOS last year where we discussed the status of running unity8 on the desktop. During the session I mentioned my own personal goal of having some brave community members running unity8 as there default desktop this cycle. Now, it's still a bit early to realize that goal, but it is getting much closer! To help get there, I would encourage you to have a look at unity8 on your desktop and start running it. The development teams are ready for feedback and anxious to get it in shape on the desktop.

So how do you get it? Check out the unity8 desktop wiki page which explains how you can run unity8, even if you are on a stable version of ubuntu like the LTS. Install it locally in an lxc container and you can login to a unity8 desktop on your current pc. Check it out! After you finish playing, please don't forget to file bugs for anything you might find. The wiki page has you covered there as well. Enjoy unity8!

Read more
Nicholas Skaggs

It's time for a testing jam!

Ubuntu Global Jam, Vivid edition is a few short weeks away. It's time to make your event happen. I can help! Here's my officially unofficial guide to global jam success.


  1. Get your jam pack! Get the request in right away so it gets to you on time. 
  2. Pick a cool location to jam
  3. Tell everyone! (be sure to mention free swag, who can resist!?)
But wait, what are you going to do while jamming? I've got that covered too! Hold a testing jam! All you need to know can be found on the ubuntu global jam wiki. The wiki even has more information for you as a jam host in case you have questions or just like details.

Ohh and just in case you don't like testing (seems crazy, I know), there are other jam ideas available to you. The important thing is you get together with other ubuntu aficionados and celebrate ubuntu! 

P.S. Don't forget to share pictures afterwards. No one will know you had the coolest jam in the world unless you tell them :-)

P.P.S. If I'm invited, bring cupcakes! Yum!

Read more
Nicholas Skaggs

PSA: Community Requests

As you plan your ubuntu related activities this year, I wanted to highlight an opportunity for you to request materials and funds to help make your plans reality. The funds are donations made by other ubuntu enthusiasts to support ubuntu and specifically to enable community requests. In other words, if you need help traveling to a conference to support ubuntu, planning a local event, holding a hackathon, etc, the community donations fund can help.

Check out the funding page for more information on how to apply and the requirements. In short, if you are a ubuntu member and want to do something to further ubuntu, you can request materials and funding to help. Global Jam is less than a month away, is your loco ready? Flavors, trying to plan events or hold other activities? I'd encourage all of you to submit requests if money or materials can help enable or enhance your efforts to spread ubuntu. Here's to sharing the joy of ubuntu this year!

Read more
Nicholas Skaggs

I thought I would add a little festivity to the holiday season, quality style. In case your holidays just are not the same without a little quality in your life, allow me to share how you can get involved.

There are opportunities for every role listed on the QA wiki. Testers and test writers are both needed. Testing and writing manual tests can be learned by anyone, no coding required. That said if you have skills or interest in technical work, I would encourage you help out. You will learn by doing and get help from others while you do it.

Now onto the good stuff! What can you do to help ubuntu this cycle from a quality perspective?

There is an ever present need for brave folks willing to simply run the development version of ubuntu and use it as a daily machine throughout the cycle. It's one of the best ways for us as a community to uncover bugs and issues, in particular things that regress from the previous release. Upgrade to vivid today and see what you can break!

This tool is written in drupal7 and runs the and sites. These sites are used to record and view the results of all of our manual testing efforts. Currently dkessel is leading the effort on implementing some needed UI changes. The code and more information about the project can be found on launchpad. The tracker is one of our primary tools and needs your help to become friendly for everyone to use.

In addition a charm would be useful to simplify setting up a development environment. The charm can be based upon the existing drupal charm. At the moment this work is ready for someone to jump in.

Running unity8 as a full-time desktop is a personal goal I have for this cycle. I hope some others might also want to be early adopters and join me in this goal. For now you can help by testing the unity8 desktop. Have a look at running unity in lxc for an easy way to run unity8 today on your machine. Use it, test it, and offer feedback. I'll be talking more about unity8 as the cycle progresses and opportunities to test new features aimed at the desktop appear.

Core Apps
The core apps project is an excellent way to get involved. These applications have been lovingly developed by community members just like you. Many of the teams are looking for help in writing tests and for someone who can help bring a testing mindset and eye to the work. As of this writing specifically the docviewer, terminal and calculator teams would love your help. The core apps hackdays are happening this week, drop by and introduce yourself to get started!

Manual Tests
Like the sound of writing tests but the idea of writing code turns you off? Manual tests are needed as well! They are written in English and are easy to understand and write. Manual tests include everything you see on the qatracker and are managed as a launchpad project. This means you can pick a bug and "fix it" by submitting a merge request. The bugs involve both fixing existing tests as well as requests for new testcases.

As always there are images that need testing. Testing milestones occur later in the cycle which involve everyone helping to test a specific set of images. In the meantime, daily images are generated that have made it through the automated tests and are ready for manual testing. Booting an image in a live session is a great way to check for regressions on your machine. Doing this early in the cycle can help make sure your hardware and others like it experience a regression free upgrade when the time comes.

After subjecting software to testing, bugs are naturally found. These bugs then need to be verified and triaged. The bugsquadders, as they are called, would be happy to help you learn to categorize or triage bugs and do other tasks.

No matter how you choose to get involved, feel free to contact me for help if needed. Most of all, Happy Testing!

Read more
Nicholas Skaggs

Creating mutli-arch click packages

Click packages are one of the pieces of new technology that drives the next version of ubuntu on the phone and desktop. In a nutshell click packages allow for application developers to easily package and deliver application updates independent of the distribution release or archive. Without going into the interesting technical merits and de-merits of click packages, this means the consumer can get faster application updates. But much of the discussion and usage of click packages until now has revolved around mobile. I wanted to talk about using click packages on the desktop and packaging clicks for multiple architectures.

The manifest file
Click packages follow a specific format. Click packages contain a payload of an application's libraries, code, artwork and resources, along with its needed external dependencies. The description of the package is found in the manifest file, which is what I'd like to talk about. The file must contain a few keys, but one of the recognized optional keys is architecture. This key allows specifying architectures the package will run on.

If an application contains no compiled code, simply use 'all' as the value for architecture. This accomplishes the goal of running on all supported architectures and many of the applications currently in the ubuntu touch store fall into this category. However, an increasing number of applications do contain compiled code. Here's how to enable support across architectures for projects with compiled code.

Fat packages
The click format along with the ubuntu touch store fully support specifying one or more values for specific architecture support inside the application manifest file. Those values follow the same format as dpkg architecture names. Now in theory if a project containing compiled code lists the architectures to support, click build should be able to build one package for all. However, for now this process requires a little manual intervention. So lets talk about building a fat (or big boned!) package that contains support for multiple architectures inside a single click package.

Those who just want to skip ahead can check out the example package I put together using clock. This same package can be found in the store as multi-arch clock test. Feel free to install the click package on the desktop, the i386 emulator and an armhf device.

Building a click for a different architecture
To make a multi-arch package a click package needs to be built for each desired architecture. Follow this tutorial on for more information on how to create a click target for each architecture. Once all the targets are setup, use the ubuntu sdk to build a click for each target. The end result is a click file specific to each architecture.

For example in creating the clock package above, I built a click for amd64, i386 and armhf. Three files were generated:

Notice the handy naming scheme allows for easy differentiation as to which click belongs to which architecture. Next, extract the compiled code from each click package. This can be accomplished by utilizing dpkg. For example,

dpkg -x amd64

Do this for each package. The result should be a folder corresponding to each package architecture.

Next copy one version of the package for use as the base of multi-arch click package. In addition, remove all the compiled code under the lib folder. This folder will be populated with the extracted compiled code from the architecture specific click packages.

cp amd64 multi
rm -rf multi/lib/*

Now there is a folder for each click package, and a new folder named multi that contains the application, minus any compiled code.

Creating the multi-arch click
Inside the extracted click packages is a lib folder. The compiled modules should be arranged inside, potentially inside an architecture subfolder (depending on how the package is built).

Copy all of the compiled modules into a new folder inside the lib folder of the multi directory. The folder name should correspond to the architecture of the complied code. Here's a list of the architectures for ARM, i386, and amd64 respectively.


You can check the naming from an intended device by looking in the application-click.conf file.

grep ARCH /usr/share/upstart/sessions/application-click.conf

To use the clock package as an example again, here's a quick look at the folder structure:


The contents of lib/* from each click package I built earlier is under a corresponding folder inside the multi/lib directory. So for example, the lib folder from became lib/i386-linux-gnu/.

Presto, magic package time! 
Finally the manifest.json file needs to be updated to reflect support for the desired architectures. Inside the manifest.json file under the multi directory, edit the architecture key values to list all supported architectures for the new package. For example to list support for ARM and x86 architectures,

"architecture": ["armhf", "i386", "amd64"],

To build the new package, execute click build multi. The resulting click should build and be named with a prefix. This click can be installed on any of the specified architectures and is ready to be uploaded to the store.

Caveats, nibbly bits and bugs
So apart from click not automagically building these packages, there is one other bug as of this writing. The resulting multi-arch click will fail the automated store review and instead enter manual review. To workaround this request a manual review. Upon approval, the application will enter the store as usual.

In summary to create a multi-arch click package build a click for each supported architecture. Then pull the compiled library code from each click and place into a single click package. Next modify the click manifest file to state all of the architectures supported. Finally, rebuild the click package!

I trust this explanation and example provides encouragement to include support for x86 platforms when creating and uploading a click package to the store. Undoubtedly there are other ways to build a multi-arch click; simply ensure all the compiled code for each architecture is included inside the click package. Feel free to experiment!

If you have any questions as usual feel free to contact me. I look forward to seeing more applications in the store from my unity8 desktop!

Read more
Nicholas Skaggs

Virtual Hugs of appreciation!

Because I was asleep at the wheel (err, keyboard) yesterday I failed to express my appreciation for some folks. It's a day for hugging! And I missed it!

I gave everyone a shoutout on social media, but since planet looks best overrun with thank you posts, I shall blog it as well!

Thank you to:

David Planella for being the rock that has anchored the team.
Leo Arias for being super awesome and making testing what it is today on all the core apps.
Carla Sella for working tirelessly on many many different things in the years I've known her. She never gives up (even when I've tried too!), and has many successes to her name for that reason.
Nekhelesh Ramananthan for always being willing to let clock app be the guinea pig
Elfy, for rocking the manual tests project. Seriously awesome work. Everytime you use the tracker, just know elfy has been a part of making that testcase happen.
Jean-Baptiste Lallement and Martin Pitt for making some of my many wishes come true over the years with quality community efforts. Autopkgtest is but one of these.

And many more. Plus some I've forgotten. I can't give hugs to everyone, but I'm willing to try!

To everyone in the ubuntu community, thanks for making ubuntu the wonderful community it is!

Read more
Nicholas Skaggs

Ubuntu Online Summit: Vivid Edition

Ubuntu Online Summit is once again upon us. This is a community event by and for the community. It's all encompassing and intends to cover a wide range of topics. You don't need to be a developer, project lead, member of a team, or even a member of ubuntu to join and participate. The only requirement is your passion for ubuntu and desire to discuss about it's future with others.

The dates are set as November 12-November 14th from 1400 UTC to 2000 UTC. I am once again privileged to be a track lead for the users track. In my opinion, this is the best track as it's the one the largest number of us within the community can easily feel a part of (just don't like Michael, David, Daniel or Alan know I said that). Do you use ubuntu? Awesome, this is the track for you.

What I'm asking for is sessions. Have an idea for a session? Please propose it! Everything you need to know about participating can be found here. If you've attended things like ubuntu open week or a classroom session in the past, all of those types of sessions are welcome and encouraged.

"The focus of the Users track is to highlight ways to get the most out of Ubuntu, on your laptop, your phone or your server. From detailed how-to sessions, to tips and tricks, and more, this track can provide something for everybody, regardless of skill level."

Regardless of your desire to contribute a session, I would encourage everyone to take a look at the schedule as it evolves and considering joining in sessions they find interesting. In addition, it's not yet too late to offer up ideas for sessions (though I would encourage you to find a way to host the session).

Ready to propose a session? Checkout this page and feel free to ping me or any track lead for help. Don't forget to register to attend and check out the currently scheduled sessions!

Read more
Nicholas Skaggs

Autopilot Feature Primer

Autopilot celebrated it's 2 year anniversary as an independent project this summer. During that time it has developed into a useful tool for testing application UI's for gtk and qt toolkits. Support has also been extended to MIR as well as phablet devices.

With this in mind, I thought it would be useful to bring attention to some new and under-used features of autopilot, along with providing a brief explanation of some companion tools you might find useful. Thus I present to you, an autopilot primer. Let's talk through some new features shall we?

Python3 Support
Autopilot started as a python2 tool but has since migrated to python3 and you should too! For now the entire source tree remains python2 compatible, but you really should migrate your tests to python3. You'll notice the autopilot3 binary in newer releases which should be used to run autopilot with python3.

Scenario Support
Scenarios are a wonderful way to keep your tests simple and easy to read while allowing you to test with many different inputs. In short, you might need to test several edge cases as part of your acceptance testing. This is most easily accomplished by keeping the test itself generic and utilizing a scenario to vary your inputs. You can check out more information on scenarios specific to autopilot in the autopilot documentation.

Screenshots / Video
Autopilot allows you to get a video recording of a test failure. To make sure autopilot records failures, install recordmydesktop and pass the -r argument to your autopilot3 run command. However, at the moment this requires X so for now it doesn't work with the MIR backend (which things like the ubuntu phone utilize). Fortunately a screenshot at the point of failure when combined with the log is generally sufficient to solving your issue. Getting those screenshots brings us to subunit support.

Subunit Support
By default autopilot generates the test output and logs straight to your console in a text format. However autopilot also supports outputting to xml and subunit. Subunit support is what I would like to highlight for a few reasons. When you set the output format as subunit, you get a few niceties. One of which is an easier to grok format for tools, and the other is screenshots of the application when failures occur. To get a subunit stream, pass the -f subunit argument to your autopilot3 run command. You will want to also pass -O with a filename to save the output to a file as the subunit stream contains binary data.

Test Result Viewer
So, with this subunit test results file, how can you enjoy all of it's goodness? Enter trv, a simple python ui that will let organize the test run in an easy to view manner, including screenshots. The tool is the creation of Thomi Richards who describes it as a quick hack (:p), and has a youtube video demonstrating it's use. It's perfect for viewing the subunit stream and visualizing your test results. For now, it's not packaged but can be easily obtained via launchpad. Grab it with bzr branch lp:trv.

autopilot3 vis
The vis tool allows you to visually interact with the introspection tree after launching an application using autopilot launch. What this means is you can visualize the application in the same way autopilot does at runtime, with live tree updates. It lets you see what autopilot sees, allowing you to interactively build your testcase.

I'll refer you the official tutorial for more information, as well as a youtube video by yours truly. It's from a livestream, but covers what you need to know. autopilot3 vis also contains a search box, and a highlight tool which didn't exist in the orignal version, so it's even nicer now than before. Give it a whirl!

I talked about this utility when I covered the test runners for autopilot. Still I would be remiss if I didn't mention it again. Everything I said in the test runners for autopilot post still applies, so go have a quick read about how to use the tool if you need more information. This tool enables you to easily run autopilot tests on the desktop in a nested xserver. What that means to you as a test author is that you can run tests without giving up your desktop session. No more waiting for autopilot to hand back control of your mouse after a test. If you are writing tests, you should be using this tool along with autopilot vis mentioned above during your test writing process.

Per test timeout
Although we all only write "good tests", sometimes you may find your test misbehaves. When this happens the test may even not exit cleanly or get stuck in a loop. The result is autopilot and the system under test to wait forever for the test to finish. To prevent a rouge test from killing a test suite run, autopilot is introducing support for per-test timeouts. This has landed in vivid; you'll need version 1.5.0+15.04.20141031-0ubuntu1 or later. To use the feature, add the --test-timeout argument to autopilot run and give is a timeout in seconds.

In conclusion
Autopilot has gotten many new features along the way, and these are but a few of the most recent and important ones. I hope this helps you take another look at what autopilot might be able to help you test. Happy Testing!

Read more
Nicholas Skaggs

Sprinting in DC: Friday

This week, my team and I are sprinting with many of the core app developers and other folks inside of Ubuntu Engineering. Each day I'm attempting to give you a glimpse of what's happening.

Friday brings an end to an exciting week, and the faces of myself and those around me reflect the discussions, excitement, fun and lack of sleep this week has entailed.

The first session of the day involved hanging out with the QA team while they heard feedback from various teams on issues with quality and process within there project. Always fun to hear about what causes different teams the most issues when it comes to testing.

Next I spent some time interviewing a couple folks for publishing later. In my case I interviewed Thomi from the QA team and Zoltan from the SDK team about the work going on within there teams and how the last cycle went. The team as a whole has been conducting interviews all week. Look for these interviews to appear on youtube in the coming weeks.

Thursday night while having a look through a book store, I came across an ad for ubuntu in Linux Voice magazine. It made me smile. The dream of running ubuntu on all my devices is becoming closer every day.

I'd like to thank all the community core app developers who joined us this week. Thanks for hanging out with us, providing feedback, and most of all for the creating the wonderful apps we have for the ubuntu phone. Your work has helped shaped the device and turn it into what it is today.

Looking back over the schedule there were sessions I wish I had been able to attend, and it was wonderful catching up with everyone. Sadly my flight home prevented me from attending the closing session and presumably getting a summary of some of these sessions. I can say I was delighted to talk and interact with the unity8 team on the next steps for unity8 on the desktop. I trust next cycle we as a community can do more around testing there work.

As I head to the airport for home, it's time to celebrate the release of utopic unicorn!

Read more
Nicholas Skaggs

Sprinting in DC: Wednesday

This week, my team and I are sprinting with many of the core app developers and other folks inside of Ubuntu Engineering. Each day I'm attempting to give you a glimpse of what's happening.

To kick off the day, I led a session on something that has been wreaking havoc for application test writers within the core apps -- environment setup. In theory, setting up the environment to run your test should be easy. In practice, I've found it increasingly difficult. The music, calendar, clock, reminders, file manager and other teams have all been quite affected by this and the canonical QA team and myself have all pitched in to help, but struggled as well. In short, a test should be easy to launch, be well behaved and not delete any user data, and be easy to setup and feed test data into for the test process. I'm happy to report that the idea of a permanent solution has been reached. Now we must implement it of course, but the result should be drastically easier and more reliable test setup for you the test author.

I also had the chance to list some grievances for application developers with the QA team. We spoke about wanting to expand the documentation on testing and specifically targeted the need to create better templates in the ubuntu sdk for new projects. When you start a new project you should have well functioning tests, and we should teach you about how to run them too!

Just before lunch the community core app developers were able to discuss post-RTM plans and features. A review of the apps was undertaken and some desire for new designs or features were discussed. Terminal is being rebuilt to be more aligned with upstream. Music is currently undergoing a re-design which is coming along great. Calculator is anxious to get some design love. Reminders potential for offline notetaking as well as potential name changes were all discussed. Overall, an amazing accomplishment by all the developers!

After lunch, I spent time confirming the fix for a longstanding bug within autopilot. The merge proposal for fixing this bug has been simmering all summer and it's time to get it fixed. The current test suites for calendar and clock have been impacted by this and have already had regressions occur that could have been caught had tests been able to be written for this area. Having myself, the autopilot team, and the calendar developers in one place made fixing this possible.

To end the day, I spent some time attending sessions for changes to CI and learning more about the coming changes to CI within ubuntu. In summary the news is wonderful. CI will test using autopkgtest, and all of ubuntu will come under this umbrella -- phone, desktop, everything. If it's a package and it has tests, we will do all of the autopkgtest goodness currently being done for the distro.

The evening closed with a bit of fun provided by a game making hackathon using bacon2d and the hilariously horrible "Turkish Star Wars". We could always use more games in the ubuntu app store, and I hear there might even still be a pioneers t-shirt or two left if you get it in early!

Read more
Nicholas Skaggs

Sprinting in DC: Tuesday

This week, my team and I are sprinting with many of the core app developers and other folks inside of Ubuntu Engineering. Each day I'm attempting to give you a glimpse of what's happening.

On Tuesday I was finally able to sit down with the team and plan our week. In addition I was able to plan some of the work I had in mind with the community folks working on the core apps. Being obsessed with testing, my primary goals this week are centered around quality. Namely I want to make it easier for developers to write tests. Asking them to write tests is much easier when it's easy to do so. Fortunately, I think (hope?) all of the community core apps developers recognize the benefits to tests and thus are motivated to drive maturity into the testing story.

I'm also keen to work on the manual testing story. The community is imperative in helping test images for not only ubuntu, but also all of it's flavors. Seriously, you should say thank you to those folks helping make sure your install of ubuntu works well. They are busy this week helping make sure utopic is as good as it can be. Rock on image testers! But the tools and process used weigh on my mind, and I'm keen to chat later in the week with the canonical QA team and get there feedback.

During the day I attended sessions regarding changes and tweaks to the CI process. For core apps developers, errors in jenkins should be easier to replicate after these changes. CI will be moving to utilizing adt-run (autopkgtest) for there test execution (and you should too!). They will also provide the exact commands used to run the test. That means you can easily duplicate the results on the dashboard locally and fix the issues found. No more works on my box excuses!

I also met the team responsible for the application store and gave them feedback on the application submission process. Submitting apps is already so simple, but even more cool things are happening on this front.

The end of the evening found us shuffling into cab's for a team dinner. We had a long table of folks eating Italian food and getting to know each other better.

After dinner, I pressured a few folks into having some dessert and ordered a sorbet for myself. After receiving no less than 4 fruit sorbets due to a misunderstanding, I began carving the fruits and sending plates of sorbet down the table. My testcase failed however when the plates all came back :-(

Read more
Nicholas Skaggs

Final Beta testing for Utopic

Can you believe final beta is here for utopic already? Where has the summer gone? The milestone and images are already prepared for the final beta testing. This is the first round of image testing for ubuntu this cycle. A final image will also be tested next month, but now is the time to try out the image on your system. Be sure to report any bugs you may find. This will help ensure there is time to fix them before the release images.

To help make sure the final utopic image is in good shape, we need your help and test results! Please, head over to the milestone on the isotracker, select your favorite flavor and perform the needed tests against the images.

If you've never submitted test results for the iso tracker, check out the handy links on top of the isotracker page detailing how to perform an image test, as well as a little about how the qatracker itself works. If you still aren't sure or get stuck, feel free to contact the qa community or myself for help. Happy Testing!

Read more
Nicholas Skaggs

Autopilot Test Runners

In my last next post, I discussed will discuss notable autopilot features and talk about how autopilot has matured since it became an independent project.

In the meantime I would be remiss if I didn't also talk about the different test runners commonly used with autopilot tests. In addition to the autopilot binary which can be executed to run the tests, different tools have cropped up to make running tests easier.

This tool ships with autopilot itself and was developed as a way to run autopilot test suites on your desktop in a sane manner. Run the autopilot3-sandbox-run command with --help to see all the options available. By default, the tests will run in an Xvfb server, all completely behind the scenes with the results being reported to you upon completion. This is a great way to run tests with no interference on your desktop. If you are a visual person like me, you may instead wish to pass -X to enable the test runs to occur in a Xephyr window allowing you to see what's happening, but still retaining control of your mouse and keyboard.

I need this tool!
sudo apt-get install python3-autopilot

I want to run tests on my desktop without losing control of my mouse!
autopilot3-sandbox-run my_testsuite_name

I want to run tests on my desktop without losing control of my mouse, but I still want to see what's happening!
autopilot3-sandbox-run -X my_testsuite_name

Autopkgtest was developed as a means to automatically test Debian packages, "as-installed". Recently support was added to also test click packages and to run on phablet devices. Autopkgtest will take care of dependencies, setting up autopilot, and unlocking the device. You can literally plug in a device and wait for the results. You should really checkout the README pages, including those on running tests. That said, here's a quick primer on running tests using autopkgtest.

I need this tool!
sudo apt-get install autopkgtest
If you are on trusty, grab and install the utopic deb from here.

I want to run tests for a click package installed on my device!
Awesome. This one is simple. Connect the device and then run:
adt-run --click --- ssh -s adb

For example,
adt-run --click --- ssh -s adb

will run the tests for the installed version of the music app on your device. You don't need to do anything else. For the curious, this works by reading the manifest file all click packages have. Read more here.

I want to run the tests I wrote/modified against an installed click package!
For this you need to also pass your local folder containing the tests. You will also want to make sure you installed the new version of the click package if needed.

adt-run my-folder/ --click --- ssh -s adb

Autopkgtest can also run in a lxc container, QEMU, a chroot, and other fun targets. In the examples above, I passed --- ssh -s adb as the target, instructing autopkgtest to use ssh and adb and thus run the tests on a connected phablet device. If you want to run autopilot tests on a phablet device, I recommend using autopkgtest as it handles everything for you.

This tool is part of the greater phablet-tools package. It was originally developed as an easy way to execute tests on your phablet device. Note however that copying the tests and any dependencies to the phablet device is left to you. The phablet-tools package provides some other useful utilities to help you with this (checkout phablet-click-test-setup for example).

I need this tool!
sudo apt-get install phablet-tools

I want to run the tests I wrote/modified against an installed click package!
First copy the tests to the device. You can use the ubuntu sdk or click-buddy for this, or even do it manually via adb. Then run phablet-test-run. It takes the same arguments as autopilot itself.

phablet-test-run -v my_testsuite

Note the tools looks for the testsuite and any dependencies of the testsuite inside the /home/phablet/autopilot folder. It's up to you to make sure everything that is needed to run your tests are located there or else it will fail.

other ways
There are of course other possible test runners that wrap around autopilot to make executing tests easier. Perhaps you've written a script yourself. Just remember at the end of the day the autopilot binary will be running the tests. It simply needs to be able to find the testsuite and all of it's dependencies in order to run. For this reason, don't be afraid to execute autopilot3 and run the tests yourself. Happy test runs!

Read more