Canonical Voices

Posts tagged with 'quality'

Nicholas Skaggs

A couple weeks ago we announced the initiative to drive up our autopilot (that is, automated) tests for our ubuntu touch core apps. The core apps are being made with the ubuntu sdk, and thus share the same language (QML) and toolkit (ubuntusdk).

With this in mind I wanted to provide an emulator, which in autopilot speak, is a utility class for writing autopilot tests that use the ubuntu SDK. The goal is to help accelerate the process for getting the testcases written, as well as standardizing best practices for testing common features. At the moment the emulator contains useful functions like tab switching, selecting from popovers, opening and closing the toolbar and clicking toolbar buttons. Please, take a look and utilize the emulator when you are contributing new tests for the ubuntu touch applications. For the moment, the emulator can be found here:

lp:~nskaggs/+junk/ubuntusdk_autopilot_emulator

The future home is hopefully in the SDK itself, but for now consider that branch your source for emulator goodness. Now, a quick FAQ.

Is it ready for use?
Yes, it's ready and tested on several core apps now including clock, calendar, terminal, and file manager. That said if you find an issue, simply contact me or propose an improvement!

How do I use it?
Inside your autopilot test subfolder, add an emulators folder if it's not already present. Next, branch my source above -- it will add ubuntusdk.py to the folder. Simply incorporate it into your __init__.py or testcase itself and call the utility functions with ubuntusdk.*. For an example check out the ubuntu-terminal-app and the merge request from today. It shows adding autopilot tests to an empty branch. In addition, the emulator (albeit an earlier version) was used in the tutorial on the ubuntu app developer portal.

Will it be updated?
Yes! Expect refinements and tweaks as we go along. Hopefully a true "stable and complete" version will appear in the not too distant feature when the emulator itself has a proper home. In the meantime, use it and as more complex tests are added, expect to update the emulator in the source branch you are working in.

Go forth and write tests!

Read more
Nicholas Skaggs

QATracker Survey + bonus mockup

Hot on the heels of our first cadence week, I wanted to take the opportunity to collect feedback about the tools we as a community utilize. Specifically the QATracker which we heavily rely on for managing our work, testcases and results. From the wiki, "The QATracker is the master repository for all our our testing within ubuntu QA. It holds our testcases, records our results, and helps coordinate our testing events."

This is a link to a brief survey asking a few simple questions about how you've used the tool. All your responses are anonymous, but I will publish the aggregate question information and share it with the community once completed. The goal is to help ensure the tool is meeting our needs and is being utilized.

I'll leave the survey up until June 24th. My hope is to encourage more folks to help test as well as make it more enjoyable for those already taking part. I want to ensure our tools and processes continue to evolve, strengthen and become more robust for everyone as we continue on our mission. Part of that is making sure the tools we use are enjoyable!

Thanks in advance everyone!

As a bonus, Pasi, aka knome, has put together some mockups on how we might be able to switch what the results page looks like. This is perhaps the most utilized page of the site, so without further ado, here's a mockup of some changes proposed to make it more usable:

Old Site
New Site Mockup


What a change eh? The add test results has been moved to the sidebar and simplified, the bugs listing has been written out, and the results have been moved to the top. Finally the links have also been moved to the sidebar and Pasi has updated the icons ;-)

SO, what does everyone think about the changes? Many thanks to Pasi for putting this together! Leave a comment, a message on the mailing list, or reflect your thoughts in the survey.

Read more
Nicholas Skaggs

Join the ubuntu quality community team's effort this week! As a community we test different things about every ~2 weeks in ubuntu, and share the results to flesh out bugs and problem areas.

So what's up for testing this week? The daily images, the default applications in ubuntu and a new version of the sound stack for testing.

Ready to help? Full details are here.

Need some help on how to contribute? Have a look at this page and the walkthroughs listed. Of particular interest is the ISO testing and Cadence Week testing walkthroughs.

Do note that you don't need anything special to participate in cadence week testing! Both an installed version of the development branch of ubuntu (aka saucy) in a VM or on a real box, or even a live session of the latest daily image will work. For more information on how to use a live session to test, check out the Cadence Week testing walkthrough or watch the youtube video of the same.
Happy Testing!

Read more
Nicholas Skaggs

A few months ago the ubuntu touch core apps project was launched. For those of you following along with Michael's regular updates have gotten to see these applications grow up rather quickly.

Autopilot Says: How can I help?
Now it's time to add some more testing around these applications as they have reached a basic functional level of usability. Automated testing via autopilot to the rescue!

To help kickstart this process we've put together a recipe for writing autopilot tests specific to QML applications and added it to developer.ubuntu.com. In addition, we'll be hosting a hackfest next week on June 13th to help add basic autopilot testcases for each of the core apps. Folks will be on-hand ready to field your questions and hack together on the autopilot testcases needed for the applications. Join us and help support the wonderful community of application developers making awesome applications for ubuntu!

So how can you help? 
  1. First, go read through the recipe on writing autopilot tests for QML applications. It's also a good idea to have a look through the official tutorial for autopilot and bookmark the API reference link so it's handy.
  2. Armed with your new knowledge, start hacking on some autopilot tests for the core apps. Here's a list of core applications along with the status of autopilot tests. Choose something that looks interesting to you and add some tests.
  3. Follow the contributing guide to help you get your work contributed into the ubuntu touch core application project you chose.
  4. Finally come out to the hackfest! It's your chance to share your work, ask questions, get your tests sorted and merged and socialize and meet other members of the community.
  5. Don't forget there is a wonderful quality community you can be a part of and get help from if you get stuck! There's a mailing list for ubuntu-touch, and ubuntu-quality as well as IRC channels #ubuntu-touch, #ubuntu-autopilot and #ubuntu-quality. Use these resources to help you!
See you next week and happy testing!

Read more
Nicholas Skaggs

This post is part of a series on the people behind the ubuntu quality team. Let me introduce you to Jackson, who loves modding his computer. He stumbled his way into ubuntu and hasn't looked back.


1) Could you provide a bit of background about yourself?

My name is Jackson Doak, i'm 14 and live in Australia. I use the nickname “Noskcaj” for everything online. I have been building PCs since i was 12, I've since attempted to make a volenti cooler, which is unfinished.

Check out that ubuntu orange!
2) How did you become invovled with the Ubuntu community?

I found Ubuntu after finishing my first PC and realizing I didn't have enough money to buy windows. I have since been using ubuntu on all my computers except my gaming PC. The ease of customization and security are to of the main reasons I still use Ubuntu.

3) What attracted you to the quality team?

I joined the QA team because I wanted to get involved with the community and it seemed the best option for me.

4) What would you say to folks new to ubuntu and/or testing?

To anyone new to the community/testing: Please try and help, any amount of time is useful, especially if you have a PowerPC computer or something that runs ubuntu touch. Even if you just tell your friends about ubuntu and get us more users, you are helping.

5) How would you describe the community and the experience of using ubuntu?

So far the community has been great, very helpful. Although there does seem to be a lack of cross group communication at times.

6) What would you like to see in the future for ubuntu?

I really do hope that canonical's vision of ubuntu on all 4 screens (PC, tablet, phone and TV) works, and UbuntuKylin's support helps a lot. I want xubuntu to be more well known and used, despite missing a real target audience at the moment.

7) Do you have a favorite experience to share from being a part of ubuntu?

I'm not sure i have a favourite experience.

8) What is your favorite activity or interest outside of computing (including ubuntu!)?

My favourite activity outside of computers would probably be playing soccer.

Read more
Nicholas Skaggs

This post is part of a series on the people behind the ubuntu quality team. Let me introduce you to Howard, an energetic and driven young man who has enjoyed being a part of the community. Howard exemplifies the ability to dive in and learn and continues to expand his horizons 

 and areas of contribution. 



1) Could you provide a bit of background about yourself?
Hello everyone, I'm Howard Chan (smartboyhw on World Wide Web), an energetic 14-year-old (K)ubuntu member from the busy harbour city of Hong Kong SAR, China. I like computers and science (especially Chemistry and Physics), and I like Ubuntu!

2) How did you become invovled with the Ubuntu community?
I was a Windows maniac before I used Ubuntu, and I liked running Beta software of things like Firefox Nightly, Microsoft Security Essentials Beta, Office Beta and such. I got fed up with Windows 8 Consumer Preview since I need to scroll the long start screen to access my applications, so I changed to Ubuntu 12.04 LTS. I liked Ubuntu, so I decided to start contributing. I joined the Ubuntu QA Team at first in June 2012, then I started to contribute to Ubuntu Studio as their tester for amd64 images. In January 2013 I started to do more development, helping out the Kubuntu Team to package KDE applications. I got my Ubuntu and Kubuntu membership respectively in February and March this year. I am now Ubuntu Studio's Release Manager, responsible for calling people to test, coordinating with the Ubuntu Release Team, marking images ready and such. I also go for some Testdrive hacking, and I am now working on different items in Ubuntu.

3) What attracted you to the quality team?
No need for programming skills. I only know beginner C, C++, Java (and more recently, Python). Except if you are writing automated testcases like Autopilot or autopkgtest, you don't need any programming skills to test applications and images for Ubuntu. You only need to like Ubuntu, that's all.

4) What would you say to folks new to ubuntu and/or testing?
Make sure you ask. If you are new in the community, try asking experienced people what to do and what NOT to do. Also, maintain a humble attitude. IRC etiquette is my biggest pain in the neck, and I don't want new contributors to make my mistakes. Never.

5) How would you describe the community and the experience of using ubuntu?
The community is awesome. Everywhere you go (for example Ubuntu Forums, the Ubuntu Studio IRC channels, Ask Ubuntu chatrooms, etc.) there is a lot of nice people willing to help you, no matter you are a novice user, or a starting contributor. Ubuntu itself is awesome too. We don't need to wait for extremely long release cycles like Debian. We have every desktop environment experiences to utilize. It's just great.

6) What would you like to see in the future for ubuntu?
A harmonized Ubuntu community. I have recently seen conflicts between many flavours, and many people (including me) disgruntling about Canonical moving to a new-styled community that doesn't seem to fit. Conflicts always happen between people, but I seriously hope that we can be more harmonized and create an awesome Linux distribution.

7) Do you have a favorite experience to share from being a part of ubuntu?
LOL that's difficult. A funny thing I've met is under-quorum. When I applied for Ubuntu membership, I almost can't get it since there was almost not enough quorum of Membership Board members until the last minute. That scared a lot of people coming to support me. When I applied for Kubuntu membership, I almost don't have enough votes for approving. That's two scary stories:)



8) What is your favorite activity or interest outside of computing (including ubuntu!)?
I love watching TV, Science and Fencing. I play fencing every Saturday. I am also a lover of teenage novels. Of course I like Ubuntu!

Read more
Nicholas Skaggs

This post is part of a series on the people behind the ubuntu quality team.  Let me introduce you to Javier who has had a hand in many different quality activitiesHe hails from Mexico and enjoys taking part in many different teams within ubuntu.

1) Could you provide a bit of background about yourself?

My name is Javier Lopez, I'm a random 2x years old guy living in Mexico city.

2) How did you become invovled with the Ubuntu community?

I become involved because I believe in the Ubuntu and open source philosophy.

3) What attracted you to the quality team?

Nicholas's blog, I used to be a Daniel Holbach super fan, but since I started reading Nicholas one, Daniel has fallen one spot, I like seeing all that energy spread over the community and its never ending enthusiasm. I'll hunt you and hug you both of you guys! ¬_¬

Contributing some iso testing results
4) What would you say to folks new to ubuntu and/or testing?

I'd encourage anyone to join any of the Ubuntu teams, you'll learn more about the system, will meet nice people and will help to build one of the most awesome and biggest projects around.

5) How would you describe the community and the experience of using ubuntu?

I'd say that the Ubuntu community is one of the most friendly communities I've ever meet, that feels great in addition to the nice system which comes with it.
As alpha tester, there are times where the ubuntu experience is not as great as an LTS ubuntu system, but I also enjoy seeing how the bugs reported get fixed and how the few patches I've sent get integrated.

6) What would you like to see in the future for ubuntu?

I'd like to see more loco teams awesomeness, people going crazy about it (something like a randall ross clone program), designers all over the place, companies funding more community programmers, more ubuntu woman members, an ubuntu skynet..

7) Do you have a favorite experience to share from being a part of ubuntu?

I like seeing people's face when they first meet an Ubuntu system, specially when they like it and they're not computer gangsters.

Javier enjoys playing the violin
8) What is your favorite activity or interest outside of computing (including ubuntu!)?

I like going out with my friends, and learning new things, I also like talking with other foreign people and traveling around my country, specially small towns. I'm trying to setup a couple of companies, so I'll be able to spend more time enjoying life en less working in an office.

Read more
Nicholas Skaggs

This post is part of a series on the people behind the ubuntu quality team. Let me introduce you to Sergio who has been an amazing contributor to the ubuntu-manual-tests. He hails from South America and is empowered to help the Spanish speaking population of ubuntu participate by making technical help available in there native language.

1) Could you provide a bit of background about yourself?

Hi! My name is Sergio Andrés Meneses as you can figure it out by my nickname (SergioMeneses), I am 24 years old, I am from Cúcuta/Colombia and I belong to the LocoCouncil and the Ubuntu QA-team.

2) How did you become invovled with the Ubuntu community?

I began with Ubuntu 7.10 and my first contact was in my university, Francisco de Paula Santander University. And later I met Ubuntu-Colombian team (My LocoTeam), where I learned a lot and especialy about how to work in community.

3) What attracted you to the quality team?

I always liked to work on testing, I have been really interesting about the quality in free software or open source projects, so this was an amazing oportunity to learn and help to do Ubuntu better. But in the Raring-cycle (13.04), I was more involved with another things like: Bugs, Reports, Test-cases, Testing-applications and as always: Testing-isos. This was my first contact officially with the ubuntu QA-team.

Contributing to the manual tests project
4) What would you say to folks new to ubuntu and/or testing?

An interesting question, let me think!... If you like technology, If you like to learn a lot and you want to share with amazing people. You're place is in the Ubuntu-Community. and why do you have to be in the testing team?... because we are the best team! we are not only users or IT engineers, we are friends with jokes, having a good time and the most important thing: we work to do Ubuntu better!

5) How would you describe the community and the experience of using ubuntu?

About the community I have the right word: Friendship, wherever you see friendship is the bigger characteristic in all the teams and it makes that your contributions on ubuntu are a good run. About my experience of using ubuntu: it's the best, I use Ubuntu in computers, its performance is amazing, as programmer and sysadmin I dont have any issue with it... I always recommend it. :)

6) What would you like to see in the future for ubuntu?

There are interesting things in the future but I'm going to put emphasis on two things: the rolling release system and ubuntu-touch especially phones, and why this? - because we had a passionate discussion in several mailinglists, blogs, forums, IRC... everybody was crazy! literally. But I like to see in the future something less technical: "I would like to see more young people working in the community, especially people from LatinAmerica"


Sergio mapping out a testcase for the software center

7) Do you have a favorite experience to share from being a part of ubuntu?

While I was doing my first merged, I didn't upload my code into my personal branch but I uploaded into +junk and I didn't what was wrong. Editors Note: uploading the branch to +junk means Sergio wouldn't be able to submit it in a merge request to the project he was contributing to. In other words, he made a contribution, but got lost trying to contribute it :-)

8) What is your favorite activity or interest outside of computing (including ubuntu!)?

I really like soccer! My favorite team is Manchester United (UK), I see many soccer games on tv and sometimes I play, when I have time enough.

Read more
Nicholas Skaggs

This post is part of a series on the people behind the ubuntu quality team. Below you'll hear from Carla Sella, who has been a wonderful spirit of can-do attitude on the team. Carla hails from Italy and enjoys being the guinea pig for new ideas and kickstarting new projects and efforts on the team. She's been a wonderful contributor to our ubuntu autopilot tests project, happily helping lead the charge towards automating our favorite desktop applications.

1) Could you provide a bit of background about yourself?


My name is Carla Sella (Letozaf_), I am Italian and I was born in South Africa. I work as a System Administrator in an Italian firm. I always loved having Linux installed on my PC's as I believe that "the only limiting factor of the Linux operating system is it's user". The thing I like most about Linux is the chance to hack and learn things.
I have tried various Linux distributions in the past, both at work and at home (maybe even Ubuntu), but it was only when a friend of mine told me about Ubuntu and the Ubuntu Community that I decided to give Ubuntu a better try and installed it on my PC, and now it's on my notebook, my husbands PC, my son's notebook and as soon as I will be able to, it will be installed on my Asus Eee Pad and my phone :D.
What I like about Ubuntu is the way Canonical and the Community are trying to make Ubuntu  an operating system for everyone, easy to use and not only for a restricted number of geeks. I think this is the best way to fix bug #1. Even if Ubuntu is "easy" to use this doesn't mean you cannot open a terminal and do some hacking too. With Ubuntu you can be a "normal user" or a hacker :).

2) How did you become invovled with the Ubuntu community?

I got involved in Ubuntu when a friend of mine, that is in the Italian Ubuntu Community, told me about the Ubuntu Community and how to contribute to Ubuntu.

3) What attracted you to the quality team?

The fact that I love Linux as you can learn a lot using it, joining the quality team is really cool and gives you the chance not only to learn a lot about Ubuntu and Linux, but also to try all the new features and "play" with them helping test Ubuntu before it's released.
You can do a lot of things from writing test cases to automatic testing, you just have to choose what you prefer doing best. Helping developers fix bugs you find is really cool as it makes you feel part of the Ubuntu world. And best of all you get to know a lot of interesting people from all over the world and you can exchange knowledge and improve your skills while helping test the latest Ubuntu release.

4) What would you say to folks new to ubuntu and/or testing?

Come and join the fun!
If you are a sort of a hacker and like to play around with Linux, you are in the right place. You can learn a lot and help Ubuntu while having fun.If you are not a hacker but a "normal" user, come an join too, you can also help by carrying out the easiest tests. There is place for everyone wanting to help Ubuntu work better and have less bugs. So stop complaining about the bugs and start helping to fix them :p.

Testing ubuntu on ARM with a pandaboard
5) How would you describe the community and the experience of using ubuntu?

The community is friendly, collaborating, fun, looking forward to the future and gives you the chance to improve yourself.
When I went to UDS-R back in October/November 2012 I got to know a lot of fantastic persons I had just known on IRC or by mail, they were all very kind and couldn't believe the warm welcome I received from everyone. Using Ubuntu is both user-friendly and hacky. I have been using Ubuntu for quite a long time now and  the improvements that have been made since then are incredible. It has changed a lot and is, to my opinion, the easiest and the most user-friendly  Linux distribution around.

6) What would you like to see in the future for ubuntu?

I wish Ubuntu will have the majority market share :p.

7) Do you have a favorite experience to share from being a part of ubuntu?

I have a lot of nice experiences, but maybe the favorite ones are testing Ubuntu 12.10 kernel on 12.04 userspace, hacking Autopilot with Nicholas, and trying out Umockdev with my camera on Shotwell with Martin.

8) What is your favorite activity or interest outside of computing (including ubuntu!)?

I love traveling, I like visiting places and seeing different cultures, how they live and how the places look like.
Most of all I like staying with my 7 year Son.

Read more
Nicholas Skaggs

I'm happy to announce a little blog mini-series, conspicuously entitled "People behind Ubuntu Quality". The next several posts will bring to you a set of interviews conducted with some ubuntu quality members. The interviews show the diversity in our team and work, along with the shared passion and interest we have.


Over the next several posts, you'll get a chance to virtually "meet" a few of the team members and witness the passion and diversity the team offers. Remember, we want you too! If you are curious to share in the excitement, have a passion to learn something, or have knowledge or skills that could help, please consider joining us. Now is an excellent time to learn the ropes, engage yourself and help ensure quality in the next Ubuntu release, Saucy Salamander!

Read more
Nicholas Skaggs

Consider this text your giant disclaimer. Just a reminder these images are not intended for end-users; please don't go flashing your device thinking you'll have a replacement for android. These images are intended for developers, enthusiasts and testers who want to help. If this describes you, please read on!

I'm happy to announce the ubuntu touch images are now available for testing on the isotracker. And further, the images are now raring based! As such, the ubuntu touch team is asking for folks to try out the new images on there devices and ensure they are no regressions or other issues.




There are 4 product listings representing each of the officially supported devices; grouper (nexus 7), maguro (galaxy nexus), mako (nexus 4), and manta (nexus 10). You can help by installing the new images following the installation instructions, and then reporting your results on the isotracker. If your device has never run a developer preview image for ubuntu touch, you might need to read and follow the steps on the touch wiki first.


There are handy links for download and bug information at the top of the testcases to help you out. If you do find a bug, please use the instructions to report it and add it to your result. Never used the tracker before? Take a look at this handy guide or watch the youtube version.

Once all the kinks and potential issues are worked out (your feedback requested!) the raring based images will become the default, and moving forward, the team will continue to provide daily images and participate in testing milestones as part of the 's' cycle.

As always please contact me if you run into issues, or have a question.
Thank you in advance for your help, and happy testing everyone!

Read more
Nicholas Skaggs

Filling the Gaps

I wanted to post briefly about the work that has been going on at the end of the cycle in the ubuntu quality team. Yes, we're testing the final images! Yes, it's been a wild ride that is nearing the finish! Yes, you can help contribute results! (And as we'll see below, you can help write tools too!)

But more than all of that, several team members have stepped out of there comfort zones and went to work on one of the testing tools we as a team utilize. The tool is called "Testdrive" and is written in python. Now, one of the great things I love to espouse on about with QA is the opportunity to work on many different things. There are needs to fit all interests, and if you are willing, the capability to learn.

In this instance, there is an opportunity to learn a little python and to work with a new team to help keep a testing tool alive. I'm happy to see that the same tool that was rendered broken in January by updates is now alive and well, with brand new contributors, fresh patches and even a release! Many thanks to smartboyhw, noskcaj, SergioMeneses, phillw, and the others who have reached out to ensure the tool that ships in raring still works. Thanks as well to the testdrive development team for engaging with us, reviewing merge proposals, and helping to ensure testdrive still works.

I look forward to a bright feature of new and improved testing tools. Specifically to those who contributed patches, with your new coding abilities, I can't wait to see what will happen next cycle! *wink, wink*

Read more
Nicholas Skaggs

The quality team invites you to a testing event for the final beta iso images. We'll be providing real-time help (IRC, or even one on one video hangouts if needed), encouraging you to download the final beta images, install, upgrade and test them out with us. You only need yourself, a machine (virtual or real!) and a bit of willingness to learn. We'll even be broadcasting for part of the event on ubuntuonair. So here's the details you need to know:

Tuesday April 2nd, 2013

  • 1800 UTC - 2200 UTC 
    • Quality team members are dedicated to hanging out in #ubuntu-quality executing testcases and helping answer questions
  •  2000 UTC: 
    • We'll be streaming live on ubuntuonair doing live testing demos and offering help
      • Basic iso test install
      • More exotic examples -- netboot, server, non-english
      • Your requests!


Interested? Great, mark the time and date on your calendar and check out the tutorial here to get a leg up on what you'll be doing during the event.


Can't make the 4 four window? Don't worry! Give testing a whirl anyways, and feel free to ask on #ubuntu-quality, and our mailing list for help.

See you on Tuesday!





Read more
Nicholas Skaggs

I wanted to write a post about the excitement of the new platform and the wonderful new challenges we face ahead of us. Now, given that this platform is being written right now from the ground up, those with a nose for quality instantly perk up. We love well tested applications, and developing with tests in mind from the start is much easier than attempting to retrofit. Seeing the first fruits of the developer effort is very exciting -- good work everyone!

So with that in mind, I started looking at some of the excellent work the core apps teams are doing with there applications. They've been working with the design community to turn the nice mockups into reality. I took the liberty of checking out and running some of the first versions of these applications. The calculator is one that stood out to me as already closing in on its specification. So armed with some of the design conversation for the calculator, I started a branch to create a set of manual tests for ubuntu touch applications, starting with the calculator. If you are interested in quality, now is the time to be involved! The applications can all be installed and run on your phone or even a ubuntu desktop.

So what can you do?

If you're a tester;

If you're a developer and have questions on writing tests for your application, feel free to contact me. I would love to see not only nice unit test driven code, but also some end user tests via autopilot, and I want to make sure you as a developer have the resources to do so. In addition, we as a quality community are happy to help test with you and write some manual tests to do so for your application.

I'm helping!


Read more
Nicholas Skaggs

After being away and enjoying some lovely downtime, I've returned to the online world to be met with the rush of a virtual UDS, a rolling release announcement, and a new windowing stack announcement. With the discussion advancing and the UDS sessions completed, it's time to weigh in and speak my thoughts as well.

I'd like to stare down the scarecrow -- that is, let us examine the straw man argument of a rolling release.



On a rolling release
I am definitely in favor of streamlining what we ship and support. The inter-LTS releases in general only make sense to run until the next one arrives. From a quality perspective, I really like what we've done with precise. I think it's an excellent solid base and the point releases we've done keep it relevant and offer a really nice way to get the latest stuff and keep a stable and long-term supported system.

As for a rolling release with LTS's sprinkled in, I have run a rolling release distro in the past (alongside ubuntu). I definitely enjoyed having access to the latest stuff, and having everyone on the same archive all the time (community-wise) kept us tighter and more able to relate and help each other. Overall, I think the pros outweigh the cons on moving towards it, but I have several caveats with the current approach.
  • Monthly snapshots
    • As Colin Watson put it, if we're presumbly releasing and testing a monthly snapshot, we failed in a rolling release sense -- we don't have daily quality.
    • In general, I don't see a target audience for a monthly snapshot. Why can't we create an installer image (do full testing milestone on it), then call it gold for a long period (until new installer changes we want to bring in, which require us making a new image). In other words, I would like to see us only generating new images for an actual release (in this case only LTS's), or for a new installer. (Note that I mean supported images (in any sense), not just an image for testing)
  • Quasi-rolling mentatility
    • It seems like we want to support the idea of users running a snapshot of our archive on a certain date, and then only update at certain days and times
      • This is insane fragmentation and defeats the purpose of being a rolling release. People should strive to run up-to-date systems, and always be current. For us, we need to ensure the archive is always upgradeable so they can do so
  • LTS point-releases
    • Continue and enhance point-releases for LTS to keep regular flows of new, well supported and stable software
      • This was discussed and well noted in discussions and at UDS. As I mentioned, I really like how precise is going, and we can continue and bolster these efforts even more in a rolling world.

On flavors
I will prefix everything I say here with the fact I have never put together and supported a flavor, but I most certainly have enjoyed utilizing them, and working with members of the community who focus there efforts on them.

I was able to catch the end of the UDS session on flavors and had an excellent discussion with some folks from xubuntu and kubuntu. Thanks to those folks who helped provide some flavor feedback on the proposal.

I would like to challenge the flavors to engage in healthy discussions about how there release process works, how to serve the needs of there users, and how to make the best use of there time and resources. I'm sure this type of introspection happens in each flavor on a regular basis, but I'd like to call special attention to how releases work.

Last cycle, ubuntu adopted a cadence for driving quality into the daily images. This work has been on-going since precise really, and rick's idea's for a rolling model continue this line of thinking. If you'll ask the community folks who helped be a part of these cadences, they can tell you it was a challenging change, but we're really hitting our stride now. The constant iterations on how we test and what we do I think had been extremely positive towards helping quality make a bigger impact.

With that in mind, our release processes shouldn't be exempted from this. QA (and development :-) ) efforts are seen as linked to the current release process, resulting in chaos if you are radical enough to unlink them. So what options (in my opinion!) of course exist for a flavor in this new world?
  • LTS only
    • Some flavors have already gone to an LTS only model, and I think it's been extremely helpful for those who've done so, in terms of what they can focus on without worrying about supporting lots of releases.
  • Rolling only
    • You can chose to go full force into a rolling release, and eschew LTS's altogether
  • 2 year "normal" releases
    • You could choose to simply push a new image out every 2 years (like an LTS), but without long term support. Instead, consider supporting until the next (2-years or so) release.
  • Keep things as-is
    • As kubuntu and others have shown this cycle by not adopting a cadence for testing, you can keep the traditional model in place. The buildbots are still there, the testing tools still exist, and the knowledge and experience in releasing on a 6 month cadence is there. Remember, ubuntu itself has synced from upstream debian every 6 months; a flavor could choose to do the same with ubuntu.
Now of all these options, at this point I would personally recommend adopting the LTS only model. Work with and sync your development to your upstream project and land your work in the rolling release. Release an LTS as normal and deliver timely point release updates to the LTS. There is nothing stopping you from even delivering these point releases every 6 months (or a different timetable!), emulating the current process but with a stable ubuntu LTS base and a simplified upgrade process.

On some alternative ideas

6-8? month-stable releases between LTS

Not a bad idea for retaining the flavor of the current system. Indeed, if you really like the idea of monthly snapshots and updating, this is probably the better way to do it. However I don't think it solves any issues for an OEM or for flavors. Namely, the release support timeline is too short for an OEM to base an image on, while for flavors, it would force an even faster cadence and churn upon users. I also don't see a target audience for it. Who would run this, but not run a rolling? Folks who want stability couldn't adopt such a small supported time-frame, and I feel like our efforts to test and release would be wasted as we throw it away as soon as the next stable is out.

Don't change anything!
This idea is just a knee-jerk reaction to change. Unless you feel like our last release was the pinnacle of perfection (I don't), we should be evaluating how we do things, iterate and try and do them better. 

On quality in a rolling release
I want to talk specifically about quality as that's what is dearest to me. How do you ensure quality in a rolling release world?
 
First, I would like to challenge you on what you mean by quality. Is older software better quality than newer software? Age != quality, even though we often traverse down that slippery slope. I wrote about this before, but simply put how we define quality is subjective. For the sake of comparison here, let's talk about quality as having a desktop that just works. That is, your hardware works and the applications and software running on it enables you to accomplish your tasks without issue.

So, with that in mind, how does that work in a rolling release? If you've run the development versions of ubuntu in the past, there have been times where a bad package may have rendered your system unbootable. For any user trying to run this as there daily system, it's obvious that level of 'quality' doesn't work. But at the same time, I've found bugs running the development version of ubuntu that cause actual crashes (see this for example), yet have little impact (if any) on my system working properly to enable me to accomplish my tasks. So how can we define quality metrics (and we should!) for each release? Here's a quick summary of my expectations in extremely simple terms:
  • LTS
    • No issue that hinders or prevents utilizing ubuntu
  • Rolling
    • No issue that would cause a crash for expected usecases and workflows
The key difference to me is usability. If I am forced to use a workaround for a crash in a minor application or task in a rolling release is probably ok. Note that I say probably, because well, we haven't defined these metrics yet as a community. Being forced to do so in an LTS is not an acceptable level of quality. And of course, causing a system to not boot is never acceptable.

On the reaction and the future
I'm excited to see these discussions taking place, and I would encourage people to think critically and take part in these discussions. There are definitely some wonderful ideas and conversation taking place.

Just remember we're a team and all part of ubuntu. Healthy debate is a very important part of continuing to better ourselves as a community and project. 


Read more
Nicholas Skaggs

Some quality resources

A couple posts ago, I mentioned the ubuntu quality team was looking for people to join the team and help out in the testing efforts we undertake. Thanks to those of you who've already answered the call and our now joining our testing ranks. We love sharing the joys of testing with others!

We're serious about wanting to make sure you are able to contribute and join the community as easily as possible. So for the last couple months, as a team we've been writing tutorials, giving classroom sessions, and hosting testing events. We really do want you as part of the team. Check out some of the resources available to you and consider becoming a part of the team!

Classroom Sessions
Video Tutorials
Written Walkthroughs


Read more
Nicholas Skaggs

Starting tomorrow February 9th, 2013 (heh, some of you reading this might already be in tomorrow), the quality community team will start testing for cadence week #6. During this week, we as a team try and help test specific packages looking for regressions, doing new feature or hardware testing, and making sure our images are in good shape. If your still confused, there's a nice wiki page that lays out what "cadence" means in a bit more detail.

So what does this mean for you, dear reader? Well, we as a team would like you to be involved in helping us test! Everyone has unique ways of interacting with software, and naturally no two computer setups are exactly the same between us. Now, I know what your thinking -- how can I help? I'm no tester, and I don't run development versions of ubuntu!

That's ok! You can still help test without needing to compromise your system. If you don't want to install the development version on your machine, you can use a virtual machine installation instead. If you are unable to run virtual machines, or are confused at the idea, you can still help test by simply running a live session and executing tests there. It's not too hard for you! Check out this walk-through for participating using only an image of the development version of ubuntu and your computer.

To help demonstrate how you can participate, I'll be hosting two live events this next week where I'll be on-hand running through the cadence week tests along with others from the quality team. There will even be a livestream, so if your a visual person (like me!), you can see for yourself how you might be able to contribute. Here's the dates you need to know:

Monday Feb 11th, 1800-1900 UTC in #ubuntu-quality. I'll also be streaming live my participation in executing the tests .

Thursday Feb 14th, 1400-1500 UTC in #ubuntu-quality. No stream, but we'll be hanging out answering questions, and working on submitting test results.


Please consider attending a session or watching the video of the stream afterwards. If you can download an image and boot your computer, you can help test. You want to be a part of ubuntu; let us help you contribute!

Read more
Nicholas Skaggs

PSA: Ubuntu Quality wants you!


NOTICE: To whom it may concern, the ubuntu quality team is seeking those with a desire to help ubuntu to contribute to the quality and testing efforts. With a little time and a willingness to learn, you too can unlock the tester within you!

Interested? Please inquire below!

If that text didn't get you, I hope the picture did. Seriously though, if you are here reading this page, I want to offer you an opportunity to help out. We as a team have expanded our activities and projects this cycle and we want to extend an offer for you to come along and learn with us. We're exploring automated testing with autopilot and autopkg, manual testing of images, and the virtues of testing in a regular cadence.

But we can't do it alone, nor do we wish to! We'd love to hear from you. Please have a look at our getting involved page (but do excuse the theme dust!) and get in touch. I offered a challenge to this community in the past, and I was blown away by the emails that flooded my inbox. Send me an email, tell me your interests, and ask me how you can help. Let me help get you started. Flood my inbox again1. Let's make ubuntu better, together!

1. If anyone is counting, I believe the record is ~100 emails in one 24 hour period :-p

Read more
Nicholas Skaggs

Introspecting with Autopilot

If you remember our post from last time, we had gone through writing our first testcase for firefox, converting a simple manual testcase to utilize autopilot instead. In this post I'd like to talk about how introspection can be used to perform some more complicated automated testcases.

First of all, let's briefly define what we mean by introspection. Specifically, we're talking about introspecting the dbus session for an application on our screen. Trust me, it sounds worse than it is. I'll let you do your own googling if you are curious to learn more. For the rest of us, let's just have a look visually at what we're talking about :-)

If you've got autopilot installed (check out the previous post; install autopilot ppa, sudo apt-get install python-autopilot), you should be able to launch the visualization tool.

autopilot vis

A window should launch, and allow you to select a connection.  This allows you to select which application you wish to introspect. Go ahead and select 'Unity'. If the bareness of the tool scares you, remember it's a development aid, not your browser ;-)

Ok, under the Tree node, you should find a giant list of of nodes and properties for Unity. It may come as a surprise that your desktop is providing this much data about what's going on right now. For instance, have a look under PanelController->Indicators. There's entries for each indicator you are running, along with properties about each one. Ok, so what if your not running Unity? Or, for our purposes, you wish to write a test about an application on our desktop?

Never fear, we can use another feature of autopilot to help launch and introspect an application using the same tool. Go ahead and close the visualization window and enter the following.

autopilot launch gedit
autopilot vis

Notice now we have a new connection called 'Root'. Select it and you'll see the node tree for the gedit window you just launched. The amount of nodes spawned is a bit overwhelming, but you can now use this data to make assertions about what's going on when you interact with the application.

As a quick example, let's say I wanted to know the size of the current gedit window. I we look under 'GeditWindow->globalRect' we can see the current position, and infer the size of the window as well. We can also see things like the title, is_active, and other 'xwindow typish' properties.

In addition, I can find out what buttons are present on the gedit toolbar of the current gedit window. I we look under 'GeditWindow->GtkToolbar' we can see several GtkToolButton nodes. Each has a set of properties, including a name and label. 

So, let's put it all together for a quick example.
 
bzr branch lp:ubuntu-autopilot-tests

Inside the resulting directory you'll notice a geditintrospection folder.

cd ubuntu_autopilot_tests
autopilot run geditintrospection

A gedit window should spawn and disappear -- and hopefully the one testcase should pass. Open up the file geditintrospection/test_geditintrospection.py. Inside you'll notice we're using some of the properties we found while introspecting to show off how we can utilize them to test gedit. Let's cover some of the new functions briefly. You can use the autopilot documentation for more information on what you see.

     def select_single(self, type_name='*', **kwargs):
        """Get a single node from the introspection tree, with type equal to
        *type_name* and (optionally) matching the keyword filters present in
        *kwargs*.


    def select_many(self, type_name='*', **kwargs):
        """Get a list of nodes from the introspection tree, with type equal to
        *type_name* and (optionally) matching the keyword filters present in
        *kwargs*.

  
These two functions allow us to get back the nodes that match our query. For example, you can see the first step of the testcase is to check for a New File button, which is on the gedit toolbar. After asserting it exists, we then click it. Load up gedit for yourself and use the vis tool to confirm. You'll find the node under GeditWindow->GtkBox->GtkToolbar->GtkToolButton. Each button is represented, in this case we pulled the one with label=_New, representing the new file button.

Later we actually interact with gedit by turning on overwrite mode. Normally we would be unable to verify if this indeed worked or not, but you'll notice we once again check for the GtkLabel change from INS to OVR. Again, you can see this under GeditWindow->GtkBox->GeditStatusbar->GtkLabel.

Finally, you'll notice some slight differences from our non-introspection testcase. We now use GtkIntrospectionTestMixin, and launch our application via the launch_test_application function, rather than using AutopilotTestCase. I've shown gtk based examples here, but introspection works with Qt too (using QtIntrospectionTestMixin), so don't be afraid to try it out on any application you are interested in. 

You may also have noticed you branched from a project; ubuntu-autopilot-tests. I'm happy to announce this is the master repository for all the autopilot testcases we'll be writing as a community. Interested in helping contribute? Come join us and get involved! We're tracking work items, including proposed tests for you to work on. In addition, the tests themselves will soon be running on jenkins for everyone in the community to benefit.

Now introspection is still new, and we as a Quality Community team are excited about adopting and utilizing the tool.  There might be some bugs and feature requests (wink, wink autopilot team!) to work out, but we are excited to build a repository of automated tests together.

NOTE: Due to an error during build, it appears the autopilot online documentation is absent or missing pieces. If this occurs, please use the local documentation installed on your machine as part of the autopilot package. You'll find a copy you can browse at /usr/share/doc/python-autopilot/html/index.html.

Read more
Nicholas Skaggs

Jamming Thursday's!

Right now as I type we have two jams going on! Last week Jono posted about enhancing the ubuntu.com/community page. If your a part of the community, join in raising the banner for your specific focus area. The fun is happening now on #ubuntu-docs. For the full details, see Jono's post. For us quality folks, the pad is here: http://pad.ubuntu.com/communitywebsite-contribute-quality. Feel free to type and edit away!

In addition, as Daniel Holbach mentioned, there is a hackathon for automated testing. Come hang out with us on #ubuntu-quality, learn, ask and write some tests. Again, the full details can be found on Daniel's post.

Come join us!

Read more