Canonical Voices

What The Quality Hour talks about

Posts tagged with 'quality'

alesage

This tutorial describes how to produce test-coverage metrics for C/C++ projects. We start by instrumenting an autotools build to produce gcov output with gcc; we then generate test coverage artifacts by running our test suite, and finally we explore our results using lcov.

Introduction

We’d like to assess the quality of the existing test suite for each Product Strategy project. A measurement of test coverage will tell us what part of the project’s code is “covered” or exercised by its tests. 50% is a start; 80% would be a good goal for a neglected project; one rarely encounters 100% test coverage, but we’d like to get as close as we can. Initially we’ll use these findings to gain an overview of the test-quality of each project; ultimately these metrics will guide the improvement of our codebase, and enable us to monitor our progress using Jenkins and associated open-source tools.

A Three-Part Process in Several Steps

We’ll enable test-coverage of a particular C/C++ project in a three-part process:

  • enabling a special build
  • running the tests
  • studying the output

Our first step will be to enable a special build. Ultimately this just means adding a few flags to our gcc invocation, but it never seems so straightforward with autotools projects</p>
            <a href=Read more

Martin Mrazik

Quality (not only) in Budapest

Last week was my first one at Canonical and I spent it in Budapest where Ubuntu Engineering and Product Strategy teams had their rally. This is the first company where I was asked to travel on my first day and it did feel strange at the beginning</p>
            <a href=Read more

ThomasVo5

Google Test & Jenkins CI

With Canonical’s growing engagement with test driven development (TDD), we recently started introducing Google Test to a number of our upstream projects. Google Test is a unit testing framework targeted towards C & C++. It is lightweight and offers a load of features that ease unit testing tremendously. Most importantly, it supports automatic test case registration and the notions of test suite environments and fixtures. Here, a developer’s efficiency is our main focus. We acknowledge that providing a decent test coverage requires time and effort and we would like to free a developer from writing boiler plate code for test case setup, teardown and for acutally assembling his test cases.

Recently, we started working on an extension to Google test that introduces a dummy (read:headless) X server environment by means of a custom Google test environment and custom Google test fixtures. In summary, the extension starts and stops a dummy xserver and makes sure that a connection to the server is established correctly as part of the test setup procedure. On tear down, the connection to the X server is terminated before the server instance is stopped. Jump over to launchpad for the actual project and take a look at utouch-frame for a real-world application of the extension.

Test Case Result Integration with Jenkins CI

Continuous integration is a hot topic at Canonical and automatic execution of unit test suites is an integral part of our development model. For this reason, we store all unit test results in Google test’s XML dialect and export it to Jenkins to provide user, developers and QA with a summary of a project’s build status.

Conclusions & Outlook

With the introduction of Google test & Jenkins CI to Canonical’s testing landscape we want to provide the plumbing layer for a TDD environment that is loved by developers. It is the first step towards supporting engineers with meaningful metrics to further understand our software ecosystem and identify blank spots on our software map. Thus, stay tuned for Allan Lesage‘s work on code coverage calculation going to land in the not too distant future</p>
            <a href=Read more