Canonical Voices

Posts tagged with 'testing'

Colin Ian King

Testing eCryptfs

Over the past several months I've been occasionally back-porting a bunch of eCryptfs patches onto older Ubuntu releases.  Each back-ported fix needs to be properly sanity checked and so I've been writing test cases for each one and adding them to the eCryptfs test suite.

To get hold of the test suite, check it out using bzr:

 bzr checkout lp:ecryptfs  
and install the dependencies so one can build the test suite:
 sudo apt-get install debhelper autotools-dev autoconf automake \
intltool libtool libgcrypt11-dev libglib2.0-dev libkeyutils-dev \
libnss3-dev libpam0g-dev pkg-config python-dev swig acl \
ecryptfs-utils
If you want to test eCrytpfs with xfs and btrfs as the lower file system onto which eCryptfs is mounted, then one needs to also install the tools for these:
 sudo apt-get install xfsprogs btrfs-tools  
And then build the test programs:
 cd ecryptfs  
autoreconf -ivf
intltoolize -c -f
./configure --enable-tests --disable-pywrap
make
To run the tests, one needs to create lower and upper mount points. The tests allow one to create ext2, ext3, ext4, xfs or btrfs loop-back mounted file systems on the lower mount point, and then eCryptfs is mounted on the upper mount point on top.   To create these, use something like:
 sudo mkdir /lower /upper  
The loop-back file system image needs to be placed somewhere too, I generally place mine in a directory /tmp/image, so this needs creating too:
 mkdir /tmp/image  
There are two categories of tests, "safe" and "destructive".  Safe tests should run in such a ways as to not lock up the machine.  Destructive tests try hard to force bugs that can cause kernel oopses or panics. One specifies the test category with the -c option.  Now to run the tests, use:
 sudo ./tests/run_tests.sh -K -c safe -b 1000000 -D /tmp/image -l /lower -u /upper  
The -K option tells the test suite to run the kernel specific tests. These are the ones I am generally interested in since I'm testing kernel patches.

The -b option specifies the size in 1K blocks of the loop-back mounted /lower file system size.  I generally use 1000000 blocks as a minimum.

The -D option specifies the path where the temporary loop-back mounted image is kept and the -l and -u options specified the paths of the lower and upper mount points.

By default the tests will use an ext4 lower filesystem. One can also run specify which file systems to run the tests on using the -f option, this can be a comma separated list of one or more file systems, for example:
 sudo ./tests/run_tests.sh -K -c safe -b 1000000 -D /tmp/image -l /lower -u /upper \
-f ext2,ext3,ext4,xfs
And also, instead of running a bunch of tests, one can just a particular test using the -t option:
 sudo ./tests/run_tests.sh -K -c safe -b 1000000 -D /tmp/image -l /lower -u /upper \
-f ext2,ext3,ext4,xfs -t lp-926292.sh
..which tests the fix for LaunchPad bug 926292
 
We also run these tests regularly on new kernel images to ensure we don't introduce and regressions.   As it stands, I'm currently adding in tests for each bug fix that we back-port and for most new bugs that require a thorough test. I hope to expand the breadth of the tests to ensure we get better general test coverage.

And finally, thanks to Tyler Hicks for writing the test framework and for his valuable help in describing how to construct a bunch of these tests.

Read more
Colin Ian King

I've been using QEMU and KVM for quite a while now for general kernel testing, for example, sanity checking eCryptfs and Ceph.   It can be argued that the best kind of testing is performed on real hardware, however, there are times when it is much more convenient (and faster) to exercise kernel fixes on a virtual machine.

I used to use the command line incantations to run QEMU and KVM, but recently I've moved over to using virt-manager because it so much simpler to use and caters for most of my configuration needs.

Virt-manager provides a very usable GUI and allows one to create, manage, clone and destroy virtual machine instances with ease.

virt-manager view of virtual machines
Each virtual machine can be easy reconfigured in terms of CPU configuration (number and type of CPUs),  memory size, boot options, disk and CD-ROM selection, NIC selection, display server (VNC or Spice), sound device, serial port config, video hardware and USB and IDE controller config.  

One can add and remove additional hardware, such serial port, parallel ports, USB and PCI host devices, watchdog controllers and much more besides.

Configuring a virtual machine

..so reconfiguring a test to run on a single core CPU to multi-core is a simple case of shutting down the virtual machine, bumping up the number of CPUs and booting up again.

By default one can view the virtual machine's console via a VNC viewer in virt-manager and there is provision to scale the screen to the window size, set to full size or resize the virt-manager window to the screen size.  For ease of use, I generally just ssh into the virtual machines and ignore the console unless I can't get the kernel to boot.

virt-manager viewing a 64 bit Natty server (for eCryptfs testing)
Virt-manager is a great tool and well worth giving a spin. For more information on virt-manager visit virt-manager.org

Read more
Colin Ian King

The Ubuntu Kernel Team has uploaded a new kernel (3.2.0-17.27) which contains an additional fix to resolve the remaining issues seen with the RC6 power saving enabled. For users with Sandy Bridge based hardware we would appreciate them to run the tests described on https://wiki.ubuntu.com/Kernel/PowerManagementRC6 and add their results to that page.

Read more
Colin Ian King

The Ubuntu Kernel Team has released a call for testing for a set of RC6 power saving patches for Ubuntu 12.04 Precise Pangolin LTS. Quoting Leann Ogasawara's email to the ubuntu kernel team and ubuntu-devel mailing lists:

"Hi All,

RC6 is a technology which allows the GPU to go into a very low power consumption state when the GPU is idle (down to 0V). It results in considerable power savings when this stage is activated. When comparing under idle loads with machine state where RC6 is disabled, improved power usage of around 40-60% has been witnessed [1].

Up until recently, RC6 was disabled by default for Sandy Bridge systems due to reports of hangs and graphics corruption issues when RC6 was enabled. Intel has now asserted that RC6p (deep RC6) is responsible for the RC6 related issues on Sandy Bridge. As a result, a patch has recently been submitted upstream to disable RC6p for Sandy Bridge [2].

In an effort to provide more exposure and testing for this proposed patch, the Ubuntu Kernel Team has applied this patch to 3.2.0-17.26 and newer Ubuntu 12.04 Precise Pangolin kernels. We have additionally enabled plain RC6 by default on Sandy Bridge systems so that users can benefit from the improved power savings by default.

We have decided to post a widespread call for testing from Sandy Bridge owners running Ubuntu 12.04. We hope to capture data which supports the the claims of power saving improvements and therefore justify keeping these patches in the Ubuntu 12.04 kernel. We also want to ensure we do not trigger any issues due to plain RC6 being enabled by default for Sandy Bridge.

If you are running Ubuntu 12.04 (Precise Pangolin) and willing to test and provide feedback, please refer to our PowerManagementRC6 wiki for detailed instructions [3]. Additionally, instructions for reporting any issues with RC6 enabled are also noted on the wiki. We would really appreciate any testing and feedback users are able to provide.

Thanks in advance,
The Ubuntu Kernel Team"

So please contribute to this call for testing by visiting https://wiki.ubuntu.com/Kernel/PowerManagementRC6 and follow the instructions.  Thank you!

Read more
Colin Ian King

Part of my focus this cycle is to see where we can make power saving improvements for Ubuntu Precise 12.04 LTS. There has been a lot of anecdotal evidence of specific machines or power saving features behaving poorly over the past few cycles.   So, armed with a 6.5 digit precision multimeter from Fluke I've been measuring the power consumption on various laptops in different test scenarios to try and answer some outstanding questions:

* Is it safe to enable Matthew Garrett's PCIe ASPM fix?
* Are the power savings suggested by PowerTop useful and can we reliably enabled any of these in pm-utils?
* How accurate are the ACPI battery readings to estimate power consumption?
* Do the existing pm-utils power.d scripts still make sense?
* Which is better for power saving: i386, i386-pae or amd64?
* How much power does the laptop backlight really use?
* Does halving the mouse input rate really save that much more power?
* Should we re-enable Aggressive Link Power Management (ALPM)?
* Are there any misbehaving applications that are consuming too much power?
* What are the root causes of HDD wake-ups
* Which applications and daemons are creating unnecessary wake events?
* How much does the MSR_IA32_ENERGY_PERF_BIAS save us?

..and many more besides!

From some of the analysis and "crowd sourcing" tests it is clear that the PCIe ASPM fix works well, so we've already incorporated that into Precise.

Aggressive Link Power Management (ALPM) is a mechanism where a SATA AHCI controller can put the SATA link that connects to the disk into a very low power mode during periods of zero I/O activity and into an active power state when work needs to be done. Tests show that this can save around 0.5-1.5 Watts of power on a typical system. However, it has been known in the past to not work on some devices, so I've put a call for testing of ALPM out to the community so we can get a better understanding of the power savings vs reliability.

Some of the PowerTop analysis has shown we can save another 1-2 Watts of power by putting USB and PCI controllers of devices like Webcams, SD card controllers, Wireless, Ethernet and Bluetooth  into a lower power state.  Again, we would like to understand the range of power savings across a large set of hardware and to see how reliable this is, so another crowd sourced call for testing has been also set up.

So, if you want to contribute to the testing, please visit the above links and spend just a few tens of minutes to see we can extend the battery life of your laptop or netbook.  And periodically visit https://wiki.ubuntu.com/Kernel/PowerManagement to see if there any new tests you can participate in.

[UPDATE]

I've written some brief notes on power saving tweaks and also some simple recommendations for application developers to follow too.

The thread continues here (part 2)

Read more
Colin Ian King

I've now competed the documentation of the Firmware Test Suite and this include documenting each of the 50+ tests (which was a bit of a typing marathon).  Each test has a brief description of what the test covers, example output from the test, how to run the test (possibly with different options) and explanations of test failure messages.

For example of the per-test documentation, check out the the suspend/resume test page and the ACPI tables test page.

I hope this is useful!


Read more
Colin Ian King

GNU EFI lib and Hello World

Writing UEFI applications takes a little bit of effort to get started, fortunately the GNU EFI library provides enough infrastructure to make it far less painful.  The crunch points are:

1. Understanding which UEFI protocols to use, fortunately these are well documented in the UEFI specification. Unfortunately the specification is a couple of thousand pages long, so be prepared to spend some time working out how it all hangs together.

2. Figuring out how to call the protocols using the GNU EFI wrapper shim.

3. Generating a UEFI PE COFF image using some objcopy runes.

So, the best way to start was to write a "Hello World" program and to work up from there.  First, I installed the gnu-efi development library using:

sudo apt-get install gnu-efi

Next I hacked up the classic "Hello World", this can be found here.  Then considerable amount of Googling was required to figure out how to use objcopy to help create the desired executable image.  The Makefile containing these runes can be found here.

So what next?  Well, this little adventure is my first step in a quest to write a bunch of tests so ensure UEFI firmware has functional protocols to allow grub to load a kernel.  I was hoping to re-use the firmware test suite framework but it may take some effort since I don't have any POSIX support on UEFI, so I'm going to have to scope out how much effort is required to port the framework to UEFI.

Anyhow, it's a start.


Read more