Canonical Voices

Posts tagged with 'upstream'

pitti

What is this?

umockdev is a set of tools and a library to mock hardware devices for programs that handle Linux hardware devices. It also provides tools to record the properties and behaviour of particular devices, and to run a program or test suite under a test bed with the previously recorded devices loaded.

This allows developers of software like gphoto or libmtp to receive these records in bug reports and recreate the problem on their system without having access to the affected hardware, as well as writing regression tests for those that do not need any particular privileges and thus are capable of running in standard make check.

After working on it for several weeks and lots of rumbling on G+, it’s now useful and documented enough for the first release 0.1!

Component overview

umockdev consists of the following parts:

  • The umockdev-record program generates text dumps (conventionally called *.umockdev) of some specified, or all of the system’s devices and their sysfs attributes and udev properties. It can also record ioctls that a particular program sends and receives to/from a device, and store them into a text file (conventionally called *.ioctl).
  • The libumockdev library provides the UMockdevTestbed GObject class which builds sysfs and /dev testbeds, provides API to generate devices, attributes, properties, and uevents on the fly, and can load *.umockdev and *.ioctl records into them. It provides VAPI and GI bindings, so you can use it from C, Vala, and any programming language that supports introspection. This is the API that you should use for writing regression tests. You can find the API documentation in docs/reference in the source directory.
  • The libumockdev-preload library intercepts access to /sys, /dev/, the kernel’s netlink socket (for uevents) and ioctl() and re-routes them into the sandbox built by libumockdev. You don’t interface with this library directly, instead you need to run your test suite or other program that uses libumockdev through the umockdev-wrapper program.
  • The umockdev-run program builds a sandbox using libumockdev, can load *.umockdev and *.ioctl files into it, and run a program in that sandbox. I. e. it is a CLI interface to libumockdev, which is useful in the “debug a failure with a particular device” use case if you get the text dumps from a bug report. This automatically takes care of using the preload library, i. e. you don’t need umockdev-wrapper with this. You cannot use this program if you need to simulate uevents or change attributes/properties on the fly; for those you need to use libumockdev directly.

Example: Record and replay PtP/MTP USB devices

So how do you use umockdev? For the “debug a problem” use case you usually don’t want to write a program that uses libumockdev, but just use the command line tools. Let’s capture some runs from libmtp tools, and replay them in a mock environment:

  • Connect your digital camera, mobile phone, or other device which supports PtP or MTP, and locate it in lsusb. For example
      Bus 001 Device 012: ID 0fce:0166 Sony Ericsson Xperia Mini Pro
  • Dump the sysfs device and udev properties:
      $ umockdev-record /dev/bus/usb/001/012 > mobile.umockdev
  • Now record the dynamic behaviour (i. e. usbfs ioctls) of various operations. You can store multiple different operations in the same file, which will share the common communication between them. For example:
      $ umockdev-record --ioctl mobile.ioctl /dev/bus/usb/001/012 mtp-detect
      $ umockdev-record --ioctl mobile.ioctl /dev/bus/usb/001/012 mtp-emptyfolders
  • Now you can disconnect your device, and run the same operations in a mocked testbed. Please note that /dev/bus/usb/001/012 merely echoes what is in mobile.umockdev and it is independent of what is actually in the real /dev directory. You can rename that device in the generated *.umockdev files and on the command line.
      $ umockdev-run --load mobile.umockdev --ioctl /dev/bus/usb/001/012=mobile.ioctl mtp-detect
      $ umockdev-run --load mobile.umockdev --ioctl /dev/bus/usb/001/012=mobile.ioctl mtp-emptyfolders

Example: using the library to fake a battery

If you want to write regression tests, it’s usually more flexible to use the library instead of calling everything through umockdev-run. As a simple example, let’s pretend we want to write tests for upower.

Batteries, and power supplies in general, are simple devices in the sense that userspace programs such as upower only communicate with them through sysfs and uevents. No /dev nor ioctls are necessary. docs/examples/ has two example programs how to use libumockdev to create a fake battery device, change it to low charge, sending an uevent, and running upower on a local test system D-BUS in the testbed, with watching what happens with upower --monitor-detail. battery.c shows how to do that with plain GObject in C, battery.py is the equivalent program in Python that uses the GI binding. You can just run the latter like this:

  umockdev-wrapper python3 docs/examples/battery.py

and you will see that upowerd (which runs on a temporary local system D-BUS in the test bed) will report a single battery with 75% charge, which gets down to 2.5% a second later.

The gist of it is that you create a test bed with

  UMockdevTestbed *testbed = umockdev_testbed_new ();

and add a device with certain sysfs attributes and udev properties with

    gchar *sys_bat = umockdev_testbed_add_device (
            testbed, "power_supply", "fakeBAT0", NULL,
            /* attributes */
            "type", "Battery",
            "present", "1",
            "status", "Discharging",
            "energy_full", "60000000",
            "energy_full_design", "80000000",
            "energy_now", "48000000",
            "voltage_now", "12000000",
            NULL,
            /* properties */
            "POWER_SUPPLY_ONLINE", "1",
            NULL);

You can then e. g. change an attribute and synthesize a “change” uevent with

  umockdev_testbed_set_attribute (testbed, sys_bat, "energy_now", "1500000");
  umockdev_testbed_uevent (testbed, sys_bat, "change");

With Python or other introspected languages, or in Vala it works the same way, except that it looks a bit leaner due to “proper” object semantics.

Packages

I have a packaging branch for Ubuntu and a recipe to do daily builds with the latest upstream code into my daily builds PPA (for 12.10 and raring). I will soon upload it to Raring proper, too.

What’s next?

The current set of features should already get you quite far for a range of devices. I’d love to get feedback from you if you use this for anything useful, in particular how to improve the API, the command line tools, or the text dump format. I’m not really happy with the split between umockdev (sys/dev) and ioctl files and the relatively complicated CLI syntax of umockdev-record, so any suggestion is welcome.

One use case that I have for myself is to extend the coverage of ioctls for input devices such as touch screens and wacom tablets, so that we can write some tests for gnome-settings-daemon plugins.

I also want to find a way to pass ioctls back to the test suite/calling program instead of having to handle them all in the preload library, which would make it a lot more flexible. However, due to the nature of the ioctl ABI this is not easy.

Where to go to

The code is hosted on github in the umockdev project; this started out as a systemd branch to add this functionality to libudev, but after a discussion with Kay we decided to keep it separate. But I kept it in git anyway, given how popular it is today. For the bzr lovers, Launchpad has an import at lp:umockdev.

Release tarballs will be on Launchpad as well. Please file bugs and enhancement requests in the git hub tracker.

Finally, if you have questions or want to discuss something, you can always find me on IRC (pitti on Freenode or GNOME).

Thanks for your attention and happy testing!

Read more
pitti

I just released a new PyGObject for GNOME 3.7.5. Unfortunately master.gnome.org is out of space right now, so I put the new tarball on my Ubuntu people account for the time being.

This again brings a nice set of memory leak and bug fixes, some more reduction of static bindings, and better support for building under Windows.

Thanks to all contributors!

  • Move various signal methods from static bindings to gi and python (Simon Feltman) (#692918)
  • GLib overrides: Support unpacking ‘maybe’ variants (Paolo Borelli) (#693032)
  • Fix ref count leak when creating pygobject wrappers for input args (Mike Gorse) (#675726)
  • Prefix names of typeless enums and flags for GType registration (Simon Feltman) (#692515)
  • Fix compilation with non-C99 compilers such as Visual C++ (Chun-wei Fan) (#692856)
  • gi/overrides/Glib.py: Fix running on Windows/non-Unix (Chun-wei Fan)
  • Do not immediately initialize Gdk and Gtk on import (Martin Pitt) (#692300)
  • Accept ±inf and NaN as float and double values (Martin Pitt) (#692381)
  • Fix repr() of GLib.Variant (Martin Pitt)
  • Fix gtk-demo for Python 3 (Martin Pitt)
  • Define GObject.TYPE_VALUE gtype constant (Martin Pitt)
  • gobject: Go through introspection on property setting (Olivier Crête) (#684062)
  • Clean up caller-allocated GValues and their memory (Mike Gorse) (#691820)
  • Use GNOME_COMPILE_WARNINGS from gnome-common (Martin Pitt)

Read more
pitti

I just released a new PyGObject, for GNOME 3.7.4 which is due on Wednesday.

This release saw a lot of bug and memory leak fixes again, as well as enabling some more data types such as GParamSpec, boxed list properties, or directly setting string members in structs.

Thanks to all contributors!

Summary of changes (see change log for complete details):

  • Allow setting values through GtkTreeModelFilter (Simonas Kazlauskas) (#689624)
  • Support GParamSpec signal arguments from Python (Martin Pitt) (#683099)
  • pygobject_emit(): Fix cleanup on error (Martin Pitt)
  • Add signal emission methods to TreeModel which coerce the path argument (Simon Feltman) (#682933)
  • Add override for GValue (Bastian Winkler) (#677473)
  • Mark caller-allocated boxed structures as having a slice allocated (Mike Gorse) (#699501)
  • pygi-property: Support boxed GSList/GList types (Olivier Crête) (#684059)
  • tests: Add missing backwards compat methods for Python 2.6 (Martin Pitt) (#691646)
  • Allow setting TreeModel values to None (Simon Feltman) (#684094)
  • Set clean-up handler for marshalled arrays (Mike Gorse) (#691509)
  • Support setting string fields in structs (Vadim Rutkovsky) (#678401)
  • Permit plain integers for “gchar” values (Martin Pitt)
  • Allow single byte values for int8 types (Martin Pitt) (#691524)
  • Fix invalid memory access handling errors when registering an enum type (Mike Gorse)
  • Fix (out) arguments in callbacks (Martin Pitt)
  • Fix C to Python marshalling of struct pointer arrays (Martin Pitt)
  • Don’t let Property.setter() method names define property names (Martin Pitt) (#688971)
  • Use g-i stack allocation API (Martin Pitt) (#615982)
  • pyg_value_from_pyobject: support GArray (Ray Strode) (#690514)
  • Fix obsolete automake macros (Marko Lindqvist) (#691101)
  • Change dynamic enum and flag gtype creation to use namespaced naming (Simon Feltman) (#690455)
  • Fix Gtk.UIManager.add_ui_from_string() override for non-ASCII chars (Jonathan Ballet) (#690329)
  • Don’t dup strings before passing them to type registration functions (Mike Gorse) (#690532)
  • Fix marshalling of arrays of boxed struct values (Carlos Garnacho) (#656312)
  • testhelpermodule.c: Do not unref called method (Martin Pitt)

Read more
pitti

I just released a new PyGObject, for GNOME 3.7.3 which is due on Wednesday.

This is mostly a bug fix release. There is one API addition, it brings back official support for calling GLib.io_add_watch() with a Python file object or fd as first argument, in addition to the official API which expects a GLib.IOChannel object. These modes were marked as deprecated in 3.7.2 (only).

Thanks to all contributors!

Summary of changes (see change log for complete details):

  • Add support for caller-allocated GArray out arguments (Martin Pitt) (#690041)
  • Re-support calling GLib.io_add_watch with an fd or Python file (Martin Pitt)
  • pygtkcompat: Work around IndexError on large flags (Martin Pitt)
  • Fix pyg_value_from_pyobject() range check for uint (Martin Pitt)
  • Fix tests to work with g-i 1.34.2 (Martin Pitt)
  • Fix wrong refcount for GVariant property defaults (Martin Pitt) (#689267)
  • Fix array arguments on 32 bit (Martin Pitt)
  • Add backwards compatible API for GLib.unix_signal_add_full() (Martin Pitt)
  • Drop MININT64/MAXUINT64 workaround, current g-i gets this right now (Martin Pitt)
  • Fix maximum and minimum ranges of TYPE_(U)INT64 properties (Simonas Kazlauskas) (#688949)
  • Ship pygi-convert.sh in tarballs (Martin Pitt) (#688697)
  • Various added and improved tests (Martin Pitt)

Read more
Michael Hall

UPDATE: A command porting walk-through has beed added to the documentation.

Back around UDS time, I began work on a reboot of Quickly, Ubuntu’s application development tool.  After two months and just short of 4000 lines of code written, I’m pleased to announce that the inner-workings of the new code is very nearly complete!  Now I’ve reached the point where I need your help.

The Recap

First, let me go back to what I said needed to be done last time.  Port from Python 2 to Python 3: Done. Add built-in argument handling: Done. Add meta-data output: Well, not quite.  I’m working on that though, and now I can add it without requiring anything from template authors.

But here are some other things I did get done. Add Bash shell completion: Done.  Added Help command (that works with all other commands): Done.  Created command class decorators: Done.  Support templates installed in any XDG_DATA_DIRS: Done.  Allow template overriding on the command-line: Done.  Started documentation for template authors: Done.

Now it’s your turn

With the core of the Quickly reboot nearly done, focus can now turn to the templates.  At this point I’m reasonably confident that the API used by the templates and commands won’t change (at least not much).  The ‘create’ and ‘run’ commands from the ubuntu-application template have already been ported, I used those to help develop the API.  But that leaves all the rest of the commands that need to be updated (see list at the bottom of this post).  If you want to help make application development in Ubuntu better, this is a great way to contribute.

For now, I want to focus on finishing the port of the ubuntu-application template.  This will reveal any changes that might still need to be made to the new API and code, without disrupting multiple templates.

How to port a Command

The first thing you need to do is understand how the new Quickly handles templates and commands.  I’ve started on some documentation for template developers, with a Getting Started guide that covers the basics.  You can also find me in #quickly in Freenode IRC for help.

Next you’ll need to find the code for the command you want to port.  If you already have the current Quickly installed, you can find them in /usr/share/quickly/templates/ubuntu-application/, or you can bzr branch lp:quickly to get the source.

The commands are already in Python, but they are stand-alone scripts.  You will need to convert them into Python classes, with the code to be executed being called in the run() method.  You can add your class to the ./data/templates/ubuntu-application/commands.py file in the new Quickly branch (lp:quickly/reboot).  Then submit it as a merge proposal against lp:quickly/reboot.

Grab one and go!

So here’s the full list of ubuntu-application template commands.  I’ll update this list with progress as it happens.  If you want to help, grab one of the TODO commands, and start porting.  Email me or ping me on IRC if you need help.

add: TODO
configure: TODO
create: DONE!
debug: TODO
design: TODO
edit: DONE!
license: TODO
package: DONE!
release: TODO
run: DONE!
save: DONE!
share: TODO
submitubuntu: TODO
test: TODO
tutorial: TODO
upgrade: TODO

Read more
Michael Hall

During this latest round of arguing over the inclusion of Amazon search results in the Unity Dash, Alan Bell pointed out the fact that while the default scopes shipped in Ubuntu were made to check the new privacy settings, we didn’t do a very good job of telling third-party developers how to do it.

(Update: I was told a better way of doing this, be sure to read the bottom of the post before implementing it in your own code)

Since I am also a third-party lens developer, I decided to add it to my come of my own code and share how to do it with other lens/scope developers.  It turns out, it’s remarkably easy to do.

Since the privacy setting is stored in DConf, which we can access via the Gio library, we need to include that in our GObject Introspection imports:

from gi.repository import GLib, Unity, Gio

Then, before performing a search, we need to fetch the Unity Lens settings:

lens_settings = Gio.Settings(‘com.canonical.Unity.Lenses’)

The key we are interested in is ’remote-content-search’, and it can have one of two value, ‘all’ or ‘none’.  Since my locoteams-scope performs only remote searches, by calling the API on http://loco.ubuntu.com, if the user has asked that no remote searches be made, this scope will return without doing anything.

And that’s it!  That’s all you need to do in order to make your lens or scope follow the user’s privacy settings.

Now, before we get to the comments, I’d like to kindly point out that this post is about how to check the privacy setting in your lens or scope.  It is not about whether or not we should be doing remote searches in the dash, or how you would rather the feature be implemented.  If you want to pile on to those argument some more, there are dozens of open threads all over the internet where you can do that.  Please don’t do it here.
&nbps;

Update

I wasn’t aware, but there is a PreferencesManager class in Unity 6 (Ubuntu 12.10) that lets you access the same settings:

You should use this API instead of going directly to GSettings/DConf.

Read more
Michael Hall

The Ubuntu Skunkworks program is now officially underway, we have some projects already staffed and running, and others where I am gathering final details about the work that needs to be done.  Today I decided to look at who has applied, and what they listed as their areas of interest, programming language experience and toolkit knowledge.  I can’t share details about the program, but I can give a general look at the people who have applied so far.

Most applicants listed more than one area of interest, including ones not listed in Mark’s original post about Skunkworks.  I’m not surprised that UI/UX and Web were two of the most popular areas.  I was expecting more people interested in the artistic/design side of things though.

Not as many people listed multiple languages as multiple areas of interest.  As a programmer myself, I’d encourage other programmers to expand their knowledge to other languages.  Python and web languages being the most popular isn’t at all surprising.  I would like to see more C/C++ applicants, given the number of important projects that are written in them.  Strangely absent was any mention of Vala or Go, surely we have some community members who have some experience with those.

The technology section had the most unexpected results.  Gtk has the largest single slice, sure, but it’s still much, much smaller than I would have expected.  Qt/QML even more so, where are all you KDE folks?  The Django slice makes sense, given the number of Python and Web applicants.

So in total, we’ve had a pretty wide variety of skills and interests from Skunkworks applicants, but we can still use more, especially in areas that are under-represented compared to the wider Ubuntu community.  If you are interested, the application process is simple: just create a wiki page using the ParticipationTemplate, and email me the link (mhall119@ubuntu.com).

Read more
Michael Hall

Now that Google+ has added a Communities feature, and seeing as how Jorge Castro has already created one for the wider Ubuntu community, I went ahead and created one specifically for our application developers.  If you are an existing app developer, or someone who is interested in getting started with app development, or thinking about porting an existing app to Ubuntu, be sure to join.

Google+ communities are brand new, so we’ll be figuring out how best to use them in the coming days and weeks, but it seems like a great addition.

Read more
pitti

just released a new PyGObject, for GNOME 3.7.2 which is due on Wednesday.

In this version PyGObject went through some major refactoring: Some 5.000 lines of static bindings were removed and replaced with proper introspection and some overrides for backwards compatibility, and the static/GI/overrides code structure was simplified. For the developer this means that you can now use the full GLib API, a lot of which was previously hidden by old and incomplete static bindings; also you can and should now use the officially documented GLib API instead of PyGObject’s static one, which has been marked as deprecated. For PyGObject itself this change means that the code structure is now a lot simpler to understand, all the bugs in the static GLib bindings are gone, and the GLib bindings will not go out of sync any more.

Lots of new tests were written to ensure that the API is backwards compatible, but experience teaches that ther is always the odd corner case which we did not cover. So if your code does not work any more with 3.7.2, please do report bugs.

Another important change is that if you build pygobject from source, it now defaults to using Python 3 if installed. As before, you can build for Python 2 with PYTHON=python2.7 or the new --with-python=python2.7 configure option.

This release also brings several marshalling fixes, docstring improvements, support for code coverage, and other bug fixes.

Thanks to all contributors!

Summary of changes (see changelog for complete details):

  • [API change] Drop almost all static GLib bindings and replace them with proper introspection. This gets rid of several cases where the PyGObject API was not matching the real GLib API, makes the full GLib API available through introspection, and makes the code smaller, easier to maintain. For backwards compatibility, overrides are provided to emulate the old static binding API, but this will throw a PyGIDeprecationWarning for the cases that diverge from the official API (in particular, GLib.io_add_watch() and GLib.child_watch_add() being called without a priority argument). (Martin Pitt, Simon Feltman)
  • [API change] Deprecate calling GLib API through the GObject namespace. This has always been a misnomer with introspection, and will be removed in a later version; for now this throws a PyGIDeprecationWarning.
  • [API change] Do not bind gobject_get_data() and gobject_set_data(). These have been deprecated for a cycle, now dropped entirely. (Steve Frécinaux) (#641944)
  • [API change] Deprecate void pointer fields as general PyObject storage. (Simon Feltman) (#683599)
  • Add support for GVariant properties (Martin Pitt)
  • Add type checking to GVariant argument assignment (Martin Pitt)
  • Fix marshalling of arrays of struct pointers to Python (Carlos Garnacho) (#678620)
  • Fix Gdk.Atom to have a proper str() and repr() (Martin Pitt) (#678620)
  • Make sure g_value_set_boxed does not cause a buffer overrun with GStrvs (Simon Feltman) (#688232)
  • Fix leaks with GValues holding boxed and object types (Simon Feltman) (#688137)
  • Add doc strings showing method signatures for gi methods (Simon Feltman) (#681967)
  • Set Property instance doc string and blurb to getter doc string (Simon Feltman) (#688025)
  • Add GObject.G_MINSSIZE (Martin Pitt)
  • Fix marshalling of GByteArrays (Martin Pitt)
  • Fix marshalling of ssize_t to smaller ints (Martin Pitt)
  • Add support for lcov code coverage, and add a lot of missing GIMarshallingTests and g-i Regress tests. (Martin Pitt)
  • pygi-convert: remove deprecated GLib ? GObject conversions (Jose Rostagno)
  • Add support for overriding GObject.Object (Simon Feltman) (#672727)
  • Add –with-python configure option (Martin Pitt)
  • Do not prefer unversioned “python” when configuring, as some distros have “python” as Python 3. Use Python 3 by default if available. Add –with-python configure option as an alternative to setting $PYTHON, whic is more discoverable. (Martin Pitt)
  • Fix property lookup in class hierarchy (Daniel Drake) (#686942)
  • Move property and signal creation into _class_init() (Martin Pitt) (#686149)
  • Fix duplicate symbols error on OSX (John Ralls)
  • [API add] Add get_introspection_module for getting un-overridden modules (Simon Feltman) (#686828)
  • Work around wrong 64 bit constants in GLib Gir (Martin Pitt) (#685022)
  • Mark GLib.Source.get_current_time() as deprecated (Martin Pitt)
  • Fix OverflowError in source_remove() (Martin Pitt) (#684526)

Read more
Michael Hall

A few weeks ago, Canonical founder Mark Shuttleworth announced a new project initiative dubbed “skunk works”, that would bring talented and trusted members of the Ubuntu community into what were previously Canonical-only development teams working on some of the most interesting and exciting new features in Ubuntu.

Since Mark’s announcement, I’ve been collecting the names and skill sets from people who were interested, as well as working with project managers within Canonical to identify which projects should be made part of the Skunk Works program.  If you want to be added to the list, please create a wiki page using the SkunkWorks/ParticipationTemplate template and send me the link in an email (mhall119@ubuntu.com).  If you’re not sure, continue reading to learn more about what this program is all about.

 What is Skunk Works?

Traditionally, skunk works programs have involved innovative or experimental projects lead by a small group of highly talented engineers.  The name originates from the Lockheed Martin division that produced such marvels as the U-2, SR-71 and F-117.  For us, it is going to focused on launching new projects or high-profile features for existing projects.  We will follow the same pattern of building small, informal, highly skilled teams that can work seamlessly together to produce the kinds of amazing results that provide those “tada” moments.

Why is it secret?

Canonical is, despite what some may say, an open source company.  Skunk Works projects will be no exception to this, the final results of the work will be released under an appropriate open license.  So why keep it secret?  One of the principle features of a traditional skunk works team is autonomy, they don’t need to seek approval for or justify their decisions, until they’ve had a chance to prove them.  Otherwise they wouldn’t be able to produce radically new designs or ideas, everything would either be watered down for consensus, or bogged down by argument.  By keeping initial development private, our skunk works teams will be able to experiment and innovate freely, without having their work questioned and criticized before it is ready.

Who can join?

Our Skunk Works is open to anybody who wants to apply, but not everybody who applies will get in on a project.  Because skunk works teams need to be very efficient and independent, all members need to be operating on the same page and at the same level in order to accomplish their goals.  Mark mentioned that we are looking for “trusted” members of our community.  There are two aspects to this trust.  First, we need to trust that you will respect the private nature of the work, which as I mentioned above is crucial to fostering the kind of independent thinking that skunk works are famous for.  Secondly, we need to trust in your ability to produce the desired results, and to work cooperatively with a small team towards a common goal.

What kind of work is involved?

We are still gathering candidate projects for the initial round of Skunk Works, but we already have a very wide variety.  Most of the work is going to involve some pretty intense development work, both on the front-end UI and back-end data analysis.  But there are also  projects that will require a significant amount of new design and artistic work.  It’s safe to say that the vast majority of the work will involve creation of some kind, since skunk works projects are by their nature more on the “proof” stage of development rather than “polish”.  Once you have had a chance to prove your work, it will leave the confines of Skunk Works and be made available for public consumption, contribution, and yes, criticism.

How to join

Still interested?  Great!  In order to match people up with projects they can contribute to, we’re asking everybody to fill out a short Wiki page detailing their skill sets and relevant experience.  You can use the SkunkWorks/ParticipationTemplate, just fill it in with your information.  Then send the link to the new page to my (mhall119@ubuntu.com) and I will add you to my growing list of candidates.  Then, when we have an internal project join the Skunk Works, I will give the project lead a list of people who’s skills and experience match the project’s need, and we will pick which ones to invite to join the team.  Not everybody will get assigned to a project, but you won’t be taken off the list.  If you revise your wiki page because you’ve acquired new skills or experience, just send me another email letting me know.

Read more
pitti

I just released PyGObject 3.4.1, in time for the GNOME 3.6.1 release on Wednesday.

This version provides a nice set of bug fixes. no API changes.

Thanks to all contributors!

Complete list of changes:

  • Skip Regress tests with –disable-cairo (Martin Pitt) (#685094)
  • _pygi_marshal_from_py_uint64: Re-fix check of negative values (Martin Pitt) (#685000)
  • Fix leak with python callables as closure argument. (Simon Feltman) (#685598)
  • Gio overrides: Handle setting GSettings enum keys (Martin Pitt) (#685947)
  • tests: Check reading GSettings enums in Gio overrides (Martin Pitt)
  • Fix unsigned values in GArray/GList/GSList/GHash (Martin Pitt) (#685860)
  • build: Fix srcdir != builddir (Colin Walters)
  • _pygi_marshal_from_py_uint64(): Use correct data type in py2.7 check (Alban Browaeys) (#685000)
  • Install an .egg-info file (Johan Dahlin) (#680138)
  • PyGProps_getattro(): Fix GObjectClass leak (Johan Dahlin) (#685218)
  • pygobject.c: Don’t leak GObjectClass reference (Olivier Crête) (#684062)
  • Fix memory leak in _pygi_argument_to_array() (Alban Browaeys) (#685082)
  • Fix error messages for out of range numbers (Martin Pitt) (#684314)
  • Kill dbus-daemon after running tests (Martin Pitt) (#685009)
  • GVariant overrides: Support empty tuple arrays (Martin Pitt) (#684928)
  • TestGVariant: Split creation test case into several smaller ones (Martin Pitt)
  • Fix unused variables and results (Martin Pitt)
  • tests: Fix wrong return type in test_int64_callback() (Martin Pitt) (#684700)
  • Fix GValue marshalling of long and unsigned long (Giovanni Campagna) (#684331)
  • Clean up deprecation message for assigning gpointers to objects. (Simon Feltman) (#683599)
  • pygi-property: Lookup property in base classes of non-introspected types (Olivier Crête) (#684058)

Read more
Michael Hall

Well, we did it.  The six members of the Canonical Community Team stayed awake and (mostly) online for 24 straight hours, all for your entertainment and generous donations.  A lot of people gave a lot over the last week, both in terms of money and time, and every one of you deserves a big round of applause.

Team Insanity

First off, I wanted to thank (blame) our fearless leader, Jono Bacon, for bringing up this crazy idea in the first place.  He is the one who thought we should do something to give back to other organizations, outside of our FLOSS eco-system.  It’s good to remind us all that, as important as our work is, there are still things so much more important.  So thanks, Jono, for giving us a chance to focus some of our energy on the things that really matter.

I also need to thank the rest of my team, David Planella, Jorge Castro, Nick Skaggs and Daniel Holbach, for keeping me entertained and awake during that long, long 24 hours.  There aren’t many people I could put up with for that long, I’m glad I work in a team full of people like you.  And most importantly, thanks to all of our families for putting up with this stunt without killing us on-air.

Upstream Awesomeness

Before we started this 24-hour marathon, I sent a challenge to the Debian community.  I said that if I got 5 donations from their community, I would wear my Debian t-shirt during the entire broadcast.  Well, I should have asked for more, because it didn’t take long before I had more than that, so I was happily sporting the Debian logo for 24 hours (that poor shirt won’t ever be the same).

I wasn’t the only one who put a challenge to the Debian community.  Nick made a similar offer, in exchange for donations he would write missing man pages, and Daniel did the same by sending patches upstream.  As a result, the Debian community made an awesome showing in support of our charities.

All of our donors

The biggest thanks, of course, go out to all of those who donated to our charities.  Because of your generosity we raised well over £5000, with the contributions continuing to come in even after we had all finally gone to bed.  As of right now, our total stands at £ 5295.70 ($8486).  In particular, I would like to thank those who helped me raise £739.13 ($1184) for the Autism Research Trust:

And a very big thank you to my brother, Brian Hall, who’s donation put us over £5000 when we only had about an hour left in the marathon.  And, in a particularly touching gesture of brotherly-love, his donation came with this personal challenge to me:

So here it is.  The things I do for charity.

Read more
Michael Hall

Back in San Francisco, during UDS-Q, we had a discussion about the need for better online documentation for the various APIs that application developers use to write apps for Ubuntu.  The Ubuntu App Showdown and subsequent AppDevUploadProcess spec work has consumed most of my time since then, but I was able to start putting together a spec for such a site.  The App Showdown feedback we got from our developers survey highlighted the need, as lack of good Gtk documentation for Python was one of the most common problems people experienced, giving it a little more urgency.

Fortunately, Alberto Ruiz was at UDS, and told me about a project he had started for Gnome called Gnome Developer Network (GDN for short).  Alberto had already done quite a bit of work on the database models and GObject Instropection parsing needed to populate it.  The plan is to use GDN as the database and import process, and build a user-friendly web interface on top of that, linking in external resources like tutorials and AskUbuntu questions, as well as user submitted comments and code snippets.

Now that the spec is (mostly) done, we need to get together some developers who can implement it.  There will be a lot of front-end work (mostly HTML, CSS and Javascript), but also enough backend work (Python and Django) to keep anybody occupied.  I’ve created a Launchpad project for the site, and a team you can join if you’re interested in helping out.

The GDN code and some very basic template are already available. You can get the code from bzr with bzr branch lp:ubuntu-api-website and following the instructions in the DEVELOPMENT file.  I’ll also be running a live App Developer Q&A Session at 1700 UTC today (September 19th), and would be happy to help anybody get the code up and running during that time.

Read more
Michael Hall

More than a few, actually. As part of our ongoing focus on App Developers, and helping them get their apps into the Ubuntu Software Center, we need to keep the Application Review Board (ARB) staffed and vibrant. Now that the App Showdown contest is over, we need people to step up and fill the positions of those members who’s terms are ending. We also want to grow the community of app reviewers that work with the ARB to process all of the submissions that are coming in to the MyApps portal.

ARB Membership

Two of the existing members, Bhavani Shankar and Andrew Mitchell, will be continuing to serve on the board, and Alessio Treglia will be joining them. But we still need four more members in order to fill the full 7 seats on the board.  ARB applicants must be Ubuntu Members, Ubuntu Developers, and should create a wiki page for their application.

ARB members help application developers get their apps into Software Center by reviewing their package, providing support and feedback where needed, and finally voting to approve the app’s publication.  You should be able to dedicate a few hours each week to reviewing apps in the queue, and discussing them on IRC and the ARB’s mailing list.

If you would like to apply, you can contact the current ARB members on #ubuntu-arb on Freenode IRC, or the team mailing list (app-review-board at lists.ubuntu.com).  The current term will expire at the end of the month, so be sure to get your applications in as soon as you can.

ARB Helpers

In addition to the 7 members of the ARB itself, we are building a community of volunteers to help review submitted packages, and work with the author to make the necessary changes.  There are no limits or restrictions on members of this community, though a rough knowledge of packaging will surely help.  This group doesn’t vote on applications, but they are essential to helping get those applications ready for a vote.

The ARB helpers community was launched in response to the overwhelming number of submissions that came in during the App Showdown competition.  Daniel Holbach put together a guide for new contributors to help them get started reviewing apps, and you can still follow those same steps if you would like to help out.

Again, if you would like to get involved with this community, you should join #ubuntu-arb on Freenode IRC, or contact the mailing list (app-review-board at lists.ubuntu.com).

Read more
Michael Hall

For the past several week, David Planella, Jono Bacon and I have been drafting a spec that proposes a radically different approach to getting desktop applications into the Ubuntu Software Center.  Now, there’s nothing that annoys me more than somebody proposing radical changes for no reason, and without giving much thought as to how it would actually be done.  So I wanted to write down, here, both the justification for this proposal, and the process that we went through in drafting it.

The current process splits submissions between closed-source and commercial apps, which get reviewed by a paid team of Canonical employees, and non-commercial open source apps which are reviewed by the Application Review Board (ARB).  The ARB consists of 7 volunteers from the Ubuntu community, who will review the source code and packaging of each submission.  Members of the ARB are very smart, very dedicated members of the community, but they also have paying jobs, or are pursuing higher education (or both), so their time is a limited resource.  The ARB process was meant to provide an easier route for app developers than the more rigorous process that distro packages must follow to get into the Universe repository or Debian’s archives, and in that respect it has been a success.  But even with eased requirements, there was a limit to how many apps they could manually review.

The recent App Developer Showdown competition, which resulted in more than 140 new apps being submitted through our MyApps portal, showed us the limits of our current process.  We even drafted a number of new volunteers to help review the incoming apps, and Daniel Holbach provided both instructions and programs to help speed things up.  It took us weeks to give an initial review to all of the apps.  Almost two months later and we still haven’t been able to publish more than a quarter of them.  Android has seen over 9,000 new apps in a month, and I can only assume that iOS has seen similar numbers.  If we can’t even scale to handle 140, something has to change.

The spec didn’t get written down all at once from some grand design. It grew organically, from a short list of general goals to the massive text it is today.  In fact, the spec we ended up with is quite a bit different than the one we initially set out to write.  We took our list of goals and started asking the obvious questions: what work is involved, who will it impact, and what could (will) go wrong?  We could have just throw these questions out to other people, but those people are busy and have their own things they are trying to do.  Before we could ask anybody else to spend time on this, we had to put in some effort ourselves.

So we answered as many of these as we could between the three of us, and those answers changed our spec accordingly.  That raised more questions, and we repeated the process, updating the spec and finding more questions that needed to be answered.  In the process we gained both a clearer idea of what we wanted, and a better understanding of how to get there.  By the time we had answered as many as we could on our own, our list of goals had transformed into a longer list of implementation items and who would most likely be doing them.

At that point, we had a more specific direction and a pretty good idea of how much work it would take.  Having done as much of the leg-work as we could, we took the implementation items, and any unanswered questions we still had, and started talking to the people who would have to implement it.  Unsurprisingly, these conversations had an even bigger impact on the spec, and it underwent some pretty drastic changes as we tried to nail down the details of the implementation.  Just like the previous stage, we iterated over this one multiple times until we had as many details as we could collect, and answered all of the questions that we could.  At the end, we had the massive spec we announced today.

But this is just the next stage, the spec isn’t final.  The three of us have answered as much as we could, the teams who will implement it have answered as much as they could, now we’re introducing it the community to gather even more details and answer even more questions.  The feedback we get in this stage will go back into the spec, and very likely generate new questions and feedback, and we’ll iterate through this stage too.

The final spec, whatever it ends up being, isn’t going to be perfect, and it’s not going to make everybody happy.  But we can be confident that is will be a very well thought out spec, it will be a very detailed spec, and it will allow us to accomplish the goals we set out to accomplish at the beginning of it all.  It will help make Ubuntu a much more attractive platform for application developers, it will make Software Center more useful to developers and users alike, and it will make Ubuntu a better OS for all of our users.

If you have any questions or comments on the spec itself, please send them to the ubuntu-devel mailing list, not the comments section here.

Read more
Michael Hall

You can watch the App Developer Q&A live stream starting at 1700 UTC (or watch the recording of it afterwards):

Questions should be asked in the #ubuntu-on-air IRC channel on freenode.

You can ask me anything about app development on Ubuntu, getting things into the Software Center, or the recent Ubuntu App Showdown competition.

Read more
Michael Hall

As you’ve probably heard already, Ubuntu is running an App Developer Showdown competition where contestants have three weeks to build an Ubuntu app from scratch.  The rules are simple: It has to be new code, it has to run on Ubuntu, and it has to be submitted to the Software Center.  The more you use Ubuntu’s tools, the better your chances of winning will be.  This week we ran a series of workshops introducing these tools and how they can be used.  It all seemed like so much fun, that I’ve decided to participate with my own submission!

Now 2 our of the 6 judges for this competition are my immediate co-workers, so let me just start off by saying that I will not be eligible for any of the prizes.  But it’s still a fun and interesting challenge, so I’m going to participate anyway.  But what is my entry going to be?  Well in my typical fashion of building tools for tools, I’ve decided to write a GUI wrapper on to of Quickly, using Quickly.

Before I started on any code, I first wanted to brainstorm some ideas about the interface itself.  For that I went back to my favorite mockup tool: Pencil.  I knew I wanted to cover all of Quickly’s functions, both for creating projects and working on them afterwards.  I also wanted something that would keep track of my projects, so I wouldn’t have to find where they are on my disk whenever I wanted to hack on them.

Now, I’ve never been a fan of GUI builders.  Even back when I was writing Java/Swing apps, and GUI builders were all the rage, I never used them.  I didn’t use one for Hello Unity, and I wasn’t planning on using Glade for this project either.  But after Jono’s fantastic workshop session about Glade, I decided to give it another chance.  I found that I was able to get a basic UI built and running in very little time.  I’m still struggling with some, and there’s a point where you need to switch from Glade to code, but all in all it has saved me a significant amount of time.

Quickly also saved me a large amount of time, both in creating the project and adding things too it.  Being able to add an Application Indicator to your app by just running “quickly add indicator” is amazing.  From there is was a simple matter to build a menu based on available Quickly commands and tie them in with callback functions.

But the part I like the best about this app so far, is that it’s useful even when you’re not using it.  Most of the time you spend developing a Quickly app is going to be in some other application, such as your code editor of choice, Glade or something.  Thanks to Unity’s HUD, and the fact that it’s smart enough to check Indicator menus in addition to the focused application’s menus, you can call your Quickly commands any time, simply by tapping ‘Alt’ and the command you want to run.  It’s like having Quickly integrated into all of your other tools.

And thanks to the developer tools available in Ubuntu, I was able to accomplish all of this in only a few hours of work.

Now it’s very, very far from being complete.  For instance, the “active” project is hard-coded to my quickly-gtk working directory, it can’t start a project yet, or support commands that take optional arguments or user input.  But in a short amount of time I was able to go from a mockup to a working layout and even some functional code.  It even packages successfully, something I found out quite by accident when I selected “Share” from the indicator menu and ended up with a package in my PPA.

Building an app in 4 hours then accidentally building a proper package and uploading it to a PPA, who’d have thought we’d ever make it that easy?  I hope you all are having as much fun and success in your showdown applications as I am.

Read more
pitti

A few weeks ago I wrote about my new role as an upstream QA engineer. I have now officially been in that role since June. Quite expectedly I had (and still have) some backlog from my previous Desktop engineer role, but I have had plenty of time to work on automatic tests and some test technology now. If you are interested in the daily details, you can look at the ramblings on my G+ page; in a nutshell I worked on integration tests for udisks2 (mostly upstream now), a mock polkit API, and a small enhancement of the scsi_debug kernel module. On the distro QA side I got the integration tests of udisks2, upower, PostgreSQL, Apport, and ubuntu-drivers-common working and added to our Jenkins autopkgtest runner, where they are executed whenever the particular package or any of its dependencies get updated. This already uncovered a surprising number of actual bugs, so I’m happy that this system starts being useful after the initial hump of getting the tests to run properly in that environment.

In that previous blog post I mentioned that Canonical will hire a second person for an upstream QA engineer role. I am pleased that the job posting is now online, so if you are familiar with how the Linux plumbing and desktop stacks work, are frantic about testing, like to work with the Linux, plumbing, GNOME, and other FOSS communities, know your way around jhbuild, Jenkins, and similar technologies, and would like to explore new possibilities like applying static code checking or creating APIs to fake hardware, please have a look at the role description! Please feel free to contact me on IRC (pitti on Freenode) or by email if you have further questions about the role.

Read more
Michael Hall

Last week we introduced a new ‘Download for Ubuntu’ campaign for upstreams to use on their websites, letting their users know that the app is available in Ubuntu already.  We event generated a list of targeted upstreams we wanted to reach out to in order to spur the adoption of these buttons.  What we didn’t go into much detail about why upstreams should use them.  I hope to remedy that here.

It’s easy

Let’s just get that out of the way, this won’t take a significant amount of work on the part of an upstream.  It’s just a one time change to a website.  You don’t even need to change it every cycle, since the buttons point to the App Directory entry for the application itself, not any specific version of it.

It makes installing your app more appealing

The button isn’t just another way of getting your app, it also tells the user that it will install correctly, all of it’s dependencies are available and will be installed, everything is configured to work with their system, and they will get be getting updates and security fixes to it through a mechanism they already use and trust.  In short, it’s a promise of a good user experience (which I’ll admit we don’t always live up to, more on that below).  Telling 20 million users (and growing) that your app is safe and easy to install is surely worth a few pixels on your website.

It’s good social exposure for your app

By sending users to the App Directory, instead of just immediately installing, new users get to see what others are saying about your app through the ratings and reviews (which will be mostly positive, because your app is awesome right?)  of other Ubuntu users.  Not only does this tell your users that other people like your app, but it’s also telling them that they can add their own ratings and reviews, which will in turn boost your app’s standing.  More reviews leads to more users, which leads to more reviews, it’s a great positive feedback loop.

Users will be looking for it

Not right now, obviously, since we just started this campaign.  But as more upstreams adopt the new button, it’s going to be one of the first things Ubuntu users will be looking for on your website (for all the reasons mentioned above).  With a majority of website visitors leaving in less than a minute (according to a lazy Google search), the promise of a quick and easy install might just be the difference between a new user and a lost opportunity.

This campaign benefits everybody: end users, upstream developers and, yes, Ubuntu too.  So let’s improve these ties, together.  If you’re an upstream, you can copy/paste the following HTML snippet directly into your website (replacing {{pkgname}} with the name of your application’s package in Ubuntu).  If you want to reach out to an upstream developer, please add them to our list so we know who’s contacting them, and what the status is.

<a href="https://apps.ubuntu.com/cat/applications/{{pkgname}}/">
 <img src="http://developer.ubuntu.com/wp-content/uploads/2012/06/downloadonubuntubutton.png"  title="Download for Ubuntu" alt="Download for Ubuntu button" width="122" height="49" />
</a>

Now I know we can’t always give the best user experience possible (see, I told you I’d get to that).  Sometimes our packagin isn’t quite right, or the default configuration of your app is sub-optimal.  Our six month release cadence and package freezes mean that rapidly developing applications will often be out of date in our main repositories.  We’ve taken on a lot of work by distributing apps the way we do, and even though we’re a very large community, it’s still hard to get every package right.  Luckily, you’re not powerless here, if you spot problems with the way we distribute your app, or you need to get a newer version out to Ubuntu users, you can do something about that.

Package fixes

Even though our process locks applications to the version in the archives for that particular release of Ubuntu, we will still allow changes to the packaging itself.  So if we’ve done something wrong on our end that is giving your app a hard time, we’ll fix it and make that available to all of your Ubuntu users as a Stable Release Update.

Backport newer versions

A six-month release cycle means that every Ubuntu release has relatively up to date versions of applications, at least compared to distros that have a longer cadence.  But for rapidly developed applications, where new versions come out more frequently than that, this means their packages can become outdated quickly.  And with the five year lifetime of our LTS releases, most packages will get to be stale by the end.  That’s why we have a special repository just for backporting new versions of packages to stable releases of Ubuntu. And starting with 11.10, this repository is enabled by default.

In order to have your application backported to a stable release, it first has to be accepted into the current development release.  If your new version was in Debian’s unstable repository at the beginning of the development cycle, chances are it’s already there.  If it’s not in Debian you’ll need to submit your package to be included in the development release.  Once it’s there, you can request that it be backported to one more more stable Ubuntu releases.  You can use the requestbackport command line tool (from ubuntu-dev-tools package) to automate much of the process, or if you’re not running Ubuntu simply file a bug to start the request.

Read more
Michael Hall

It’s no secret anymore, we want apps in Ubuntu.  Big app, small apps, shiny apps, cute apps,  apps that play a catchy tune while avian avengers explode porcine poachers.  You name it, we want it in Ubuntu.  And we need your help getting them.

It doesn’t matter if you’re not a programmer, there are plenty of programmers in the world, and they’re making all kinds of really cool apps that you want, that I want, that we all want.  But they’re not putting them in Ubuntu, and that’s why I need your help.

You see I’m just one person, and with David Planella we’ve got, well, two people.  Now, two people can find an awful lot of apps on the internet, that’s true.  Especially when that’s part of your job.  But even still that would hardly amount to a drop in the bucket of apps that have been made.  And we’re not happy with just a drop in the bucket, are you?

No, of course you’re not.  So right about now you’re probably thinking of a bunch of cool apps you’ve used or seen or read about recently (and if you weren’t before, you certainly are now, aren’t you).  And you may be wondering how you can help us get them into the Ubuntu Software Center.   Well wonder no more my friend, because we have a Trello board.

Not terribly impressive, I’ll admit, but it is terribly useful.  You see, David and I will be contacting each and every one of these app developers about submitting their apps to the Software Center, and moving the card from column to column as that conversation progresses.  And while we have a nice long “To Contact” list right now, it’s not going to stay that way without your help.

We’re opening up this board to anybody who wants to add new upstream apps for us to contact.  All you need to do is follow David’s instructions on this wiki page to get started.  It’s quick, mostly painless, and best of all you get to start telling us what to do (at least a little bit).

But wait! There’s more!  Didn’t I say we were just two people?  There’s no way we could contact all these upstreams on our own.  And hey, you’ve just signed up to join the Trello board (you did do that, right?), which means you can start contacting them too, and moving their cards, and make Ubuntu even better.  We’ve even created a reusable email template that you can use for that initial contact.

If you contact one of the app developers, make sure you move their card to the “Conversation Started” column, assign it to yourself, and add comments about who you contacted and any other relevant information.  This of course requires that you joined the board, which you definitely should do if you are going to be working on this.  And please note that is says “Started”, not “Finished”.  Once you contact the developer, keep that line of communication open and help them through the process of submitting their app.  If you get to a point where you can’t help them, let David or I know and we’ll be happy to pick it up.

Read more