One of the key aspects in developing the Ubuntu Touch core apps has been Quality Assurance. With that goal in mind, we’ve been adding functional tests to each and every one of the applications, using Autopilot.

We want to ensure our core apps are rock-solid, and we’d like to invite each of you who want to help make it happen to participate in the Autopilot Hackfest today. Here’s how:

1. Join the #ubuntu-quality IRC channel ›
2. Read the Autopilot tutorial ›
3. Read Nick Skagg’s blog post for more details ›

Looking forward to the new autopilot tests for core apps. See you there!

Image: Autopilot Engaged CC-BY-SA by Mike Miley

Time does fly, and we’re alread on the last day of the Ubuntu Developer Summit. Lots of content covered and still lots of interesting discussions to be had. We’re thrilled to bring you the summary on what’s on today on the App Development track.

Here’s the list of app development sessions for today at UDS:

• At 14:00 UTC: App Development Roundtable
• At 14:00 UTC: SDK UI Toolkit Responsive Layouting
• At 15:05 UTC: Core Apps plans for 13.10
• At 15:05 UTC: Contact Service for Ubuntu Touch
• At 16:05 UTC: LoCo Team Coding Challenge
• At 16:05 UTC: Refocus the Ubuntu App Developer site to go mobile
• At 18:05 UTC: Building a set tutorials for App Developers
• At 19:00 UTC: Closing Plenary and Track Summaries

Hope to see you there!

After a very productive kick off, we’re back with the second day of the Ubuntu Developer Summit on the App Development track and the summary of sessions for today. Thank you everyone who participated in the sessions yesterday, either in hangouts or in IRC.

Here’s the list of app development sessions for today:

• At 14:00 UTC: App Development Roundtable
• At 15:05 UTC: SDK Tools and Qt Creator
• At 16:05 UTC: The Next Ubuntu App Showdown
• At 18:05 UTC: New API Documentation Website
• At 18:05 UTC: QML Friends API and UI Components
• At 19:05 UTC: Core Apps in Ubuntu Touch Preview Images
• At 19:05 UTC: Calendar Application and Backend for Ubuntu Touch

See you there!

UDS, the Ubuntu Developer Summit, is here again, starting in just a few hours. A week packed with content that will define the plans for the new Ubuntu development cycle, and as usual, a with a full track dedicated to application development.

So for all of you interested in helping and being part of the effort of making Ubuntu a platform of choice for application developers, here’s a quick list with an overview of the sessions we’ve got in store for today.

The links in the list below will take you to the each session, ready to participate on the live hangout or on IRC. You can also check out the full UDS schedule.

So, without further ado, here’s the list of app development sessions for today:

• At 15:05 UTC: App Development Roundtable
• At 16:05 UTC SDK UI Toolkit Theming
• At 16:05 UTC SDK Feedback from App Developers
• At 18:05 UTC SDK Roadmap
• At 19:05 UTC Growing the Ubuntu SDK Apps Collection

Looking forward to seeing you there!

That time has once again arrived…the Top 10 for April. Stormcloud continues to reign the top with Fluendo DVD moving into the second spot in paid applications. The Top 10 Free apps has not changed much from last month with Steam continuing to dominate the Free Top 10.

Want to develop for the new Phone and Tablet OS, Ubuntu Touch? Be sure to check out the “Go Mobile” site for details.

Top 10 paid apps

1. Stormcloud
2. Fluendo DVD Player
3. War in a Box – Paper Tanks
4. Splice [NEW]
5. Filebot
6. UberWriter [NEW]
7. Quick ‘n Easy Web Builder
8. Braid
9. Drawers
10. Bastion

Top 10 free apps

1. Steam
2. Master PDF Editor
3. Youtube to MP3
4. Nitro
5. Plex Media Server
6. CrossOver (Trial)
7. Motorbike
8. IntelliJ IDEA 12 Community Edition
9. flareGet
10. Splashtop Streamer

Would you like to see your app featured in this list and on millions of user’s computers? It’s a lot easier than you think:

• If you’re just getting your feet wet, check out how to get started with your first app in Ubuntu ›
• If you have an existing app you’d like to deliver to the world, submit it to the Ubuntu Software Center ›

Notes:

• The lists of top 10 app downloads includes only those applications submitted through My Apps on the Ubuntu App Developer Site. For more information about of usage of other applications in the Ubuntu archive, check out the Ubuntu Popularity Contest statistics.
• The top 10 free apps list contains gratis applications that are distributed under different types of licenses, some of which may not be open source. For detailed license information, please check each application’s description in the Ubuntu Software Center.

Follow Ubuntu App Development on:

Social Media Icons by Paul Robert Lloyd

A few years ago, when I started pondering about the possibility of porting juju to the Go language, one of the first pieces of the puzzle that were put in place was goyaml: a Go package to parse and serialize a yaml document. This was just an experiment and, as a sane route to get started, a Go layer that does all the language-specific handling was written on top of the libyaml C scanner, parser, and serializer library.

This was a good initial plan, but for a number of reasons the end goal was always to have a pure Go implementation. Having a C layer in a Go program slows down builds significantly due to the time taken to build the C code, makes compiling in other platforms and cross-compiling harder, has certain runtime penalties, and also forces the application to drop the memory safety guarantees offered by Go.

For these reasons, over the last couple of weeks I took a few hours a day to port the C backend to Go. The total time, considering full time work days, would be equivalent to about a week worth of work.

The work started on the scanner and parser side of the library. This took most of the time, not only because it encompassed more than half of the code base, but also because the shared logic had to be ported too, and there was a need to understand which patterns were used in the old code and how they would be converted across in a reasonable way.

The whole scanner and parser plus header files, or around 5000 code lines of C, were ported over in a single shot without intermediate runs. To steer the process in a sane direction, gofmt was called often to reformat the converted code, and then the project was compiled every once in a while to make sure that the pieces were hanging together properly enough.

It’s worth highlighting how useful gofmt was in that process. The C code was converted in the most convenient way to type it, and then gofmt would quickly put it all together in a familiar form for analysis. Not rarely, it would also point out trivial syntactic issues. A double win.

After the scanner and parser were finally converted completely, the pre-existing Go unmarshaling logic was shifted to the new pure implementation, and the reading side of the test suite could run as-is. Naturally, though, it didn’t work out of the box.

To quickly pick up the errors in the new implementation, the C logic and the Go port were put side-by-side to run the same tests, and tracing was introduced in strategic points of the scanner and parser. With that, it was easy to spot where they diverged and pinpoint the human errors.

It took about two hours to get the full suite to run successfully, with a handful of bugs uncovered. Out of curiosity, the issues were:

• An improperly dropped parenthesis affected the precedence of an expression
• A slice was being iterated with copying semantics where a reference was necessary
• A pointer arithmetic conversion missed the base where there was base+offset addressing
• An inner scoped variable improperly shadowed the outer scope

The same process of porting and test-fixing was then repeated on the the serializing side of the project, in a much shorter time frame for the reasons cited.

The resulting code isn’t yet idiomatic Go. There are several signs in it that it was ported over from C: the name conventions, the use of custom solutions for buffering and reader/writer abstractions, the excessive copying of data due to the need of tracking data ownership so the simple deallocating destructors don’t double-free, etc. It’s also been deoptimized, due to changes such as the removal of macros and in many cases its inlining, and the direct expansion of large unions which causes some core objects to grow significantly.

At this point, though, it’s easy to gradually move the code base towards the common idiom in small increments and as time permits, and cleaning up those artifacts that were left behind.

This code will be made public over the next few days via a new goyaml release. Meanwhile, some quick facts about the process and outcome follows.

Lines of code

According to cloc, there was a total of 7070 lines of C code in .c and .h files. Of those, 6727 were ported, and 342 were 12 functions that were left unconverted as being unnecessary right now. Those 6727 lines of C became 5039 lines of Go code in a mostly one-to-one dumb translation.

That difference comes mainly from garbage collection, lack of forward declarations, standard helpers such as append, range-based for loops, first class slice type with length and capacity, internal OOM handling, and so on.

Future work code can easily increase the difference further by replacing some of the logic ported with more sensible options available in Go, such as standard abstractions for readers and writers, buffered writing support as availalbe in the standard library, etc.

Code clarity and safety

In the specific context of the work done, which is of a scanner, parser and serializer, the slice abstraction is responsible for noticeable clarity gains in the code, when compared to the equivalent logic based on pointer arithmetic. It also gives a much more comforting guarantee of correctness of the written code due to bound-checking.

Performance

While curious, this shouldn’t be taken as a performance comparison between the two languages, as it is comparing a fine tuned C implementation with something that is worse than a direct one-to-one port: not only it hasn’t seen any time at all on preventing waste, but the original logic was deoptimized due to changes such as the removal of inlining macros and the expansion of large unions. There are many obvious changes to be done for improving performance.

With that out of the way, in a simple decoding benchmark the C-backed decoder runs on about 37% of the time taken by the out-of-the-box deoptimized Go port.

Output size

The previous goyaml.a Go package file had 1463kb. The new one has 1016kb. This difference includes glue code generated for the integration.

Considering only the .c and .h files involved in the port, the C object code generated with the standard flags used by the go build tool (-g -O2) sums up to 789kb. The equivalent Go code with the standard settings compiles to 664kb. The 12 functions not ported are also part of that difference, so the difference is pretty much negligible.

Build time

Building the 8 .c files alone takes 3.6 seconds with the standard flags used by the go build tool (-g -O2). After the port, building the entire Go project with the standard settings takes 0.3 seconds.

Mechanical changes

Many of the mechanical changes were done using regular expressions. Excluding the trivial ones, about a dozen regular expressions were used to swap variable and type names, drop parenthesis, place brackets in the right locations, convert function declarations, and so on.

Last week I was part of a rant with a couple of coworkers around the fact Go handles errors for expected scenarios by returning an error value instead of using exceptions or a similar mechanism. This is a rather controversial topic because people have grown used to having errors out of their way via exceptions, and Go brings back an improved version of a well known pattern previously adopted by a number of languages — including C — where errors are communicated via return values. This means that errors are in the programmer’s face and have to be dealt with all the time. In addition, the controversy extends towards the fact that, in languages with exceptions, every unadorned error comes with a full traceback of what happened and where, which in some cases is convenient.

All this convenience has a cost, though, which is rather simple to summarize:

Exceptions teach developers to not care about errors.

A sad corollary is that this is relevant even if you are a brilliant developer, as you’ll be affected by the world around you being lenient towards error handling. The problem will show up in the libraries that you import, in the applications that are sitting in your desktop, and in the servers that back your data as well.

Raymond Chen described the issue back in 2004 as:

Writing correct code in the exception-throwing model is in a sense harder than in an error-code model, since anything can fail, and you have to be ready for it. In an error-code model, it’s obvious when you have to check for errors: When you get an error code. In an exception model, you just have to know that errors can occur anywhere.

In other words, in an error-code model, it is obvious when somebody failed to handle an error: They didn’t check the error code. But in an exception-throwing model, it is not obvious from looking at the code whether somebody handled the error, since the error is not explicit.
(…)
When you’re writing code, do you think about what the consequences of an exception would be if it were raised by each line of code? You have to do this if you intend to write correct code.

That’s exactly right. Every line that may raise an exception holds a hidden “else” branch for the error scenario that is very easy to forget about. Even if it sounds like a pointless repetitive task to be entering that error handling code, the exercise of writing it down forces developers to keep the alternative scenario in mind, and pretty often it doesn’t end up empty.

It isn’t the first time I write about that, and given the controversy that surrounds these claims, I generally try to find one or two examples that bring the issue home. So here is the best example I could find today, within the pty module of Python’s 3.3 standard library:

def spawn(argv, master_read=_read, stdin_read=_read):
    """Create a spawned process."""
    if type(argv) == type(''):
        argv = (argv,)
    pid, master_fd = fork()
    if pid == CHILD:
        os.execlp(argv[0], *argv)
    (...)

Every time someone calls this logic with an improper executable in argv there will be a new Python process lying around, uncollected, and unknown to the application, because execlp will fail, and the process just forked will be disregarded. It doesn’t matter if a client of that module catches that exception or not. It’s too late. The local duty wasn’t done. Of course, the bug is trivial to fix by adding a try/except within the spawn function itself. The problem, though, is that this logic looked fine for everybody that ever looked at that function since 1994 when Guido van Rossum first committed it!

Here is another interesting one:

$ make clean
Sorry, command-not-found has crashed! Please file a bug report at:

https://bugs.launchpad.net/command-not-found/+filebug

Please include the following information with the report:

command-not-found version: 0.3
Python version: 3.2.3 final 0
Distributor ID: Ubuntu
Description:    Ubuntu 13.04
Release:        13.04
Codename:       raring
Exception information:

unsupported locale setting
Traceback (most recent call last):
  File "/.../CommandNotFound/util.py", line 24, in crash_guard
    callback()
  File "/usr/lib/command-not-found", line 69, in main
    enable_i18n()
  File "/usr/lib/command-not-found", line 40, in enable_i18n
    locale.setlocale(locale.LC_ALL, '')
  File "/usr/lib/python3.2/locale.py", line 541, in setlocale
    return _setlocale(category, locale)
locale.Error: unsupported locale setting

That’s a pretty harsh crash for the lack of locale data in a system-level application that is, ironically, supposed to tell users what packages to install when commands are missing. Note that at the top of the stack there’s a reference to crash_guard. This function has the intent of catching all exceptions right at the edge of the call stack, and displaying a detailed system specification and traceback to aid in fixing the problem.

Such “parachute catching” is a fairly common pattern in exception-oriented programming and tends to give developers the false sense of having good error handling within the application. Rather than actually guarding the application, though, it’s just a useful way to crash. The proper thing to have done in the case above would be to print a warning, if at all, and then let the program run as usual. This would have been achieved by simply wrapping that one line as in:

try:
    locale.setlocale(locale.LC_ALL, '')
except Exception as e:
    print("Cannot change locale:", e)

Clearly, it was easy to handle that one. The problem, again, is that it was very natural to not do it in the first place. In fact, it’s more than natural: it actually feels good to not be looking at the error path. It’s less code, more linear, and what’s left is the most desired outcome.

The consequence, unfortunately, is that we’re immersing ourselves in a world of brittle software and pretty whales. Although more verbose, the error result style builds the correct mindset: does that function or method have a possible error outcome? How is it being handled? Is that system-interacting function not returning an error? What is being done with the problem that, of course, can happen?

A surprising number of crashes and plain misbehavior is a result of such unconscious negligence.

Hot on the heels of the Game Developer’s Conference in San Francisco we bring you the Top 10 Ubuntu App downloads for March 2013.
Stormcloud continues its rule at the top of the charts and the “far out” puzzle game, Machinarium is right behind it at #2. No surprise that Steam continues to dominate the top Free chart.
We saw some really cool technology at the Game Developer’s Conference, met some super nice people, and demonstrated the Ubuntu Phone and Tablet to a ton of people at the show.

Top 10 paid apps

1. Stormcloud
2. Machinarium
3. Fluendo DVD Player
4. War in a Box – Paper Tanks [NEW]
5. Filebot
6. Quick ‘n Easy Web Builder
7. Braid
8. Legend of Grimrock
9. Mini Minecraft Launcher
10. Linux Tycoon [NEW]

Top 10 free apps

1. Steam
2. Master PDF Editor
3. Youtube to MP3
4. Splashtop Streamer
5. Plex Media Server
6. Motorbike
7. CrossOver (Trial)
8. Nitro
9. flareGet
10. IntelliJ IDEA 12 Community Edition

Would you like to see your app featured in this list and on millions of user’s computers? It’s a lot easier than you think:

• If you’re just getting your feet wet, check out how to get started with your first app in Ubuntu ›
• If you have an existing app you’d like to deliver to the world, submit it to the Ubuntu Software Center ›

Notes:

• The lists of top 10 app downloads includes only those applications submitted through My Apps on the Ubuntu App Developer Site. For more information about of usage of other applications in the Ubuntu archive, check out the Ubuntu Popularity Contest statistics.
• The top 10 free apps list contains gratis applications that are distributed under different types of licenses, some of which may not be open source. For detailed license information, please check each application’s description in the Ubuntu Software Center.

Follow Ubuntu App Development on:

Social Media Icons by Paul Robert Lloyd

Canonical is again attending the Game Developer Conference in San Francisco. This year we will be in the ever popular Unity Partner Pavilion located in the South Hall #1002. The show floor is open from Wednesday to Friday March 27 – 29.

Stop by our kiosk for a demonstration of the Unity3d export for Ubuntu and see how easy it is to submit games to the Ubuntu Software Center via our App Developer Program.  Let us show you the benefits of Ubuntu, the painless submission process, and how we can help you access millions of Ubuntu users. We will have Ubuntu running on phones and tablets so you can touch the future of Ubuntu on mobile.

You will not want to miss a talk about monetising games and how to go mobile on the Ubuntu platform to be presented Wednesday, 27 March at 1:55 p.m. in the Unity Booth Theater on the exhibit floor in the South Hall.

Follow us on Twitter, Facebook or on G+ for updates while we are there.

See you at the Game Developer’s Conference!

Hot on the heels of the announcements of the Ubuntu SDK and the Touch Developer Preview, we bring you the first ever Ubuntu SDK Days.

On Thursday, 14th March and Friday, 15th March a number of app developers and Ubuntu SDK creators will get you started writing apps for Ubuntu on multiple devices. It’s surprisingly simple, and since the announcement we’ve seen many early adopters try out the SDK and the first apps up and running. We will  answer your questions, talk about best practises and show you the power of the SDK.

Here a quick overview over the sessions we’ll run:

• Installing and Configuring the SDK
• Writing your first app with the SDK
• Writing games with QML and Javascript
• Live update from the development progress of the Touch Core Apps
• Several Q&A sessions
• Making the best of the Ubuntu App Design guidelines
• More about the SDK skunkworks projects
• Introducing Friends and Gwibber QML
• Writing a new generation of Scopes
• Lightning talks and Project demos

How to join

Participating is easy: just head to http://ubuntuonair.com to watch the sessions on the schedule. Videos will be available after the event, to ensure you can watch the content even if you couldn’t make it to the session you wanted.

You can ask your questions on the chat widget on http://ubuntuonair.com or join the #ubuntu-app-devel IRC channel on Freenode directly.

Check out https://wiki.ubuntu.com/UbuntuSDKDays/ to see the timetable of the event, be there for lots of fun and bring your friends – and your questions too!

Another month in the books and this time we’re on time giving you the top 10 for February 2013. Stormcloud still rules the top spot and Machinarium is back in the charts at #2 with its far out puzzle game. Steam steamed its way to the top in the free category after it’s debut in late February.
Come see us at GDC ’13 in the Unity Technologies pavilion and we’ll show you how to make money on your app in Ubuntu quick! Stop by, say hi and check out the latest Ubuntu Touch devices, too.

Top 10 paid apps

1. Stormcloud
2. Machinarium
3. Braid
4. Fluendo DVD Player
5. Quick ‘n Easy Web Builder
6. Legend of Grimrock
7. Bastion
8. RC Mini Racers
9. Drawers
10. Filebot [NEW]

Top 10 free apps

1. Steam [NEW]
2. Youtube to MP3 [NEW]
3. Motorbike [NEW]
4. Plex Media Server
5. Splashtop Streamer
6. Command & Conquer: Tiberium Alliances
7. Lord of Ultima
8. CrossOver (Trial)
9. Master PDF Editor
10. Nitro

Would you like to see your app featured in this list and on millions of user’s computers? It’s a lot easier than you think:

• If you’re just getting your feet wet, check out how to get started with your first app in Ubuntu ›
• If you have an existing app you’d like to deliver to the world, submit it to the Ubuntu Software Centre ›

Notes:

• The lists of top 10 app downloads includes only those applications submitted through My Apps on the Ubuntu App Developer Site. For more information about of usage of other applications in the Ubuntu archive, check out the Ubuntu Popularity Contest statistics.
• The top 10 free apps list contains gratis applications that are distributed under different types of licence, some of which may not be open source. For detailed licence information, please check each application’s description in the Ubuntu Software Centre.

Follow Ubuntu App Development on:

Social Media Icons by Paul Robert Lloyd

Yesterday we released Ubuntu Touch Preview images for four devices. This is a huge milestone for Ubuntu. We always wanted Ubuntu to be everywhere and the Preview shows quite nicely how well the vision of a design family across different form factors works.

There is quite a bit of work to be done, we all know that, but it’s a giant opportunity for us, the Ubuntu community. Everybody can contribute to the effort and we can show the world how we believe software should look like.

How you can help? Easy.

• You can install the Ubuntu Touch preview images on a device and test them.
• You can help out designing and shaping the Ubuntu Touch Core Apps.
• If you are a bit more experienced with bringing software up on new devices, you can help us porting Ubuntu Touch to new devices.

Did the last point find your interest? Excellent, because we just took the wraps of our Ubuntu Touch Porting guide. This also marks the start of our Ubuntu Touch Port-a-thon. We want to get Ubuntu Touch up and running on as many devices as possible.

If you don’t mind some tinkering, maybe some kernel building, some configuration meddling and flashing your device repeatedly, you might just the person we’re looking for.

The porting guide should help you understand

• how Ubuntu Touch works internally,
• which bits are generally involved and where to find them
• how to submit patches
• how images are put together
• how to test them and
• where to find help

To get you started and into the mood, you might want to join us today, at Friday 22nd February at 15:00 UTC on http://ubuntuonair.com when two super heroes of the Ubuntu Touch project, namely Ricardo Salveti and Sergio Schvezov, are going to talk to us about the technical aspects of the phone and the tablet.

Reliable sources tell us, there’s going to be a surprise announce during the hangout as well.

This is the opportunity we always wanted. Let’s make it happen. Bring Ubuntu to the world in all its beauty.

We’re thrilled to announce yet another significant milestone in the history of the Ubuntu project. After having recently unveiled the Ubuntu Touch Developer Preview, today we’re publishing the full source code and images for supported devices.

For developers and enthusiasts only

While a huge amount of Engineering and Design work has been put into ensuring that the foundations for our user experience vision are in place, we want to stress that the Ubuntu Touch Developer Preview is currently work in progress. We are releasing the full code at this point to align to our philosophy of transparency and open source development.

We recommend to install the Touch Developer Preview only if you are a developer or enthusiast who wants to test or contribute to the platform. It is not intended to replace production devices or the tablet or handset you use every day.

Flash your device

All that said, let’s get on to how to install Touch Developer Preview from a public image on your device.

What to expect after flashing

Not all functionality from a production device is yet available on the Touch Preview. The list of functions you can expect after installing the preview on your handset or tablet are as follows. For detailed information check the release notes.

• Shell and core applications
• Connection to the GSM network (on Galaxy Nexus and Nexus 4)
• Phone calls and SMS (on Galaxy Nexus and Nexus 4)
• Networking via Wifi
• Functional camera (front and back)
• Device connectivity through the Android Developer Bridge tool (adb)

Supported devices

The images we are making available today support the following devices:

• Galaxy Nexus
• Nexus 4
• Nexus 7
• Nexus 10

I’m all set, show me how to flash!

You will find the detailed instructions to flash on the Ubuntu wiki.

Install the Touch Developer Preview >

Contributing and the road ahead

These are exciting times for Ubuntu. We’re building the technology of the future, this time aiming at a whole new level of massive adoption. The Touch Developer Preview means the first fully open source mobile OS developed also in the open. True to our principles this milestone also enables our community of developers to contribute and be a key part of this exciting journey.

In terms of the next steps, today we’re making the preview images available for the Ubuntu 12.10 stable release. In the next few days we’re going to switch to Raring Ringtail, our development release, which is where development will happen on the road to our convergence story.

You’ll find the full details of how the infrastructure and the code are being published and used on the Ubuntu wiki.

Contribute to the Touch Developer Preview >

Presenting the Ubuntu SDK Alpha

But there’s more! To further celebrate the Touch Preview, we’re very proud to bring some exciting news that app developers will surely enjoy: the Ubuntu SDK Alpha release.

In fact, development of the SDK still keeps happening in the open and on a rolling release basis. But coinciding with the Touch Developer Preview, we thought that the latest release came with so much goodness, that we decided to label it in celebration.

Feature highlight: remote app deployment

Perhaps the coolest feature ever since the SDK was released: you can now deploy and execute the apps you create straight from the IDE.

Applications developed with Qt Creator can now be seamlessly and securely transferred and executed to a device just moving two fingers. Remember this shortcut: Ctrl+F12.

Inline with how easy and lightweight the process of creating a phone app is, a lot of work has been put into ensuring all complexity is hidden from the developer, yet it works solidly. Behind the scenes, SSH key pairing with the remote device works on-the-fly.

Here’s the lowdown:

1. Plug in your mobile device running Ubuntu on the USB port of your computer
2. Make sure your device is also connected to a wireless network (SSH key pairing happens over the air)
3. Start Qt Creator from the Dash, and select the new Devices tab
4. Press the Enable button to activate Developer Mode
5. Once the device is connected, you can develop your QML projects as usual (check out the new project wizard as well) and press Ctrl+F12 to install and execute your app on the remote device

Tooling updates

With Qt Creator at its heart, the set of tools app developers use on an everyday basis to author their software, have seen major improvements:

• Qt Creator has been updated to the bleeding edge version: 2.7. We expect this version to continue maturing together with the platform and the SDK.
• Ubuntu application templates and wizard are now available to easily start creating apps that run on the phone and tablets.
• The visual user interface designer in Qt Creator now works with QtQuick 2, the framework upon the Ubuntu SDK is based.

User Interface Toolkit updates

The UI Toolkit is the part of the SDK that provides the graphical components (such as buttons, text entries, and others) as building blocks that enable the basic user interaction with the underlying system. A new component, polishing and bug fixing have set the theme for this release:

• The MainView component has been added, with closer integration to the HUD and the shell. Learn how to use MainView.
• PopOver component polishing, general bug fixes and improvements

Install the Ubuntu SDK Alpha

By now we’re pretty certain you’re looking forward to installing and putting all of that development goodness to the test.

That’s an easy one, if you haven’t yet install the Ubuntu SDK.

If you already installed the SDK, just run Update Manager from the Dash and update the Ubuntu SDK package as prompted. Or alternatively, if you prefer the command line, just fire up a terminal and run ‘sudo apt-get update && sudo apt-get install ubuntu-sdk’.

And that’s pretty much it! Be sure to check out the release notes for any additional technical details too.

Let us know what you think

We’d be delighted to hear what you think and get your feedback on how you are using the SDK and ways in which it could be improved. So do get in touch with us or report a bug if you find any issues.

Time to start developing beautiful apps now!

We’re thrilled to announce one of the most expected resources for Ubuntu app developers: the App Design Guides.

The App Design Guides site is the first installment of a live resource that will organically grow to provide guidance and enable app developers to build stunning, consistent and usable applications on a diversity of Ubuntu devices.

Together with the Ubuntu SDK preview, the App Design Guides complete yet another chapter in the Ubuntu app developer story. Developers have now the tools to create beautiful software, along with all the information related to UX, behaviour, patterns and visual design to ensure their apps provide a solid, clean and enjoyable user experience.

And consistent with the Ubuntu philosophy and our beliefs, all of these tools and guides are available to everyone as open source and for free.

Show me the Ubuntu App Design Guides! ›

Updating the core app designs for Ubuntu App Guides compliance

We have recently kicked off a community-driven process to design and implement a set of 12 core apps for Ubuntu running on phones. The first stage of the project consisted in asking community members to participate in the submission of designs to be used as input and food for thought for the core app developers.

The response so far has been overwelming:  over 50 community designers signed up for this initiative, submitting nearly 90 mockups on the Ubuntu MyBalsamiq site we set up for this project.

Following the App Design Guides go-live, it is now a great opportunity to ensure those designs follow the guidelines for a consistent app experience on Ubuntu. Therefore, we’d like to ask everyone who submitted a design to review them and update them to make sure they are inline with the App Design Guides.

Reminder: if you want to collaborate in this design project, just drop an e-mail to David Planella <david(dot)planella(at)canonical(dot)com> and Michael Hall <michael(dot)hall(at)canonical(dot)com>.

Open design and collaboration

Continuing with the trend of open and collaborative design, we want to hear from you!

The Guides are a resource that will grow together with the needs of app developers, so we’ll greatly appreciate your feedback on the Ubuntu Phone mailing list (remember to prepend the subject with [Design]) and if you’ve got any questions about them, just ask on Ask Ubuntu.

Stay tuned for updates and for some visual designs for core apps from the Canonical Design team coming soon!

The new year is in full swing and we’re happy to give you the top 10 for January 2013. Stormcloud moved into the top spot while Legend of Grimrock jumped into the 3rd spot after it’s debut in the Humble Indie Bundle. Plex Media Server took over the top spot in the free category with the EA games not far behind.

Top 10 paid apps

1. Stormcloud
2. RC Mini Racers
3. Legend of Grimrock [NEW]
4. Braid
5. Bastion
6. Fluendo DVD Player
7. Torchlight
8. Quick ‘n Easy Web Builder
9. Drawers [NEW]
10. MC Launcher

Top 10 free apps

1. Plex Media Server
2. Lord of Ultima
3. Command & Conquer: Tiberium Alliances
4. Nitro
5. Master PDF Editor
6. CrossOver (Trial)
7. IntelliJ IDEA Community Edition
8. Ryzom
9. flareGet
10. Full Circle Magazine #68 [NEW]

Would you like to see your app featured in this list and on millions of user’s computers? It’s a lot easier than you think:

• If you’re just getting your feet wet, check out how to get started with your first app in Ubuntu ›
• If you have an existing app you’d like to deliver to the world, submit it to the Ubuntu Software Centre ›

Notes:

• The lists of top 10 app downloads includes only those applications submitted through My Apps on the Ubuntu App Developer Site. For more information about of usage of other applications in the Ubuntu archive, check out the Ubuntu Popularity Contest statistics.
• The top 10 free apps list contains gratis applications that are distributed under different types of licence, some of which may not be open source. For detailed licence information, please check each application’s description in the Ubuntu Software Centre.

Follow Ubuntu App Development on:

Social Media Icons by Paul Robert Lloyd

In the previous post, I explored a bit how ephemeral most of the artifacts of software development processes are. One of these processes is code reviewing, which is arguably a major player in code quality, knowledge acquisition, and even team dynamics.

Even being so important, the outcome of the code review process — the review itself — tends to reach a very limited audience and have a short life time. It’ll be hard to change that picture given the nature of reviews: they are conversational, and address specific issues for the integration of a change in the project. At the same time, even if code reviews are not generally useful as permanent documentation, we can increase their value as reference material by improving the quality of those conversations. Having a good conversation has many other great side effects, of course.

As a small step in that direction, what follows are personal guidelines that I have been evolving empirically over the years as a software developer and code reviewer. They may not bring you fortune and fame, and are not always easy to apply, but hopefully they will help improving your experience as a member of your team and the value of those reviews.

Explain why

Unless the change is about an extremely obvious mistake, explain why you’re suggesting it. If the reasoning was natural to the author, he’d have done it in the first place. Good explanations also help avoiding the same mistake over and over again, and are much more rewarding to the listener. They also become a target for future references.

If you don’t have enough time to justify it and would rather provide the review sooner than later, one approach is to just recommend the change and invite the author for a conversation later if that would be helpful. That said, try to have that conversation over a media that may be shared with the rest of the team, or recollected whenever necessary.

Be respectful

Always keep in mind that there’s a person on the other side of the wire, not a machine, and that it’s hard to understand written words with little context. Avoid letting anger and frustration leak into the review, even if you feel it is justified. There’s no good outcome in those situations.

It doesn’t matter who broke it, or who coded that silly piece of code. If there is broken code, and the project has reviews, multiple people were in the pipeline for that result, and they were trying to get it right. Take shared ownership of the problem, and look for the solution and for how to avoid such issues in the future.

Praise the good work

Reviews carry some low energy feel by their very nature. No matter how positive you are about them, and how much the whole team understands and agrees it is for the best, you are in fact looking for places to put your finger in someone else’s work. For that reason, it is very helpful to take every chance you can of praising logic, design, code organization, or whatever else that you honestly felt was well done. It won’t ever balance it out, but it will at least remind the author that the contributions are welcome.

Suggestions are appreciated

Perhaps a longer variable name would be helpful, or that constant could have a more descriptive name? In many circumstances, the change is indeed subjective, and the gain is pretty marginal. In those cases, if you really can’t resist the urge to say something, a good approach is a suggestion that may be exercised or not at the author’s discretion. Ideally, suggest several options that would feel better to you, so that your point is better understood and agreement is easier. That said, read on.

Avoid trivialities

When reviewing that very simple point, think to yourself: all things considered, does it actually matter? Is the cost of the author’s time, and the potential debate, really worth it? You surely have your opinion about whether to spell “min <= count” or “count >= min“, but so does everybody else. When it’s purely a matter of preference, the author is entitled to have one after all.

Small branches win

Code reviews are useful for a number of secondary reasons, but the primary goal of the code review is to analyze a proposed change, to fix it for inclusion, or to reject it. It’s often tempting to recommend further changes to be bundled onto the same review, but it’s important to keep some focus. Are these additional changes tightly related to the original idea, or would they rather be more appropriate on a future branch?

Also keep an eye on large review submissions. It’s quite rare to see changes of a thousand lines or more that are really an indivisible unit. More often, it ends up like that organically, as a result of the workflow followed by the author. These branches may be very frustrating, both for the author and for reviewers. For the reviewer, it’s hard to keep the necessary level of attention and enthusiasm for the problem over expanded periods of time. For the author, it will be equally problematic to run over a large review. In some extreme cases, it may be worth going back and breaking down the change into more change sets.

Overall, fast iterations on small branches are much more rewarding to work with.

Work with inline comments

This is about tooling, but doing anything else should be considered unethical really. If you don’t have a system that allows the change diff to be seen within the rest of the content, and comments to be made inlined right where you see the issue, implement one right now. Moving to such a system was the most dramatic change in productivity I had as a reviewer in the past several years, and makes the whole experience a lot more bearable for everyone.

Enjoy!

Make sure you’re enjoying what you do, and appreciate what your code reviews are achieving. There’s little point in playing the role of an intelligent computer over extended periods of time if you are unhappy about it. Get yourself your preferred slow-drinking beverage (chimarrão?), perhaps some snacks, a comfortable chair, and relax.

Lately I’ve been considering the amount of waste we produce during software development, and how to increase the amount of recycled content. I’m not talking about actual trash, though, but rather about software development artifacts.

Over the years, we’ve learned about and put in practice several means for improving the quality and success rate of projects we create or contribute to. We have practices such as sprints to get people together with high communication bandwidth; we have code reviews for sharing knowledge and improving project quality; we’ve got technical leadership roles to mentor developers and guide the progress of projects; we’ve created kanban boards and burndown charts to help people visualize what they’re going through; and so on.

While all of that seems to have helped tremendously, there’s a sad fact about where we stand: the artifacts of most of these processes are local to their context, and very sensitive to time. That burndown chart is meaningless after it’s burned, and a kanban has no relevant history. Our technical leads indeed guide their teams, but their wisdom stays with the few people that had the chance to interact with them, and subjectively so. That brilliant code review from our best developers has a very limited audience, and rarely carries any meaning just days after it has been accomplished.

That last one is specially interesting. The process of reviewing code is an intense task, very expensive, and that takes a significant portion of the life of an active developer, and even then very little is carried forward as the outcome of that process. We have no effective means or even culture of sharing the generated wisdom to other teams. In fact, we rarely share these details even within the team itself. Why was that line changed like this? Why an interface like that is a bad idea? Who will instruct the new guy next week, and where did we record a bit of the wisdom of the brilliant guy that has left the company recently?

Unfortunately there’s probably no easy solution for this problem. At this point, I mainly recognize that most of the efforts I’ve lead to improve software development for the past several years had a very limited scope. The software itself became immediately better as a result of my efforts, its design became more sensible, and hopefully I contributed a bit to the growth of people around me, but at a company or even community-wide scope, all of these code reviews, sprints, and IRC conversations are buried for very rare revives.

I want to start doing something about this, though. There must be a way to shape these conversations in a more reusable format; in a way that knowledge and agreement can be more proactively preserved and scattered. Perhaps it’s more about how than it is about what. Perhaps we just need to write more posts like this, and cover more topics related to daily development findings. Not sure. I’ll be thinking…

Our son Otávio was born recently. Right in the first few days, we decided to keep tight control on the feeding times for a while, as it is an intense routine pretty unlike anything else, and obviously critical for the health of the baby. I imagined that it wouldn’t be hard to find an Android app that would do that in a reasonable way, and indeed there are quite a few. We went with Baby Care, as it has a polished interface and more features than we’ll ever use. The app also includes some basic statistics, but not enough for our needs. Luckily, though, it is able to export the data as a CSV file, and post-processing that file with the R language is easy, and allows extracting some fun facts about what the routine of a healthy baby can look like in the first month, as shown below.

The first thing to do is to import the raw data from the CSV file. It is a one-liner in R:

> info = read.csv("baby-care.csv", header=TRUE)

Then, this file actually comes with other events that won’t be processed now, so we’ll slice it and grab only the rows and columns of interest:

> feeding <- info[info$Event.type == "Breast",
        c("Event.subType", "Start.Time", "End.Time", "Duration")]

This is how it looks like:

> feeding[100:103,]
    Event.subType       Start.Time         End.Time Duration
129          Left 2013/01/04 13:45 2013/01/04 14:01    00:16
132          Left 2013/01/04 16:21 2013/01/04 16:30    00:09
134         Right 2013/01/04 17:46 2013/01/04 17:54    00:08

Now things get more interesting. Let’s extract that duration column into a more useful vector, and do some basic analysis:

> duration <- as.difftime(as.vector(feeding$Duration), "%H:%M")

> length(duration)
[1] 365

> total = sum(duration)
> units(total) = "hours"
> total
Time difference of 63.71667 hours

> mean(duration)
Time difference of 10.47397 mins
> sd(duration)
[1] 5.937172

A total of 63 hours surprised me, but the mean time of around 10 minutes per feeding is within the recommendation, and the standard deviation looks reasonable. It may be more conveniently pictured as a histogram:

> hist(as.numeric(duration), breaks="FD",
    col="blue", main="", xlab="Minutes")

Another point we were interested on is if both sides are properly balanced:

> sides <- c("  Right", "  Left")
> tapply(duration, feeding$Event.subType, mean)[sides]
   Right     Left 
10.72283 10.22099

Looks good.

All of the analysis so far goes over the whole period, but how has the daily intake changed over time? We’ll need an additional vector to compute this and visualize in a chart:

> day <- format(strptime(feeding$Start.Time, "%Y/%m/%d %H:%M"),
                "%Y/%m/%d")
> perday <- tapply(duration, day, sum)
> mean(perday)
[1] 136.5357
> sd(perday)
[1] 53.72735
> sd(perday[8:length(perday)])
[1] 17.49735

> plot(perday, type="h", col="blue", xlab="Day", ylab="Minutes")

The mean looks good, with about two hours every day. The standard deviation looks high on a first look, but it’s actually not that bad if we take off the first few days. Looking at the graph shows why: the slope on the left-hand side, which is expected as there’s less milk and the baby has more trouble right after birth.

The chart shows a red flag, though: one day seems well below the mean. This is something to be careful about, as babies can get into a loop where they sleep too much and miss being hungry, the lack of feeding causes hypoglycemia, which causes more sleep, and it doesn’t end up well. A rule of thumb is to wake the baby up every two hours in the first few days, and at most every four hours once he stabilizes for the following weeks.

So, this was another point of interest: what are the intervals between feedings?

> start = strptime(feeding$Start.Time, "%Y/%m/%d %H:%M")
> end = strptime(feeding$End.Time, "%Y/%m/%d %H:%M")
> interval <- start[-1]-end[-length(end)]

> hist(as.numeric(interval), breaks="FD", col="blue",
       main="", xlab="Minutes")

Seems great, with most feedings well under two hours. There's a worrying outlier, though, of more than 6 hours. Unsurprisingly, it happened over night:

> feeding$End.Time[interval > 300]
[1] 2013/01/07 00:52

It wasn't a significant issue, but we don't want that happening often while his body isn't yet ready to hold enough energy for a full night of sleep. That's the kind of reason we've been monitoring him, and is important because our bodies are eager to get full nights of sleep, which opens the door for unintended slack. As a reward for that kind of control, we've got the chance to enjoy not only his health, but also an admirable mood.

Love, Dad.

I’m glad to announce experimental support for multi-document transactions in the mgo driver that integrates MongoDB with the Go language. The support is done via a driver extension, so it works with any MongoDB release supported by the driver (>= 1.8).

Features

Here is a quick highlight list to get your brain ticking before the details:

• Supports sharding
• Operations may span multiple collections
• Handles changes, inserts and removes
• Supports pre-conditions
• Self-healing
• No additional locks or leases
• Works with existing data

Let’s see what these actually mean and how the goodness is done.

The problem being addressed

The typical example is a bank transaction: imagine you have two documents representing accounts for different people, and you want to transfer 100 bucks from Aram to Ben. Despite the apparent simplicity in that description, there are a number of edge cases that turn it into a non-trivial change.

Imagine an agent processing the change following these steps:

1. Is Ben’s account valid?
2. Take 100 bucks out of Aram’s account if its balance is above 100
3. Insert 100 bucks into Ben’s account

Note that this description already assumes the availability of some single-document atomic operations as supported by MongoDB. Even then, how many race conditions and crash-related problems can you count? Here are some spoilers that hint at the problem complexity:

• What if Ben cancels his account after (1)?
• What if the agent crashes after (2)?

How it works

Thanks to the availability of single-document atomic operations, it is be possible to craft a sequence of changes that manipulate documents in a way that supports multi-document transactional behavior. This works as long as the clients agree to use the same conventions.

This isn’t exactly news, though, and there’s even documentation describing how one can explore these ideas. The challenge is in crafting a generic mechanism that not only does the basics but goes beyond by supporting inserts and removes, being workload agnostic, behaving correctly on crashes (!), and yet remaining pleasant to use. That’s the territory being explored.

The implemented semantics offers an isolation level that allows non-repeatable reads to occur (a partially committed transaction is visible), but the changes are guaranteed to only be visible in the order specified in the transaction, and once any change is done the transaction is guaranteed to be applied completely without intervening changes in the affected documents (no dirty reads). Among other things, this means one can use any existing mechanism at read time.

When writing documents that are affected by the transaction mechanism, one must necessarily use the API of the new mgo/txn package, which ended up surprisingly thin and straightforward. In other words for emphasis: if you modify fields that are affected by the transaction mechanism both with and without mgo/txn, it will misbehave arbitrarily. Fields that are read or written by mgo/txn must only be changed using mgo/txn.

Using the example described above, the bank account transfer might be done as:

runner := txn.NewRunner(tcollection)
ops := []txn.Op{{
        C:      "accounts", 
        Id:     "aram",
        Assert: M{"balance": M{"$gte": 100}},
        Update: M{"$inc": M{"balance": -100}},
}, {
        C:      "accounts",
        Id:     "ben",
        Assert: M{"valid": true},
        Update: M{"$inc": M{"balance": 100}},
}}
id := bson.NewObjectId() // Optional
err := runner.Run(ops, id, nil)

The assert and update values are usual MongoDB querying and updating documents. The tcollection is a MongoDB collection that is used to atomically insert the transaction details into the database. As long as that document makes it into the database, the transaction is guaranteed to be eventually entirely applied or entirely aborted. The exact moment when this happens is defined by whether there are other transactions in progress and whether a communication problem occurs and when it occurs, as described below.

Concurrency and crash-proofness

Perhaps the most interesting piece of the puzzle when coming up with a nice transaction mechanism is defining what happens when an agent misbehaves, even more in a world where there are multiple distributed transaction runners. If there are locks, someone must unlock when a runner crashes, and must know the difference between running slowly and crashing. If there are leases, the lease boundary becomes an issue. In both cases, the speed of the overall system would become bounded by the speed of the slowest runner.

Instead of falling onto those issues, the implemented mechanism observes the transactions being attempted on the affected documents, orders them in a globally agreed way, and pushes all of their operations concurrently.

To illustrate the behavior, imagine again the described scenario of bank transferences:

In this diagram there are two transactions being attempted, T1 and T2. The first is a transference from Aram to Ben, and the second is a transference from Ben to Carl. If a runner starts executing T2 while T1 is still being applied by a different runner, the first runner will pick T1 up and complete it before starting to work on T2 which is its real goal. This works even if the original runner of T1 died while it was in progress. In reality, there’s little difference between the original runner of T1 and another runner that observes T1 on its way.

There’s a chance that T1′s runner died too soon, though, and it hasn’t had a chance to even start the transaction by tagging Ben’s account document as participating in it. In that case, T2 will be pushed forward by its own runner independently, since there’s nothing on its way. T1 isn’t lost, though, and it may be resumed at any point by calling the runner’s Resume or ResumeAll methods.

The whole logic is implemented without introducing any new globally shared point of coordination. It works if documents are in different collections, different shards, and it works even if the transaction collection itself is sharded across multiple backends for scalability purposes.

The testing approach

While a lot of thinking was put onto the way the mechanism works, this is of course non-trivial and bug-inviting logic. In an attempt to nail down bugs early on, a testing environment was put in place to simulate multiple runners in a conflicting workload. To make matters more realistic, this simulation happens in a harsh scenario with faults and artificial slowdowns being randomly injected into the system. At the end, the result is evaluated to see if the changes performed respected the invariants established.

While hundreds of thousands of transactions have been successfully run in this fashion, the package should still be considered experimental at this point, and its API is still prone to change.

There’s one race

There’s one known race that’s worth mentioning, and it was consciously left there for the moment as a tradeoff. The race shows itself when inserting a new document, at the point in time when the decision has been made that the insert was genuinely good. At this exact moment, if that runner is frozen for long enough that would allow for a different runner to insert the document and remove it again, and then the original runner is unfrozen without any errors or timeouts, it will naturally go on and insert the new document.

There are multiple solutions for this problem, but they present their own disadvantages. One solution would be to manipulate the document instead of removing it, but that would leave the collection with ghost content that has to be cared for, and that’s an unwanted side effect. A second solution would be to use the internal applyOps machinery that MongoDB uses in its sharding implementation, but that would mean that collections affected by transactions couldn’t be sharded, which is another unwanted side effect (please vote for SERVER-1439 so we can use it).

Have fun!

I hope the package serves you well, and if you would like to talk further about it, please join the mgo-users mailing list and drop a message.

Last week Benjamin Otte shared some thoughts about GNOME that were pretty stark. It gathered some steam and hit Slashdot and this all happened the week GUADEC was taking place in A Coruña. I wasn’t at GUADEC but I can imagine there was some fervent discussion about the blog entry.

The gist of Benjamin’s blog was that people are leaving GNOME, that the project is understaffed, and arguably the reason for this is that GNOME has lost its direction and Red Hat have overtaken the project as the primary contributor-base. Of course I am summarizing, but check out the original post if you feel I am not representing Benjamin’s views fairly.

I wanted to share a few thoughts. To be clear: these thoughts are my own, and I am not speaking on behalf of Canonical, but I am speaking from my experiences as someone who has primarily been affiliated with Ubuntu and as a Canonical employee. My feedback is going to be frank but I really do care about GNOME as a project, and this feedback is intended from a position of love for the project and to be open and transparent about my own experiences as just one set of eyeballs in this story.

Fortunately, I think all of these problems are solvable, but for them to be solved GNOME is going to need to do a little soul searching to discover and focus on the right problems and explore and deliver the right solutions.

A Little History

To provide a little context, my interest in GNOME pre-dates my involvement in Ubuntu. I have worked on a few applications that use the GNOME platform (Jokosher, Acire, Lernid, and most recently Ubuntu Accomplishments) and I have had a long interest in where the project is moving forward and as a core part of Ubuntu. I used to go to GUADEC every year, and I consider many folks in the GNOME project to be good friends.

While I care about where the project moves forward I too have also become concerned about the direction it is going in, not in terms of the design and user experience of GNOME (there are other, better versed people to assess this work), but instead in terms of how the project works with others such as companies, developers, and other partners.

In my mind GNOME has become bittersweet. I remember back at GUADEC in Stuttgart in 2005, discussions started happening about what form GNOME 3 would be in. As the years progressed the project struggled to decide on a final vision for what GNOME 3 would look like. This is not surprising: GNOME 2 was such a smashing success that GNOME 3 was going to be difficult second album time. Ideas were shared and bike-shedding occurred, but ultimately it seemed that the project was lacking leadership to take take all of these ideas and flip the switch to a vision and design and move it forward.

Around this time Ubuntu had become arguably the most popular way in which people were consuming GNOME and we (Canonical) were hiring more and more people to perform this integration work (which is no light task, as any distro developer will tell you).

Back then Canonical was taking quite a bit of heat for “never writing code and just shipping other people’s work” (which I always found a misguided viewpoint as integrating and delivering a solid Free Software Operating System is significant work and a great contribution to the wider Free Software commons).

We were starting to find though that there were areas of GNOME 2 that we felt could be improved and expanded (largely based on feedback from our users). We started growing a design competence and hiring developers to build new code to add improvements to the experience. Many technologies were created such as the messaging menu, notify-osd, dbusmenu and the global menu, control center improvements, and ultimately Unity as an additional shell for GNOME.

I remember this time vividly. I was in weekly discussions with Mark Shuttleworth, Rick Spencer (Ubuntu desktop team leader), Ivanka Majic (head of design), and David Barth (head of engineering these components). Our goal was simple: be able to showcase these technologies in Ubuntu and bring value to Ubuntu users, but to also ensure they were contributed to the wider GNOME project as technology that could help the general project in moving forward.

I personally saw this all boil down into pretty simple parts: Canonical and GNOME were partners and it was a mutually beneficial relationship – the GNOME desktop with barely any users defeats it’s purpose and Canonical was helping to deliver it to millions of users in Ubuntu, but Canonical could not build an awesome Ubuntu without the wonderful components in the GNOME desktop to fill in the many different pieces in an OS.

My simple philosophy was also marinaded in the gift culture of Open Source and Free Software: Canonical was paying designers and developers to produce new code that could be of value (and thus offered as a gift to the GNOME commons) and as with all gifts, while it may not be exactly what you want (and may need some adjustments and improvements), I presumed there would be a polite, respectful, and open discourse to take these contributions and bring them into the shared commons that was GNOME, particularly as they were created with GNOME in mind.

This was not my experience of what happened.

Partners

I was really disappointed with what resulted. After years of Canonical and Ubuntu being criticized for not contributing code, when we then engaged in writing code we were met with a frosty, suspicious, and at times, frankly entitled attitude from some parts of the GNOME camp.

Now, don’t get me wrong, Canonical was not perfect here either. I fully admit that some of this relationship could have been handled better (and I am partially to blame here). We made some mistakes early on in which code was released too late and there was sometimes not enough open discussion. Retrospectively, we could definitely have done better in being more pro-active in some parts of the relationship too. At the time we were still learning how to do this, and as such we made some mistakes too.

Canonical wanted to strike the right balance of bringing innovation to Ubuntu releases with new features, but to also openly engage and contribute that innovation to upstreams such as GNOME. My goal here is not to open up a blame game of who did what and when (I will leave that to the commentators ), but what disappointed me most about the whole situation was that from my personal perspective it seemed that some influential members of the GNOME project were treating Canonical’s contributions more critically and suspiciously than others.

Now I haver never subscribed to conspiracy theories, and I don’t believe that there was a shadowy GNOME Illuminati that was meeting together in a hollowed out volcano to plan how to keep Canonical and their contributions out of GNOME, but I was surprised and disappointed at the attitude that came out of parts of the GNOME project to us, when we were ultimately delivering GNOME to millions of users as well as writing new code that could enhance GNOME. It just seemed incredibly entitled.

There were three things that really blew my mind about all of this:

1. From my experience of working on volunteer Open Source projects, new volunteers and their code contributions are tremendously valuable. As an example, if someone comes to my current project (Ubuntu Accomplishments) and is willing to propose new, disruptive ideas, and willing to contribute chunks of code, I will treat those people with open arms. Being challenged is a good thing: it keeps us fresh, and a challenging, innovative idea followed up with running code is awesome. Now, of course, this is not to say that writing code automatically gets the contribution into the core project, but I would treat the entire social engagement with someone offering such a gift with positive open discussion to see how we could find a great solution that makes everyone happy. This seems an area where things could be improved with GNOME.
2. If I was also running a project that was understaffed and struggling to define its direction (which I would argue was the case with GNOME at the time) I would treat such new contributions as wonderful ways of solving problems and building a new direction for the project, particularly if our major distributor was going to be delivering that technology anyway. Code is the currency of Open Source, and rejecting chunks of this currency because they don’t fit an as-yet incomplete jigsaw puzzle of a vision just doesn’t make sense.
3. Without sounding egotistical from the perspective of an Ubuntu guy, I would argue that the vast majority of GNOME consumers were getting GNOME in Ubuntu. Of course, there was and continues to be the wonderful work going into Debian, Fedora, OpenSuSE and others, but it seemed that Ubuntu was the most commonly-used GNOME distribution (I suspect it still is). Again, I saw this as a partnership but from my perspective it seemed like parts of the GNOME project saw Ubuntu as fundamentally subservient to GNOME; as if we had an obligation to deliver whatever the GNOME project saw fit, irrespective of our own ideas and feedback from our users. In my position as an Ubuntu guy, I have always tried to treat our upstreams with maximum respect as they are a big part of who we are; Ubuntu is nothing without awesome apps, and a wonderful integrated experience. I guess I just expected a more positive and collaborative experience with GNOME than I experienced…the kind of collaborative experience that I had known and loved in the earlier days of GNOME.

Of course, it takes two to tango and we at Canonical could have no doubt done better to improve our relationship with GNOME, but I remember back then feeling like no matter what we tried to do, we came up against resistance from the GNOME project, and this was de-motivating and no-doubt added stress to our relationship.

GNOME 3

To shift gears a little, one of the points in Benjamin’s post was that GNOME 3 is a Red Hat project. To me this is a bit of a double-edged sword.

On one hand, the crux of his point is entirely valid: most people contributing to GNOME seem to be a clique of Red Hat folks. What concerns me a little are the concerns in parts of the community that Red Hat is “running the show” and that much of the decision-making has been private to Red Hat staff.

Here’s the thing: I don’t doubt that this is probably happening, but this is not necessarily a bad thing. These concerns again highlight what I think continues to be an unrealistic expectation in parts of the GNOME project with those who are willing to invest in the platform (in this case, Red Hat). If Red Hat have decided to invest in a team of developers to work on and bring value to GNOME, building Free Software that can be shared with everyone, these contributions should be received with open arms. Leadership is leadership, irrespective of the employer.

Of course, there needs to be a culture of openness and transparency, and I suspect a certain amount of internal water-cooler chat is happening in Red Hat, but you will find that with any commercial team that is actively engaged in a Free Software project; we just need to always try to keep things as open as possible. GNOME is definitely going to need to ensure that the openness and values of open collaboration are not compromised, and an open and frank discussion with the Red Hat team about resolving these concerns is no doubt the best step forward.

I personally think it is wonderful that Red Hat are investing so much in GNOME and they have arguably led in much of the direction and leadership in delivering GNOME Shell and the various other parts of the platform. What seems ironic to me is that the same criticisms that were thrown at Canonical with Unity (as a perceived competitor to GNOME Shell, which it was never intended to be) are now being leveled at GNOME Shell (“you don’t care about our needs”, “you are pushing your own agenda” etc).

Maybe a solution to this problem is to be open and frank about the relationship with Red Hat. As an example, we always try to be open about our relationship between Ubuntu and Canonical; there is no doubt that Canonical drives a lot of the development and innovation in Ubuntu, although this leadership and innovation is firmly rooted in expectations around openness and collaboration. We don’t try to hide the influence Canonical has on Ubuntu, and I wonder whether the wider GNOME community feels comfortable in accepting the influence Red Hat has on the project. This is always a delicate balance.

I would agree with Benjamin that GNOME is essentially a Red Hat project these days, but as I say this is double-edged: the wonderful benefits of the investment from Red Hat will be tinged with the challenges of how vendor-neutral the project wants to remain.

The Future

So what is the future of GNOME and how can these problems be solved? Can they even be solved in the first place?

I think so.

I love GNOME as a project, and I love the folks involved in it. While we don’t always agree, the core ethos and goal of GNOME is admirable: to bring an awesome Free Software desktop to everyone. While I personally prefer Unity as a shell, I think the work that has gone into GNOME Shell has been a wonderful rebirth of the motivation and focus of GNOME. The architects of this vision should be credited in getting GNOME out of the slump I mentioned earlier that seemed to stem from 2005. Of course, I will always be disappointed that GNOME seemed quite so resistant to much of the contributions we wished to make, and I think we could have helped to have moved things along a little faster, but I am delighted that GNOME 3 has got to the point it has got to.

As I mentioned earlier, my feedback here really has nothing to do with the design and technical direction of GNOME, and others can provide more insightful commentary than me. I do though think this people-problem issue of GNOME being a rather difficult project to work and interface with at times is a problem that has not yet been confronted and resolved. While this problem continues to exist, I worry that it will eat away at GNOME more and more.

GNOME is blessed with some wonderful leaders, and I hope that the content in this post can act as some food for thought: I am not expecting everyone to agree with me, but if this opens up a discussion about these topics I will be happy.

What is not a solution is for us to give up on GNOME. I know some folks are moving on from the project and moving onto other things, and we have more competition than ever for desktops, but I still see GNOME as an important foundational component of the Free Software and Open Source desktop today.

Now, I am sure this blog entry is going to result in some folks screaming from the rafters that I am misrepresenting GNOME and it is all Canonical’s fault, and you are entitled to your view. Traditionally I have not wanted to raise these concerns publicly as I didn’t want to cause any further harm in the relationship between Ubuntu and GNOME, but Benjamin’s blog post seemed to offer a good opportunity to throw out some feedback that might be helpful in constructing a solution.

While I don’t have much time to contribute to GNOME formally these days, I am more than happy to talk more, provide any further feedback, and help where else I can. I would love to see the GNOME project that we know and love be back in a healthier state. Thanks for reading.