Bazaar is a great tool for distributed development, but distros are built on packages, and so packages are what distro developer care about.  That’s why many of you who have followed my previous blogs have probably been asked for patches to the package itself, not to the bzr branch.

Why the difference?  Well for package maintainers, it’s easier and faster to import  upstream changes if they keep their source code clean.  To do that, any changes made by the distro are applied on top of the unmodified upstream code in the form of patches.  There are many tools designed specifically to make this easy for the package maintainers.

Below I’m going to show you how to turn your code change into a package patch that is easy for Ubuntu developers to add to the distro’s packages.  Only do this if your submitted branch is to a package in main and it hasn’t already been merged.

0) Check your source package format

The following instructions will only work on source packages using quilt 3.0 for managing patches.  Before you do anything else, check that the file debian/source/format contains the following:

3.0 (quilt)

1) Find your revisions

Starting from your existing code branch, we first need to identify which revisions in your branch we need to turn into a patch.  To do that, we simply check for revisions in your branch that don’t exist in the main one.  Here is what I used for geany:

bzr missing --mine-only ubuntu:geany

You just need to replace ‘geany’ with your application’s branch name (the same you bzr branched in my earlier articles).  The –mine-only will limit the result to only revisions in your branch just to keep things simple.  You’ll want to make note of the first and last revisions in this output.  If, like me, you only had one revision missing, that makes it even easier.

2) Generate the patch

Fortunately the package “bzr-builddeb” provides a command that makes this step easy.

mkdir -p debian/patches
bzr dep3-patch -d ubuntu:geany . > debian/patches/add_keywords.patch

Again, just replace ‘geany’ with your application’s branch name, and dep3-patch will find the differences in your branch and convert them into a patch file.

Now that you have a patch file, we need to add it to the list of patches for this package.  To do that, all you need is to add it’s name to the end of the debian/patches/series file like this:

echo add_keywords.patch >> debian/patches/series

3) Convert your source changes

Now that your changes are in a patch file, we need remove those changes from the source code itself.  This is where those revision numbers from step 1 come in, you will need the highest revision number and one less than the lowest.  Since I only had one revision, rev 32, my numbers are 32 and 31.

bzr diff -r 32..31 | bzr patch

This causes bzr to generate a reverse-diff of your changes (by going from the higher to the lower revision), and then apply that reverse-diff to your current code, effectively undoing your changes.

Now you need to apply your new patch file using quilt, so that quilt knows about it:

quilt push -a

Which should give you the following output if everything applies cleanly (if not, then your package is going to need some extra work, and you should ask for help from someone in #ubuntu-devel on freenode IRC).

Applying patch add_keywords.patch
patching file geany.desktop.in

Now at patch add_keywords.patch

4) Log your changes

Since you are making changes to the package itself now, you need to add that information to the debian/changelog:

export DEBFULLNAME="Michael Hall"
export DEBEMAIL="mhall119@ubuntu.com"
dch -i

You will, of course, want to replace my name and email with your own (Hint: you can put those 2 export lines into ~/.bashrc for future packaging work). This will create a new entry in the chanelog for you, with one higher version number.  All you need to do it add in the comments:

* Add search keywords to .desktop file (LP: #942154)

Be sure to use the proper bug number for your changes.  Also, if you are not running on Precise, you  will need to change the release target at the top of the file to ‘precise’.  Here’s what my new record looks like:

geany (0.21.dfsg-1ubuntu4) precise; urgency=low

* Add search keywords to .desktop file (LP: #942154)

-- Michael Hall <mhall119@ubuntu.com> Wed, 07 Mar 2012 14:40:32 -0500

5) Commit and push

Now it’s time to put everything back into your bzr branch.  First you need to add your patch file:

bzr add debian/patches/add_keywords.patch
bzr add debian/patches/series
bzr add .pc/

If your package branch didn’t already have a ‘series’ file, my instructions in step 2 will have created one, so I’m adding it here just in case.  If it already existed, bzr add won’t do anything.

Next, commit and push your changes back to your submitted branch:

bzr commit -m "Convert source changes into a package patch file"
bzr push lp:~mhall119/ubuntu/precise/geany/add_keywords

There has been a fantastic amount of feedback and contributions being made by folks who have been following my recent posts, and we’ve already had some of those contributions landing in Ubuntu 12.04.  But we’re not done yet!

Ubuntu means “I am who I am, because of who we all are”.  In a very literal sense, Ubuntu is what it is, because of what other distros are.  We all bring together the best open source software from upstream developers, and we should all be giving our improvements back to those developers.  Unity may be the default desktop on Ubuntu, but it’s also been ported to ArchLinux and SuSe, and is making it way to Fedora too.  That means that your Quicklist contribution can help more than just Ubuntu, all you need to do is submit it upstream.

Step 1: Find your Upstream

While all of Ubuntu’s package sources use Launchpad, and are therefore all in the same place, your upstream developers will all of their own separate spaces.  Some are rather easy to find, for example Firefox is developed over on mozilla.org.  Others may use GitHub, SourceForge, or their own sites.  So you first task if to find out where the code is developed.

Step 2: Talk to your Upstream

Once you’ve found their development home, look for ways of contacting the developers.  Usually there will me an email/mailing list available, an IRC channel or a forum.  Whatever method they prefer, us it to ask the developers how they want you to submit your patch.  Most developers love being contacted by somebody who already has a patch written.

Step 3: Submit your changes

Depending on your upstream’s development process, they may want a patch file with your changes, or they might ask you to use their version control system of choice (svn, git, etc) to submit your change in a way that is easy for them to merge.  Some, especially Debian, may request a patch for the package itself.  This is why Step 2 is so important, you need to know how to submit it in order for it to be accepted.  Don’t worry if you don’t know how do what they want, ask them to help you learn it.  Again, developers like people who come to them with patches.

For Geany, I first found their website, geany.org, and from there a link to their Feature Tracker on SourceForge.  I also found their IRC channel, #geany on freenode, and was in contact with their developers there.  In the end, I was asked to create a fork of their code on GitHub, and submit my change as a Pull Request.

Step 4: Updated your Merge Proposal

Once you’ve submitted you changes upstream, leave a comment to that effect in your merge proposal.  Ubuntu developers don’t like to keep custom changes in Ubuntu, and your merge proposal is much more likely to be accepted if they know the change will eventually be made in the upstream project as well.

Step 5: Stay involved with your Upstream

If you were working on a program you like, don’t let this be the end of your contribution!  You now have experience making and submitting changes (to both Ubuntu and Upstream), and you are forming a relationship with the upstream developers.  Keep up with both of those, as they will be invaluable assets to you.

The Inkscape folks are looking for someone to help finish off some editing for their website introduction video.

Hop on over if you want to lend a hand!

(via Ted Gould) 

The Yorba folks have released a new version of Shotwell. You can snag it here.

New features include:

• Video support for Ogg, AVI, MP4, Quicktime, and WMV files
• Upload videos to major Web services, including YouTube, Facebook, Flickr, and PicasaWeb
• Runtime monitoring of library directory
• Background writing of metadata to master files
• Flagging of photos for batch operations
• Set multiple photos to desktop background slideshow

Congrats to the Yorba team!

In case you’ve missed out, smspillaz has started working fulltime at Canonical on working on Compiz. 

Sam’s been shuffling getting things fixed in the code and needs a bit of help, so I asked him to put together a list of things Compiz could use help with.

• Finish writing the compiz code doxygen
• Provide a support for sound events using libcanberra in gnomecompat
• Package compiz-plugins-extra and compiz-plugins-unsupported
• Figure out why evince is placing it’s window wrongly
• Figure out what is up with gtk-window-decorator on virtualbox
• Figure out what is up with resizing xterm in one direction

Here’s the full list with details, if more issues come up we’ll add them there.

Also a special thanks to the moderators of the Ubuntu Forums who volunteered to help the Compiz forums clean out some spam and tidy things up a bit. High five!

Thanks to PiTiVi I was able to take the webm files from GUADEC and put together a video of Gabriel’s Banshee talk.

Those of you wondering how to make Banshee extensions (Stuart!) will want to pay attention around the 5 minute mark, where he goes into an example on how easy it is to make an extension. 

So, go forth and make great extensions!

Did you know that stackoverflow.com offers free advertising to open source projects?

Every project can always use some more new blood. And if you don’t think that’s a good enough reason, as of this writing here is a list of 344,881 other reasons.

Ok I promise not to blog about StackExchange anymore, until tomorrow anyway.

Allison Randall has blogged about her new role in Ubuntu; working at Canonical as the Technical Architect for Ubuntu. One thing which I think is awesome is how she mentions people who encouraged her:

To give credit where credit is due, there have been 4 great influences on my career over the years, mentors, friends, people who believed in me, encouraged me to dream big dreams and try big things, who taught me that I’m better, smarter, wiser, more dynamic, and resilient than I ever imagined. In alphabetical order: Damian Conway, Greg Kroah-Hartman, Mark Shuttleworth, and Nathan Torkington. Thanks guys, I wouldn’t be here without you!

Everyone has mentors in Free Software, so Allison’s post inspired me to talk about mine. People who made you who you are. I hope this encourages you to talk about the people who inspired you.

While Allison’s post motivated me, this next month I get to celebrate 3 years at Canonical and I would like to thank people who have motivated me to do what I do. Think of them as the root of me:

• Dave Camp - Dave is one of the older Ximian folk, a former co-maintainer of Nautilus, and later he worked on Mozilla. He taught me to ignore the haters, and rock on by your bad self. He’s a hacker’s hacker, a dude who puts his head down and skates. He doesn’t care about fame, fortune, or any of that jazz, he’s just a dude. He is also an amazing guitar player.
• Luis Villa - Not much to say here, he’s brought so many of us here that I have no words that could ever be kind enough to explain what Luis has done for so many of us.
• Jeff Waugh and Benjamin “Mako” Hill - I am going to mush them together, since at the time they were the collective first “Ubuntu Community managers”. Jeff for being the spark plug of motivation and getting me to a UDS, and Mako for being the Free Software advocate who leads by example, not by shoving the GPL down people’s throat. I would love to hear much more from both of you. Both of you believed in me from the beginning, and I will always be grateful.
• Asa Dotzler - many years ago when I first started wondering what Mozilla was he took the time to explain open source to me. I also love that he flames Linux for what it is, maybe someday we’ll get our act together. :) I’m not even sure if he’s looking at Linux these days but I’ll always appreciate his first interaction with me.

These people pointed me in the right direction, and these are the ones who focused me into a fine instrument of Ubuntu laser-destruction. If you hate me, then it’s probably these people’s fault:

• Luke Kanies from Puppet Labs (at the time from Reductive Labs), who convinced me to think about applying for my current job at Canonical.
• Oliver Grawert, who made me actually do that or he promised to punch me in the face.
• Daniel Holbach. The name mentioned by Jono Bacon when he said “I am starting a new team” that made me apply for the job on the spot and totally not care what the consequences are. He has that effect on people.
• … and of course the rest of you on the community team (and I just don’t mean Canonical folk) who have been supportive of me over the years, even when I quit the team and had a temper tantrum a few years ago.)

And on top of that I’ve got the old folks, the mentors who are just awesome by being there, they’re all good friends and good mentors. I am pretty sure that everyone who has worked with these people are not surprised:

• Chris Blizzard - old school GNOME, now at Mozilla. He builds airplanes ffs.
• Vincent Untz - old school GNOME, now at Novell. He builds ice cream ffs.
• Miguel de Icaza - old school GNOME, now at Novell. He builds stack exchanges ffs.
• Ryan Lortie - old school GNOME, now at Codethink. He builds dconf ffs.

I’d also like to take a moment to thank the new people — those of you who are new here and want to rock. Spend some time researching my generation’s heroes, and learn from what they have to say.

Hah, I just thought “Some day this entire mess will all be yours”, but I don’t know if that’s a nice thing to say or not, so heh to you.

A couple of weeks ago we launched the Desktop Testing Program. You can read more about it in the original announcement but, basically, we have some infrastructure to track test results for desktop applications, a wiki that stores the testcases description and a large community willing to test every Ubuntu milestone.

The Alpha 3 testing cycle went very well, but we still need more testcases to make the Beta testing cycle event better.

Mathieu Trudel-Lapierre, one of the Network Manager upstream developers, stepped ahead and wrote some testcases for Network Manager. He, as an upstream, wanted Network Manager to be part of the testing program, to have the opportunity to get test results every Ubuntu milestone. His tests will be part of the Desktop Testing Program starting on Maverick Beta.

If you are an upstream (or would like to collaborate somehow with your favourite upstream project), you can review the available tests in our testcases wiki, and, if the application is already there, make sure that the tests still apply and write more to cover new features. If your application is not there, just create a new page and start adding new testcases. In both cases you can follow our syntax guidelines.

I think this is a great opportunity for upstreams to have their project tested on a regular basis by a great community, with results they can browse, in a repeatable way. I just hope more upstreams could know about it. If only this blog was syndicated in Planet Gnome…

The last few months we’ve been doing a lot of work to enable direct import of translations from different upstream VCS systems. For now, we’ve focused on getting one very important case right first (GNOME), and then we’ll extend it to supporting other upstreams as well.

How are we going to do it? First off, we’ve split it all into two separate stages:

• get upstream translations into Launchpad
• push upstream translations from Launchpad into Ubuntu

For some upstreams, getting them into Launchpad is trivial: they might already be hosted in Launchpad. For majority of them, however, it means pulling from different VCS systems. Thanks to Launchpad Code and Bazaar teams, getting the code in the form of bazaar branch is not that big a deal. However, when pulling translations from a VCS instead of getting them from tarballs means one slight complication. Translation templates (POT files) won’t be there, and we’ll have to regenerate them.

Regenerating templates differs from project to project. And doing it should be considered an unsafe operation. So, in the first step we are only going to support intltool-based modules, and generation of templates will happen inside a sandboxed environment. This will enable us to import upstream translations directly into read-only Launchpad projects: this is marked with green-coloured arrows on the diagram.

After that is done, we’ll start pushing all these translations directly into Ubuntu (blue-coloured arrows), minimizing the time it takes for translations to get from upstream translators to Ubuntu users.

I’ve written a more thorough explanation in my personal blog, so check it out.

Parts of this will be rolled out this cycle, but more will come in the coming months.

I finally listened to Sebastien Bacher and applied for GNOME commit rights yesterday, after hassling Seb once more about committing an approved patch for me. Surprisingly, it only took some 4 hours until my application was approved and my account created, wow! Apparently 71 patches are enough.

With my new powers, I fixed a crash in gdm, and applied two stragglers into gvfs’ build system today.

More to come!

When you present yourself on the web, you have 15 seconds to make an impression, so aspiring champions of the web 2.0 industry have converged on a good recipe for success:

1. Make your site visually appealing,
2. Do something different and do it very, very well,
3. Call users to action and give them an immediate, rewarding experience.

We need the same urgency, immediacy and elegance as part of the free software desktop experience, and that’s is an area where Canonical will, I hope, make a significant contribution. We are hiring designers, user experience champions and interaction design visionaries and challenging them to lead not only Canonical’s distinctive projects but also to participate in GNOME, KDE and other upstream efforts to improve FLOSS usability.

Fortunately, we won’t be working in a vacuum. This is an idea that is already being widely explored. It’s great to see that communities like GNOME and KDE have embraced user experience as a powerful driver of evolution in their platforms. Partly because of the web-2.0 phenomenon and the iPhone, there’s a widely held desire to see FLOSS leap forward in usability and design. We want to participate and help drive that forward.

There’s also recognition for the scale of the challenge that faces us. When I laid out the goal of “delivering a user experience that can compete with Apple in two years” at OSCON, I had many questions afterwards about how on earth we could achieve that. “Everyone scratches their own itch, how can you possibly make the UI consistent?” was a common theme. And it’s true – the free software desktop is often patchy and inconsistent. But I see the lack of consistency as both a weakness (GNOME, OpenOffice and Firefox all have different UI toolkits, and it’s very difficult to make them seamless) and as a strength – people are free to innovate, and the results are world-leading. Our challenge is to get the best of both of those worlds.

I don’t have answers to all of those questions. I do, however, have a deep belief in the power of the free software process to solve seemingly intractable problems, especially in the long tail. If we articulate a comprehensive design ethic, a next-generation HIG, we can harness the wisdom of crowds to find corner cases and inconsistencies across a much broader portfolio of applications than one person or company could do alone. That’s why it’s so important to me that Canonical’s design and user experience team also participate in upstream projects across the board.

In Ubuntu we have in general considered upstream to be “our ROCK”, by which we mean that we want upstream to be happy with the way we express their ideas and their work. More than happy – we want upstream to be delighted! We focus most of our effort on integration. Our competitors turn that into “Canonical doesn’t contribute” but it’s more accurate to say we measure our contribution in the effectiveness with which we get the latest stable work of upstream, with security maintenance, to the widest possible audience for testing and love. To my mind, that’s a huge contribution.

Increasingly, though, Canonical is in a position to drive real change in the software that is part of Ubuntu. If we just showed up with pictures and prototypes and asked people to shape their projects differently, I can’t imagine that being well received! So we are also hiring a team who will work on X, OpenGL, Gtk, Qt, GNOME and KDE, with a view to doing some of the heavy lifting required to turn those desktop experience ideas into reality. Those teams will publish their Bzr branches in Launchpad and of course submit their work upstream, and participate in upstream sprints and events. Some of the folks we have hired into those positions are familiar contributors in the FLOSS world, others will be developers with relevant technical expertise from other industries.

One strong meme we want to preserve is the idea that Ubuntu, the platform team, is still primarily focused on integration and distribution. We will keep that team and the upstream work distinct to minimise the conflict of interest inherent in choosing the patches and the changes and the applications that actually ship each six months as part of an Ubuntu release.

Of course, there’s a risk to participation, because you can’t easily participate without expressing opinions, visions, desires, goals, and those can clash with other participants. It’s hard to drive change, even when people agree that change is needed. I hope we can find ways to explore and experiment with new ideas without blocking on consensus across diverse and distributed teams. We have to play to our strengths, which include the ability to diverge for experimental purposes to see what really works before we commit everyone to a course of action. It will be a challenge, but I think it’s achievable.

All of this has me tapdancing to work in the mornings, because we’re sketching out really interesting ideas for user interaction in Launchpad and in the desktop. The team has come together very nicely, and I’m thoroughly enjoying the processes, brainstorming and prototyping. I can’t wait to see those ideas landing in production!