Canonical Voices

What Something driven development talks about

Michael

For quite a while I’ve been wanting to get stuck into learning the Go programming language, after a colleague at Canonical (Gustavo Niemeyer) wrote enthusiastically about it… and today I took a bit of time to get started.

Getting setup on my Ubuntu Maverick VM was pretty straight forward:

  • Add Gustavo’s software channel:
sudo add-apt-repository ppa:niemeyer/ppa
sudo apt-get update
sudo apt-get install golang
export GOROOT=/usr/lib/go
  • This also adds ‘gorun‘, which effectively saves you having to compile and link each time you edit your go source file – nice
  • I had to then grab the go source to get the vim syntax highlighting etc. (files are provided for most editors in the source).
Whenever learning a new language I always find it incredibly helpful to be able to see all the options available – code auto-completion. So finally, I installed gocode, but had to make the following changes to get it to `make install`:
  • Installed golang-weekly (rather than golang) to get a more recent version…
  • Reverted the latest commit to gocode which was an update for the latest golang weekly release, which hadn’t yet hit the PPA.
With those changes, gocode installed and worked beautifully – learning with auto-completion is like seeing with peripheral vision for the first time.
Next week I hope to checkout the gocheck testing library as well as the Effective Go tutorial. As much as I love Python, it’s great getting stuck into a new language!

Filed under: golang

Read more
Michael

I really enjoyed the emphasis of Ben’s Code-reviews as relationship builders post over the weekend. The focus on healthy learning relationships both for new and old is great – something I’ve tried to focus on myself over the past few years working in distributed teams.

On a related note, I’ve wondered of late whether I could use “Agreed” rather than “Approved” when reviewing a branch. Agreed (IMO) implies one type of relationship – peer discussion resulting in agreement – rather than an approval from an authority (perhaps without interaction). It is a small and maybe insignificant distinction…


Filed under: Uncategorized

Read more
Michael

One of the things I love is reviewing smaller targeted branches – 400-500 lines focused on one or two particular points. But you don’t want to be blocked on the review of one branch to start the next dependent branch. Here’s a quick screencast (with text version below) that I made for some colleagues showing how Bzr’s pipeline plugin can help your workflow in this situation:

Text version

Assuming you’ve already got a lightweight checkout of your project code, create a new branch for part-1 of your feature and switch to it:

$ bzr branch trunk part-1
$ cd current_work;bzr switch ../part-1

Make your changes, commit and push to Launchpad:

$ bzr commit -m "Finished part 1 of feature-x that does x, y and z";bzr push

While that branch is being reviewed, you can start on the next part of the feature:

$ bzr add-pipe part-2
Created and switched to pipe "part-2"
$ bzr pipes
    part-1
*   part-2

Now make your further changes for the feature and commit/push those as above (Launchpad will create a new branch for part-2). Now let’s say that the review for part-1 comes back, requiring a small change, so we switch back to part-1, make the change, commit and pump it (so that our part-2 branch will include the change also):

$ bzr switch part-1
$ bzr pipes
*   part-1
    part-2
(make changes)
$ bzr commit -m "Changes from review";bzr pump

Now we can see the changes have been merged into the part-2 branch:

$ bzr switch part-2
$ bzr log --limit 1 --line
102: Michael Nelson 2011-01-13 [merge] Merged part-1 into part-2.

So, let’s say we’re now ready to submit part-2 for review. The handy lp-submit bzr plugin will automatically know to set part-1 as a prerequisite branch, so that the diff shown on the part-2 merge proposal includes only the changes from part-2 (as well as making sure that your branches are pushed up to LP):

$ bzr lp-submit
Pushing to lp:~michael.nelson/rnr-server/part-1
Pushing to lp:~michael.nelson/rnr-server/part-2

Nice! Now, what if there have been some changes to trunk in the mean-time that you want to include in your branch? Easy, just:

$ bzr pump --from-submit

and pipelines will automatically merge trunk into part-1, commit, then merge part-1 into part-2.

More info with `bzr help pipeline` (once it’s installed), or more information (including how to get it) on the Bazaar Pipeline wiki page.


Filed under: bzr, python

Read more
Michael

My direct team-mates at Canonical live everywhere from the US west-coast through to Poland, so we don’t have the option of doing a ‘traditional’ stand-up each day.

In the past we’ve done IRC standups – each pasting our notes at one point in the day when we’re all online – which helps a few points (recording what we’re up to) but has quite a few drawbacks in my opinion (see Vincent’s thoughts on irc standups also). A lot of us on the team admitted that we didn’t always give the irc standups our full-attention.

Right now we’re trying an experiment which is kind-of neat. We’re doing voice standups in pairs/triplets with people in similar timezones, but adding our standup notes to a wiki page before the call. The advantages are that the calls are easy – I can read through Lukasz’s notes and we just discuss any issues or successes as well as the plan for today, and vice-versa. Also, as we all subscribe to the wiki page for email notifications, we actually read everyone else’s notes from different time-zones asynchronously (and all stakeholders can take part in calls or read the notes as desired).

Personally, having a brief voice call as part of my day is a huge benefit – just touching base with another voice from work, and having a bit of a chat after going through our stand-up gives a more human touch to working from home. It’ll be interesting to see whether others on the team find it more or less helpful.


Filed under: Uncategorized

Read more
Michael

Moving on from Launchpad

I’ve been working at Canonical for nearly two years now, in a role that I think of as a support role for all the awesome Debian, Ubuntu and application developers out there who use Launchpad to publish their software in Ubuntu. Most of my time has been spent making it easier to get software to users of Ubuntu (technically, working on Launchpad’s archive features such as Personal Package Archives), but I’ve also enjoyed improving processes for designing new features collaboratively in an open environment.

It’s been a great two years and I’ve learned an incredible amount while working with the Launchpad community. And although I’m not leaving the community, I will be spending the majority of my time developing tools on a different team in Canonical. I’m excited to be spending more time doing Django development, and am hoping that the skills I’ll be building there will give me opportunities to work on a bunch of open-source education-related tools in my own time to support life-based open-learning – something I’m quite passionate about…

The work that we all do in the open-source world impacts thought and philosophy far beyond the circles we work in. Principles from open source software have already been changing the way people think about copyright in education (the free high-school science texts project is an incredible example of that), but I believe they will also change the way we facilitate the learning of the next generations (from my edu-focus blog). Educators in schools and universities are already adapting to new mediums and principles but – in the same way that early film commercials re-implemented the stage on film - educators are often just re-implementing the classrooms of the past online (with “Learning management systems” that deliver content and keep the teacher/institution managing the learning). Schools and other education institutions need either a revolution or speedy evolution, and I’m excited to be a part of that both indirectly through my work and in my own time.


Filed under: Uncategorized

Read more
Michael

Thanks to the excellent work from the Launchpad Code team and others (James and William, to name just two), here’s a preview of how easy it will soon be to build a branch into published binaries in your PPA:

http://blip.tv/file/3738068

(I’ll update with the inline flash version once it’s been converted).


Filed under: launchpad

Read more
Michael

Having just been through the process of trying to involve people in open source design and development, Maris Fogels (another Launchpad engineer) and I took some time to document/reflect on the process. The strongest point that came out of the Q&A for me is the importance of voice conversations for iterating a design like this.

In this Q&A Michael shares his experience helping design the ‘Build Branch to Archive’ user interface:

Ubuntu is beginning to manage its source packages as Bazaar branches, as part of the DistributedDevelopment project.

One incredibly important activity for source packages is building them into binary packages that can be downloaded and installed. This spec explains how Launchpad can make this a wonderful experience.

Mars: Have you done any work designing user interfaces before this?

Michael: A little – we went through a similar process for the PPA redesign as
part of Launchpad 3.0. Prior to that, no, only your typical “Oh this
needs a UI – let’s just start creating the UI that *we* think the
target users will want.”

We had tried (in the Launchpad Soyuz team) doing something similar to
this design process previously (particularly, defining stories for an
interaction before coding, and getting feedback from the people
representing the target users), but no matter how hard we tried, we
weren’t able to get the time with them or any feedback, and so we
ended up implementing the design and landing it without their
feedback, which was a mistake – and a lesson learned which influenced
the process I used here.

Mars: What tools and resources did you use as you went through the project?

Michael: Balsamiq, wikis, email discussions and phone calls. Balsamiq was great
as a way to create UI’s quickly, modify them easily as we learned
more, share them for others to modify (I just pushed a bzr branch that
I kept up to date).

Mars: Was there a particular process that you followed?

Michael: Basically I tried to:

  1. Identify and describe the use-cases,
  2. Create mockups for each use-case,
  3. Get feedback from as many people as possible,

and then constantly update or add new use-cases and mockups as more
info came in. Looking back, it seems we went through three definite
iterations.

Mars: How long did it take to design the page?

Michael: We just designed the main interactions to build a branch into a PPA,
which is 2 pages/overlays (they’ll be overlays if js is enabled). I
think the process (3 iterations above of gathering feedback,
phone-calls etc.) spanned 3 weeks, on and off.

Mars: Do you think the problem was well-understood before you started working on the user interface?

Michael: I think different people understood their own domains of the
build-from-branch interaction, and everyone had ideas about how the
other parts should work, but no, I don’t think any of us understood
the use-cases properly – particularly the issues around re-using
recipes in the UI – until all the ideas and feeback was collected and
put together.

Mars: Did the design of the interface change much as work progressed?

Old vs. New

Michael: Yes, quite a bit. Initially I had initially (mis)understood that we
needed to work with the current implementation of bzr-builder, but
after the first iteration, James Westby pointed out that bzr-builder
can be updated to support the requirements of the UI. This freed us to
consider sharing recipes in the UI in a way that would not have been
possible otherwise.

Collecting and expanding the use-cases naturally required updates to
the design. Also feedback and questions from various people helped to
keep refocusing on what people want to be able to do (for example,
“This recipe supports building to Ubuntu Lucid, Ubuntu 9.10 and Ubuntu
9.04″, while still allowing the person using the recipe to select just
the distroseries they want).

Mars: How did you get feedback on your designs? Was there a good response?

Michael: (Just an aside, there’s no ‘y’ there – they are really our designs, I
was just filtering and focusing the collected thoughts and ideas into
something visible).

So as above, email to the dev list, then irc chats, but the biggest
throughput of communication and feedback came via actual calls. It’s
just so much easier to clear misunderstandings on the spot if you walk
through the process together. I had multiple calls with 6 different
people, all which were invaluable.

Mars: Looking back, what was the most difficult part of the process?

Michael: (1) Getting real user input and feedback and (2) getting feedback from
professional designers.

With (1) the best I could do with the time I had was to communicate
with representatives of target users, which I think worked quite well.
And (2) was difficult mostly due to the nature of the problem for
build-from-branch. IMO it requires someone to sit physically in the
office and walk through the process with a designer, something I
couldn’t do, and hasn’t been done yet as far as I know.

Mars: What did you most enjoy about the work?

Michael: Bringing lots of ideas together from lots of people, clarifying the
different needs to different people to build a common understanding of
the issues involved.

But, to be honest, I am also enjoying getting back to soyuz coding. I
enjoy doing this kind of thing for a week or two, but after that I’m
always keen to get back to doing some coding.

I think I (and James?) also enjoyed the fact that we could work on
defining the interaction *before* anything had been coded. Too often
we get in there and just start coding something without thinking
through the use-cases properly (or even defining them), and it ends up
causing a lot of pain. Like everything, it’s a learning process.

Mars: Any final words for developers looking to try this themselves?

Michael: Sure, if you’d like a change of rhythm and enjoy facilitating
conversations and ideas, give it a go – it’s a great chance to
interact with other people in different areas and build your
understanding of what people actually want to do with your product.


Filed under: design, launchpad

Read more
Michael

Every time that I upgrade my operating system, I feel like I’m receiving a whole bunch of gifts from people I don’t know – which actually is what’s happening! As I learn more about the Debian and Ubuntu communities, and Launchpad (which I’ve been working on for the past year), I never stop feeling awed at the amount of effort that is being filtered through Launchpad (in both directions) in terms of bug-fixes, new packages, translations etc., into end-products that rock!

Having upgraded my netbook a few weeks ago, over the weekend I decided to upgrade my work machine to test the latest development release, Ubuntu Lucid, and the only difficult part of the process was having to install Windows XP to update my bios (because Samsung, the manufacturer of my work laptop, only provide bios updates as win-32 applications). Other than that, it’s been a joy seeing all the improvements (like the open-source nvidia support – a few clicks and my dual monitors worked perfectly, something I could only do previously with the proprietary drivers).

So, whether it is heard or not, I want to say a big thanks to the millions involved for the gift – my Lucid experience has been wonderful.


Read more
Michael

A few weeks ago at our Launchpad BuildFromBranch sprint, I gave a brief overview of the process I use when doing a significant design change – whether it’s a code architecture or UI design (hint: don’t try to do a presentation after flying for 26 hours with two kids).

The main point that I was keen to communicate is how important it is to involve people and get early feedback. Launchpad bugs provide a great way to invite this involvement too, as you’ve already got many of the interested people already subscribed, and if you add information early other people – such as the person who created the bug – have an opportunity to clarify and contribute to the design or solution.

As a result of our sprint, we’re now focusing on how we can provide the best user experience for automating the process of building code into installable packages that will be updated on peoples machines the next day. In particular, we want to enable easy setup of daily builds for projects, allowing people who like to live on the edge and contribute QA time to their favourite open-source projects (building on the excellent work of bzr-builder).

To encourage involvement in this process, I’m trying a combination of relevant email lists for discussion, phone calls where helpful, and a wiki page that I try to keep up-to-date with the current status of those discussions. The aim being that anyone can read the wiki page and, in addition to seeing the current UI mockups, can see all the significant questions that have been brought up already. We’ll see how it goes…


Read more
Michael

When hacking on Launchpad’s soyuz application – and creating tests to verify that your new functionality works, you’ll often need sources or binaries published in very specific scenarios.

This demo of using Launchpad’s SoyuzTestPublisher for testing shows you how you can setup those scenarios using the Soyuz Test Publisher which – as the name suggests – was created for this exact reason (by Celso).

To save you typing (or squinting if you can’t view the hi-res version of the demo), the python script for the demo is available also.


Read more
Michael

After reading lots of reviews, I went out and bought an EeePC 1008HA (‘Seashell’) netbook last weekend and installed the alpha Ubuntu 9.10 Netbook Remix… and everything just worked. Very impressed.

I haven’t decided whether I’ll stick with the Netbook Remix launcher interface, or just go for a cut-down desktop with Gnome-do launcher, but eitherway it’s lovely to use.


Read more
Michael

During the Ubuntu Developer Summit there were a number of discussions around how debian operating systems, such as Ubuntu, can best enable people to add trusted software archives to their computer (see the blueprint AptURL-policy for Ubuntu Karmic).

As far as I understood, there are three competing issues:

  1. There is currently no easy way for a person to evaluate whether a software archive might be trustworthy – other than trusting a centralized white-list,
  2. Installing software directly from a web-browser click should be minimized and discouraged,
  3. There should be an easy way for people to share (and maintain) software that they develop

I’m only going to discuss the first point here. DoctorMo has some great mockups demonstrating gpg-keys being used for identity and software trustworthiness, but a few people have commented on the problem of confusing the “web-of-(identity)-trust” that gpg keys currently provide with some sort of general trust that the identified person will be responsible with my computer.

Reading all the comments there leaves me wondering whether we should stop using the phrase “web-of-trust” and instead use the more verbose “web-of-identity-trust” to avoid confusion – even though it doesn’t roll off the tounge so easily. I love Martin’s idea – the concern is real – but am wondering how it could be implemented in a decentralised way.

Of course, it would be possible to add this as an api feature of Launchpad – my feeling is that Launchpad should be a source of the social value of an archive (ie. user ratings etc.), but not necessarily a source of this fundamental trust-relationship.

So here is an idea: what if the repository contained this infrastructure itself, such as a directory of meta-information, ‘identities-who-trust-this-archive’, which contained signed trust files ‘trust-of-canonical.txt’ or ‘trust-of-joe-bloggs.txt’. The text could just be a standard paragraph, or be modified by the signer if needed. The installer on the operating system would allow the user to check the trust text etc. Could this work? Can anyone see any issues? (hmm… revoking trust would not be possible without revoking the key used to sign the document?)


Read more
Michael

For over 6 months now I’ve been working on launchpad.net – collaborative development for open source software – using Python (Zope). It’s been an unreal experience seeing various testing strategies on such a large code-base, but the two things I miss the most while developing on launchpad are:

  1. The ability to do outside-in development properly – as is possible with ruby’s Cucumber project (although, this could be used, but it’d be adding yet-another-technology to the stack – I haven’t tried pyCucumber, but it doesn’t look too active), and
  2. Something similar to rspactor – allowing tests to be run automatically as I edit files, and reporting them via the OS’s notification system.

But trying to communicate exactly what I mean by that to people who haven’t used rspactor or Cucumber is kind-of tricky, so here’s a video demo’ing how I’d love to do inside-out development on Ubuntu…


Read more
Michael

Voidspace just released Mock 0.4.0 – an update to the excellent Mock library which brings a few conveniences and easier patching of module and class-level attributes.

This makes tests using mocks easier to read, for example inside a test case:

self.assertEquals(my_mock.my_method.call_args,     (('goodbye',),{'hello': False}))


Can now be written as:

my_mock.my_method.assert_called_with('goodbye', hello=False)

Check it out!


Read more
Michael

Since working with RSpec over the past 6 months – a Behaviour-Driven Development framework for ruby – I’ve been wondering if there’s anything comparable in Python (my preferred tool for development!). One of the things I love about RSpec is the ease with which Mock objects can be used to keep tests focused.

While there are a number of mock libraries around for Python most don’t result in particularly readable test code. But I was pleasantly suprised to discover Michael Foord’s Mocking and Testing utilities.

A simple example: I’ve got a Django application hosted on Google AppEngine and say I want to write a simple test to verify that my view does require the user to be logged in with their Google account and if not, redirects appropriately – but I don’t want to have to manually log a user in, or even use the Google api as part of my test. Here’s a snippet showing how easy this is with Mock:

from mock import Mockfrom google.appengine.api import users

class MySpecialView(TestCase):

    def setUp():        """ Create the required mocks for the view tests """        # Mock the get_current_user method of the google users api        users.get_current_user = Mock()

        # Create a mock user that we'll pretend is logged in        mock_user = Mock()

        # Just for readability, save the special app-engine login url as         # an instance variable        self.url = reverse('my-special-view-name')        self.login_url = "http://testserver/_ah/login?continue=http%%3A//testserver%s" % self.url

    def test_logged_in_user_can_access_page(self):        """A logged in user should not be redirected to the login page"""        # Set the return value for the mocked         # get_current_user method:        users.get_current_user.return_value = mock_user

        response = do_request()

        # Make sure the mock method was called        self.assertTrue(users.get_current_user.called)

        # And the redirect did not take place, but the         # normal template was rendered...        self.assertTemplateUsed(response, 'myapp/overview.html')

    def test_anonymous_user_is_redirected_to_login(self):        """ An anonymous user should be redirected to the login page"""        # Set the google api's get_current_user method to return None        users.get_current_user.return_value = None

        response = self.do_request()

        # Make sure the mock method was called        self.assertTrue(users.get_current_user.called)

        # And that the redirect took place... note we can't use        # the normal assertRedirects due to the app-engine specific        # login url.        self.assertEquals(response.status_code, 302)        self.assertEquals(            response['location'],            "http://testserver/_ah/login?continue=http%%3A//testserver%s" % self.url        )


Easy! Thanks Michael. The Mock object has lots of other goodies of course (such as auto-setting all the mock-methods from the real object, testing for call parameters etc.).


Read more