Canonical Voices

Posts tagged with 'code'

Colin Watson

Launchpad has had Git-to-Bazaar code imports since 2009, along with imports from a few other systems.  These form part of Launchpad’s original mission to keep track of free software, regardless of where it’s hosted.  They’re also very useful for automatically building other artifacts, such as source package recipes or snap packages, from code hosted elsewhere.  Unfortunately they’re quite complicated: they need to be able to do a full round-trip conversion of every revision from the other version control system, which has made it difficult to add support for Git features such as signed commits or submodules.  Once one of these features is present anywhere in the history of a branch, importing it to Bazaar becomes impossible.  This has been a headache for many users.

We can do better nowadays.  As of last year, we have direct Git hosting support in Launchpad, and we can already build snaps and recipes straight from Git, so we can fulfil our basic goal more robustly now with a lot less code.  So, Launchpad now supports Git-to-Git code imports, also known as Git mirroring.  You can use this to replace many uses of Git-to-Bazaar imports (note that there’s no translations integration yet, and of course you won’t be able to branch the resulting import using bzr).

See our Git documentation for more details.

Read more
Colin Watson

We just rolled out a new feature for Launchpad’s Git repository hosting: Git-based merge proposals can now be linked to Launchpad bugs.  This can be done manually from the web UI for the merge proposal, but normally you should just mention the Launchpad bug in the commit message of one of the commits you want to merge.  The required commit message text to link to bugs #XXX and #YYY looks like this:


This is the same form used for Launchpad bug references in debian/changelog files in source packages, and the general approach of mentioning bugs in commit messages is similar to that of various other hosting sites.

Bugs are not automatically closed when merge proposals land, because the policy for when that should happen varies from project to project: for example, projects often only close bugs when they make releases, or when their code is deployed to production sites.

Users familiar with Bazaar on Launchpad should note that the model for Git bug linking is slightly different: bugs are linked to merge proposals rather than to individual branches.  This difference is mainly because individual branches within a Git repository are often much more ephemeral than Bazaar branches.

Documentation is here, along with other details of Launchpad’s Git hosting.

Read more
Colin Watson

If you are a member of Launchpad’s beta testers team, you can now try out Git-based recipes.  These work very similarly to the Bazaar-based recipes that Launchpad has supported for a long time, with a few minor changes to handle differences in the underlying version control system: the main thing you’ll probably notice is that you almost always need to explicitly specify a branch name.  The recipes documentation explains what to do; if you’re already familiar with how recipes work for Bazaar, then you should probably skip straight to the recipes guide to look over the differences for Git.

Please try this out and report any bugs you find.  If all goes well, we’ll open this up to all users in a couple of weeks.

Update: as of 2016-03-15, this feature is enabled for all Launchpad users.

Read more
Colin Watson

If you are a member of Launchpad’s beta testers team, you can now try out webhooks for Bazaar branches and Git repositories. These can be used to set up integration with external sites for various purposes, such as running CI jobs or publishing documentation. We expect to open this up to all Launchpad users soon, but in the meantime please do file a bug against Launchpad itself if you encounter any problems.

See our webhooks documentation for more details.

Update: as of 2015-11-20, this feature is enabled for all Launchpad users.

Read more
Colin Watson

It’s been a while since we posted much regularly on this team blog, not least because for a while Launchpad was running more or less in maintenance mode.  That’s no longer the case and we’re back to the point where we can do feature development work again, as exemplified by our recent addition of Git code hosting support.

Lots of other things have been happening in the Launchpad world lately, though, and the half-way point in the year seems like a good time to start talking about them.  I’m going to try to do this a bit more regularly, aiming for about once a month when we also update our internal stakeholders.  This post covers roughly the last three months.


Of course, you don’t get to release a major new feature and then have everything be perfect.  In fact, we’d released it pretty much as soon as we had a minimum viable product, so we knew there was plenty more to do.

Now that the basics work reasonably well, we’ve been focusing on tying off loose ends, and on getting to the point where Launchpad itself can be self-hosted in Git: this is partly because most of the things we need to do for that are relevant to many other projects considering a migration, partly because that ensures that any problems with it will affect Launchpad developers directly, and partly because this will allow us to trim a very large amount of database cruft associated with the representation of Launchpad’s own branches.

Here’s a brief changelog since our initial public announcement:

  • Leaving the “Target reference path” field empty when proposing a Git-based merge proposal no longer crashes (#1451068)
  • Git-based merge proposals can be deleted
  • Pushing to private repositories no longer hangs (#1451107)
  • The backend now stores reflogs so that we can help users recover from mistakes
  • Pushing a new default repository for a project now sets the repository owner to the project owner rather than to the person performing the push
  • The project +sharing view now handles Git repositories
  • Branches of a project default repository are now shown on the project’s code page, although we expect to be making significant further improvements in this area soon
  • Individual Git commits under “Recent Commits” now link directly to a full view of the corresponding commit
  • Merge proposal listings now cover Git (#1453020)
  • Branches in private repositories now show a privacy banner (#1457553)
  • Repositories can now be deleted (#1456583)
  • Private repositories are now browseable and show a privacy banner in the browsing interface
  • The URLs for Git-based merge proposals have moved to be directly under the source repository rather than under the source branch, so that historical MPs can still be accessed after deleting the branch (#1456589)
  • Launchpad now sends mail for Git-based merge proposals
  • There are various new Git repository listing views, chiefly for projects, packages, and people
  • Repositories now have a complete edit form allowing you to rename them, move them between targets, and set their default branch (“HEAD”) (#1456625)
  • We’ve exported a few more bits and pieces on the webservice API to support Git merge robots
  • We’ve brought various bits of automation for Git-based merge proposals up to par with their Bazaar equivalents: when a Git branch is updated, Launchpad now automatically updates the preview diffs for any MPs where that is the source branch, and detects whether any MPs where that is the target branch can now be marked as Merged
  • The Code settings view for projects now allows you to set the project’s default Git repository and to choose whether the default revision system for the project is Bazaar or Git

We’ve started on support for webhooks, and hope to be able to tell you more about this soon. Git-to-Git repository mirroring and native Git recipes are also frequent requests that we know are important to people, and we hope to get to those soon.

Other code review changes

Many of the changes needed for Git touched on our code review infrastructure, but we’ve been making some other improvements there too.  Launchpad has supported “inline comments” on merge proposals for a year or so now, allowing you to comment on parts of a diff without having to manually quote it; but the feature hadn’t been quite finished and lots of people had nits to pick with it.  We’ve made a couple of improvements here lately:

  • Preview diffs in merge proposals now have keyboard navigation to move between files, diff hunks, and inline comments
  • Links to diffs now include a per-file diffstat under an expander triangle
  • The mails sent when a reviewer adds inline comments now only include relevant parts of the diff context rather than the entire diff, making them much easier to read and making it harder to miss comments (#1334577)

We threw in a couple of bonus features here as well.  You can now see side-by-side diffs for merge proposals as well as unified diffs (#917502): there’s a link at the top of each preview diff allowing you to switch it to side-by-side mode (we may add a user preference for this later).  And resubmitting a merge proposal now automatically preserves the commit message (#676769).

Package build infrastructure

When we first added support for building debug symbol packages (ddebs) in Ubuntu, we handled the build and publication side of things with a temporary hack involving a custom hack to our sbuild fork that stashed the ddebs on the builder for a while, and a job that periodically fetched them to As is the way of temporary hacks, it proceeded to stay that way for eight and a half years. It mostly worked, but was rather fragile, contributed to tying us to an old sbuild, wasn’t very extensible to PPAs, and couldn’t work with virtualised builders where all the builder’s state is reset before every build.

A couple of years ago we put together the Launchpad changes required to store ddebs straightforwardly in Launchpad instead, but weren’t able to deploy this at the time due to needing better librarian infrastructure. We finally got all this cleared out in April and deployed the new ddeb publishing mechanism. The main user-visible effect of this is that isn’t liable to lose packages any more, and ddebs are also available from Launchpad build pages.

Having done this, we were able to upgrade from our 2004-era fork of sbuild to a modern version, which fixed a number of long-standing bugs.

All architectures are now optional for PPAs (#1244868), so, for example, a PPA that only needs to support armhf can save builder cycles by disabling amd64 and i386. At the moment this can only be done by Launchpad administrators, so ask a question on Launchpad if you think this would be worth doing for any of your PPAs.

We rewrote the way that we install build-dependencies for source package recipes. This allows recipes to handle the “:native” qualifier in build-dependencies which is used in some places as part of supporting cross-building.

The next step in this modernisation programme will be to move all builds into ScalingStack and decommission the corresponding bare-metal builders, which we’ll be starting soon and working on an architecture-by-architecture basis as that cloud gains support for them. In the short term, this will give us better build capacity on the migrated architectures; in the longer term, it will let us support PPA builds on more architectures, and erase the cumbersome distinction between “non-virtualised” and “virtualised” builders. More news on this as it happens.


We upgraded to significantly more modern database servers earlier this year, which meant that a lot of hitherto difficult timeout problems suddenly either disappeared or became tractable. There are still some bad queries and a few mysterious problems, but as a general trend our timeout rates are very significantly down from where they were six months ago. We have some more general plans in this area, and will continue to spot-fix bad pages as they show up and as time permits.


We have lots more to do across the whole application, but still have a rather limited number of developers. If any of this kind of thing sounds interesting and you’d like to help, you can!

Read more
William Grant

Today we’re announcing early support for hosting Git repositories directly on Launchpad, as well as or instead of Bazaar branches. This has been by far the single most commonly requested feature from Launchpad code hosting for a long time; we’ve been working hard on it for several months now, and we’re very happy to be able to release it for general use.

This is distinct from the facility to import code from Git (and some other systems) into Bazaar that Launchpad has included for many years. Code imports are useful to aggregate information from all over the free software ecosystem in a unified way, which has always been one of the primary goals of Launchpad, and in the future we may add the facility to import code into Git as well. However, what we’re releasing today is native support: you can use git push to upload code to Launchpad, and your users and collaborators can use git clone to download it, in the same kind of way that you can with any Git server.

Our support is still in its early stages, and we still have several features to add to bring it up to parity with Bazaar hosting in Launchpad, as well as generally making it easier and more pleasant to use. We’ve released it before it’s completely polished because many people are clamouring to be able to use it and we’re ready to let you all do so. From here on in, we’ll be adding features, applying polish, and fixing bugs using Launchpad’s normal iterative deployment process: changes will be rolled out to production once they’re ready, so you’ll see the UI gradually improving over time.

What’s supported?

You can:

  • push Git repositories to Launchpad over SSH
  • clone repositories over git://, SSH, or HTTPS
  • see summary information on repositories and the branches they contain in the Launchpad web UI
  • follow links from the Launchpad web UI to a full-featured code browser (cgit)
  • push and clone private repositories, if you have a commercial subscription to Launchpad
  • propose merges from one branch to another, including in a different repository, provided that they are against the same project or package

What will be supported later?

Launchpad’s Bazaar support has grown many features over the years, and it will take some time to bring our Git support up to full parity with it and beyond. Git repositories use a somewhat different model from Bazaar branches, which we’ve had to account for in many places, and some facilities will require complete reimplementation before we can support them with Git.

Here’s an incomplete list of some of the features we hope to add:

  • useful subscriptions (currently only attribute change notifications work, which are not usually very interesting in themselves)
  • RSS feeds
  • mirroring
  • webhooks
  • an integrated code browser

See our help page for more known issues and instructions on using Launchpad with Git.

Helping out

This is a new service, and we welcome your feedback: you can ask questions in #launchpad on freenode IRC, on our launchpad-users mailing list, or on Launchpad Answers, and if you find a bug then please tell us about that too.

Launchpad is free software, licensed under the GNU AGPLv3. We’d be very happy to mentor people who want to help out with parts of this service, or to build things on top of it using our published API. Some preliminary documentation on this is on our developer wiki, and you can always contact us for help.

Read more

A couple of people have reached out to me via LinkedIn and reminded me that my three year work anniversary happened last Friday. Three years since I left my job at a local place to go work for the Canonical where I got the chance to be paid to work on open source software and better my Python skills with the team working on Launchpad. My wife wasn’t quite sure. “You’ve only been at your job a year and a half, and your last one was only two years. What makes this different?”

What’s amazing, looking back, is just how *right* the decision turned out to be. I was nervous at the time. I really wasn’t Launchpad’s biggest fan. However, the team I interviewed with held this promise of making me a better developer. They were doing code reviews of every branch that went up to land. They had automated testing, and they firmly believed in unit and functional tests of the code. It was a case of the product didn’t excite me, but the environment, working with smart developers from across the globe, was exactly what I felt like I needed to move forward with my career, my craft.

2013-09-02 18.17.47

I joined my team on Launchpad in a squad of four other developers. It was funny. When I joined I felt so lost. Launchpad is an amazing and huge bit of software, and I knew I was in over my head. I talked with my manager at the time, Deryck, and he told me “Don’t worry, it’ll take you about a year to get really productive working on Launchpad.” A year! Surely you jest, and if you’re not jesting…wtf did I just get myself into?

It was a long road and over time I learned how to take a code review (a really hard skill for many of us), how to do one, and how to talk with other smart and opinionated developers. I learned the value of the daily standup, how to manage work across a kanban board. I learned to really learn from others. Up until this point I’d always been the big fish in a small pond and suddenly I was the minnow hiding in the shallows. Forget books on how to code, just look at the diff in the code review you’re reading right now. Learn!

My boss was right, it was nearly ten months before I really felt like I could be asked to do most things in Launchpad and get them done in an efficient way. Soon our team was moved on from Launchpad to other projects. It was actually pretty great. On the one hand, “Hey! I just got the hang of this thing” but, on the other hand, we were moving on to new things. Development life here has never been one of sitting still. We sit down and work on the Ubuntu cycle of six month plans, and it’s funny because even that is such a long time. Do you really know what you’ll be doing six months from now?


Since that time in Launchpad I’ve gotten work on several different projects and I ended up switching teams to work on the Juju Gui. I didn’t really know a lot about this Juju thing, but the Gui was a fascinating project. It’s a really large scale JavaScript application. This is no “toss some jQuery on a web page” thing here.

I also moved to work under a new manager Gary. As my second manager since starting at Canonical and I was amazed at my luck. Here I’ve had two great mentors that made huge strides in teaching me how to work with other developers, how to do the fun stuff, the mundane, and how to take pride in the accomplishments of the team. I sit down at my computer every day and I’ve got the brain power of amazing people at my disposal over irc, Google Hangouts, email, and more. It’s amazing to think that at these sprints we do, I’m pretty much never the smartest person in the room. However, that’s what’s so great. It’s never boring and when there’s a problem the key is that we put our joint brilliant minds to the problem. In every hard problem we’ve faced I’ve never found that a single person had the one true solution. What we come up with together is always better than what any of us had apart.

When Gary left and there was a void for team lead and it was something I was interested in. I really can’t say enough awesome things about the team of folks I work with. I wanted to keep us all together and I felt like it would be great for us to try to keep things going. It was kind of a “well I’ll just try not to $#@$@# it up” situation. That was more than nine months ago now. Gary and Deryck taught me so much, and I still have to bite my tongue and ask myself “What would Gary do” at times. I’ve kept some things the same, but I’ve also brought my own flavor into the team a bit, at least I like to think so. These days my Github profile doesn’t show me landing a branch a day, but I take great pride in the progress of the team as a whole each and every week.

The team I run now is as awesome a group of people, the best I could hope to work for. I do mean that, I work for my team. It’s never the other way around and that’s one lesson I definitely picked up from my previous leads. The projects we’re working on are exciting and new and are really important to Canonical. I get to sit in and have discussions and planning meetings with Canonical super genius veterans like Kapil, Gustavo, and occasionally Mark Shuttleworth himself.

Looking back I’ve spent the last three years becoming a better developer, getting an on the job training course on leading a team of brilliant people, and crash course on thinking about the project, not just as the bugs or features for the week, but for the project as it needs to exist in three to six months. I’ve spent three years bouncing between “what have I gotten myself into, this is beyond my abilities” to “I’ve got this. You can’t find someone else to do this better”. I always tell people that if you’re not swimming as hard as you can to keep up, find another job. I feel like three years ago I did that and I’ve been swimming ever since.


Three years is a long time in a career these days. It’s been a wild ride and I can’t thank the folks that let me in the door, taught me, and have given me the power to do great things with my work enough. I’ve worked by butt off in Budapest, Copenhagen, Cape Town, Brussels, North Carolina, London, Vegas, and the bay area a few times. Will I be here three years from now? Who knows, but I know I’ve got an awesome team to work with on Monday and we’ll be building an awesome product to keep building. I’m going to really enjoy doing work that’s challenging and fulfilling every step of the way.


Read more
Matt Fischer

Warning: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead in /home/mfischer/ on line 68

Read more
Matt Fischer

Warning: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead in /home/mfischer/ on line 68

Read more
Matt Fischer

Warning: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead in /home/mfischer/ on line 68

Read more
Matt Fischer


According to Alex Chiang, being in Copenhagen just means “more work while being tired”. Well we’ve been hard at work, and so this will be coming soon…

Read more
Matt Fischer

A bug triager’s best friend is ubuntu-bug. Special per-package hooks in /usr/share/apport/package-hooks collect logs and config files that make bug triage and analysis much easier. The scripts also help to avoid the “Can you please post the … file?” back and forth that tends to waste time and many time ends up in an incomplete bug being marked expired. (As an aside when a bug submitter is asked for more info, the bug is marked Incomplete until it is updated, if it’s not updated in 60 days, it expires).

What does this have to do with lightdm? So tonight I shared my lightdm blog post with some people and Bryce Harrington suggested that I write an apport script for lightdm. I mentioned this brilliant plan to Robert Ancell and it turns out that lightdm has one already! Well my work is done!  Except…  The existing script wasn’t being packaged or installed, was collecting logs with the wrong (old) names, and didn’t collect the configs. All of that is fixed tonight with a MP I made, based on the code that Xorg’s apport script uses. Once it’s merged and released (probably not until R opens) we will have an apport script for lightdm. Once that happens, you can file a bug by simply typing:

ubuntu-bug lightdm

There’s no apport script for any of the the greeters, at least not that I know of for unity-greeter or lightdm-gtk-greeter, but they’d need the same data. The only thing that this script doesn’t do is get a screencap/picture, that’s up to you guys.

Read more
Matthew Revell

On the 5th of October we’ll be ending our beta of Mercurial imports in Launchpad. On that day your existing Mercurial imports will cease and you won’t be able to create new ones.

This doesn’t affect Bazaar, Git, Subversion or CVS imports.

You’re probably wondering why. During the beta, we found that not many people wanted to import Mercurial branches into Launchpad. Today there are only around forty people using the facility. It’s also fair to say that our importer wasn’t of the quality we want for Launchpad.

So, with low demand for the feature we decided to focus engineering effort elsewhere rather than continue to maintain, or fix up, a less than satisfactory feature.

I’m sorry if you currently rely on Launchpad to import code from Mercurial into Bazaar. You can, though, still use the bzr-hg plugin locally.

Read more
Matt Fischer

Last week, I found myself having to dive into the udev code to figure out how it determines whether what is a mouse or a keyboard. To solve the problem I was working on, I ended up having to replicate some of that logic in python, which is posted at the bottom. Let me explain how it works.

Device Info in /sys

To start, let’s look in /sys and see what input devices we have. If you look in /sys/class/input on your system, you’ll see many symlinks to devices. I’ll pick /sys/class/input/event12 to look at in more detail, so cd to event12/device

mfisch@caprica:/sys/class/input/event12/device$ ls -al
total 0
drwxr-xr-x 7 root root 0 Jul 8 16:05 .
drwxr-xr-x 3 root root 0 Jul 8 16:05 ..
drwxr-xr-x 2 root root 0 Jul 8 16:05 capabilities
lrwxrwxrwx 1 root root 0 Jul 8 20:50 device -> ../../../serio1
drwxr-xr-x 3 root root 0 Jul 8 16:05 event12
drwxr-xr-x 2 root root 0 Jul 8 16:05 id
-r--r--r-- 1 root root 4096 Jul 8 20:50 modalias
drwxr-xr-x 3 root root 0 Jul 8 16:05 mouse0
-r--r--r-- 1 root root 4096 Jul 8 16:05 name
-r--r--r-- 1 root root 4096 Jul 8 20:50 phys
drwxr-xr-x 2 root root 0 Jul 8 20:50 power
-r--r--r-- 1 root root 4096 Jul 8 20:50 properties
lrwxrwxrwx 1 root root 0 Jul 8 16:05 subsystem -> ../../../../../../class/input
-rw-r--r-- 1 root root 4096 Jul 8 16:05 uevent
-r--r--r-- 1 root root 4096 Jul 8 20:50 uniq

There are some interesting things in here, but the two I’m interested in are “name” and the “capabilities” directory. Let’s look at name first.

mfisch@caprica:/sys/class/input/event12/device$ cat name
PS/2 Generic Mouse

Using Capabilities

Okay, so this looks like a mouse, but udev doesn’t use the name to figure this out, it uses the capabilities directory. Let’s look there:

mfisch@caprica:/sys/class/input/event12/device/capabilities$ ls -al
total 0
drwxr-xr-x 2 root root 0 Jul 8 16:05 .
drwxr-xr-x 6 root root 0 Jul 8 16:05 ..
-r--r--r-- 1 root root 4096 Jul 8 16:05 abs
-r--r--r-- 1 root root 4096 Jul 8 16:05 ev
-r--r--r-- 1 root root 4096 Jul 8 20:43 ff
-r--r--r-- 1 root root 4096 Jul 8 16:05 key
-r--r--r-- 1 root root 4096 Jul 8 20:39 led
-r--r--r-- 1 root root 4096 Jul 8 20:43 msc
-r--r--r-- 1 root root 4096 Jul 8 16:05 rel
-r--r--r-- 1 root root 4096 Jul 8 20:43 snd
-r--r--r-- 1 root root 4096 Jul 8 16:05 sw

The aptly named capabilities provide information to udev on what “capabilities” the device has. udev is specifically interested in the following ones: abs (Absolute axes), ev (Event types), key (Keys and Buttons), and rel (Relative axes). Let’s examine one of these to determine what data it contains:

mfisch@caprica:/sys/class/input/event12/device/capabilities$ cat ev

So what does 7 mean? This is a bitmask (111) who’s bits are defined in /usr/include/linux/input.h. That file has bits defined for each of the types listed in the capabilities directory, the #define prefix matches the name in the capabilities directory, so we’re looking at EV_ (Note: key is represented by defines KEY_ and BTN_, since it’s for keys and buttons).

Since this mask is 0×7 or 111, we have bits 0, 1, and 2 set and the rest are false. Judging from the #defines, the following bits are set for this device:

#define EV_SYN 0x00
#define EV_KEY 0x01
#define EV_REL 0x02

This tells us that our device gets key events, relative movement event, and synchronization events. (Figuring this out required poking around the header a bit more).

A Quick Note About the ‘key’ Capability

The entry ‘key’ is much longer than ev and requires more explanation. Let’s look at what the ‘key’ capabilities shows for my keyboard:

20000 20000000020 0 0 500f02100002 3803078f900d401 feffffdfffefffff fffffffffffffffe

So that’s much larger than ev was. It’s still a bitmask, but it’s been split into words, using the word-size sense, so 8 bytes on my system. Since this mask is little endian, the ’20000′ represents the highest set bits. This makes processing need to take place in word-size chunks, starting at the right. Another way to think of the output above is to think about it as one large mask, as long as you pad each “word” to be 16 digits, for example, what you really see above is this:

00000000000200000000020000000020000000000000000000000000000000000000500f02100002 03803078f900d401feffffdfffeffffffffffffffffffffe

To solve my problem, I split the output up on the spaces and set 64-bits at a time in my mask starting from the right-most word.

Now that we’ve read all these bitmasks, we need to do something with it.

What udev Does with this Info

If you look at the udev source code, specifically udev/udev-builtin-input_id.c, you can see how this data is used.

For mice, the code is fairly straightforward, look at the code for the test_pointers function. We really want to know where they set “is_mouse”. The code is also looking for other stuff like touchpads, so let’s ignore that. It basically boils down to this:

if EV_REL and REL_X and REL_Y and BTN_MOUSE:
you have a mouse

There’s also a similar block where you can replace REL above with ABS, which the comment claims is for VMWare’s USB mouse, but I kept it in my solution because I didn’t quite trust the comment and it seemed harmless otherwise.

For keyboards, the code is a bit more complex because it’s basically checking to see if you have a bunch of keys defined, this code is in the function test_key. then test_key looks for a keyboard it boils down to this:

If any bits are set in 'ev' and
if bits 1-31 (but not 0) are set:
you have a keyboard

Note: bits 1-31 represent the Escape key, numbers, and Q through D.

Putting it All Together

So now that we know how it works, let’s see it in action! Here it is run on my dev box:

mfisch@caprica:~/tmp/find_input$ ./
INFO:root:/dev/input/event3 is a keyboard (AT Translated Set 2 keyboard)
INFO:root:/dev/input/event12 is a mouse (PS/2 Generic Mouse)

Here it is run on another system with more devices attached:

INFO:root: /dev/input/event3 is a keyboard (AT Translated Set 2 keyboard)
INFO:root: /dev/input/event13 is a mouse (PS/2 Generic Mouse)
INFO:root: /dev/input/event15 is a keyboard (BTC USB Keyboard)
INFO:root: /dev/input/event16 is a mouse (Primax HP Wireless Laser Mini Mouse)

My python code including unit tests! is avaialble in bzr at lp:~mfisch/+junk/find_input

Comments, bugs, and fixes all gladly accepted.

Read more
Daniel Holbach

This morning I woke up and found the sponsoring queue at 103 items! I mailed the ubuntu-devel and ubuntu-motu lists and the current count is down to 81. This is great, but I’m sure we can get it down to 0.

Jani Monoses also filed these bugs to discuss how we can improve our sponsoring strategy:

You might want to join the conversation.

What we need most though is that if you can review code and upload changes, you head over to the sponsoring queue and help reviewing. It’s understandable that after UDS everybody is busy doing merges or jumps head-first into work items, but we also need to help newcomers get their changes reviewed. If you need some help, review our sponsorship best-practices.

If you should want to help on a regular basis, ping me or drop me an email and I’ll add you to the patch pilot schedule and you’ll get monthly reminders.

Rock on everybody! We can be happy we have so many new contributors, let’s don’t let them down! :-)

Read more
Matt Fischer

This morning I was debugging a problem in unity-greeter and I suspected that it might be related to the resolution of the system I was using. Since I was live booting it was a pain to make changes, mainly because I didn’t have all my standard tools (or web access). I ended up messing around with xorg.conf files for awhile, without much success (this reminded me of college, but not pleasantly). After some other searches, I ended up settling on this method:

For this example, I wanted a resolution of 1024×600.

  1. Login to my system
  2. Open a terminal and run cvt 1024 600
  3. Copy the resulting output, minus ModeLine portion
  4. mfisch@caprica:~$ cvt 1024 600
    # 1024x600 59.85 Hz (CVT) hsync: 37.35 kHz; pclk: 49.00 MHz
    Modeline "1024x600_60.00"   49.00  1024 1072 1168 1312  600 603 613 624 -hsync +vsync
  5. Run xrandr –newmode
  6. mfisch@caprica:~$ xrandr --newmode "1024x600_60.00"   49.00  1024 1072 1168 1312  600 603 613 624 -hsync +vsync
  7. Run xrandr –addmode LVDS 1024x600_60.00. (Note that there are several displays that xrandr shows, I wanted to change my main laptop display, LVDS, you may want to change another. Run xrandr without arguments to see all the displays.)
  8. Run xrandr –output LVDS –mode 1024x600_60.00
  9. If that switches your resolution and looks right, then lets make it permanent.
  10. Edit: /etc/lightdm/lightdm.conf, add this line (note you can choose a different path:
  11. display-setup-script=/usr/bin/lightdmxrandr
  12. Make a new file called /usr/bin/lightdmxrandr, make it a shell script that just executes lines 4, 5, and 6 from above
  13. #!/bin/sh
    xrandr --newmode "1024x600_60.00"   49.00  1024 1072 1168 1312  600 603 613 624 -hsync +vsync
    xrandr --addmode LVDS 1024x600_60.00
    xrandr --output LVDS --mode 1024x600_60.00
  14. Make /usr/bin/lightdmxrandr executable, logout, and restart lightdm (sudo initctl restart lighthdm)
  15. Enjoy your new resolution!

Here are the two links that helped me piece this together:

How change display resolution settings using xrandr
How can I make xrandr customization permanent?

Read more
Matt Fischer

Over the past week I’ve been playing with DHCP option tags, and using them to send data to DHCP client systems, and then do something with the data. I didn’t know much about DHCP option tags before doing this, I figure DHCP just gave you an IP address and maybe a nameserver. In reality, DHCP can be customized to send a couple hundred different settings, including where the IRC or SMTP servers are, VOIP settings, and lots more. What makes it really powerful is that you can use the unassigned options to create your own settings, and then do actions on the client when they get set. In this simple example, I’m going to send and set a new “message of the day” via dhcp. The message of the day is a (usually) short text message shown when you login via the terminal and is stored in /etc/motd. If you are trying this on a production system, you should back-up your /etc/motd first.

Server Setup

On the server, after installing dhcpd, edit /etc/dhcp/dhcpd.conf, create a valid subnet and then in the global space, add your new option. Since options 151-174 are unassigned, I will use option 151. You could also use a ‘site-local’ option between 224-254.

Add the following outside any subnet sections:
option motd code 151 = text;
option motd "Happy Easter from your DHCP server team";

Client Setup

On the client, manually run dhclient -d and make sure you’re getting a valid IP from the server. Once you know that is working, edit /etc/dhcp/dhclient.conf and add two parts, first you need to name the new option and second you need to request it:

Add this to the top, you may see another option code there, add it near that:
option motd code 151 = text;

Then look for the big request block, add your new option into that block:
request subnet-mask, broadcast-address, time-offset, routers,
domain-name, domain-name-servers, domain-search, host-name,
netbios-name-servers, netbios-scope, interface-mtu,
rfc3442-classless-static-routes, ntp-servers,
dhcp6.domain-search, dhcp6.fqdn,, dhcp6.sntp-servers,

Now, let’s see if we get it. Edit /etc/dhcp/dhclient-exit-hooks.d/debug and switch RUN=”no” to RUN=”yes”. Then edit the large block of variables and add your new option, motd.

Re-run dhclient -d and then check /tmp/dhclient-script.debug, do you see your motd option? If so, your setup is correct. It should look liks this:

new_motd='Happy Easter from your DHCP server team'

Doing Something When You Get an Option

dhclient defines exit-hooks, one of which is the debug script you edited above. You can create your own shell script in /etc/dhcp/dhclient-exit-hooks.d/ and follow some of the other examples in there. DHCP options come in as variables, so in this case, the script would use ${new_motd}. From what I can tell everything in here has to be a shell script, a python script I tried did not work. Here is my script to set motd:

if [ -n "${new_motd+x} ]; then
echo ${new_motd} > /etc/motd

Re-run dhclient -d and you should see that the script has run and your message of the day is now a Happy Easter message.

What About NetworkManager?

If you are running NetworkManager on your system, and you probably are, then NetworkManager starts dhclient during boot and subverts the running of dhclient’s exit hook scripts. You can use NetworkManager’s own version of “exit hooks” which live in /etc/NetworkManager/dispatcher.d to respond to DHCP events. Scripts in that directory can be written directly in python. The scripts run from NetworkManager’s dispatcher use slightly different environment variables for DHCP options, they’re in all caps and are pre-pended with DHCP4_ (and presumably DHCP6_ if you’re serving IPv6 addresses).

if [ -n "${DHCP4_MOTD+x} ]; then
echo ${DHCP4_MOTD} > /etc/motd

See the man NetworkManager(8) for more details on actions that your scripts can respond to.

Options as Expressions

There are more advanced things you can do. For example, you set a setting on the server based on an expression, for example:

if option dhcp-user-class = "fortcollins-office" {
option smtp-server "";

You then configure the client to send up a dhcp-user-class of “fortcollins-office” for all your PCs in Fort Collins.

Using my example, you could change the holiday in the message based on the date that the lease was given out.

See man dhcp-eval(5) for more info on expressions.

Encapsulated Options

Another thing you can do is vendor encapsulated options. Vendor encapsulated options take a bunch of vendor specific info and encapsulate it into one option. This means that you can pack a bunch of data into one option. I am not sure if all dhcpd servers support this construct however.

Testing This
If you want to try DHCP options without messing up your network, the quickest way is to create two virtual machines, a client and a server, and give them each two NICs. One NIC on each machine should be on a private network, and the server should have a static IP on this private network NIC and serve DHCP on that interface. The other non-private NIC is used by you to copy files to and from your virtual machines, but is otherwise not specifically required. I did this setup using Ubuntu 12.04 precise and qemu and it works great.

Customized DHCP options let you transmit data to your DHCP clients and then write scripts to act on that data. Although my example is contrived, you could use DHCP to transmit useful system settings and then configure them via script.


Read more
Curtis Hovey

We are reimagining the nature of privacy in Launchpad. The goal of the disclosure feature is to introduce true private projects, and we are reconciling the contradictory implementations of privacy in bugs and branches.

Launchpad will use policies instead of roles to govern who has access to a kind of privacy

We are implementing three kinds of policies, proprietary, embargoed security, and user data. The maintainer is the default member of these policies. The maintainer can share a kind or private data by adding a user or team to a policy.

For proprietary projects, the maintainer can add their organisational teams to the proprietary policy to share all the project information with the team members.

For Ubuntu, the maintainer will set the apport bot to be the only user in the user data policy; user data is only shared with a bot that removes personal data so that the bug can be made public. The Ubuntu security team will be the only users in the security policy.

Most maintainers will want to add project drivers to the policies if they use drivers. Bug supervisors can be added as well, though the team must be exclusive (moderated or restricted).

You can still subscribe a user or team to a private bug or branch and Launchpad will also permit the user to access it without sharing everything with the user. The existing behaviour will continue to work but it will be an exception to the normal rules.

Polices replace the bug-subscription-on-privacy-change rules. If you have every had to change the bug supervisor for a project with many private bugs, you can rejoice. You will not need to manually update the subscriptions to the private bugs to do what Launchpad implied would happen. Subscriptions are just about notification. You will not be notified of proprietary changes is proprietary information is not shared with you. Sharing kinds or information via policy means that many existing private bugs without subscribers will finally be visible to project members who can fix the issue.

Read more
Matt Fischer

Fun With Lenses and Scopes

I’ve been fooling around with lenses over the past couple of weeks in my spare time. My idea was to write a stock ticker lens/scope. I wrote a quick backend in python in an hour or so, but the lens/scope proved more difficult. The main issue was that as I made progress, Michael Hall announced Singlet and then Singlet 0.2. These were compelling enough to drop my work and start over. Initially I had some issues getting singlet based scopes to work (some of which I fixed), but then I got better at debugging them and figuring out file placement. (Of course after I was well into this project, Michael Hall announced another easier way of doing lenses and scopes!)

Announcing the Stock Quote Lens

The lens I’m pushing today is a stock quote & news lens and scope combo. It relies on Michael Hall’s singlet 0.2 and will only work for precise.

Here are some screenshots:
multi-symbols one_symbol

As you can see it has two modes. If you enter one symbol, you get a quote and news. If you enter more than one, you get quotes. I could not figure out a good wait to show news for multiple symbols in the constraints of a lens. However, all the icons are clickable and direct you to a page with more quote info and news for each ticker, so use that method to get more info. The icons themselves are static, they’re not real charts.

For now, this is US markets only and it’s only in English. Given the architecture, you could write your own scope back-end for this and populate results for the stock markets in whatever country you’re in.

Where to Get It

I pushed the code up to launchpad today. You can package it yourself, but I have not published the package. I will be doing so next week when I can get into the scopes packagers PPA. The package is published in the Scopes Packagers PPA. You can install it by doing:

sudo add-apt-repository ppa:scopes-packagers/ppa
sudo apt-get update
sudo apt-get install unity-stock-ticker-lens

You can also pull the code from bzr directly:

bzr branch lp:~mfisch/onehundredscopes/unity-stock-ticker-lens

Some tricks to debugging Unity scopes & lenses

These are some methods I used during development.

  1. Remember to run setsid unity and watch the terminal output. This is where you’ll see all your missing icons and broken .scope and .lens issues. You’ll need to re-run this when you change a .scope or .lens file IIRC.
  2. When debugging, fire up your scope in a terminal, then your lens in another terminal. Any errors or prints will dump to the terminal. Seeing a dumb error like a python indent error here is much more helpful than trying to use the scope and wondering why you have no output.
  3. If you want to run the scope/lens manually, be sure to kill the old processes first.
  4. Check your paths and dbus info. If the stuff runs fine manually, but won’t autostart, then something is wrong in your dbus service files.

PS – Thanks to Bob Davis for help with sourcing those icons.

Read more
Curtis Hovey

We are reimagining the nature of privacy in Launchpad. The goal of the disclosure feature is to introduce true private projects, and we are reconciling the contradictory implementations of privacy in bugs and branches.

We are adding a new kind of privacy called “Proprietary” which will work differently than the current forms of privacy.

The information in proprietary data is not shared between projects. The conversations, client, customer, partner, company, and organisation data are held in confidence. proprietary information is unlikely to every be made public.

Many projects currently have private bugs and branches because they contain proprietary information. We expect to change these bugs from generic private to proprietary. We know that private bugs and branches that belong to projects that have only a proprietary license are intended to be proprietary. We will not change bugs that are public, described as security, or are shared with another project.

This point is a subtle change from what I have spoken and written previously. We are not changing the current forms of privacy. We do not assume that all private things are proprietary. We are adding a new kind of privacy that cannot be shared with other projects to ensure the information is not disclosed.

Launchpad currently permits projects to have default private bugs and branches. These features exist for proprietary projects. We will change the APIs to clarify this. eg:

    project.private_bugs = True  => project.default_proprietary_bugs = True
    project.setBranchVisibilityTeamPolicy(FORBIDDEN) => project.default_proprietary_branches = True

Projects with commercial subscriptions will get the “proprietary” classification. Project contributors will be able to classify their bugs and branches as proprietary. The maintainers will be able to enable default proprietary bugs and branches.

Next part: Launchpad will use policies instead of roles to govern who has access to a kind of privacy.

Read more