According to Alex Chiang, being in Copenhagen just means “more work while being tired”. Well we’ve been hard at work, and so this will be coming soon…

A bug triager’s best friend is ubuntu-bug. Special per-package hooks in /usr/share/apport/package-hooks collect logs and config files that make bug triage and analysis much easier. The scripts also help to avoid the “Can you please post the … file?” back and forth that tends to waste time and many time ends up in an incomplete bug being marked expired. (As an aside when a bug submitter is asked for more info, the bug is marked Incomplete until it is updated, if it’s not updated in 60 days, it expires).

What does this have to do with lightdm? So tonight I shared my lightdm blog post with some people and Bryce Harrington suggested that I write an apport script for lightdm. I mentioned this brilliant plan to Robert Ancell and it turns out that lightdm has one already! Well my work is done!  Except…  The existing script wasn’t being packaged or installed, was collecting logs with the wrong (old) names, and didn’t collect the configs. All of that is fixed tonight with a MP I made, based on the code that Xorg’s apport script uses. Once it’s merged and released (probably not until R opens) we will have an apport script for lightdm. Once that happens, you can file a bug by simply typing:

ubuntu-bug lightdm

There’s no apport script for any of the the greeters, at least not that I know of for unity-greeter or lightdm-gtk-greeter, but they’d need the same data. The only thing that this script doesn’t do is get a screencap/picture, that’s up to you guys.

On the 5th of October we’ll be ending our beta of Mercurial imports in Launchpad. On that day your existing Mercurial imports will cease and you won’t be able to create new ones.

This doesn’t affect Bazaar, Git, Subversion or CVS imports.

You’re probably wondering why. During the beta, we found that not many people wanted to import Mercurial branches into Launchpad. Today there are only around forty people using the facility. It’s also fair to say that our importer wasn’t of the quality we want for Launchpad.

So, with low demand for the feature we decided to focus engineering effort elsewhere rather than continue to maintain, or fix up, a less than satisfactory feature.

I’m sorry if you currently rely on Launchpad to import code from Mercurial into Bazaar. You can, though, still use the bzr-hg plugin locally.

Last week, I found myself having to dive into the udev code to figure out how it determines whether what is a mouse or a keyboard. To solve the problem I was working on, I ended up having to replicate some of that logic in python, which is posted at the bottom. Let me explain how it works.

Device Info in /sys

To start, let’s look in /sys and see what input devices we have. If you look in /sys/class/input on your system, you’ll see many symlinks to devices. I’ll pick /sys/class/input/event12 to look at in more detail, so cd to event12/device

mfisch@caprica:/sys/class/input/event12/device$ ls -al
total 0
drwxr-xr-x 7 root root 0 Jul 8 16:05 .
drwxr-xr-x 3 root root 0 Jul 8 16:05 ..
drwxr-xr-x 2 root root 0 Jul 8 16:05 capabilities
lrwxrwxrwx 1 root root 0 Jul 8 20:50 device -> ../../../serio1
drwxr-xr-x 3 root root 0 Jul 8 16:05 event12
drwxr-xr-x 2 root root 0 Jul 8 16:05 id
-r--r--r-- 1 root root 4096 Jul 8 20:50 modalias
drwxr-xr-x 3 root root 0 Jul 8 16:05 mouse0
-r--r--r-- 1 root root 4096 Jul 8 16:05 name
-r--r--r-- 1 root root 4096 Jul 8 20:50 phys
drwxr-xr-x 2 root root 0 Jul 8 20:50 power
-r--r--r-- 1 root root 4096 Jul 8 20:50 properties
lrwxrwxrwx 1 root root 0 Jul 8 16:05 subsystem -> ../../../../../../class/input
-rw-r--r-- 1 root root 4096 Jul 8 16:05 uevent
-r--r--r-- 1 root root 4096 Jul 8 20:50 uniq


There are some interesting things in here, but the two I’m interested in are “name” and the “capabilities” directory. Let’s look at name first.

mfisch@caprica:/sys/class/input/event12/device$ cat name
PS/2 Generic Mouse


Using Capabilities

Okay, so this looks like a mouse, but udev doesn’t use the name to figure this out, it uses the capabilities directory. Let’s look there:

mfisch@caprica:/sys/class/input/event12/device/capabilities$ ls -al
total 0
drwxr-xr-x 2 root root 0 Jul 8 16:05 .
drwxr-xr-x 6 root root 0 Jul 8 16:05 ..
-r--r--r-- 1 root root 4096 Jul 8 16:05 abs
-r--r--r-- 1 root root 4096 Jul 8 16:05 ev
-r--r--r-- 1 root root 4096 Jul 8 20:43 ff
-r--r--r-- 1 root root 4096 Jul 8 16:05 key
-r--r--r-- 1 root root 4096 Jul 8 20:39 led
-r--r--r-- 1 root root 4096 Jul 8 20:43 msc
-r--r--r-- 1 root root 4096 Jul 8 16:05 rel
-r--r--r-- 1 root root 4096 Jul 8 20:43 snd
-r--r--r-- 1 root root 4096 Jul 8 16:05 sw


The aptly named capabilities provide information to udev on what “capabilities” the device has. udev is specifically interested in the following ones: abs (Absolute axes), ev (Event types), key (Keys and Buttons), and rel (Relative axes). Let’s examine one of these to determine what data it contains:

mfisch@caprica:/sys/class/input/event12/device/capabilities$ cat ev
7


So what does 7 mean? This is a bitmask (111) who’s bits are defined in /usr/include/linux/input.h. That file has bits defined for each of the types listed in the capabilities directory, the #define prefix matches the name in the capabilities directory, so we’re looking at EV_ (Note: key is represented by defines KEY_ and BTN_, since it’s for keys and buttons).

Since this mask is 0×7 or 111, we have bits 0, 1, and 2 set and the rest are false. Judging from the #defines, the following bits are set for this device:

#define EV_SYN 0x00
#define EV_KEY 0x01
#define EV_REL 0x02


This tells us that our device gets key events, relative movement event, and synchronization events. (Figuring this out required poking around the header a bit more).

A Quick Note About the ‘key’ Capability

The entry ‘key’ is much longer than ev and requires more explanation. Let’s look at what the ‘key’ capabilities shows for my keyboard:

20000 20000000020 0 0 500f02100002 3803078f900d401 feffffdfffefffff fffffffffffffffe

So that’s much larger than ev was. It’s still a bitmask, but it’s been split into words, using the word-size sense, so 8 bytes on my system. Since this mask is little endian, the ’20000′ represents the highest set bits. This makes processing need to take place in word-size chunks, starting at the right. Another way to think of the output above is to think about it as one large mask, as long as you pad each “word” to be 16 digits, for example, what you really see above is this:

00000000000200000000020000000020000000000000000000000000000000000000500f02100002 03803078f900d401feffffdfffeffffffffffffffffffffe

To solve my problem, I split the output up on the spaces and set 64-bits at a time in my mask starting from the right-most word.

Now that we’ve read all these bitmasks, we need to do something with it.

What udev Does with this Info

If you look at the udev source code, specifically udev/udev-builtin-input_id.c, you can see how this data is used.

For mice, the code is fairly straightforward, look at the code for the test_pointers function. We really want to know where they set “is_mouse”. The code is also looking for other stuff like touchpads, so let’s ignore that. It basically boils down to this:

if EV_REL and REL_X and REL_Y and BTN_MOUSE:
you have a mouse


There’s also a similar block where you can replace REL above with ABS, which the comment claims is for VMWare’s USB mouse, but I kept it in my solution because I didn’t quite trust the comment and it seemed harmless otherwise.

For keyboards, the code is a bit more complex because it’s basically checking to see if you have a bunch of keys defined, this code is in the function test_key. then test_key looks for a keyboard it boils down to this:

If any bits are set in 'ev' and
if bits 1-31 (but not 0) are set:
you have a keyboard


Note: bits 1-31 represent the Escape key, numbers, and Q through D.

Putting it All Together

So now that we know how it works, let’s see it in action! Here it is run on my dev box:

mfisch@caprica:~/tmp/find_input$ ./find_input.py
INFO:root:/dev/input/event3 is a keyboard (AT Translated Set 2 keyboard)
INFO:root:/dev/input/event12 is a mouse (PS/2 Generic Mouse)


Here it is run on another system with more devices attached:

INFO:root: /dev/input/event3 is a keyboard (AT Translated Set 2 keyboard)
INFO:root: /dev/input/event13 is a mouse (PS/2 Generic Mouse)
INFO:root: /dev/input/event15 is a keyboard (BTC USB Keyboard)
INFO:root: /dev/input/event16 is a mouse (Primax HP Wireless Laser Mini Mouse)


My python code including unit tests! is avaialble in bzr at lp:~mfisch/+junk/find_input

Comments, bugs, and fixes all gladly accepted.

This morning I woke up and found the sponsoring queue at 103 items! I mailed the ubuntu-devel and ubuntu-motu lists and the current count is down to 81. This is great, but I’m sure we can get it down to 0.

Jani Monoses also filed these bugs to discuss how we can improve our sponsoring strategy:

• Bug #1008449: explicit forward-to-debian tags?
• Bug #1008445: Support for blacklisting packages
• Bug #1008441: Allow grouping by contributor

You might want to join the conversation.

What we need most though is that if you can review code and upload changes, you head over to the sponsoring queue and help reviewing. It’s understandable that after UDS everybody is busy doing merges or jumps head-first into work items, but we also need to help newcomers get their changes reviewed. If you need some help, review our sponsorship best-practices.

If you should want to help on a regular basis, ping me or drop me an email and I’ll add you to the patch pilot schedule and you’ll get monthly reminders.

Rock on everybody! We can be happy we have so many new contributors, let’s don’t let them down!

This morning I was debugging a problem in unity-greeter and I suspected that it might be related to the resolution of the system I was using. Since I was live booting it was a pain to make changes, mainly because I didn’t have all my standard tools (or web access). I ended up messing around with xorg.conf files for awhile, without much success (this reminded me of college, but not pleasantly). After some other searches, I ended up settling on this method:

For this example, I wanted a resolution of 1024×600.

1. Login to my system
2. Open a terminal and run cvt 1024 600
3. Copy the resulting output, minus ModeLine portion

mfisch@caprica:~$ cvt 1024 600
# 1024x600 59.85 Hz (CVT) hsync: 37.35 kHz; pclk: 49.00 MHz
Modeline "1024x600_60.00"   49.00  1024 1072 1168 1312  600 603 613 624 -hsync +vsync

4. Run xrandr –newmode

mfisch@caprica:~$ xrandr --newmode "1024x600_60.00"   49.00  1024 1072 1168 1312  600 603 613 624 -hsync +vsync

5. Run xrandr –addmode LVDS 1024x600_60.00. (Note that there are several displays that xrandr shows, I wanted to change my main laptop display, LVDS, you may want to change another. Run xrandr without arguments to see all the displays.)
6. Run xrandr –output LVDS –mode 1024x600_60.00
7. If that switches your resolution and looks right, then lets make it permanent.
8. Edit: /etc/lightdm/lightdm.conf, add this line (note you can choose a different path:

display-setup-script=/usr/bin/lightdmxrandr

9. Make a new file called /usr/bin/lightdmxrandr, make it a shell script that just executes lines 4, 5, and 6 from above

#!/bin/sh
xrandr --newmode "1024x600_60.00"   49.00  1024 1072 1168 1312  600 603 613 624 -hsync +vsync
xrandr --addmode LVDS 1024x600_60.00
xrandr --output LVDS --mode 1024x600_60.00

10. Make /usr/bin/lightdmxrandr executable, logout, and restart lightdm (sudo initctl restart lighthdm)
11. Enjoy your new resolution!

Here are the two links that helped me piece this together:

How change display resolution settings using xrandr
How can I make xrandr customization permanent?

Over the past week I’ve been playing with DHCP option tags, and using them to send data to DHCP client systems, and then do something with the data. I didn’t know much about DHCP option tags before doing this, I figure DHCP just gave you an IP address and maybe a nameserver. In reality, DHCP can be customized to send a couple hundred different settings, including where the IRC or SMTP servers are, VOIP settings, and lots more. What makes it really powerful is that you can use the unassigned options to create your own settings, and then do actions on the client when they get set. In this simple example, I’m going to send and set a new “message of the day” via dhcp. The message of the day is a (usually) short text message shown when you login via the terminal and is stored in /etc/motd. If you are trying this on a production system, you should back-up your /etc/motd first.

Server Setup

On the server, after installing dhcpd, edit /etc/dhcp/dhcpd.conf, create a valid subnet and then in the global space, add your new option. Since options 151-174 are unassigned, I will use option 151. You could also use a ‘site-local’ option between 224-254.

Add the following outside any subnet sections:
option motd code 151 = text;
option motd "Happy Easter from your DHCP server team";

Client Setup

On the client, manually run dhclient -d and make sure you’re getting a valid IP from the server. Once you know that is working, edit /etc/dhcp/dhclient.conf and add two parts, first you need to name the new option and second you need to request it:

Add this to the top, you may see another option code there, add it near that:
option motd code 151 = text;

Then look for the big request block, add your new option into that block:
request subnet-mask, broadcast-address, time-offset, routers,
domain-name, domain-name-servers, domain-search, host-name,
netbios-name-servers, netbios-scope, interface-mtu,
rfc3442-classless-static-routes, ntp-servers,
dhcp6.domain-search, dhcp6.fqdn,
dhcp6.name-servers, dhcp6.sntp-servers,
motd;


Now, let’s see if we get it. Edit /etc/dhcp/dhclient-exit-hooks.d/debug and switch RUN=”no” to RUN=”yes”. Then edit the large block of variables and add your new option, motd.

Re-run dhclient -d and then check /tmp/dhclient-script.debug, do you see your motd option? If so, your setup is correct. It should look liks this:

new_motd='Happy Easter from your DHCP server team'

Doing Something When You Get an Option

dhclient defines exit-hooks, one of which is the debug script you edited above. You can create your own shell script in /etc/dhcp/dhclient-exit-hooks.d/ and follow some of the other examples in there. DHCP options come in as variables, so in this case, the script would use ${new_motd}. From what I can tell everything in here has to be a shell script, a python script I tried did not work. Here is my script to set motd:

if [ -n "${new_motd+x} ]; then
echo ${new_motd} > /etc/motd
fi

Re-run dhclient -d and you should see that the script has run and your message of the day is now a Happy Easter message.

What About NetworkManager?

If you are running NetworkManager on your system, and you probably are, then NetworkManager starts dhclient during boot and subverts the running of dhclient’s exit hook scripts. You can use NetworkManager’s own version of “exit hooks” which live in /etc/NetworkManager/dispatcher.d to respond to DHCP events. Scripts in that directory can be written directly in python. The scripts run from NetworkManager’s dispatcher use slightly different environment variables for DHCP options, they’re in all caps and are pre-pended with DHCP4_ (and presumably DHCP6_ if you’re serving IPv6 addresses).

if [ -n "${DHCP4_MOTD+x} ]; then
echo ${DHCP4_MOTD} > /etc/motd
fi

See the man NetworkManager(8) for more details on actions that your scripts can respond to.

Options as Expressions

There are more advanced things you can do. For example, you set a setting on the server based on an expression, for example:

if option dhcp-user-class = "fortcollins-office" {
option smtp-server "smtp.ftc-office.net";
}


You then configure the client to send up a dhcp-user-class of “fortcollins-office” for all your PCs in Fort Collins.

Using my example, you could change the holiday in the message based on the date that the lease was given out.

See man dhcp-eval(5) for more info on expressions.

Encapsulated Options

Another thing you can do is vendor encapsulated options. Vendor encapsulated options take a bunch of vendor specific info and encapsulate it into one option. This means that you can pack a bunch of data into one option. I am not sure if all dhcpd servers support this construct however.

Testing This
If you want to try DHCP options without messing up your network, the quickest way is to create two virtual machines, a client and a server, and give them each two NICs. One NIC on each machine should be on a private network, and the server should have a static IP on this private network NIC and serve DHCP on that interface. The other non-private NIC is used by you to copy files to and from your virtual machines, but is otherwise not specifically required. I did this setup using Ubuntu 12.04 precise and qemu and it works great.

Conclusion
Customized DHCP options let you transmit data to your DHCP clients and then write scripts to act on that data. Although my example is contrived, you could use DHCP to transmit useful system settings and then configure them via script.

References

• DHCP Options Manpage
• NetworkManager Manpage

We are reimagining the nature of privacy in Launchpad. The goal of the disclosure feature is to introduce true private projects, and we are reconciling the contradictory implementations of privacy in bugs and branches.

Launchpad will use policies instead of roles to govern who has access to a kind of privacy

We are implementing three kinds of policies, proprietary, embargoed security, and user data. The maintainer is the default member of these policies. The maintainer can share a kind or private data by adding a user or team to a policy.

For proprietary projects, the maintainer can add their organisational teams to the proprietary policy to share all the project information with the team members.

For Ubuntu, the maintainer will set the apport bot to be the only user in the user data policy; user data is only shared with a bot that removes personal data so that the bug can be made public. The Ubuntu security team will be the only users in the security policy.

Most maintainers will want to add project drivers to the policies if they use drivers. Bug supervisors can be added as well, though the team must be exclusive (moderated or restricted).

You can still subscribe a user or team to a private bug or branch and Launchpad will also permit the user to access it without sharing everything with the user. The existing behaviour will continue to work but it will be an exception to the normal rules.

Polices replace the bug-subscription-on-privacy-change rules. If you have every had to change the bug supervisor for a project with many private bugs, you can rejoice. You will not need to manually update the subscriptions to the private bugs to do what Launchpad implied would happen. Subscriptions are just about notification. You will not be notified of proprietary changes is proprietary information is not shared with you. Sharing kinds or information via policy means that many existing private bugs without subscribers will finally be visible to project members who can fix the issue.

Fun With Lenses and Scopes

I’ve been fooling around with lenses over the past couple of weeks in my spare time. My idea was to write a stock ticker lens/scope. I wrote a quick backend in python in an hour or so, but the lens/scope proved more difficult. The main issue was that as I made progress, Michael Hall announced Singlet and then Singlet 0.2. These were compelling enough to drop my work and start over. Initially I had some issues getting singlet based scopes to work (some of which I fixed), but then I got better at debugging them and figuring out file placement. (Of course after I was well into this project, Michael Hall announced another easier way of doing lenses and scopes!)

Announcing the Stock Quote Lens

The lens I’m pushing today is a stock quote & news lens and scope combo. It relies on Michael Hall’s singlet 0.2 and will only work for precise.

Here are some screenshots:

As you can see it has two modes. If you enter one symbol, you get a quote and news. If you enter more than one, you get quotes. I could not figure out a good wait to show news for multiple symbols in the constraints of a lens. However, all the icons are clickable and direct you to a page with more quote info and news for each ticker, so use that method to get more info. The icons themselves are static, they’re not real charts.

For now, this is US markets only and it’s only in English. Given the architecture, you could write your own scope back-end for this and populate results for the stock markets in whatever country you’re in.

Where to Get It

I pushed the code up to launchpad today. You can package it yourself, but I have not published the package. I will be doing so next week when I can get into the scopes packagers PPA. The package is published in the Scopes Packagers PPA. You can install it by doing:

sudo add-apt-repository ppa:scopes-packagers/ppa
sudo apt-get update
sudo apt-get install unity-stock-ticker-lens

You can also pull the code from bzr directly:

bzr branch lp:~mfisch/onehundredscopes/unity-stock-ticker-lens

Some tricks to debugging Unity scopes & lenses

These are some methods I used during development.

1. Remember to run setsid unity and watch the terminal output. This is where you’ll see all your missing icons and broken .scope and .lens issues. You’ll need to re-run this when you change a .scope or .lens file IIRC.
2. When debugging, fire up your scope in a terminal, then your lens in another terminal. Any errors or prints will dump to the terminal. Seeing a dumb error like a python indent error here is much more helpful than trying to use the scope and wondering why you have no output.
3. If you want to run the scope/lens manually, be sure to kill the old processes first.
4. Check your paths and dbus info. If the stuff runs fine manually, but won’t autostart, then something is wrong in your dbus service files.

PS – Thanks to Bob Davis for help with sourcing those icons.

We are reimagining the nature of privacy in Launchpad. The goal of the disclosure feature is to introduce true private projects, and we are reconciling the contradictory implementations of privacy in bugs and branches.

We are adding a new kind of privacy called “Proprietary” which will work differently than the current forms of privacy.

The information in proprietary data is not shared between projects. The conversations, client, customer, partner, company, and organisation data are held in confidence. proprietary information is unlikely to every be made public.

Many projects currently have private bugs and branches because they contain proprietary information. We expect to change these bugs from generic private to proprietary. We know that private bugs and branches that belong to projects that have only a proprietary license are intended to be proprietary. We will not change bugs that are public, described as security, or are shared with another project.

This point is a subtle change from what I have spoken and written previously. We are not changing the current forms of privacy. We do not assume that all private things are proprietary. We are adding a new kind of privacy that cannot be shared with other projects to ensure the information is not disclosed.

Launchpad currently permits projects to have default private bugs and branches. These features exist for proprietary projects. We will change the APIs to clarify this. eg:

    project.private_bugs = True  => project.default_proprietary_bugs = True
    project.setBranchVisibilityTeamPolicy(FORBIDDEN) => project.default_proprietary_branches = True

Projects with commercial subscriptions will get the “proprietary” classification. Project contributors will be able to classify their bugs and branches as proprietary. The maintainers will be able to enable default proprietary bugs and branches.

Next part: Launchpad will use policies instead of roles to govern who has access to a kind of privacy.

We are reimagining the nature of privacy in Launchpad. The goal of the disclosure feature is to introduce true private projects, and we are reconciling the contradictory implementations of privacy in bugs and branches.

We must change the UI to accommodate the a kind of privacy, and we must change some existing terms because to avoid confusion.

We currently have two checkboxes, Private and Security that create 4 combined states:

• Public
• Public Security
• Private Security
• Private something else

Most private bugs in Launchpad are private because they contain user data. You might think at first that something that is just private is proprietary. This is not the so. Ubuntu took advantage of defects in Launchpad’s conflation of subscription and access to address a kind of privacy we did not plan for. Most private bugs in Launchpad are owned by Ubuntu. They were created by the apport bug reporting process and may contain personal user data. These bugs cannot be made public until they are redacted or purged of user data. We reviewed a sample of private bugs that belong to public projects and discovered more than 90% were made private because they contained user data. Since project contributors cannot hide or edit bug comments, they chose to make the bug private to protect the user. Well done. Launchpad needs to clarify when something contains user data so that everyone knows that it cannot
be made public without removing the personal information.

Public and private security bugs represent two states in a workflow. The goal of every security bug is to be resolved, then made public so that users are informed. People who work on these issues do not use ”public” and “private”, they use “unembargoed” and “embargoed”.

Also, when I view something that is private, Launchpad needs to tell me why. The red privacy banner shown on Launchpad pages must tell me why something is private. Is it because the page contains user data, proprietary information, or an embargoed security issue? This informs me if the thing could become public.

When I want to change somethings visibility, I expect Launchpad to show me a choice that clearly states my options. Launchpad’s pickers currently shows me a term without an explanation, yet Launchpad’s code does contain the term’s definition. Instead of making me search help.launchpad.net (in vain), the picker must inform me. Given the risks of disclosing personal user data or proprietary information, I think an informative picker is essential. I expect to see something like this when I open the visibility picker for a bug:

Branches require a similar, if not identical way of describing their kind of information. I am not certain branches contain user data, but if one did, it would be clear that the branch should not be visible to everyone and should not be merged until the user data is removed.

Next post: We are adding a new kind of privacy called “Proprietary” which will work differently than the current forms of privacy.

What is a LightDM Greeter?

LightDM is a lightweight display manager that splits the responsibilities of a display manager between the server and the UI, which is known as a greeter.  When you login to Ubuntu you are shown the Unity Greeter, but all the back-end session management and authentication is provided by the LightDM daemon, and functionality that a greeter needs is provided by liblightdm. LightDM exposes a set of APIs that allow you easily and quickly write your own greeter using your choice of languages.  As long as you can interact with LightDMs GObject API, you can use any language you want.

Why Write a Greeter?

Given all that, why would you want to write a greeter?  The Unity Greeter looks pretty nice after all.  In my case, I had special requirements that necessitated a change to the UI and some extra processing for the session.  During the course of this work, I wrote a greeter based on the sample GTK greeter that comes with LightDM and then applied my modifications.  I originally wrote it in C, but then moved to Python for easier integration into my other code.  My hope is that you can re-use some of my learnings below and write your own greeter that suits the needs of your distro or project.

The Unity Greeter

Let’s Get Started

This post is designed to give you an overview of the greeter and how it works. The code for my very simple example greeter, which is available here, is commented to the point where I hope it will answer your code questions. After reading this post, walk through the code and I think the process will make sense. Once you understand this code and want to do more advanced things, you should check out the GTK example greeter, which is part of the LightDM GTK+ Greeter project.

Note: You can also get my code from launchpad here, or via bzr with bzr branch lp:~mfisch/+junk/example-greeter

How LightDM Authentication Works

The first thing to understand is the way LightDM authenticates is a multi-step process and that this process will guide how we write our greeter.

1. User enters a username and clicks the Login button.
2. LightDM passes the username to PAM and responds back to the greeter with a prompt, such as “Password:”.
3. User enters the password and the greeter passes it back to LightDM.
4. LightDM authenticates with PAM and returns success or failure back to the greeter.
5. LightDM may also return a message, such as “Login failed”, which needs to be displayed to the user.
6. If the authentication succeeds, the greeter starts a session.

Program Flow

The diagram below shows the flow between the greeter, the top row, and the LightDM backend, the bottom section. This flow assumes a successful login as we don’t show the failure case for simplicity’s sake.

Flow of the greeter during login

Looking at the Code

At this point you probably want to look at the code and compare it to the flow in the diagram. The code is heavily commented and hopefully the diagram makes it easy to follow.

Screenshots of the Example Greeter

As you can see our greeter isn’t pretty, but it works. Hopefully this gives you an idea of the prompt re-use, which is pretty standard in greeter’s I’ve seen. Making this look nice is left as an exercise to the artistic reader.

Installing Your Greeter

Your greeter will minimally include an executable and a desktop file. The executable is usually installed somewhere in the path, like /usr/bin. The .desktop file goes into /usr/share/xgreeters and tells LightDM how to launch your greeter. The name of the .desktop file defines what you need to put in LightDM’s conf file. Additionally your greeter may install a configuration file, typically in /etc/lightdm and a Gtk UI file, if you’re using GTK. Once your greeter is in place you need to tell LightDM to use it. If you’re just testing things, you can edit /etc/lightdm/lightdm.conf and set greeter-session=example-greeter. What you set here has to match the name of your desktop file as mentioned above. If you plan to do this in a package install, you need to use /usr/lib/lightdm-set-defaults, which allows you to set a few defaults via a script. In this case we’d call /usr/lib/lightdm/lightdm-set-defaults –greeter=example-greeter. You can run that script with no arguments to see it’s full capabilities and options.

Once you install your greeter, you need to restart LightDM with sudo initctl restart lightdm. If it works, great! If not, check out the Debugging section below.

Debugging

A few hints on debugging your greeter. Anything you print to stderr in your greeter ends up in /var/log/lightdm/x-0-greeter.log. I find that keeping that open in a separate console dedicated to looking at this log always helps. Since the logs get deleted every time the greeter restarts, this is your best option: sudo tail -f /var/log/lightdm/x-0-greeter.log

LightDM itself may have useful logs if your configuration is broken, they’re in /var/log/lightdm/lightdm.log, and again you need sudo to view the logs.

Debugging a greeter with a debugger can be tricky, so I’ve found that printing debug into to stderr is your best option. However, the greeter seems simple enough that this method usually works fine.

Also you almost always want to do your development of a greeter in a VM. Alternatively, you can run lightdm in test mode with lightdm –test-mode, which allows you to run lightdm from the command-line as an unprivileged user. This will start your greeter in a window inside your existing session.

Finally you may get your greeter into a state where it keeps trying to launch and keeps failing. In a VM this will result in the window popping between X and a text terminal as it keeps failing. Just wait a bit and LightDM will eventually give up on your greeter and you can drop into a console and see what’s wrong.

Documentation

LightDM’s API is defined here. Everything I used in the example greeter focuses on the Greeter section. If you want to do more advanced things like allow the user to shutdown the system from the greeter, or query the list of available sessions, please check the other sections.

I also highly recommend walking through the GTK Greeter for seeing how some of these more advanced options work.

A word of caution if you’re going to use Python, in version 1.0.6 of LightDM, the introspection bindings for Python were broken. They’re fixed on trunk, but I don’t think a fix has made it into a release yet (as of Feb 6, 2012).

Summary

This example code barely covers the surface of what you need to do in order to make a good greeter, but it should help you get started. I recommend for follow-up that you read through example the GTK example greeter, which comes in the source for the LightDM package. You can use that knowledge to make your greeter look much nicer and add features like accessibility (a11y) and options like shutdown/restart/etc from your greeter.

Launchpad is ending support for multi-tenancy for branches and bugs to ensure that projects can manage the disclosure of private information. This is a fundamental change to how launchpad permits communities to share projects. Very few users will be affected by this change, but several communities will need to change how they work with private bugs and branches.

Launchpad currently permits users to create private bugs or branches that cannot be seen by the project maintainers, or the project’s other communities. This feature permits communities and companies to develop features in secret until they are ready to share their work with the other communities. This exclusivity feature is difficult to use, difficult to maintain, and makes Launchpad slow. This feature also contradicts the project maintainer’s need to be informed and to manage the disclosure of confidential information. When multiple parties can control privacy on a project, important information is lost.

While discussing the proposed changes with Launchpad stakeholders, I was surprised that most believed that the project maintainers could see the private bugs they were reporting — they wanted to collaborate, but the subscription-as-access mechanism is faulty. There are thousands of private bugs reported in Launchpad that cannot be fixed because the person who can fix the issue is not subscribed.

A contributing reason to drop support for private branches on project you do not maintain is that the feature is fundamentally broken. Privacy is inherited from the base branch. If you cannot access the base branch your branch is stacked on, you are locked out. Project owners can, and accidentally do, lock out contributors. You cannot subscribe someone to review the security concerns in your branch if that user does not have access to the base branch. Project contributors must subscribe each other to their respective branches to collaborate on a fix or feature.

This change is a part of a super-feature called Disclosure. To ensure that confidential data is not accidentally disclosed, project maintainers will be able to view and change who has access to confidential project information. Maintainers can add users to security or proprietary policies to grant access to all the information in the respective policy. You will not need to subscribe users to individual bugs or branches, unless you want to grant an exception to a user to access one confidential piece of information.

Just a heads up, this week’s CoffeeHouseCoders (CHC) Detroit-ish will be a bit different. One of the goals of moving the location to the new Caribou was that we get access to the meeting room. This opens up the opportunity for us to have some group discussion and such around various topics. We’re going to give that a shot this week with a group viewing of YUI Theater video viewings and JavaScript discussion.

Most of us do at least some JavaScript in our work and projects so I think it’s relevant and should be fun to geek out before the holidays start up. I’ll have a little projector and speaker and with the new videos from YUIConf 2011 going up, it’ll be nice to set aside some time to catch up on some of the recorded presentations. Take a peek and set aside one or two "must watch" videos for Wed night! Not all of the videos are YUI specific, so it should be useful for all of us doing JavaScript.

Edit 2011-11-15 08:18 UTC: The problem is now fixed and we’ve re-enabled the new menu.

Edit 2011-11-11 13:42 UTC: We’ve temporarily disabled the new menu while we fix some unfortunate side effect.

We’ve just deployed a new, simplified version of the branch menu displayed on the right hand side of personal code pages (e.g. personal page for the Launchpad team). It looks like this:

Old menu

New menu

Calculating the number of branches took way too much time for people/teams with a huge number of branches (e.g. https://code.launchpad.net/~ubuntu-branches), up to the point that they were getting timeouts.

The new design, along with optimisations we’ve made to the database queries, should improve performance for everyone.

We’ve just upgraded Launchpad’s builder machines to Bazaar 2.4. Most importantly, this means that recipe builds of very large trees will work reliably, such as the daily builds of the Linaro ARM-optimized gcc. (This was bug 746822 in Launchpad).

We are going to do some further rollouts over the next week to improve supportability of recipe builds, support building non-native packages, handle muiltiarch package dependencies, improve the buildd deployment story etc.

I recently posted about Initializing page JavaScript from the JSONCache. Now I’m pleased to announce that you can also get updated copies of the IJSONRequestCache, to make it easier to update your page.

Brad Crittenden and I started work on this at the Dublin Thunderdome, and it’s finally been deployed. What this means is that for basically any page on Launchpad, you can append /++model++ to the URL, to get a fresh copy of the IJSONRequestCache. With ++model++, a change will typically require only two roundtrips; one to make a change, and one to retrieve an updated model. Future work may reduce this to a single roundtrip.

Why ++model++, not ++cache++? Cache is a really poor name for what the IJSONRequestCache is. Rather than providing fast access to whatever data has been previously retrieved, it is a complete collection of all the relevant data.

In Launchpad, the IJSonRequestCache is associated with the view, so we’re trying to rebrand it as the “view model”. This may seem strange from an MVC (Model, View Controller) perspective, but MVC can be recursive. A view may use a model to render itself.

Here’s a quick pub quiz:

Question: How do you make packages for Ubuntu?

You can choose from the following answers:

1. learn Debian packaging through hours of study and practice
2. borrow existing packaging from elsewhere, throw a couple of Bazaar branches together and let Launchpad handle the rest
3. Uruguay in both 1930 and 1950.

If you selected either of the first answers you’d be right.

Okay, so, if you want to do it for real — i.e. become an Ubuntu MOTU or otherwise create Debian-style packages from scratch — then you still need to go through the hard work.

However, for everyone else who really just needs to get something out there and working for, say, a group of beta testers, we now have Launchpad’s source package recipes.

How it works, in three steps

It’s almost ridiculously easy to set up a source package build:

And that’s it. Within a few minutes you can set up a daily build direct from your trunk or any other buildable branch in Launchpad.

Watch how it works in our screencast:

An example

Let’s say you’re the developer of a home finance application called Alvin. You track your project’s code using Git and host it on your own server. For the past couple of years Alvin has been packaged in the Ubuntu universe and your trunk has also been imported from Git to a Bazaar branch in Launchpad at lp:alvin.

Just as you’re approaching Alvin’s next release, you want to get some wider testing. In the past, you’ve published a nightly tarball and provided instructions on manual installation. That’s given you a handful of dedicated beta testers but you’re worried that you’re asking too much of people.

With Launchpad’s source package recipes, you write a short recipe that pulls in your trunk branch, adds the packaging from Alvin’s existing Ubuntu package and then builds an installable Ubuntu package in the PPA of your choice:


# bzr-builder format 0.3 deb-version 2.0beta+{revno}
lp:alvin
nest-part packaging lp:ubuntu/alvin debian debian


Paste the recipe into Launchpad and with a couple of clicks you have a daily build of your trunk, that’s published as an Ubuntu package in your PPA.

Now you can ask people to test the latest Alvin code by doing no more than adding your PPA to their system. Launchpad will build a new version of the package on each day it spots a change in your trunk (or the Ubuntu packaging). For your beta testers, any changes will show up just like any other Ubuntu update.

Simple as that!

Here’s a quick recap of how it works: you can take any buildable branch — whether hosted in directly Launchpad or imported from Git, Subversion, CVS or Bazaar hosted elsewhere — merge or nest other branches, add packaging and then leave it to Launchpad to create a daily build that it publishes in your chosen PPA.

Seeing it in action

During the beta, people added a whole range of source package recipes, with a list of more than 350 daily builds as I write this.

Daily builds on Launchpad right now include Project Neon, who have around sixty recipes providing daily builds of KDE and Amarok. There are also daily builds of the Scribus DTP app, Audacity and the scriptable screen reader Gnome Orca.

Try it out

It’s easy to get your own source package recipes and daily builds up and running.

Start at our Getting Started guide and screencast.

I’ll leave the last word to Luke Benstead, who has been using source package recipes while developing a set of game libraries:

I’ve been using LP to develop some small open source game libraries. Because there are quite a few of them, packaging them all is a pain, so the package builds have worked out pretty well for them.

Now I get nightly builds delivered to a PPA, so I know that if I fix a bug it’s reflected to all my machines. And my recipes are only a single line so they’ve been really easy to use. I’m not really sure how they could be easier.

Images:
Beer photo by dearbarbie. CC-BY-SA.
Alvin Hall photo by Phil Guest. CC-BY-SA.

Continuing with the recent improvements to the build farm – Jelmer has made another massive one.

The last major scalability problem that we had was one where the whole farm was blocked when a single builder was ready to upload a build. In the case of large packages, like the kernel, the manager process could block for over a minute while it waited for the upload processor to unpack the package and verify its contents.

Jelmer’s work has decoupled the upload processing from the build farm manager process. What happens now is that the files collected from the builder are thrown into a staging queue area and then the manager process immediately continues with polling the rest of the builders, unblocked. A cron job will then process the builder upload queue at 1 minute intervals.

You can see the dramatic effect this has had on the overall queue for the PPA builders here:

This is quite an incredible improvement as you can see! But we’re not stopping there, we’re currently doing a massive refactoring of the builder dispatching code so it’s all fully asynchronous. When this is all done we’re going to be in superb shape to support an increase in load that’s anticipated from the increasing number of people using the package recipes.

As I’ve said before, Launchpad is pretty big. Getting to know everything you can to do in Launchpad can take a while.

Of course, there are the user guide, the tour and the dev wiki even has a feature check-list. It’s still easy to miss things.

So, I’m turning the fifth day of each week into Feature Friday

I’m gonna kick off with something I use a lot and that, to be honest, is more a Bazaar feature, than a Launchpad feature:

bzr commit -m "Adds email functionality to the client, thereby obeying Zawinski's law." --fixes lp:1234

Adding --fixes lp:1234 to a commit tells Bazaar that the branch contains a fix for bug 1234 tracked in Launchpad.

The next time you push the branch to Launchpad, Launchpad will create a link between the branch and bug 1234.