Canonical Voices

Posts tagged with 'python'

Matt Fischer

Last week, I found myself having to dive into the udev code to figure out how it determines whether what is a mouse or a keyboard. To solve the problem I was working on, I ended up having to replicate some of that logic in python, which is posted at the bottom. Let me explain how it works.

Device Info in /sys

To start, let’s look in /sys and see what input devices we have. If you look in /sys/class/input on your system, you’ll see many symlinks to devices. I’ll pick /sys/class/input/event12 to look at in more detail, so cd to event12/device

mfisch@caprica:/sys/class/input/event12/device$ ls -al
total 0
drwxr-xr-x 7 root root 0 Jul 8 16:05 .
drwxr-xr-x 3 root root 0 Jul 8 16:05 ..
drwxr-xr-x 2 root root 0 Jul 8 16:05 capabilities
lrwxrwxrwx 1 root root 0 Jul 8 20:50 device -> ../../../serio1
drwxr-xr-x 3 root root 0 Jul 8 16:05 event12
drwxr-xr-x 2 root root 0 Jul 8 16:05 id
-r--r--r-- 1 root root 4096 Jul 8 20:50 modalias
drwxr-xr-x 3 root root 0 Jul 8 16:05 mouse0
-r--r--r-- 1 root root 4096 Jul 8 16:05 name
-r--r--r-- 1 root root 4096 Jul 8 20:50 phys
drwxr-xr-x 2 root root 0 Jul 8 20:50 power
-r--r--r-- 1 root root 4096 Jul 8 20:50 properties
lrwxrwxrwx 1 root root 0 Jul 8 16:05 subsystem -> ../../../../../../class/input
-rw-r--r-- 1 root root 4096 Jul 8 16:05 uevent
-r--r--r-- 1 root root 4096 Jul 8 20:50 uniq

There are some interesting things in here, but the two I’m interested in are “name” and the “capabilities” directory. Let’s look at name first.

mfisch@caprica:/sys/class/input/event12/device$ cat name
PS/2 Generic Mouse

Using Capabilities

Okay, so this looks like a mouse, but udev doesn’t use the name to figure this out, it uses the capabilities directory. Let’s look there:

mfisch@caprica:/sys/class/input/event12/device/capabilities$ ls -al
total 0
drwxr-xr-x 2 root root 0 Jul 8 16:05 .
drwxr-xr-x 6 root root 0 Jul 8 16:05 ..
-r--r--r-- 1 root root 4096 Jul 8 16:05 abs
-r--r--r-- 1 root root 4096 Jul 8 16:05 ev
-r--r--r-- 1 root root 4096 Jul 8 20:43 ff
-r--r--r-- 1 root root 4096 Jul 8 16:05 key
-r--r--r-- 1 root root 4096 Jul 8 20:39 led
-r--r--r-- 1 root root 4096 Jul 8 20:43 msc
-r--r--r-- 1 root root 4096 Jul 8 16:05 rel
-r--r--r-- 1 root root 4096 Jul 8 20:43 snd
-r--r--r-- 1 root root 4096 Jul 8 16:05 sw

The aptly named capabilities provide information to udev on what “capabilities” the device has. udev is specifically interested in the following ones: abs (Absolute axes), ev (Event types), key (Keys and Buttons), and rel (Relative axes). Let’s examine one of these to determine what data it contains:

mfisch@caprica:/sys/class/input/event12/device/capabilities$ cat ev
7

So what does 7 mean? This is a bitmask (111) who’s bits are defined in /usr/include/linux/input.h. That file has bits defined for each of the types listed in the capabilities directory, the #define prefix matches the name in the capabilities directory, so we’re looking at EV_ (Note: key is represented by defines KEY_ and BTN_, since it’s for keys and buttons).

Since this mask is 0×7 or 111, we have bits 0, 1, and 2 set and the rest are false. Judging from the #defines, the following bits are set for this device:

#define EV_SYN 0x00
#define EV_KEY 0x01
#define EV_REL 0x02

This tells us that our device gets key events, relative movement event, and synchronization events. (Figuring this out required poking around the header a bit more).

A Quick Note About the ‘key’ Capability

The entry ‘key’ is much longer than ev and requires more explanation. Let’s look at what the ‘key’ capabilities shows for my keyboard:

20000 20000000020 0 0 500f02100002 3803078f900d401 feffffdfffefffff fffffffffffffffe

So that’s much larger than ev was. It’s still a bitmask, but it’s been split into words, using the word-size sense, so 8 bytes on my system. Since this mask is little endian, the ’20000′ represents the highest set bits. This makes processing need to take place in word-size chunks, starting at the right. Another way to think of the output above is to think about it as one large mask, as long as you pad each “word” to be 16 digits, for example, what you really see above is this:

00000000000200000000020000000020000000000000000000000000000000000000500f02100002 03803078f900d401feffffdfffeffffffffffffffffffffe

To solve my problem, I split the output up on the spaces and set 64-bits at a time in my mask starting from the right-most word.

Now that we’ve read all these bitmasks, we need to do something with it.

What udev Does with this Info

If you look at the udev source code, specifically udev/udev-builtin-input_id.c, you can see how this data is used.

For mice, the code is fairly straightforward, look at the code for the test_pointers function. We really want to know where they set “is_mouse”. The code is also looking for other stuff like touchpads, so let’s ignore that. It basically boils down to this:

if EV_REL and REL_X and REL_Y and BTN_MOUSE:
you have a mouse

There’s also a similar block where you can replace REL above with ABS, which the comment claims is for VMWare’s USB mouse, but I kept it in my solution because I didn’t quite trust the comment and it seemed harmless otherwise.

For keyboards, the code is a bit more complex because it’s basically checking to see if you have a bunch of keys defined, this code is in the function test_key. then test_key looks for a keyboard it boils down to this:

If any bits are set in 'ev' and
if bits 1-31 (but not 0) are set:
you have a keyboard

Note: bits 1-31 represent the Escape key, numbers, and Q through D.

Putting it All Together

So now that we know how it works, let’s see it in action! Here it is run on my dev box:

mfisch@caprica:~/tmp/find_input$ ./find_input.py
INFO:root:/dev/input/event3 is a keyboard (AT Translated Set 2 keyboard)
INFO:root:/dev/input/event12 is a mouse (PS/2 Generic Mouse)

Here it is run on another system with more devices attached:

INFO:root: /dev/input/event3 is a keyboard (AT Translated Set 2 keyboard)
INFO:root: /dev/input/event13 is a mouse (PS/2 Generic Mouse)
INFO:root: /dev/input/event15 is a keyboard (BTC USB Keyboard)
INFO:root: /dev/input/event16 is a mouse (Primax HP Wireless Laser Mini Mouse)

My python code including unit tests! is avaialble in bzr at lp:~mfisch/+junk/find_input

Comments, bugs, and fixes all gladly accepted.

Read more
Matt Fischer

Over the past week I’ve been playing with DHCP option tags, and using them to send data to DHCP client systems, and then do something with the data. I didn’t know much about DHCP option tags before doing this, I figure DHCP just gave you an IP address and maybe a nameserver. In reality, DHCP can be customized to send a couple hundred different settings, including where the IRC or SMTP servers are, VOIP settings, and lots more. What makes it really powerful is that you can use the unassigned options to create your own settings, and then do actions on the client when they get set. In this simple example, I’m going to send and set a new “message of the day” via dhcp. The message of the day is a (usually) short text message shown when you login via the terminal and is stored in /etc/motd. If you are trying this on a production system, you should back-up your /etc/motd first.

Server Setup

On the server, after installing dhcpd, edit /etc/dhcp/dhcpd.conf, create a valid subnet and then in the global space, add your new option. Since options 151-174 are unassigned, I will use option 151. You could also use a ‘site-local’ option between 224-254.

Add the following outside any subnet sections:
option motd code 151 = text;
option motd "Happy Easter from your DHCP server team";

Client Setup

On the client, manually run dhclient -d and make sure you’re getting a valid IP from the server. Once you know that is working, edit /etc/dhcp/dhclient.conf and add two parts, first you need to name the new option and second you need to request it:

Add this to the top, you may see another option code there, add it near that:
option motd code 151 = text;

Then look for the big request block, add your new option into that block:
request subnet-mask, broadcast-address, time-offset, routers,
domain-name, domain-name-servers, domain-search, host-name,
netbios-name-servers, netbios-scope, interface-mtu,
rfc3442-classless-static-routes, ntp-servers,
dhcp6.domain-search, dhcp6.fqdn,
dhcp6.name-servers, dhcp6.sntp-servers,
motd;

Now, let’s see if we get it. Edit /etc/dhcp/dhclient-exit-hooks.d/debug and switch RUN=”no” to RUN=”yes”. Then edit the large block of variables and add your new option, motd.

Re-run dhclient -d and then check /tmp/dhclient-script.debug, do you see your motd option? If so, your setup is correct. It should look liks this:

new_motd='Happy Easter from your DHCP server team'

Doing Something When You Get an Option

dhclient defines exit-hooks, one of which is the debug script you edited above. You can create your own shell script in /etc/dhcp/dhclient-exit-hooks.d/ and follow some of the other examples in there. DHCP options come in as variables, so in this case, the script would use ${new_motd}. From what I can tell everything in here has to be a shell script, a python script I tried did not work. Here is my script to set motd:

if [ -n "${new_motd+x} ]; then
echo ${new_motd} > /etc/motd
fi

Re-run dhclient -d and you should see that the script has run and your message of the day is now a Happy Easter message.

What About NetworkManager?

If you are running NetworkManager on your system, and you probably are, then NetworkManager starts dhclient during boot and subverts the running of dhclient’s exit hook scripts. You can use NetworkManager’s own version of “exit hooks” which live in /etc/NetworkManager/dispatcher.d to respond to DHCP events. Scripts in that directory can be written directly in python. The scripts run from NetworkManager’s dispatcher use slightly different environment variables for DHCP options, they’re in all caps and are pre-pended with DHCP4_ (and presumably DHCP6_ if you’re serving IPv6 addresses).

if [ -n "${DHCP4_MOTD+x} ]; then
echo ${DHCP4_MOTD} > /etc/motd
fi

See the man NetworkManager(8) for more details on actions that your scripts can respond to.

Options as Expressions

There are more advanced things you can do. For example, you set a setting on the server based on an expression, for example:

if option dhcp-user-class = "fortcollins-office" {
option smtp-server "smtp.ftc-office.net";
}

You then configure the client to send up a dhcp-user-class of “fortcollins-office” for all your PCs in Fort Collins.

Using my example, you could change the holiday in the message based on the date that the lease was given out.

See man dhcp-eval(5) for more info on expressions.

Encapsulated Options

Another thing you can do is vendor encapsulated options. Vendor encapsulated options take a bunch of vendor specific info and encapsulate it into one option. This means that you can pack a bunch of data into one option. I am not sure if all dhcpd servers support this construct however.

Testing This
If you want to try DHCP options without messing up your network, the quickest way is to create two virtual machines, a client and a server, and give them each two NICs. One NIC on each machine should be on a private network, and the server should have a static IP on this private network NIC and serve DHCP on that interface. The other non-private NIC is used by you to copy files to and from your virtual machines, but is otherwise not specifically required. I did this setup using Ubuntu 12.04 precise and qemu and it works great.

Conclusion
Customized DHCP options let you transmit data to your DHCP clients and then write scripts to act on that data. Although my example is contrived, you could use DHCP to transmit useful system settings and then configure them via script.

References

Read more
Matt Fischer

Fun With Lenses and Scopes

I’ve been fooling around with lenses over the past couple of weeks in my spare time. My idea was to write a stock ticker lens/scope. I wrote a quick backend in python in an hour or so, but the lens/scope proved more difficult. The main issue was that as I made progress, Michael Hall announced Singlet and then Singlet 0.2. These were compelling enough to drop my work and start over. Initially I had some issues getting singlet based scopes to work (some of which I fixed), but then I got better at debugging them and figuring out file placement. (Of course after I was well into this project, Michael Hall announced another easier way of doing lenses and scopes!)

Announcing the Stock Quote Lens

The lens I’m pushing today is a stock quote & news lens and scope combo. It relies on Michael Hall’s singlet 0.2 and will only work for precise.

Here are some screenshots:
multi-symbols one_symbol

As you can see it has two modes. If you enter one symbol, you get a quote and news. If you enter more than one, you get quotes. I could not figure out a good wait to show news for multiple symbols in the constraints of a lens. However, all the icons are clickable and direct you to a page with more quote info and news for each ticker, so use that method to get more info. The icons themselves are static, they’re not real charts.

For now, this is US markets only and it’s only in English. Given the architecture, you could write your own scope back-end for this and populate results for the stock markets in whatever country you’re in.

Where to Get It

I pushed the code up to launchpad today. You can package it yourself, but I have not published the package. I will be doing so next week when I can get into the scopes packagers PPA. The package is published in the Scopes Packagers PPA. You can install it by doing:

sudo add-apt-repository ppa:scopes-packagers/ppa
sudo apt-get update
sudo apt-get install unity-stock-ticker-lens

You can also pull the code from bzr directly:

bzr branch lp:~mfisch/onehundredscopes/unity-stock-ticker-lens

Some tricks to debugging Unity scopes & lenses

These are some methods I used during development.

  1. Remember to run setsid unity and watch the terminal output. This is where you’ll see all your missing icons and broken .scope and .lens issues. You’ll need to re-run this when you change a .scope or .lens file IIRC.
  2. When debugging, fire up your scope in a terminal, then your lens in another terminal. Any errors or prints will dump to the terminal. Seeing a dumb error like a python indent error here is much more helpful than trying to use the scope and wondering why you have no output.
  3. If you want to run the scope/lens manually, be sure to kill the old processes first.
  4. Check your paths and dbus info. If the stuff runs fine manually, but won’t autostart, then something is wrong in your dbus service files.

PS – Thanks to Bob Davis for help with sourcing those icons.

Read more