Canonical Voices

Posts tagged with 'debian'

pitti

PostgreSQL 9.1 has had its first release candidate out for some two weeks without major problem reports, so it’s time to promote this more heavily. If you use PostgreSQL, now is the time to try it out and report problems.

We always strive to minimize the number of major versions which we have to support. They not only mean more maintenance for developers, but also more upgrade cycles for the users.

9.0 has not been in any stable Debian or Ubuntu release, and 9.1 final will be released soon. So we recently updated the current Ubuntu development release for 11.10 (“oneiric”) to 9.1. In Debian, the migration from 8.4/9.0 to 9.1 is making good progress, and there is not much which is left until postgresql-9.0 can be removed.

Consequently, I also removed 9.0 from my PostgreSQL backports PPA, as there is nothing any more to backport it from. However, that mostly means that people will now set up installations with 9.1 instead of 9.0, and won’t magically make your already installed 9.0 packages go away. They will just be marked as obsolete in the postgresql-common debconf note.

If you want to build future 9.0 packages yourself, you can do this based on the current branch: bzr branch lp:~pitti/postgresql/debian-9.0, get a the new upstream tarball, name it accordingly, add a new changelog with a new upstream version number, and run bzr bd to build the package (you need to install the bzr-builddeb package for this).

Update 2011-09-09: As I got a ton of pleas to continue the 9.0 backports for a couple of months, and to keep it in Debian unstable for a while longer, I put them back now. I also updated the removal request in Debian to point out that I’m mainly interested in getting 9.0 out of testing. I don’t mind much maintaining it for a couple of more months in unstable. My dear, I had no idea that my backports PPA was that popular!

Read more
Barry Warsaw

sbuild is an excellent tool for locally building Ubuntu and Debian packages.  It fits into roughly the same problem space as the more popular pbuilder, but for many reasons, I prefer sbuild.  It's based on schroot to create chroot environments for any distribution and version you might want.  For example, I have chroots for Ubuntu Oneiric, Natty, Maverick, and Lucid, Debian Sid, Wheezy, and Squeeze, for both i386 and amd64.  It uses an overlay filesystem so you can easily set up the primary snapshot with whatever packages or prerequisites you want, and the individual builds will create a new session with an overlaid temporary filesystem on top of that, so the build results will not affect your primary snapshot.  sbuild can also be configured to save the session depending on the success or failure of your build, which is fantastic for debugging build failures.  I've been told that Launchpad's build farm uses a customized version of sbuild, and in my experience, if you can get a package to build locally with sbuild, it will build fine in the main archive or a PPA.

Right out of the box, sbuild will work great for individual package builds, with very little configuration or setup.  The Ubuntu Security Team's wiki page has some excellent instructions for getting started (you can stop reading when you get to UMT :).

One thing that sbuild doesn't do very well though, is help you build a stack of packages.  By that I mean, when you have a new package that itself has new dependencies, you need to build those dependencies first, and then build your new package based on those dependencies.  Here's an example.

I'm working on bug 832864 and I wanted to see if I could build the newer Debian Sid version of the PySide package.  However, this requires newer apiextractor, generatorrunner, and shiboken packages (and technically speaking, debhelper too, but I'm working around that), so you have to arrange for the chroot to have those newer packages when it builds PySide, rather than the ones in the Oneiric archive.  This is something that PPAs do very nicely, because when you build a package in your PPA, it will use the other packages in that PPA as dependencies before it uses the standard archive.  The problem with PPAs though is that when the Launchpad build farm is overloaded, you might have to wait several hours for your build.  Those long turnarounds don't help productivity much. ;)

What I wanted was something like the PPA dependencies, but with the speed and responsiveness of a local build.  After reading the sbuild manpage, and "suffering" through a scan of its source code (sbuild is written in Perl :), I found that this wasn't really supported by sbuild.  However, sbuild does have hooks that can run at various times during the build, which seemed promising.  My colleague Kees Cook was a contributor to sbuild, so a quick IRC chat indicated that most people create a local repository, populating it with the dependencies as you build them.  Of course, I want to automate that as much as possible.  The requisite googling found a few hints here and there, but nothing to pull it all together.  With some willful hackery, I managed to get it working.

Rather than post some code that will almost immediately go out of date, let me point you to the bzr repository where you can find the code.  There are two scripts: prep.sh and scan.sh, along with a snippet for your ~/.sbuildrc file to make it even easier.  sbuild will call scan.sh first, but here's the important part: it calls that outside the chroot, as you (not root). You'll probably want to change $where though; this is where you drop the .deb and .dsc files for the dependencies.  Note too, that you'll need to add an entry to your /etc/schroot/default/fstab file so that your outside-the-chroot repo directory gets mapped to /repo inside the chroot.  For example:

# Expose local apt repository to the chroot
/home/barry/ubuntu/repo    /repo    none   rw,bind  0 0
An apt repository needs a Packages and Packages.gz file for binary packages, and a Sources and Sources.gz file for the source packages.  Secure APT also requires a Release and Release.gpg file signed with a known key.  The scan.sh file sets all this up, using the apt-ftparchive command.  The first apt-ftparchive call creates the Sources and Sources.gz file.  It scans all your .dsc files and generates the proper entries, then creates a compressed copy, which is what apt actually "downloads".  The tricky thing here is that without changing directories before calling apt-ftparchive, your outside-the-chroot paths will leak into this file, in the form of Directory: headers in Sources.gz.  Because that path won't generally be available inside the chroot, we have to get rid of those headers.  I'm sure there's an apt-ftparchive option to do this, but I couldn't find it.  I accidentally discovered that cd'ing to the directory with the .dsc files was enough to trick the command into omitting the Directory: headers.

The second call to apt-ftparchive creates the Packages and Packages.gz files.  As with the source files, we get some outside-the-chroot paths leaking in, this time as path prefixes to the Filename: header value.  Again, we have to get rid of these prefixes, but cd'ing to the directory with the .deb files doesn't do the trick.  No doubt there's some apt-ftparchive magical option for this too, but sed'ing out the paths works well enough.

The third apt-ftparchive file creates the Release file.  I shameless stole this from the security team's update_repo script.  The tricky part here is getting Release signed with a gpg key that will be available to apt inside the chroot.  sbuild comes with its own signing key, so all you have to do is specify its public and private keys when signing the file.  However, because the public file from
/var/lib/sbuild/apt-keys/sbuild-key.pub
won't be available inside the chroot, the script copies it to what will be /repo inside the chroot.  You'll see later how this comes into play.

Okay, so now we have the repository set up well enough for sbuild to carry on.  Later, before the build commences, sbuild will call prep.sh, but this script gets called inside the chroot, as the root user.  Of course, at this point /repo is mounted in the chroot too.  All prep.sh needs to do is add a sources.list.d entry so apt can find your local repository, and it needs to add the public key of the sbuild signing key pair to apt's keyring.  After it does this, it needs to do one more apt-get update.  It's useful to know that at the point when sbuild calls prep.sh, it's already done one apt-get update, so this does add a duplicate step, but at least we're fortunate enough that prep.sh gets called before sbuild installs all the build dependencies.  Once prep.sh is run, the chroot will have your overriding dependent packages, and will proceed with a normal build.

Simple, huh?

Besides getting rid of the hackery mentioned above, there are a few things that could be done better:
  • Different /repo mounts for each different chroot
  • A command line switch to disable the /repo
  • Automatically placing .debs into the outside-the-chroot repo directory

Anyway, it all seems to hang together.  Please let me know what you think, and if you find better workarounds for the icky hacks.
 

Read more
Timo Jyrinki

Debian MeeGo mixup. MeeGo screenshot CC-BY wiki.meego.com, Debian screenshot by me
FreeSmartphone.Org (FSO), Openmoko, Debian's FSO group, SHR, QtMoko et cetera are a few of the community based intertwined projects to bring free software to smartphones. They have a relatively long and colorful history of doing this, and have nowadays been approaching multiple target devices despite limited resources and for example the losing of Openmoko Inc. in 2009. I've been using Debian on my Neo FreeRunner phone for over two years now, and over three years of FreeRunner use altogether. FSO2, the next generation freesmartphone.org stack, is finally coming into Debian now, extending the basic phone support besides Openmoko phones to eg. Palm Pre, Nexus One, Nokia N900 and a few HTC phones. It needs a lot of tweaking and eg. a proper kernel, but still.

Four years after beginning of sales of the first Openmoko device (Neo1973), we're still in the pioneering phase of free distributions for mobile phones. There is no "Ubuntu for phones" so to speak, not for even a selected models. Meanwhile, like so often in the wide world, competing free software approaches have arrived. Android is the obvious one, and has seen a port to Neo Freerunner among else. Android is not as open a project as one could like, and replaces everything we've known with its own code, but nevertheless it requires to be noted and is completely usable in its free software form. But since there are limitations to its approach, and since it's more of an own separate world from the rest of Linux distributions, it is not as interesting to me as the others in the long run, at least in its current shape.

A more similar competitor to FSO and distributions using FSO is MeeGo and its middleware, and to be more precise so far specifically Nokia's efforts on it. Obviously there is the strong competitor for the best general population smartphone of the year, the Qt based Nokia N9, but its default software is more of a proprietary thing even though it has a neatly rock solid free software foundation with separate free/non-free repositories and all that. Nice and great for the (GNU/)Linux in general, but the Harmattan software is not exactly on topic for this blog post. It's however in my opinion the best marketing to vendors around the world GNU/Linux + Qt can get as Android Linux has been taking most of the limelight. But meanwhile, with significantly smaller focus and resources, Nokia has also been sponsoring to try to create a truly community based mobile phone software stack at the MeeGo upstream, nowadays called "MeeGo Community Edition" or MeeGo CE for short. It co-operates with the MeeGo Handset target of MeeGo (and Intel) that hasn't got actual target consumer hardware at the moment, although that might change soon (?), but has been doing some nice applications recently. CE has the target hardware and additional device specific and non-specific software cooking. It used to target Nokia N900 only, but nowadays the project has added N950 (the for-developers-only phone) and N9 to the targets, and it is starting to seem there should be no showstoppers to bring MeeGo CE (or other distributions later on) to them, despite some earlier doubts. A few needed bits to void the warranty we all want to do are still missing, though, but coming. After that it's just developing free software. Usual caveats about specifics of a few kernel drivers apply, as the devices were not designed with the sole purpose of free software drivers in mind. Hopefully mobile 3D gets into a better shape in the coming years, but that's another story.

MeeGo (CE) smartphone middleware, of course, shares nothing with FSO *). While FSO is a "handle everything" in itself with its wide variety of daemons, MeeGo consists of more separate software like Intel's and Nokia's oFono for modem support. FSO demo UIs and SHR UIs have traditionally focused on using Enlightenment 17 for its suitability to low power machines, while in MeeGo everything is written in Qt. As Qt/QML is becoming faster and faster, and it's very powerful to write, there might be quite a bit of useful software emerging from there also for other distributions besides MeeGo itself. Actually, there is already a Debian MeeGo stack maintainer group available, although it hasn't yet focused on the UIs as far as I can see (if I'll have free time I'll join the effort and see for myself in more detail). There is also the QtMoko distribution, based on the original, canceled Trolltech/Nokia Qt Extended (Qtopia) project, but ported to newer Qt:s and put on top of Debian.

*) Although, correct me if I'm wrong and I might be, the Nokia N900 isi modem driver in FSO was ported/learned from oFono.

MeeGo CE is not only a project to bring a proper MeeGo distribution to a few smartphones, but also to shake out bugs in the MeeGo project's contributions and community processes. It is acting as a completely open MeeGo product contributor, and investigating how things should be optimally done so that everything gets integrated into the MeeGo properly, and that proper MeeGo software releases can be done for the target devices. Therefore it's also an important project for the vitality of the whole MeeGo.

MeeGo CE has so far had a whole team in Nokia working on it, but for obvious strategy related reasons community cannot rely on it lasting forever. The real hurdle is for the wider free smartphones community to be ready for really embracing a project that is already a community project but to outsiders might seem like a company project. I believe it's never easy to grow a volunteer community if the starting setup is a paid company team, but it mainly requires a) the interested people and b) the few smart ones to take control and communication responsibility so that it's not anymore seen as a company project. MeeGo CE never was a traditional company project, everything being done in the open and together with volunteers, but I just know how people perceive these things by default. People tend to assume stuff as someone else's responsibility

Whatever happens, I hope there are enough people interested in free software for mobile phones to carry on all these different approaches to the software needed. I hope MeeGo / MeeGo CE will have a great future, and I hope that both the middleware like FSO and MeeGo's components, and the UIs like FSO, SHR, QtMoko and MeeGo Handheld UIs continue to develop. I also hope other distributions like Debian will gather a strong suite of packaged software for smartphones. I know I have had some hard time to find suitable apps for my phone at times.

For those interested about how I use Debian as my mobile phone OS, see http://wiki.openmoko.org/wiki/User:TimoJyrinki

Read more
Barry Warsaw

So, yesterday (June 21, 2011), six talented and motivated Python hackers from the Washington DC area met at Panera Bread in downtown Silver Spring, Maryland to sprint on PEP 382. This is a Python Enhancement Proposal to introduce a better way for handling namespace packages, and our intent is to get this feature landed in Python 3.3. Here then is a summary, from my own spotty notes and memory, of how the sprint went.

First, just a brief outline of what the PEP does. For more details please read the PEP itself, or join the newly resurrected import-sig for more discussions. The PEP has two main purposes. First, it fixes the problem of which package owns a namespace's __init__.py file, e.g. zope/__init__.py for all the Zope packages. In essence, it eliminate the need for these by introducing a new variant of .pth files to define a namespace package. Thus, the zope.interfaces package would own zope/zope-interfaces.pth and the zope.components package would own zope/zope-components.pth.  The presence of either .pth file is enough to define the namespace package.  There's no ambiguity or collision with these files the way there is for zope/__init__.py.  This aspect will be very beneficial for Debian and Ubuntu.

Second, the PEP defines the one official way of defining namespace packages, rather than the multitude of ad-hoc ways currently in use.  With the pre-PEP 382 way, it was easy to get the details subtly wrong, and unless all subpackages cooperated correctly, the packages would be broken.  Now, all you do is put a * in the .pth file and you're done.

Sounds easy, right?  Well, Python's import machinery is pretty complex, and there are actually two parallel implementations of it in Python 3.3, so gaining traction on this PEP has been a hard slog.  Not only that, but the PEP has implications for all the packaging tools out there, and changes the API requirements for PEP 302 loaders.  It doesn't help that import.c (the primary implementation of the import machinery) has loads of crud that predates PEP 302.

On the plus side, Martin von Loewis (the PEP author) is one of the smartest Python developers around, and he's done a very good first cut of an implementation in his feature branch, so there's a great place to start.

Eric Smith (who is the 382 BDFOP, or benevolent dictator for one pep), Jason Coombs, and I  met once before to sprint on PEP 382, and we came away with more questions than answers.  Eric, Jason, and I live near each other so it's really great to meet up with people for some face-to-face hacking.  This time, we made a wider announcement, on social media and the BACON-PIG mailing list, and were joined by three other local Python developers.  The PSF graciously agreed to sponsor us, and while we couldn't get our first, second, third, or fourth choices of venues, we did manage to score some prime real-estate and free wifi at Panera.

So, what did we accomplish?  Both a lot, and a little.  Despite working from about 4pm until closing, we didn't commit much more than a few bug fixes (e.g. an uninitialized variable that was crashing the tests on Fedora), a build fix for Windows, and a few other minor things.  However, we did come away with a much better understanding of the existing code, and a plan of action to continue the work online.  All the gory details are in the wiki page that I created.


One very important thing we did was to review the existing test suite for coverage of the PEP specifications.  We identified a number of holes in the existing test suite, and we'll work on adding tests for these.  We also recognized that importlib (the pure-Python re-implementation of the import machinery) wasn't covered at all in the existing PEP 382 tests, so Michael worked on that.  Not surprisingly, once that was enabled, the tests failed, since importlib has not yet been modified to support PEP 382.


We also came up with a number of questions where we think the PEP needs clarification.  We'll start discussion about these on the relevant mailing lists.


Finally, Eric brought up a very interesting proposal.  We all observed how difficult it is to make progress on this PEP, and Eric commented on how there's a lot of historical cruft in import.c, much of which predates PEP 302.  That PEP defines an API for extending the import machinery with new loaders and finders.  Eric proposed that we could simplify import.c by removing all the bits that could be re-implemented as PEP 302 loaders, specifically the import-from-filesystem stuff.  The other neat thing is that the loaders could probably be implemented in pure-Python without much of a performance hit, since we surmise that the stat calls dominate. If that's true, then we'd be able to refactor importlib to share a lot of code with the built-in C import machinery.  This could have the potential to greatly simplify import.c so that it contains just the PEP 302 machinery, with some bootstrapping code.  It may even be possible to move most of the PEP 382 implementation into the loaders.  At the sprint we did a quick experiment with zipping up the standard library and it looked promising, so Eric's going to take a crack at this.


This is briefly what we accomplished at the sprint.  I hope we'll continue the enthusiasm online, and if you want to join us, please do subscribe to the import-sig!

Read more
Timo Jyrinki

Just reminding myself (and you) of the following pretty important tip that I got 2.5 years ago and already forgot: use alt=rss in blogger feeds when giving the feed to planetplanet. Planetplanet treats the "updated" field of Atom feeds similar to "published".

So hopefully planet Debian fixed now as well.

Read more
pitti

Hot on the heels of the Announcement of the second 9.1 Beta release there are now packages for it in Debian experimental and backports for Ubuntu 10.04 LTS, 10.10. and 11.04 in my PostgreSQL backports for stable Ubuntu releases PPA.

Warning for upgrades from Beta 1: The on-disk database format changed since Beta-1. So if you already have the beta-1 packages installed, you need to pg_dumpall your 9.1 clusters (if you still need them), and pg_dropcluster all 9.1 clusters before the upgrade. I added a check to the pre-install script to make the postgresql-9.1 package fail early to upgrade if you still have existing 9.1 clusters to avoid data loss.

Read more
pitti

Two weeks ago, PostgreSQL announced the first beta version of the new major 9.1 version, with a lot of anticipated new features like synchronous replication or better support for multilingual databases. Please see the release announcement for details.

Due to my recent moving and the Ubuntu Developer Summit it took me a bit to package them for Debian and Ubuntu, but here they are at last. I uploaded postgresql-9.1 to Debian experimental; currently they are sitting in the NEW queue, but I’m sure our restless Debian archive admins will get to it in a few days. I also provided builds for Ubuntu 10.04 LTS, 10.10. and 11.04 in my PostgreSQL backports for stable Ubuntu releases PPA.

I provided full postgresql-common integration, i. e. you can use all the usual tools like pg_createcluster, pg_upgradecluster etc. to install 9.1 side by side with your 8.4/9.0 instances, attempt an upgrade of your existing instances to 9.1 without endangering the running clusters, etc. Fortunately this time there were no deprecated configuration options, so pg_upgradecluster does not actually have to touch your postgresql.conf for the 9.0 ?9.1 upgrade.

They pass upstream’s and postgresql-common’s integration test suite, so should be reasonably working. But please let me know about everything that doesn’t, so that we can get them in perfect shape in time for the final release.

I anticipate that 9.1 will be the default (and only supported) version in the next Debian release (wheezy), and will most likely be the one shipped in the next Ubuntu LTS (in 12.04). It might be that the next Ubuntu release 11.10 will still ship with 9.0, but that pretty much depends on how many extensions get ported to 9.1 by feature freeze.

Read more
Timo Jyrinki

It was good that I didn't hold up my blog post in November until the videos from FSCONS 2010 (Free Society Conference and Nordic Summit) are out, but now they finally are:


Timo Jyrinki - Tuning an old but free phone (pt 1/2)
Timo Jyrinki - Tuning an old but free phone (pt 2/2)

Definitely see also all videos and since Vimeo doesn't work in Gnash, use a script to download.

ps. As a related item to the talk's future oriented aspects, while waiting for GTA04A3 boards to arrive, GTA04A2 has been patched to run Debian/LXDE.

Read more
pitti

As a followup action to my recent Talk about PyGI I now re-used my notes to provide some real wiki documentation.

It would be great if you could add package name info for Fedora/SUSE/etc., and perhaps add more example links for porting different kinds of software! Please also let me know if you have suggestions how to improve the structure of the page.

Read more
pitti

On next Monday this cycle’s Ubuntu Application Developer Week classes will start.

The topic that kept me busy most in this cycle was Python gobject-introspection, and porting pygtk2 apps to PyGI (see my initial steps and my report from the PyGI hackfest.)

To spread the love, there will be two talks about this next week: On Monday 17:00 UTC the very Tomeu Vizoso himself will explain what gobject-introspection (“GI”) is, why we need it, and how library developers use it to ship a good and useful GI binding (“typelib”) for application developers. I will then follow up on Tuesday 16:00 UTC about the app developer side, in particular how to use the GI typelibs in Python, and how to port PyGTK2 applications to PyGI.

For the most part these sessions are distribution neutral (we don’t have any special sauce for this in Debian/Ubuntu, it all happened right upstream :-) ); only a very small fraction of it (where I explain package names, etc.) will be specific to Debian/Ubuntu, but shouldn’t be hard to apply to other distributions as well.

So please feel invited to join, and bombard us with questions!

Read more

Photo



Read more
Timo Jyrinki

The 2008 product release, Openmoko Inc's – now in other business areas than mobile phones – Neo FreeRunner is finally starting to get a ”spiritual” successor, in form of the GTA04 project (not to forget about gta02-core project, either). Yes, the ”even schematics and cover CAD files are CC-BY-SA” free mobile phone is back... or at least if the German company Golden Delicious finishes what's it has been doing lately. Of course, the original FreeRunner is also still on sale as an improved version.

Since I have two FreeRunners, I look forward to replacing one of those with the new innards while keeping the other one as a daily phone while kernel and modem drivers get ready with the new platform. With a newer platform with ARMv7 instruction set, there would be easily also other ”traditional” distributions choices besides Debian to choose from, like Ubuntu or MeeGo. Both have oFono (+ telepathy etc.) packaged up, while Ubuntu also has a lot of the FreeSmartphone.Org (FSO) stack that has come to Ubuntu from Debian's pkg-fso team that I'm part of. Both software stacks are capable of getting updated with new modem drivers, although I think currently FSO has more daily / only phone users than oFono at this point of time still (I haven't yet heard of many Nokia N900 users using only free software distribution while using the phone functionality of such a distro as the one to count on).

So, a quote from today's Openmoko Community Updates:

---cut---

GTA04

GTA04 is a project by the long time distributor and hw developer, German company Golden Delicious. The name is loaned from Openmoko project because of the spiritual continuation - GTA01 was the codename for Neo1973, GTA02 was the Neo FreeRunner, and GTA03 was the canceled successor product. Besides offering improved versions of Neo FreeRunner (better battery life, better audio output), they've a complete replacement board planned to fit an existing Neo FreeRunner case and use the existing display.

The key details of GTA04 include among else:

  • OMAP3530 ARMv7 CPU
  • UMTS/3G (HSPA)
  • USB 2.0 OTG
  • WLAN, BT, FM transceiver
  • Barometric Altimeter, Accelerometer, Compass, Gyroscope
  • Optionally camera

Find your GTA04 information at the following addresses:

Latest news:

Visit the FOSDEM in Brussels, Belgium next weekend (5th/6th of February, 2011) to see GTA04 in action and discuss about it! See http://fosdem.org/2011/ , http://wiki.openmoko.org/wiki/FOSDEM_2011 , http://lists.openmoko.org/pipermail/community/2010-December/063899.html

---cut---

Read more
pitti

(Update: Link to Tomeu’s blog post, repost for planet.gnome.org)

Last week I was in Prague to attend the GNOME/Python 2011 Hackfest for gobject-introspection, to which Tomeu Vizoso kindly invited me after I started working with PyGI some months ago. It happened at a place called brmlab which was quite the right environment for a bunch of 9 hackers: Some comfy couches and chairs, soldering irons, lots of old TV tubes, chips, and other electronics, a big Pirate flag, really good Wifi, plenty of Club Mate and Coke supplies, and not putting unnecessary effort into mundane things like wallpapers.

It was really nice to get to know the upstream experts John (J5) Palmieri and Tomeu Vizoso (check out Tomeu’s blog post for his summary and some really nice photos). When sitting together in a room, fully focussing on this area for a full week, it’s so much easier to just ask them about something and getting things done and into upstream than on IRC or bugzilla, where you don’t know each other personally. I certainly learned a lot this week (and not only how great Czech beer tastes :-) )!

So what did I do?

Application porting

After already having ported four Ubuntu PyGTK applications to GI before (apport, jockey, aptdaemon, and language-selector),
my main goal and occupation during this week was to start porting a bigger PyGTK application. I picked system-config-printer, as it’s two magnitudes bigger than the previous projects, exercises quite a lot more of the GTK GI bindings, and thus also exposes a lot more GTK annotation and pygobject bugs. This resulted in a new pygi s-c-p branch which has the first 100 rounds of “test, break, fix” iterations. It now at least starts, and you can do a number of things with it, but a lot of functionality is still broken.

As a kind of “finger exercise” and also to check for how well pygi-convert works for small projects now, I also ported computer-janitor. This went really well (I had it working after about 30 minutes), and also led me to finally fixing the unicode vs. str mess for GtkTreeView that you got so far with Python 2.x.

pygobject and GTK fixes

Porting system-config-printer and computer-janitor uncovered a lot of opportunities to improve pygi-convert.sh, a big “perl -e” kind of script to do the mechanical grunt work of the porting process. It doesn’t fix up changed signatures (such as adding missing arguments which were default arguments in PyGTK, or the ubiquitous “user_data” argument for signal handlers), but at least it gets a lot of namespaces, method, and constant names right.

I also fixed three annotation fixes in GTK+. We also collaboratively reviewed and tested Pavel’s annotation branch which helped to fix tons of problems, especially after Steve Frécinaux’s excellent reference leak fix, so if you play around with current pygobject git head, you really also have to use the current GTK+ git head.

Speaking of which, if you want to port applications and always stay on top of the pygobject/GTK development without having to clutter your package system with “make install”s of those, it works very well to have this in your ~/.bashrc:

export GI_TYPELIB_PATH=$HOME/projects/gtk/gtk:$HOME/projects/gtk/gdk
export PYTHONPATH=$HOME/projects/pygobject

Better GVariant/GDBus support

The GNOME world is moving from the old dbus-glib python bindings to GDBus, which is integrated into GLib. However, dbus-python exposed a really nice and convenient way of doing D-Bus calls, while using GDBus from Python was hideously complicated, especially for nontrivial arguments with empty or nested arrays:

from gi.repository import Gio, GLib
from gi._gi import variant_type_from_string

d = Gio.bus_get_sync(Gio.BusType.SESSION, None)
notify = Gio.DBusProxy.new_sync(d, 0, None, 'org.freedesktop.Notifications',
    '/org/freedesktop/Notifications', 'org.freedesktop.Notifications', None)

vb = GLib.VariantBuilder()
vb.init(variant_type_from_string('r'))
vb.add_value(GLib.Variant('s', 'test'))
vb.add_value(GLib.Variant('u', 1))
vb.add_value(GLib.Variant('s', 'gtk-ok'))
vb.add_value(GLib.Variant('s', 'Hello World!'))
vb.add_value(GLib.Variant('s', 'Subtext'))
# add an empty array
eavb = GLib.VariantBuilder()
eavb.init(variant_type_from_string('as'))
vb.add_value(eavb.end())
# add an empty dict
eavb = GLib.VariantBuilder()
eavb.init(variant_type_from_string('a{sv}'))
vb.add_value(eavb.end())
vb.add_value(GLib.Variant('i', 10000))
args = vb.end()

result = notify.call_sync('Notify', args, 0, -1, None)
id = result.get_child_value(0).get_uint32()
print id

So I went to making the GLib.Variant constructor work properly with nested types and boxed variants, adding Pythonic GVariant iterators and indexing (so that you can treat GVariant dictionaries/arrays/tuples just like their Python equivalents), and finally a Variant.unpack() method for converting the return value of a D-Bus call back into a native Python data type. This looks a lot friendlier now:

from gi.repository import Gio, GLib

d = Gio.bus_get_sync(Gio.BusType.SESSION, None)
notify = Gio.DBusProxy.new_sync(d, 0, None, 'org.freedesktop.Notifications',
    '/org/freedesktop/Notifications', 'org.freedesktop.Notifications', None)

args = GLib.Variant('(susssasa{sv}i)', ('test', 1, 'gtk-ok', 'Hello World!',
    'Subtext', [], {}, 10000))
result = notify.call_sync('Notify', args, 0, -1, None)
id = result.unpack()[0]
print id

I also prepared another patch in GNOME#640181 which will provide the icing on the cake, i. e. handle the variant building/unpacking transparently and make the explicit call_sync() unnecessary:

from gi.repository import Gio, GLib

d = Gio.bus_get_sync(Gio.BusType.SESSION, None)
notify = Gio.DBusProxy.new_sync(d, 0, None, 'org.freedesktop.Notifications',
    '/org/freedesktop/Notifications', 'org.freedesktop.Notifications', None)

result = notify.Notify('(susssasa{sv}i)', 'test', 1, 'gtk-ok', 'Hello World!',
            'Subtext', [], {}, 10000)
print result[0]

I hope that I can get this reviewed and land this soon.

Thanks to our sponsors!

Many thanks to the GNOME Foundation and Collabora for sponsoring this event!

Read more
Timo Jyrinki

Just a note that the slides are available (non-slideshare link) for my presentation ”Tuning an old but free phone” (description) that I held in the tremendously great event FSCONS 2010. It could be described as a smaller scale FOSDEM, but that would be actually down-playing it since the free software effects on society are something that I've actually never seen elsewhere on such a scale. My talk was among the purely technical ones, though.

I was planning to hold on with this blog post until the recorded videos arrive, but since it seems it might not be during this year I will just post this now that slides are available.

I've shared a few photos as well at Flickr...


Keynote: Karin Kosina, The Inanna Project. A tech + art workshop for female artists in Damascus, Syria. An experiment in art, technology, and the transformative power of Free Hardware and Software.


Erik de Bruijn, The Future of RepRap, a self-replicating open source 3D printer that fabricates arbitrary objects including parts of itself.


Social event at the Berg 211.


Malin Nilsson on Gender, class and global flows. Using free software to fuel a revolution in home based industrial work.



Keynote: Glyn Moody, Ethics of Intellectual Monopolies.


Keynote: Glyn Moody, Ethics of Intellectual Monopolies (audience).

A few summaries available on a Qaiku seminar channel.

Read more
Timo Jyrinki

(many exciting news from this old project/community working on the "Free Your Phone" idea, so as an publisher of the update sharing via blog as well. FreeRunner still available at eg. http://www.pulster.de/engl/index.html and the improved "+" versions at http://www.handheld-linux.com/wiki.php?page=Neo%20Freerunner, if an old/slow hardware is enough for you when you know you can tweak it to your liking)
Period 2010-09-01 to 2010-10-31

Distributions

Debian GNU/Linux

Debian is a universal operating system used on many embedded devices, servers and home computers. Using Debian on the FreeRunner gives access to the huge army of software packaged in the Debian repositories, already compiled for the Neo's ARM(v4) processor. Moreover, one can build one's own source files for programs without having to learn the OpenEmbedded way. For an existing Debian/Ubuntu user, choosing Debian for Neo FreeRunner makes phone a very familiar, trustworthy and flexible place to hack in.

General news:



Codename: 'sid'
Homepage: http://wiki.debian.org/DebianOnFreeRunner
Image: http://wiki.openmoko.org/wiki/Debian

Hardware Works
Neo 1973 yes
FreeRunner yes
HTC-Dream yes
Other yes


Applications

New Applications

aTrack 0.8

APRS tracker and communicator for mobile devices. It turns your Neo into bidirectional APRS unit and besides others it allows you to track your position, do text messaging, object creation or display stations around.


Homepage: http://atrack.googlecode.com/
Package: [1]
Tested on: SHR-Unstable


Application Updates

Gamerunner GnuBoy 0.8

A gameboy emulator which runs very nice, even with sound (sometimes it freezes, then you have to press the A button and everything is ok). You need the gamerunner distro or a gamepad to use it. Using frameskip to run smooth at 320*240 pixel. In second controll mode in gamerunner you can use savestates by pressing top right corne to save and left lower corner to load. Select is right lower corner.


Homepage: http://jlime.com/forum/viewtopic.php?f=117&t=3005
Package: [nopacket no packet sorry]
Tested on: Gamerunner, QTMoko


FoxtrotGPS 1.0.0

FoxtrotGPS is an offshoot of Marcus Bauer's excellent Free & Open Source tangoGPS application, with a focus on cooperation and fostering community innovation. 1.0.0 announcement.

  • Gracefully recovering from gpsd shutting down
  • GPX routepoints support
  • Integration of distribution patches
  • Map tiles are displayed immediately after downloading
  • GeoRSS points can be imported as POIs (script)
  • Various other fixes and improvements
  • Since no feedback gotten from the sister project and some changes will not be imported, version number 1.0.0 was selected to show that there is divergence


Homepage: http://www.foxtrotgps.org/
Package: foxtrotgps
Tested on: Debian


General News

Most important and change making mails on the mailing lists, blogs etc.. Coolest hacks, screenshots, themes etc..

GPRS

  • GPRS on FreeRunner is unstable? Too many connections hang the modem? Unfixable? NO MORE! Our magician lindi has conjured a tc (traffic control) command that makes the data flow more stable even under heavy load. Huge thanks! http://docs.openmoko.org/trac/ticket/2264#comment:21 (use the lower one for faster operation)

Kernels

Spin-off Hardware Projects

Event News

  • 2010-10-12 Next "FYP.de / Openmoko Stammtisch" in Munich, Germany [2]
  • 2011-01-24 Mobile FOSS MiniConf at LCA2011 announced a call for papers that closes on Friday 22nd October 2010. So submit something about OpenMoko today!
  • 2011-02-05/06 FOSDEM 2011 calls for Main Speakers and Devrooms [3] that closes on Saturday 16th October 2010. So submit something about OpenMoko today!

Read more
pitti

After 20 days of final polishing and maturing since the release candidate, the PostgreSQL team released the final 9.0 version today.

Hot off the press, I uploaded postgresql-9.0 final into Debian unstable; they will not go into Debian Squeeze, because Squeeze is frozen and it will take a long time to port all the packaged server side extensions to 9.0.

If you are on Ubuntu 10.04 LTS or Ubuntu 10.10, you can add my PostgreSQL backports for stable Ubuntu releases PPA, which will carry 9.0 until it can be moved to the official Ubuntu backports (i. e. when 9.0 goes into Ubuntu Natty).

Enjoy, and kudos to the PostgreSQL team!

Read more
pitti

It’s been a decade ago when I did my first steps with contributing to Free Software, about seven years when I joined Debian, and about 6 with Canonical and Ubuntu. Time for some reflection what I have done over these years!

Distribution Packaging and Maintenance

My first sponsored Debian upload ever was cracklib2, which seriously needed some love and was looking for a new maintainer. So in that upload I managed to close all outstanding bugs. Thanks to my mentor Martin Godisch about providing a lot of guidance for this!

Since then I’ve maintained various packages, where the most popular ones are certainly the free database server “PostgreSQL” (see next section) and the e-book management software “Calibre”.

“Maintaining” by and large means “making it really easy to get and use this software”. This decomposes to:

  • Packaging it in a way that a simple apt-get install makes the software work out of the box (as far as possible)
  • Provide a default configuration/customizations so that it integrates and plays well with the rest of the system; this includes the paths and permissions for log files, log rotation, debconf, configuration file standards, etc.
  • Be the front line for bug reports from users, sort, answer, and de-duplicate them, and either fix them myself, or forward useful bugs to upstream.
  • Providing security updates for stable releases
  • To some extent, help with the development of the software; this gets mostly driven by user demand, and of course my personal interests.

In August 2004 I got employed by Canonical to work full time on Ubuntu, which pretty much turned a hobby into a profession. I never regretted this in the past years, it’s an awesome job to do!

In principle I’m doing the same thing in Ubuntu as well: Bring stuff from developers to the people out there. Except with a different focus, in Ubuntu my daily bread and butter is the GNOME desktop and stuff around (and immediately below) it. And even though after a long day of bug triaging and debugging I feel a bit low-hearted (“50.000 bugs away from perfection”), when I take a step back and see how much the usage of Free Software in the world has grown since 2004, I am very proud of being part of Ubuntu, which certainly has its fair contribution and share in this growth. So what seemed like a crazy idea from Mark back in 2004 actually has made a remarkable progress.

In the beginning of Ubuntu I mostly sent back patches to the Debian bug tracker, but this evolved quite a bit on both sides: These days I try to keep “my” packages in sync and commit stuff directly to Debian, which works very well with e. g. the pkg-utopia team, which is responsible for HAL, udisks, upower, PolicyKit, and related packages. At this point I want to thank Michael Biebl for being such an awesome guy on the Debian side! Also, it seems that Debian has moved a fair bit away from the strong “Big Maintainer Lock” towards team based maintenance, so these days it is easier than ever to commit stuff directly to Debian for a lot of packages, without much fuss.

PostgreSQL

I have done a handful of changes to PostgreSQL, but these mostly concerned easy packaging and crash fixes, nothing out of the ordinary. I’m not really a PostgreSQL upstream developer.

The thing I am really proud of is the postgresql-common package, which is a very nice example what a distro can provide on top of upstream: If you install the upstream tarball, you have to manually care for creating clusters, providing a sensible configuration for them, set up SSL, set up log rotation, etc. With postgresql-common, this is all done automatically. The biggest feature it provides is a robust and automatic way of upgrading between major releases with the pg_upgradecluster tool, which takes care of a dozen corner cases and the nontrivial process of dumping the old cluster and reloading the new one. Also, you can effortlessy run several instances of the same version in parallel, so that you can e. g. have a production and a development instance, or try the new 9.0 RC1 while still running the 8.4 production one. (more details)

Crash Reporting with Apport

This has been a pet peeve of mine pretty much from day 1. Back in the old days, crashes in software were a pain to track down: many crashes are hard to reproduce, it takes ages to get useful information from bug reporters, and a lot of data cannot be recovered any more when you try to reproduce and analyze a crash after the fact.

With the growing demand for QA from both Canonical and Ubuntu, in 2006 I finally got some time to start Apport, which would make all this a lot easier: It intercepts crashes as they happen, collects the data that we as a developer need, and makes it very easy for the user to submit them as a bug report. This is accompanied by a backend service (called “retracers”) which would reprocess the bug reports by taking the core dump, reproducing a chroot with the packages and versions that the reporter had, installing the debugging symbols, and re-running gdb, to produce a fully symbolic stack trace.

See this bug report for how this looks like. Since then, tons of crashes were fixed, way more than we could ever have done “the old way” with asking users to rebuild with “-g -O0″, running gdb, etc.

By today, Apport has grown quite a bit: rich bug reports, per-package hooks, automatic duplication of crashes, interactive GUI elements in hooks, etc.

Plumbing Development

Handling hotpluggable hardware has always interested me, since the day when I got my first USB stick and it was ridiculously hard (from an user perspective) to use it:

    $ su -
    # mount -t vfat -o uid=1000 /dev/sda1 /mnt

My first go at this was to write pmount which would allow normal users to mount hotpluggable storage without root privileges and worrying about mount paths and options, and then integrate it into GNOME and HAL. Personally I abandoned it years ago, but it seems other people still use it, so I’m glad that Vincent Fourmond took over the maintenance now.

Since then, the entire stack evolved quite a bit: HAL grew to something useful and rather secure, and finally into some monstrous unmaintainable beast, which is why it was declared dead in 2008, and replaced with the “U” stack: udev, udisks, upower, etc. I enjoy hacking on that a lot, and since it’s part of my Desktop Team Tech Lead/Developer role in Canonical, I can spend some company time on it. so far I worked on bug fixes, small new features, and writing a rather comprehensive test suite for udisks (see my udisks commits so far). I also did a fair share of porting stuff away from HAL to the new stack, including some permanent commitments like maintaining the keymaps in udev.

GNOME

Before I started with Canonical I was never much of a GUI person: I was fully content with fvwm and a few xterms around it. But as an Ubuntu developer I do dogfooding, and thus I switched to GNOME for my day to day work. It didn’t take long before I really fell in love with it!

Similar to my Debian packages, my upstream involvement with GNOME is mostly integration and bug fixing. As already explained, we get a looooot of bug reports, so my focus is mostly on bug fixing. To date, I sent 93 patches to bugzilla. Since January 2010 I became a committer, so that it’s easier for me to get patches upstream.

Debugging problems and fixing bugs is a pretty tedious task, but I still enjoy the rewarding nice feeling when you finally tracked down something and can close a bug with 50 duplicates, and you have made people’s life a little bit easier from now on.

Read more
beuno

For the first year and a half in Canonical I worked with the amazing Launchpad team, with the ambitious goal of building a new user interface, introducing AJAX in an established code base and rolling it all out on time. While all of that was overwhelming in itself, what was more important to me was making sure the UI remained consistent across time.
Long story short, it was a success and it's been 8 months since I've left the team and the established process is still on-going.

I wrote a paper on the whole experience and presented it at the agile conference XP2010 in Norway.

Here's the introduction:

When I started working with the Launchpad team I was tasked with designing and rolling out a new interface using cutting-edge technology on a well established product and team. The existing processes and team structure made it very hard to roll out big changes while also ensuring consistency as time went by.
While the general designs and work ow changes were being eshed out, I started to drive some change to the existing processes, enabling me to be successful at an objective that would take a year to accomplish, and unexpectedly, beyond that.
The project was 4 years old and had over 500 dynamic pages with different templates and layouts that had been left untouched at different points in time. The goal for the next year was to make the application easier to use, even enjoyable. I also had to make the UI consistent across the board, take the project from static HTML pages into the wonderful world of in-line editing, streamlined work-flows and predictable interactions. In parallel, fundamental features that had been developed were going completely unused and we needed to turn that around. A re-usable AJAX infrastructure had to be developed from the ground up, new features needed to be designed and delivered, and general navigation issues needed to be addressed.
However, this story isn't about the success of the roll out of a new interface, but rather the success in the process changes that evolved during that year and how the project went from nobody feeling ownership over the user interface, to the developers taking strong ownership.

I feel very passionate about this subject, and hope this experience can help other projects and teams.

Here's the paper for download: xp2010_paper.pdf

Read more
pitti

PostgreSQL 9.0 with a whole lot of new features and improvements is nearing completion. The first release candidate was just announced.

As with the beta versions, I uploaded RC1 to Debian experimental again. If you want to test/use them on Ubuntu 10.04 (Lucid Lynx), you can get packages from my “PostgreSQL backports for stable Ubuntu releases” PPA. Please let me know if you need them for other releases.

Just for the records, both Debian 6.0 “Squeeze” and Ubuntu 10.10 “Maverick Meerkat” will release and officially support 8.4 only, as 9.0 is too late for the feature freezes of both. Also, it will take quite some time to update all the packaged extensions to 9.0. As usual, 9.0 will be provided as official backports for both Debian and Ubuntu.

Happy testing!

Read more

During our Debconf we were wondering how many people shared the common link of being both Ubuntu Developers and working in Debian. The initial list is 62! (Make that 59, see below) Thanks to Lucas Nussbaum and Michel Bienia for this first cut.

Ond?ej Surý, Fabio Tranchitella, Kees Cook, LI Daobing, Fathi Boudra, Steve Kowalik, Benjamin Mako Hill, Scott Kitterman, Alexander Sack, Colin Watson, Sebastien Bacher, Martin Meredith, Andrew Mitchell, Daniel Silverstone, LaMont Jones, Fabio Massimo Di Nitto, Loïc Minier, ???? ???????? (Ahmed El-Mahmoudy), Luca Falavigna, Laurent Bigonville, Philipp Kern, Martin Pitt, Matthias Urlichs, Sebastian Dröge, Luke Faraone, Steve Langasek, Thom May, Matthias Klose, Alessio Treglia, Iulian Udrea, Timo Jyrinki, Emilio Pozuelo Monfort, Albin Tonnerre, Stefan Potyra, Andrea Veri, Scott Howard, Nicolas Valcárcel, Raphaël Hertzog, Reinhard Tartler, Julian Andres Klode, Barry deFreese, Benjamin Drung, Michael Banck,  Riccardo Setti, Lucas Nussbaum, Adam Conrad, Michael Vogt, Jelmer Vernooij, Adrian Perez, Robert Collins, Tollef Fog Heen, Gerfried Fuchs, Mark Shuttleworth, Andrew Pollock, Sylvestre Ledru, Michael Casadevall, Ben Collins, Jo Shields, and Chris Cheney

This is a list of people who are in ubuntu-dev or ubuntu-core-dev AND have their key in the Debian keyring, it’s not an indicator of how active that person may or may not be. Here are the scripts they used if you’re interested in working on this sort of thing.

UPDATE: mdz, keybuk, and Kyle McMartin are emeritus, I’ve removed them from the list.

Read more