Canonical Voices

Posts tagged with 'debian'

Timo Jyrinki

Do you ever have those afternoons you get a ”great” idea and you've all the evening time for that task. The task is a relaxing one and won't need much attention, and you can watch a movie or something.  But then, it happens that the evening turns into night as you realize a couple of little details adding complexity to the idea, and the task turns out to be much more invasive to your evening than you thought?

In this example, I got the great idea to upgrade my Debian running NAS device (thanks Martin for everything!) to use ext4 instead of ext3. The kind of idea that takes a long time for relatively little practical benefit, but it just feels like a nice thing do when you've the extra amounts of nerd time available. It's basically just opening the NAS device up, mounting its hard disk to a laptop via external case, running the tune2fs and fsck then putting the disk back. It just takes a long time for the initial fsck (to make sure everything's intact) and then the required fsck run to get ext4 mountable.

Only in this situation, it would have been beneficial to have the ext4 support in the flashed initramfs before the migration. So... before the photo below, I've already:

  1. done the ext4 migration and fsck:s
  2. screwed the disk back to the NAS case, attached cables and found that it doesn't boot
  3. (on the laptop with the hard disk attached again tried manually unpacking initramfs and adding ext4 module... also had time to bind mount everything and chroot into the ARM system to run update-initramfs manually... also tried booting with those... until remembered the simple fact that the /boot partition is only for show and also the initramfs is loaded directly from flash)
  4. copied the main root filesystem content from the original disk to another external disk with ext3 partition
  5. attached the another disk (with same UUID:s) to the QNAP NAS device, booted, double-checked that I have now ext4 specified under /etc/initramfs-tools/modules, reconfigured the linux image that also regenerates initramfs and flashes it
And in the photo, what's happening is that:
  1. I've again the original disk reattached and system booted with the initramfs generated and flashed from the ext3 disk
  2. the NAS device is hanging in the air, cover open, from the closet where I've things stuffed in (normally secured with cable ties), and I need to support it with a knee or one hand since the 2TB disk is much heavier than the small SSD I used as the ext3 disk so the power cable and RJ-45 cable would have pretty heavy load
  3. Since I've only one hand in use and can't use a laptop, I'm logging in via my Nokia N9 and then reflashing the kernel + initramfs from this original disk, just to make sure everything is now alright and also after that flashing it still boots (it does!). Note that I feel like the setup is secure enough for non-interrupted flashing so that I can indeed support the NAS with a knee, use one hand to keep N9 and another hand to take a photo with a camera.

And so we have had a productive and educating afternoon/evening/night once again. Does this ever happen to you?

Read more
pitti

New PostgreSQL microreleases with two security fixes and several bug fixes was just announced publically.

I spent the morning with the packaging orgy for Debian unstable and experimental (now uploaded), Debian Wheezy (update sent to security team), Ubuntu hardy, lucid, natty, oneiric, precise (LP #1008317) and my backports PPA.

I tested these fairly thoroughly, but please let me know if you encounter any problem with these.

Read more
pitti

The first Beta of the upcoming PostgreSQL 9.2 was released yesterday (see announcement). Your humble maintainer has now created packages for you to test. Please give them a whirl, and report any problems/regressions that you may see to the PostgreSQL developers, so that we can have a rock solid 9.2 release.

Remember, with the postgresql-common infrastructure you can use pg_upgradecluster to create a 9.2 cluster from your existing 8.4/9.1 cluster and run them both in parallel without endangering your data.

For Debian the package is currently waiting in the NEW queue, I expect them to go into experimental in a day or two. For Ubuntu 12.04 LTS you can get packages from my usual PostgreSQL backports PPA. Note that you need at least postgresql-common version 0.130, which is available in Debian unstable and the PPA now.

I (or rather, the postgresql-common test suite) found one regression: Upgrades do not keep the current value of sequences, but reset them to their default value. I reported this upstream and will provide updated packages as soon as this is fixed.

Read more
beuno

While a lot of you are at UDS, several Latin American LoCos are working hard to organize a local Ubuntu conference.
Things are going really well, we're 4 weeks away, but we're a little short on funds. Every year the same people who organize it end up having to pay for many things themselves despite have a few generous sponsors, and this year I'd like to change it so I set up a small but valuable fund raising campaign and we could really use your help.
The site is in Spanish, so it may take a bit of blind surfing to get around but it should be fairly easy once you've been sent to PayPal  :)

If you have a some spare change, head on over here: http://www.groofi.com/profile/beuno/projects/ubuconla-2012-conferencia-de-ubuntu-en-argentina

Read more
Barry Warsaw

So, now all the world now knows that my suggested code name for Ubuntu 12.10, Qwazy Quahog, was not chosen by Mark.  Oh well, maybe I'll have more luck with Racy Roadrunner.

In any case, Ubuntu 12.04 LTS is to be released any day now so it's time for my semi-annual report on Python plans for Ubuntu.  I seem to write about this every cycle, so 12.10 is no exception.  We've made some fantastic progress, but now it's time to get serious.

For Ubuntu 12.10, we've made it a release goal to have Python 3 only on the desktop CD images.  The usual caveats apply: Python 2.7 isn't going away; it will still probably always be available in the main archive.  This release goal also doesn't affect other installation CD images, such as server, or other Ubuntu flavors.  The relatively modest goal then only affects packages for the standard desktop CD images, i.e. the alternative installation CD and the live CD.

Update 20120425: To be crystal clear,  if you depend on Python 2.7, the only thing that changes for you is that after a fresh install from the desktop CD on a new machine, you'll have to explicitly apt-get install python2.7.  After that, everything else will be the same.

This is ostensibly an effort to port a significant chunk of Ubuntu to Python 3, but it really is a much wider, Python-community driven effort.  Ubuntu has its priorities, but I personally want to see a world where Python 3 rules the day, and we can finally start scoffing at Python 2 :).

Still, that leaves us with about 145 binary packages (and many fewer source packages) to port.  There are a few categories of packages to consider:

  • Already ported and available.  This is the good news, and covers packages such as dbus-python.  Unfortunately, there aren't too many others, but we need to check with Debian and make sure we're in sync with any packages there that already support Python 3 (python3-dateutil comes to mind).
  • Upstream supports Python 3, but it is not yet available in Debian or Ubuntu.  These packages should be fairly easy to port, since we have pretty good packaging guidelines for supporting both Python 2 and Python 3.
  • Packages with better replacements for Python 3.  A good example is the python-simplejson package.  Here, we might not care as much because Python 3 already comes with a json module in its standard library, so code which depends on python-simplejson and is required for the desktop CD, should be ported to use the stdlib json module.  python-gobject is another case where porting is a better option, since pygi (gobject-introspection) already supports Python 3.
  • Canonical is the upstream.  Many packages in the archive, such as python-launchpadlib and python-lazr.restfulclient are developed upstream by Canonical.  This doesn't mean you can't or shouldn't help out with the porting of those modules, it's just that we know who to lean on as a last resort.  By all means, feel free to contribute to these too!
  • Orphaned by upstream.  These are the most problematic, since there's essentially no upstream maintainer to contribute patches to.  An example is python-oauth.  In these cases, we need to look for alternatives that are maintained upstream, and open to porting to Python 3.  In the case of python-oauth, we need to investigate oauth2, and see if there are features we're using from the abandoned package that may not be available in the supported one.
  • Unknowns.  Well, this one's the big risky part because we don't know what we don't know.
We need your help!  First of all, there's no way I can personally port everything on our list, including both libraries and applications.  We may have to make some hard choices to drop some functionality from Ubuntu if we can't get it ported, and we don't want to have to do that.  So here are some ways you can contribute:
  • Fill in the spreadsheet with more information.  If you're aware of an upstream or Debian port to Python 3, let us know.  It may make it easier for someone else to enable the Python 3 version in Debian, or to shepherd the upstream patch to landing on their trunk.
  • Help upstream make a Python 3 port available.  There are lots of resources available to help you port some code, from quick references to in-depth guides.  There's also a mailing list (and Gmane newsgroup mirror) you can join to get help, report status, and have other related discussions. Some people have asked Python 3 porting questions on StackOverflow, using the tags #python, #python-3.x, and #porting
  • Join us on the #python3 IRC channel on Freenode.
  • Subscribe to the python-porting mailing list.
  • Get packages ported in Debian.  Once upstream supports Python 3, you can extend the existing Debian package to expose this support into Debian.  From there, you or we can make sure that gets sync'd into Ubuntu.
  • Spread the word!  Even if you don't have time to do any ports yourself, you can help publicize this effort through social media, mailing lists, and your local Python community.  This really is a Python-wide effort!
Python 3.3 is scheduled to be released later this year.  Please help make 2012 the year that Python 3 reached critical mass!

 -----------------------------

On a more personal note, I am also committed to making Mailman 3 a Python 3 application, but right now I'm blocked on a number of dependencies.  Here are the list of dependencies from the setup.py file, and their statuses.  I would love it if you help get these ported too!
Of course, these are only the direct dependencies.  Others that get pulled in include:


Read more
Timo Jyrinki

I have a few thoughts based on the yet another round of debate where marketing clashes with free software advocation and technical details. Nothing new in the debate itself, but I'm adding a couple of insights.

Marketing is not highly respected by many technical people, and neither by the people wanting more advocating than the messing up with facts and feelings that the marketing does. I'm all for advocating free software, but it's currently not something you can use for marketing to win big markets. If we advance to a world where free software is as wanted as the green values today are, it can be used in marketing as well similar to all the ecological (according to the market department at least) products today, but alas the benefits of free software are not yet as universally known. Since it doesn't say much that touches the masses, advocating has a negative marketing effect since it takes space away from the potentially "hitting" marketing moves, in those cases where you target the big masses in the first place. "Open" this and that has some marketing power in it nowadays, but it's a mess of different meanings that probably doesn't advance libre software freedoms as such. Wikipedia has probably been the biggest contributor to advancing general knowledge of software and culture libre. Disclaimer: I'm not a proper marketing person, some more professional might have better insights in this area. If I'd be a proper marketing person, I'd decorate this blog post with fancy pictures so that more people would actually read it.

Some of the marketing can be done without sacrificing any of the advocation. The fabulous Fedora campaigns, graphics, slogans and materials are a great example of those and do an important job, even though Fedora isn't reaching the big masses with it (and Fedora isn't targeted for OEMs to ship to millions either). But they do hopefully reach a lot of level people on the grassroots level. I hope as a Debian Developer that more people valuing the freedoms of free software would help also Debian as a project to reach more of its advocation potential and developers from the more proprietary world. But I'm happy that at least some free software projects have nowadays true graphical and marketing talent. Even though not as widely known, freedoms of the software users including for example privacy aspects are a potential good marketing tool toward a portion of the developer pool.

Let's not forget that Android conquered the mobile market without using the brand power of Linux. The 30+ million people who know Linux a bit deeper than just "I've heard it" already run a Linux distribution like Ubuntu, but we need hotter brands than a project name of a kernel to reach the bigger masses. Call it Ubuntu, Fedora, or something, but no matter do what it needs to ship it to millions. AFAIK mostly Ubuntu and SUSE are shipping currently via desktop/notebook OEMs, and more Ubuntu than SUSE. Others aren't concentrating on the market, which is a very difficult one. Ubuntu is doing a lot more mass population advocating for free software than Android has ever done. Note for example the time any user tries to play a video for the first time in a non-free format - Ubuntu will tell the user about the problems related to those formats and asking a permission to install a free software player for those (or buy licensed codecs), and the Ubuntu Help texts describe a lot of details about restricted formats, DRM et cetera. Not to mention what happens if the person actually wanders into the community, discovering Debian, other free software projects, free software licenses and so on. Meanwhile Android users never notice anything being wrong while watching H.264 videos or watching DRM Flash videos. Granted, like Boot2Gecko debate shows, it may be a partially similar situation on shipping desktop Linux variants as well, in order to actually ship them via partners.

Anyway, the best example of brands is MeeGo. Even the LWN editor does not get into dissecting the meaning of MeeGo on Nokia N9, because "there has been no real agreement on that in the past", but just uses the brand name as is. That is  the power of brands. Technical people debate that it's not really MeeGo, it's maemo GNU/Linux with a special permission from Linux Foundation to use the MeeGo brand name, and then counter-argument with that the MeeGo is an API and maemo matches the MeeGo 1.2 API which is actually just Qt 4.7 API. And actually, as proven by the LWN example, it's not even "technical people". For most of technical people Nokia N9 is MeeGo as well. Only the people who have actually worked on it plus the couple of other people migrated from maemo and MeeGo.com communities to Mer project understand the legacy, history and the complete difference between the Maemo Harmattan platform and what MeeGo.com was. Yet at the same time like all GNU/FreeDesktop.org/Linux distributions, they are 95% same code, just all the infrastructure and packaging and polishing and history is different.

For 99.9% of people who know what MeeGo is, MeeGo is the cool Nokia smartphone, one of its kind and not sold in some of the major western markets, and/or the colorful sweet characters of meego.com shown at one time on a couple of netbooks. That is the brand, and the technical details do not matter even to the technical people unless they actually get into working on the projects directly.

To most people, the Linux brand is a mess of a lot of things, while other brands have the possibility at least to have a more differentiating and unique appearance.

Read more
Matt Fischer

A couple weeks ago I participated in the Ubuntu Global Jam/Bug Fix session. I started by looking at all the “fails to build from source” aka ftbfs list. I read through the list and found a few interesting packages, but ended up working on live-manual, because I use live-build and also it seems odd that we can’t build a user-manual.

At first I was confused, because I pulled the source and ran dpkg-buildpackage on it and it worked fine on my amd64 box. So I tried an i386 VM that I had running, works fine there too! Next, the litmus test, a build in a pbuilder chroot. Guess what, that one failed. I traced the errors back to this one error when building the German user manual:

ERROR occurred, message:”invalid byte sequence in US-ASCII”

It was odd, why are we processing a German user manual file using a US ASCII locale?

I compared the environment between my system, where it built, and the pbuilder chroot, where it failed and found that inside a pbuilder chroot, the LC_ALL variable is set to “C”. C is equivalent of US-ASCII. After trying in vain to change the locale to something that understands utf-8, I realized that I needed to install a locale that could handle UTF-8 inside my pbuilder chroot.

debian/control:
-Build-Depends: debhelper (>= 8), ruby, libnokogiri-ruby
+Build-Depends: debhelper (>= 8), ruby, libnokogiri-ruby, language-pack-en

debian/rules:

%:
- dh ${@}
+ LC_ALL=en_US.UTF-8 dh ${@}

Now the build was working, so I submitted my merge proposal.

My fix was accepted but discussions revealed that the fix wouldn’t work in debian because their locale packages are different. With that in mind, I worked on a more generic fix, based on what python-djvulibre does. After some tweaking (not shown in the merge proposal), I settled on this fix:

debian/rules:

+override_dh_auto_build:
+ mkdir -p debian/tmp/locale/
+ localedef -f UTF-8 -i en_US ./debian/tmp/locale/en_US.UTF-8/
+ export LOCPATH=$(CURDIR)/debian/tmp/locale/ && \
+ export LC_ALL=en_US.UTF-8 && \
+ dh_auto_build

Note: During the development of that fix, I had trouble finding all the different steps in the debhelper build and what I could override. A debian developer pointed me to this great presentation which answered a lot of questions: Joey Hess’ Not Your Grandpa’s Debhelper.

So there are two ways to solve a locale issue during a build. Do you know of any others? If so I’d love to hear them in the comments.

Read more
Timo Jyrinki

Just a quick note that the merry Finnish localization folks are organizing an (extended) localization weekend, starting today. As a nice step towards ease of use, they're utilizing the long developed, maybe even underused Translatewiki.net platform, or to be precise a separate instance of it. Translatewiki.net is used by MediaWiki (Wikimedia Foundation), StatusNet and other high profile projects. Co-incidentally the main developer of Translatewiki.net is Finnish as well.

Anyway enough of the platform, join the translation frenzy at http://l10n.laxstrom.name/wiki/Gnome_3.4, but do make sure to read the notes at http://muistio.tieke.fi/IYZxesy9uc.

I've promised to help in upstreaming those to git.gnome.org on Sunday. There is additionally a new report about Ubuntu 12.04 LTS translations schedule (to which these GNOME contributions will find their way as well) at the ubuntu-l10n-fin mailing list by Jiri.

Ja sama suomeksi.

Read more
pitti

Part of our efforts to reduce power consumption is to identify processes which keep waking up the disk even when the computer is idle. This already resulted in a few bug reports (and some fixes, too), but we only really just began with this.

Unfortunately there is no really good tool to trace file access events system-wide. powertop claims to, but its output is both very incomplete, and also wrong (e. g. it claims that read accesses are writes). strace gives you everything you do and don’t want to know about what’s going on, but is per-process, and attaching strace to all running and new processes is cumbersome. blktrace is system-wide, but operates at a way too low level for this task: its output has nothing to do any more with files or even inodes, just raw block numbers which are impossible to convert back to an inode and file path.

So I created a little tool called fatrace (“file access trace”, not “fat race” :-) ) which uses fanotify, a couple of /proc lookups and some glue to provide this. By default it monitors the whole system, i. e. all mounts (except the virtual ones like /proc, tmpfs, etc.), but you can also tell it to just consider the mount of the current directory. You can write the log into a file (stdout by default), and run it for a specified number of seconds. Optional time stamps and PID filters are also provided.

$ sudo fatrace
rsyslogd(967): W /var/log/auth.log
notify-osd(2264): O /usr/share/pixmaps/weechat.xpm
compiz(2001): R device 8:2 inode 658203
[...]

It shows the process name and pid, the event type (Rread, Write, Open, or Close), and the path. Sometimes its’ not possible to determine a path (usually because it’s a temporary file which already got deleted, and I suspect mmaps as well), in that case it shows the device and inode number; such programs then need closer inspection with strace.

If you run this in gnome-terminal, there is an annoying feedback loop, as gnome-terminal causes a disk access with each output line, which then causes another output line, ad infinitum. To fix this, you can either redirect output to a file (-o /tmp/trace) or ignore the PID of gnome-terminal (-p `pidof gnome-terminal`).

So to investigate which programs are keeping your disk spinning, run something like

  $ sudo fatrace -o /tmp/trace -s 60

and then do nothing until it finishes.

My next task will be to write an integration program which calls fatrace and powertop, and creates a nice little report out of that raw data, sorted by number of accesses and process name, and all that. But it might already help some folks as it is right now.

The code lives in bzr branch lp:fatrace (web view), you can just run make and sudo ./fatrace. I also uploaded a package to Ubuntu Precise, but it still needs to go through the NEW queue. I also made a 0.1 release, so you can just grab the release tarball if you prefer. Have a look at the manpage and --help, it should be pretty self-explanatory.

Read more
Martin Pool

Jelmer writes:

bzr-builddeb 2.8.1 has just landed on Debian Sid and Ubuntu Precise. This version contains some of my improvements from late last year for the handling of quilt patches in packaging branches. Most of these improvements depend on bzr 2.5 beta 5, which is also in Sid/Precise.

The most relevant changes (enabled by default) are:

  • ‘bzr merge-package’ is now integrated into ‘bzr merge’ (it’s just a hook that fires on merges involving packages)
  • patches are automatically unapplied in relevant trees before merges
  • before a commit, bzr will warn if you have some applied and some unapplied quilt patches

Furthermore, you can now specify whether you would like bzr to automatically apply all patches for stored data and whether you would like to automatically have them applied in your working tree by setting ‘quilt-tree-policy‘ and ‘quilt-commit-policy‘ to either ‘applied‘ or ‘unapplied‘. This means that you can have the patches unapplied in the repository, but automatically have them applied upon checkout or update. Setting these configuration options to an empty string causes bzr to not touch your patches during commits, checkout or update.

We’ve done some testing of it, as well as running through a package merge involving patches with Barry, but none of us do package merges regularly. If you do run into issues or if you think there are ways we can improve the quilt handling further, please comment here or file a bug report against the UDD project.

Caveats:

  • If there are patches to unapply for the OTHER tree, bzr will currently create a separate checkout and unapply the patches there. This may have performance consequences for big packages. The best way to prevent this is to set ‘quilt-commit-policy = unapplied‘.
  • bzr merge‘ will now fail if you are merging in a packaging tree that is lacking pristine tar metadata; I’m submitting a fix for this, but it’s not in 2.8.1.
  • if you set ‘quilt-commit-policy‘ and ‘quilt-tree-policy‘ but have them set to a different value, bzr will consider the tree to have changes.

To disable the automatic unapplying of patches and fall back to the previous behaviour, set the following in your builddeb configuration:

quilt-smart-merge = False

Read more
Timo Jyrinki

I have a new GPG key


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1,SHA512

Hello,

I'm transitioning from my 2003 GPG key to a new one.

The old key will continue to be valid for some time, but I eventually
plan to revoke it, so please use the new one from now on. I would also
like this new key to be re-integrated into the web of trust. This message
is signed by both keys to certify the transition.

The old key was:

pub 1024D/FC7F6D0F 2003-07-10
Key fingerprint = E6A8 8BA0 D28A 3629 30A9 899F 82D7 DF6D FC7F 6D0F

The new key is:

pub 4096R/90BDD207 2012-01-06
Key fingerprint = 6B85 4D46 E843 3CD7 CDC0 3630 E0F7 59F7 90BD D207

To fetch my new key from a public key server, you can simply do:

gpg --keyserver pgp.mit.edu --recv-key 90BDD207

If you already know my old key, you can now verify that the new key is
signed by the old one:

gpg --check-sigs 90BDD207

If you don't already know my old key, or you just want to be double
extra paranoid, you can check the fingerprint against the one above:

gpg --fingerprint 90BDD207

If you are satisfied that you've got the right key, and the UIDs match
what you expect, I'd appreciate it if you would sign my key:

gpg --sign-key 90BDD207

Lastly, if you could send me these signatures, i would appreciate it.
You can either send me an e-mail with the new signatures by attaching
the following file:

gpg --armor --export 90BDD207 > timojyrinki.asc

Or you can just upload the signatures to a public keyserver directly:

gpg --keyserver pgp.mit.edu --send-key 90BDD207

Please let me know if there is any trouble, and sorry for the inconvenience.

(this post has been modified from the example at
http://www.debian-administration.org/users/dkg/weblog/48)

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAk8GuZoACgkQgtffbfx/bQ9nqACglWyHnDTFQfdKmz8OCd3oL6iR
hcEAmgKJ7RZsgwxwkRGPhygy5y1Ztb+3iQIcBAEBCgAGBQJPBrmaAAoJEOD3WfeQ
vdIHdVQQAMT1yvIogzbtK6sUnWqwbrXI9pDEFk7AzJTb80R+wzxsw7gu9gcBDk8G
BL2O26GKUqKWA3ytuApSl42FJam/Lusi9npT3XNkmHs6FaBMNuLYrqEXmCwXwWr/
OrLyeeLiF4yxgbNWbv+600BqAWqFlo6NeTgQKsJWtCjR3RVMxX3R8nzjDnKJuF+z
c6+2JKBWyx/HVUKcJpJrFDDR36HRFvVJomTuma2JCQ/RAl9vzAguqNYOi1QkuuQv
EF1gXH7gLifukGuwquP1DHP6SWWkj77jtRWr5ewC0xymbrArzAwKbvMQl3VpKBHh
MmpJjYP3ECyL14AKi/TY2Lidi0Sf6yqFMcPcreoih01N0OU0NXmD4IrHMT24/ssb
okDUe1o3YImjGq1jTACvlzC8s54EfLsqDgSP98SGVpuoDqPJUwVk4nuHj8q0vDSs
qZox26gVwB2FAOUi1BFiZbIzM5rsyYfCGyWUGiAwBFf54lYRAeCDCt8iAOOL1Ov/
TumIGYdLoXnDuOJq1VjXLGx2OFDrpyU8SPGoa3zNEVz39tgxQ48ASJEqcqt7HvBy
IW+TTsMLdJ1Ait9aCM3mzzr1iwP8TrL0qUsdRLOE6AKdAqocIfqXY8OeDKhbUiOJ
CXWk5q3xheK3sDWUXX7J63bAAUH4jFnpQEOVMJKBUNMKsWa0iXDS
=mklN
-----END PGP SIGNATURE-----

Read more

Zeeshan sends along that he’s looking for help bring GNOME Boxes to Ubuntu and Debian.

If you read any of my previous blog entries, you must be now familiar with this ‘express installation’ concept we have in Boxes. Its pretty neat actually, you just set a few options at the beginning and then you can leave Boxes (or your machine) and when you are back, everything is setup for you automatically in a new box.

I have invested a lot of time/efforts on this already and will be spending a lot more time in future as well but I am just one man so can not possibly cover all operating systems out there. That is why I am asking for help from anyone who will be interested in adding express installation support for Ubuntu and Debian while I focus on Fedora and Windows variants. Oh and if you are interested in adding support for some other distribution/OS, that contribution will also be more than welcomed.

In any case, happy hacking!

If you’re interested in doing this (it would be great to get Boxes in 12.04) let me know. You’ll likely need to link up with the Desktop Team, I can help get you talking to the right people if you want to rock this.

Read more

This weekend, we held a combined Debian Bug Squashing Party and Ubuntu Local Jam in Portland, OR. A big thank you to PuppetLabs for hosting!

Thanks to a brilliant insight from Kees Cook, we were able to give everyone access to their own pre-configured build environment as soon as they walked in the door by deploying schroot/sbuild instances in "the cloud" (in this case, Amazon EC2). Small blips with the mirrors notwithstanding, this worked out pretty well, and let people start to get their hands dirty as soon as they walked in the door instead of spending a lot of time up front doing the boring work of setting up a build environment. This was a big win for people who had never done a package build before, and I highly recommend it for future BSPs. You can read about the build environment setup in the Debian wiki, and details on setting up your own BSP cloud in Kees's blog.

(And the cloud instances were running Ubuntu 11.10 guests, with Debian unstable chroots - a perfect pairing for our joint Debian/Ubuntu event!)

So how did this curious foray into a combined Ubuntu/Debian event go? Not too shabby:

  • Roughly 25 people participated in the event - a pretty good turnout considering the short notice we gave. Thanks to everyone who turned up!
  • Multiarch patches were submitted for 14 library packages by 9 distinct contributors
  • Four of these people submitted their first patch to Debian!
  • Three more contributors worked on patches that were not submitted to Debian by the end of the event, but we will stalk them and see to it that their patches make it in ;)
  • 8 Ubuntu Stable Release Updates were looked at for verification of fixes
  • 7 of these fixes were successfully verified (one bug was not reproducible)
  • 6 of those packages have already been moved to the -updates pocket, where all of Ubuntu's users can now benefit from them

When all was said and done, we didn't get a chance to tackle any wheezy release critical bugs like we'd hoped. That's ok, that leaves us something to do for our next event, which will be bigger and even better than this one. Maybe even big enough to rival one of those crazy, all-weekend BSPs that they have in Germany...

Read more
Timo Jyrinki

Since I think I just summarized a few thoughts of mine well at LWN, I'll copy-paste it here:

I can grumble about Android from time to time, but I do not say that it sucks. Extreme views are what are annoying. Android is what it is and it's great as it is, even though it could be different as well.

When it comes to discussing about free software and mobile phones, I'm especially annoyed by two types of comments:

1. People essentially saying that there is no value in an open project, ie. free software code dumps should be enough for everybody. I'm interested in the long term viability of free software projects, and it is hard to have successful projects without there being all sorts of factors that make up a good project - like transparency, inclusion, meritocracy. Even though the mobile projects have had little resources and a hard road, it's not useful to forget about these goal in the longer term. For example Debian, Mer, SHR, KDE Plasma Active have some of these in the mobile sector. I hope the best for them (and participate).

2. People complaining about something being not 100% free software, while not themselves actually even interested in it for other sake than complaining. When I've been talking about free software mobile phones, from time to time there is someone complaining about eg. not open GSM stack, wlan firmwares etc.. and to put it sharply probably writing the message from iPhone, while I'm reading it on Neo FreeRunner. If the complainer would be Harald Welte, I'd probably listen and agree with him.

 So there. For more civilized discussion.

Read more
Timo Jyrinki

Almost forgot to post this. My mobile phones running free software in photos. From left to right:



All of that software running on the devices is more or less free software, with Harmattan obviously being by far the least free, especially applications, but still better than any other on-the-shelf phone software *), and the others being 99% or "Ubuntu like" free ie. possibly with firmware and a few driver exceptions. N9 needs some bootloader work still before Nemo, Debian, Ubuntu etc. can be run there. I've collected a few things about N9 from this point of view at a wiki page.
*) Not sure about every Android phone, but Android is not openly developed anyway so it's hardly a similar free software project such as freedesktop.org projects or Qt

I gave my N900 away now since obviously I cannot make full use of each one of these. I'm multi-SIMming my N9 and the GTA02a7 Neo FreeRunner for daily use, while the other FreeRunner and N950 are purely for tinkering related purposes. The development FreeRunner will get on upgrade to GTA04 once it's available, and then hopefully that can be made into a daily usable phone as well.

By the way, see you in FSCONS in Gothenburg next weekend. Even rms will be there, which is always interesting of course :)

Read more

Raphaël Hertzog recently announced a new dpkg-buildflags interface in dpkg that at long last gives the distribution, the package maintainers, and users the control they want over the build flags used when building packages.

The announcement mail gives all the gory details about how to invoke dpkg-buildflags in your build to be compliant; but the nice thing is, if you're using dh(1) with debian/compat=9, debhelper does it for you automatically so long as you're using a build system that it knows how to pass compiler flags to.

So for the first time, /usr/share/doc/debhelper/examples/rules.tiny can now be used as-is to provide a policy-compliant package by default (setting -g -O2 or -g -O0 for your build regardless of how debian/rules is invoked).

Of course, none of my packages actually work that way; among other things I have a habit of liberally sprinkling DEB_MAINT_CFLAGS_APPEND := -Wall in my rules, and sometimes DEB_LDFLAGS_MAINT_APPEND := -Wl,-z,defs and DEB_CFLAGS_MAINT_APPEND := $(shell getconf LFS_CFLAGS) as well. And my upstreams' build systems rarely work 100% out of the box with dhauto* without one override or another somewhere. So in practice, the shortest debian/rules file in any of my packages seems to be 13 lines currently.

But that's 13 lines of almost 100% signal, unlike the bad old days of cut'n'pasted dh_* command lists.

The biggest benefit, though, isn't in making it shorter to write a rules file with the old, standard build options. The biggest benefit is that dpkg-buildflags now also outputs build-hardening compiler and linker flags by default on Debian. Specifically, using the new interface lets you pick up all of these hardening flags for free:

-fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security -Wl,-z,relro

It also lets you get -fPIE and -Wl,-z,now by adding this one line to your debian/rules (assuming you're using dh(1) and compat 9):

export DEB_BUILD_MAINT_OPTIONS := hardening=+pie,+bindnow

Converting all my packages to use dh(1) has always been a long-term goal, but some packages are easier to convert than others. This was the tipping point for me, though. Even though debhelper compat level 9 isn't yet frozen, meaning there might still be other behavior changes to it that will make more work for me between now and release, over the past couple of weekends I've been systematically converting all my packages to use it with dh. In particular, pam and samba have been rebuilt to use the default hardening flags, and openldap uses these flags plus PIE support. (Samba already builds with PIE by default courtesy of upstream.)

You can't really make samba and openldap out on the graph, but they're there (with their rules files reduced by 50% or more).

I cannot overstate the significance of proactive hardening. There have been a number of vulnerabilities over the past few years that have been thwarted on Ubuntu because Ubuntu is using -fstack-protector by default. Debian has a great security team that responds quickly to these issues as soon as they're revealed, but we don't always get to find out about them before they're already being exploited in the wild. In this respect, Debian has lagged behind other distros.

With dpkg-buildflags, we now have the tools to correct this. It's just a matter of getting packages to use the new interfaces. If you're a maintainer of a security sensitive package (such as a network-facing daemon or a setuid application), please enable dpkg-buildflags in your package for wheezy! (Preferably with PIE as well.) And if you don't maintain security sensitive packages, you can still help out with the hardening release goal.

Read more
Timo Jyrinki

The MeeGo community is frustrated with the news of the MeeGo brand being abandoned. Some are understandably angry or otherwise not happy about how Linux Foundation, Intel handled the Tizen announcement and community in general - or more like how they didn't handle it at all. Last week Openmind 2011 happened to be arranged in Tampere on the very same day as Tizen announcement came alive. It was good in the way that it lead to the fact that Nomovok's CEO Pasi Nieminen was able to initiate the "Reigniting MeeGo" session not just by talking vague things about future, but actually about the process which led to Tizen and the unfortunately brief initial PR about it. Pasi is intense on emphasizing the quality and role of Qt in Tizen as well, even though officially Tizen is all about HTML5 and apparently from Samsung's part at least EFL is provided as a native toolkit. However, the promise of Tizen compared to MeeGo is reportedly that the toolkit is not specified in compliancy documents, so HTML5 with WAC is the main/only "3rd party apps" layer whereas others can be offered case-by-case. This means that unlike before, the underlying system can be built on top of practically any distribution (theoretically) and using whatever toolkits and other techniques wanted. Obviously the "Nordic System Integrators" are probably all very keen of using Qt to produce more of Nokia N9 quality user experiences in various products.

Taking the corporate hat off, I as a community member am also puzzled. The only reason I was not completely blown by the news was that I didn't yet manage to get involved in MeeGo community on a daily basis, since I'm involved with a dozen communities already. Instead I've been more like scratching the surface with MeeGo Network Finland meetings, IRC activity, OBS usage for building a few apps for MeeGo Harmattan and MeeGo proper etc. But I can somewhat understand how people like Jarkko Moilanen from meego-fi feel. They have given a _lot_ to the MeeGo community and brand, all taken away without hearing or pre-notice.

So where to now for MeeGo community? Tizen is one obvious choice. However, for all the talks that even I started this post with, Tizen is still vaporware today, and the dislike of how community is being treated might make it easy to consider other options. Also, if Tizen's reference implementation has lesser meaning, it might also mean less to actually be "in" the Tizen community than in MeeGo. I met Jos Poortvliet at Openmind, and he invited people to openSUSE. There is a lot of common ground with MeeGo and openSUSE - strong OBS usage, RPM packaging, community side focused on KDE and therefore Qt.

I would like to now point similarly to Debian! If one is tired about corporate interests and not listening to community, there is no match for Debian's 15+ years history, purely volunteer based, trust based organization, and first of all scope. While openSUSE has traditionally focused on desktop (even though like Jos pointed out they are open to all new contributions and projects), Debian has always had the "universal" scope, ie. no boundaries besides producing free software operating system for various purposes. There are over 10 architectures maintained at the moment, including the ARM (different ports for ARMv4 and hard-float ARMv7) and x86 from MeeGo world. There are even alternative kernels to Linux, mainly the GNU/kFreeBSD port. There are multiple relevant plans and projects like the Smartphones wiki area, most noticeably Debian on Neo FreeRunner. I have run Debian on my primary mobile phone for over 2.5 years, although now in the recent months I've had dual-SIM in my Nokia N950 as well (Debian not yet running on Nokia N950 or Nokia N9 - but it can and will be done!).

What Debian may lack in both good and bad is corporate funding, if you don't count the still quite respectful contributions from Ubuntu to Debian (it's in Ubuntu's interests to contribute as much possible back to Debian, so that the delta remains small). For each and every aspect, it needs a volunteer - there are a thousand volunteer Debian Developers, and at least a double of that of people without the official DD status but who still maintain a package or two among the 25000+ packages in Debian. That means also that one my find it more lucrative to join a project that has paid people to do some of the "boring parts", more of fancy web tools, including for bug handling and build systems like the OBS (which I do love by the way). On the other hand, there is no other project in my opinion where what you do really matters as much.

To find out more about Debian from MeeGo perspective, please see the recent mailing list post Mobile UXes - From the DebConf11 BoF to the stars where I wrote most of the MeeGo (CE) part when I was asked to and known of my MeeGo involvement.

Last but not certainly least, there is the Mer project - originally "maemo reconstructed", ie. making Nokia's "not really distro" into a real distro by filling in the void places. Now it's obviously MeeGo reconstructed, and they aim to be the MeeGo they always wanted MeeGo to be! Read the post for details from Carsten Munk and other key Mer people. They share the love for Qt, and want the core to be as lean as possible. They also aim to incorporate the most community like aspect from MeeGo - MeeGo CE - as the reference vendor in Mer. They also aim to be Tizen compliant - and when Tizen comes alive, I wouldn't see why the Tizen reference implementation couldn't be used for saving resources. Maybe Nomovok and/or others could offer the Qt maintaining part.

So, it might be that Tizen itself is enough for most people's needs. The key point however in this post is not to fall in agony if one corporate based project takes big turns - it has happened before, it will happen in the future. There are always enough political and business reasons from some points of view to do Big Changes. But the wider community is out there, always, and it's bigger than you think. You should consider where you want to contribute by asking yourself why you are/were part of for example the MeeGo community. Aaron Seigo from KDE asked us all this question in the Openmind MeeGo Reignited session, and I think it's good to repeat.

Read more
pitti

Hot on the heels of the PostgreSQL 9.1.0 release I am happy to announce that the final version is now packaged for Debian unstable, the current Ubuntu development version “Oneiric”, and also in my Ubuntu backports PPA for Ubuntu 10.04 LTS, 10.10, and 11.04.

Enjoy trying out all the cool new features like builtin synchronous replication or per-column collation settings for correctly handling international strings, or an even finer-grained access control for large environments. Please see the detailled explanation of the new features.

As already announced a few days ago, 9.0 is gone from Ubuntu 11.10, as it is still only a development version and not an LTS. 9.1 will be the version which the next 12.04 LTS will support, so this slightly reduces the number of major upgrades Ubuntu users will need to do. However, 9.0 will still be available in Debian unstable and backports, and the Ubuntu backports PPA for a couple of months to give DB administrators some time to migrate.

Read more
pitti

PostgreSQL 9.1 has had its first release candidate out for some two weeks without major problem reports, so it’s time to promote this more heavily. If you use PostgreSQL, now is the time to try it out and report problems.

We always strive to minimize the number of major versions which we have to support. They not only mean more maintenance for developers, but also more upgrade cycles for the users.

9.0 has not been in any stable Debian or Ubuntu release, and 9.1 final will be released soon. So we recently updated the current Ubuntu development release for 11.10 (“oneiric”) to 9.1. In Debian, the migration from 8.4/9.0 to 9.1 is making good progress, and there is not much which is left until postgresql-9.0 can be removed.

Consequently, I also removed 9.0 from my PostgreSQL backports PPA, as there is nothing any more to backport it from. However, that mostly means that people will now set up installations with 9.1 instead of 9.0, and won’t magically make your already installed 9.0 packages go away. They will just be marked as obsolete in the postgresql-common debconf note.

If you want to build future 9.0 packages yourself, you can do this based on the current branch: bzr branch lp:~pitti/postgresql/debian-9.0, get a the new upstream tarball, name it accordingly, add a new changelog with a new upstream version number, and run bzr bd to build the package (you need to install the bzr-builddeb package for this).

Update 2011-09-09: As I got a ton of pleas to continue the 9.0 backports for a couple of months, and to keep it in Debian unstable for a while longer, I put them back now. I also updated the removal request in Debian to point out that I’m mainly interested in getting 9.0 out of testing. I don’t mind much maintaining it for a couple of more months in unstable. My dear, I had no idea that my backports PPA was that popular!

Read more
Barry Warsaw

sbuild is an excellent tool for locally building Ubuntu and Debian packages.  It fits into roughly the same problem space as the more popular pbuilder, but for many reasons, I prefer sbuild.  It's based on schroot to create chroot environments for any distribution and version you might want.  For example, I have chroots for Ubuntu Oneiric, Natty, Maverick, and Lucid, Debian Sid, Wheezy, and Squeeze, for both i386 and amd64.  It uses an overlay filesystem so you can easily set up the primary snapshot with whatever packages or prerequisites you want, and the individual builds will create a new session with an overlaid temporary filesystem on top of that, so the build results will not affect your primary snapshot.  sbuild can also be configured to save the session depending on the success or failure of your build, which is fantastic for debugging build failures.  I've been told that Launchpad's build farm uses a customized version of sbuild, and in my experience, if you can get a package to build locally with sbuild, it will build fine in the main archive or a PPA.

Right out of the box, sbuild will work great for individual package builds, with very little configuration or setup.  The Ubuntu Security Team's wiki page has some excellent instructions for getting started (you can stop reading when you get to UMT :).

One thing that sbuild doesn't do very well though, is help you build a stack of packages.  By that I mean, when you have a new package that itself has new dependencies, you need to build those dependencies first, and then build your new package based on those dependencies.  Here's an example.

I'm working on bug 832864 and I wanted to see if I could build the newer Debian Sid version of the PySide package.  However, this requires newer apiextractor, generatorrunner, and shiboken packages (and technically speaking, debhelper too, but I'm working around that), so you have to arrange for the chroot to have those newer packages when it builds PySide, rather than the ones in the Oneiric archive.  This is something that PPAs do very nicely, because when you build a package in your PPA, it will use the other packages in that PPA as dependencies before it uses the standard archive.  The problem with PPAs though is that when the Launchpad build farm is overloaded, you might have to wait several hours for your build.  Those long turnarounds don't help productivity much. ;)

What I wanted was something like the PPA dependencies, but with the speed and responsiveness of a local build.  After reading the sbuild manpage, and "suffering" through a scan of its source code (sbuild is written in Perl :), I found that this wasn't really supported by sbuild.  However, sbuild does have hooks that can run at various times during the build, which seemed promising.  My colleague Kees Cook was a contributor to sbuild, so a quick IRC chat indicated that most people create a local repository, populating it with the dependencies as you build them.  Of course, I want to automate that as much as possible.  The requisite googling found a few hints here and there, but nothing to pull it all together.  With some willful hackery, I managed to get it working.

Rather than post some code that will almost immediately go out of date, let me point you to the bzr repository where you can find the code.  There are two scripts: prep.sh and scan.sh, along with a snippet for your ~/.sbuildrc file to make it even easier.  sbuild will call scan.sh first, but here's the important part: it calls that outside the chroot, as you (not root). You'll probably want to change $where though; this is where you drop the .deb and .dsc files for the dependencies.  Note too, that you'll need to add an entry to your /etc/schroot/default/fstab file so that your outside-the-chroot repo directory gets mapped to /repo inside the chroot.  For example:

# Expose local apt repository to the chroot
/home/barry/ubuntu/repo    /repo    none   rw,bind  0 0
An apt repository needs a Packages and Packages.gz file for binary packages, and a Sources and Sources.gz file for the source packages.  Secure APT also requires a Release and Release.gpg file signed with a known key.  The scan.sh file sets all this up, using the apt-ftparchive command.  The first apt-ftparchive call creates the Sources and Sources.gz file.  It scans all your .dsc files and generates the proper entries, then creates a compressed copy, which is what apt actually "downloads".  The tricky thing here is that without changing directories before calling apt-ftparchive, your outside-the-chroot paths will leak into this file, in the form of Directory: headers in Sources.gz.  Because that path won't generally be available inside the chroot, we have to get rid of those headers.  I'm sure there's an apt-ftparchive option to do this, but I couldn't find it.  I accidentally discovered that cd'ing to the directory with the .dsc files was enough to trick the command into omitting the Directory: headers.

The second call to apt-ftparchive creates the Packages and Packages.gz files.  As with the source files, we get some outside-the-chroot paths leaking in, this time as path prefixes to the Filename: header value.  Again, we have to get rid of these prefixes, but cd'ing to the directory with the .deb files doesn't do the trick.  No doubt there's some apt-ftparchive magical option for this too, but sed'ing out the paths works well enough.

The third apt-ftparchive file creates the Release file.  I shameless stole this from the security team's update_repo script.  The tricky part here is getting Release signed with a gpg key that will be available to apt inside the chroot.  sbuild comes with its own signing key, so all you have to do is specify its public and private keys when signing the file.  However, because the public file from
/var/lib/sbuild/apt-keys/sbuild-key.pub
won't be available inside the chroot, the script copies it to what will be /repo inside the chroot.  You'll see later how this comes into play.

Okay, so now we have the repository set up well enough for sbuild to carry on.  Later, before the build commences, sbuild will call prep.sh, but this script gets called inside the chroot, as the root user.  Of course, at this point /repo is mounted in the chroot too.  All prep.sh needs to do is add a sources.list.d entry so apt can find your local repository, and it needs to add the public key of the sbuild signing key pair to apt's keyring.  After it does this, it needs to do one more apt-get update.  It's useful to know that at the point when sbuild calls prep.sh, it's already done one apt-get update, so this does add a duplicate step, but at least we're fortunate enough that prep.sh gets called before sbuild installs all the build dependencies.  Once prep.sh is run, the chroot will have your overriding dependent packages, and will proceed with a normal build.

Simple, huh?

Besides getting rid of the hackery mentioned above, there are a few things that could be done better:
  • Different /repo mounts for each different chroot
  • A command line switch to disable the /repo
  • Automatically placing .debs into the outside-the-chroot repo directory

Anyway, it all seems to hang together.  Please let me know what you think, and if you find better workarounds for the icky hacks.
 

Read more