Canonical Voices

Posts tagged with 'apt'

Matt Fischer

EDIT: As several people have pointed out, there is a script to already to this, pull-lp-source. Perfect! I’ve asked numerous people over the last year about whether there was a tool that could do this and nobody mentioned this one (even asked on AskUbuntu last week). So out of all this I end up with a link to a great new tool and got to write some python yesterday. pull-lp-source looks like it will meet all my needs.

During my work on bug triage and trying to become MOTU, I’ve found myself wanting to be able to pull source packages for a specified release, for example, download source for lxc on precise, even if I’m using raring. Although you can do this if you setup apt with all the releases and then use pinning, or doing a setup like this, I wanted an easier way. So I decided to glue together rmadison and dpkg-source and create a tool called “get_source”. This is how it works.

get_source.py -r <release> -p <package>

Pulling the source for bc on oneiric:

get_source.py -r oneiric -p bc

Grabbing lxc on precise:

get_source.py -r precise -p lxc

Seems pretty simple and it is!

The tool relies on outside helpers to do the hard work, namely rmadison and dpkg-source, so you’ll need those installed to use it. Please give it a try and send in feedback and fixes. If you’re a developer you’ll note that I even have unit-ish tests, please add more if you make some fixes for corner-cases.

bzr branch lp:~mfisch/+junk/get_source

How It Works

  1. Run rmadison and build a list of packages + versions per release
  2. Find the release we care about. We now know the package name, version, and release name.
  3. Using some hueristics, download the dsc file.
  4. Read and parse the dsc file to find the filenames for the orig file and diff and/or debdiff
  5. Download the orig and diff/debdiff files
  6. Use dpkg-source -x to extract it

Alternatives and Issues

When I started this, I figured it would be simple, but I was mistaken. There is lots of variation on filenames and locations in the archives, for example:

  1. I had originally planned to just go grab http://url/pool/main/<package first letter>/package/package_version.<extension>, but it’s not quite that simple. First, not all packages use standard names, some have a diff.gz, some a debian.tar.gz. Then some packages use xz and some use gz.  Native packages won’t have a diff at all (I think), and right now I know my code won’t support that.
  2. There’s also the question of package directory. alsa-base for example comes from the directory “alsa-driver”. I plan on grabbing this information from apt-cache show, but even that will not solve the issue if I’m on raring and the package was elsewhere in precise. This is also not yet supported in this version.
  3. Packages like angband have a version of 1:3.0.9-1, and the “1:” portion is not included in the filename. The code now supports this.

I found these cases by making this app work for a package and then randomly trying more and more packages to find and hopefully fix new cases. The worry I have is that there are hundreds more corner-cases that I don’t handle. Given all these issues, I’m still releasing this code for other people to test, but perhaps someone has simpler solutions to the problems above? Even better, maybe someone has already written a better tool, which I’ll gladly use!

Read more
pitti

PackageKit has a “WhatProvides” API for mapping distribution independent concepts to particular package names. For example, you could ask “which packages provide a decoder for AC3 audio files?

$ pkcon what-provides  "gstreamer0.10(decoder-audio/ac3)"
[...]
Installed   	gstreamer0.10-plugins-good-0.10.30.2-2ubuntu2.amd64	GStreamer plugins from the "good" set
Available  	gstreamer0.10-plugins-ugly-0.10.18-3ubuntu4.amd64	GStreamer plugins from the "ugly" set

This is the kind of question your video player would ask the system if it encounters a video it cannot play. In reality they of course use the D-BUS or the library API, but it’s easier to demonstrate with the PackageKit command line client.

PackageKit provides a fair number of those concepts; I recently added LANGUAGE_SUPPORT for packages which provide dictionaries, spell checkers, and other language support for a given language or locale code.

However, PackageKit’s apt backend does not actually implement a lot of these (only CODEC and MODALIAS), and aptdaemons’s PackageKit compatibility API does not implement any. That might be because their upstreams do not know enough how to do the mapping for a particular distro/backend, because doing so involves distro specific code which should not go into upstreams, or simply because of the usual chicken-egg problem of app developers rather doing their own thing instead of using generic APIs.

So this got discussed between Sebastian Heinlein and me, and voila, there it is: it is now very easy to provide Python plugins for “what-provides” to implement any of the existing types. For example, language-selector now ships a plugin which implements LANGUAGE_SUPPORT, so you can ask “which packages do I need for Chinese in China” (i. e. simplified Chinese)?

$ pkcon what-provides "locale(zh_CN)"
[...]
Available   	firefox-locale-zh-hans-10.0+build1-0ubuntu1.all	Simplified Chinese language pack for Firefox
Available   	ibus-sunpinyin-2.0.3-2.amd64            	sunpinyin engine for ibus
Available   	language-pack-gnome-zh-hans-1:12.04+20120130.all	GNOME translation updates for language Simplified Chinese
Available   	ttf-arphic-ukai-0.2.20080216.1-1.all    	"AR PL UKai" Chinese Unicode TrueType font collection Kaiti style
[...]

Rodrigo Moya is currently working on implementing the control-center region panel redesign in a branch. This uses exactly this feature.

In Ubuntu we usually do not use PackageKit itself, but aptdaemon and its PackageKit API compatibility shim python-aptdaemon.pkcompat. So I ported that plugin support for aptdaemon-pkcompat as well, so plugins work with either now. Ubuntu Precise got the new aptdaemon (0.43+bzr769-0ubuntu1) and language-selector (0.63) versions today, so you can start playing around with this now.

So how can you write your own plugins? This is a trivial, although rather nonsense example:

from packagekit import enums

def my_what_provides(apt_cache, provides_type, search):
    if provides_type in (enums.PROVIDES_CODEC, enums.PROVIDES_ANY):
        return [apt_cache["gstreamer-moo"]]
    else:
        raise NotImplementedError('cannot handle type ' + str(provides_type))

The function gets an apt.Cache object, one of enums.PROVIDES_* and the actual search type as described in the documentation (above dummy example does not actually use it). It then decides whether it can handle the request and return a list of apt.package.Package objects (i. e. values in an apt.Cache map), or raise a NotImplementedError otherwise.

You register the plugin through Python pkg-resources in your setup.py (this needs setuptools):

   setup(
       [....]

       entry_points="""[packagekit.apt.plugins]
what_provides=my_plugin_module_name:my_what_provides
""",
       [...])

You can register arbitrarily many plugins, they will be all called and their resulting package lists joined.

All this will hopefully help a bit to push distro specifics to the lowest possible levels, and use upstream friendly and distribution agnostic APIs in your applications.

Read more
Martin Pool

You may start getting “Failed to fetch” error messages when updating your software sources (e.g. through “apt-get update” or “Reload package information” in Synaptic), which may be due to a bug we’ve just cleaned up in Launchpad’s PPAs.

The error looks like this:

  W: Failed to fetch http://ppa.launchpad.net/chromium-daily/ppa/ubuntu/dists/maverick/Release
  Unable to find expected entry  restricted/binary-i386/Packages in Meta-index file (malformed Release file?)

  E: Some index files failed to download, they have been ignored, or old ones used instead.

Fixing the cause of the error

Here’s how to fix it. In your terminal, type:

 sudo gedit /etc/apt/sources.list

In the editor, look for PPA sources — these are URLs that feature the ppa.launchpad.net domain. In this example, someone has set up the ~ubuntu-x-swat/x-updates PPA incorrectly:

 deb http://ppa.launchpad.net/ubuntu-x-swat/x-updates/ubuntu maverick main
 deb http://ppa.launchpad.net/ubuntu-x-swat/x-updates/ubuntu maverick restricted
 deb http://ppa.launchpad.net/ubuntu-x-swat/x-updates/ubuntu maverick universe
 deb http://ppa.launchpad.net/ubuntu-x-swat/x-updates/ubuntu maverick multiverse

All of these refer to different components within the same PPA. PPAs only have the first component, so you should delete lines for PPAs that don’t end in “main”. Watch out for lines that wrap, as with the “restricted”, “universe” and “multiverse” examples above.

You may also need to check source lists under the /etc/apt/sources.list.d directory.

Note: you should leave your standard Ubuntu sources, and any non-PPA sources, just as they are.

If you’re not sure how to do this, pop into #launchpad on freenode and one of the Launchpad community will help.

Why this has happened

The Debian-style archives used by Ubuntu are often divided into different components. With Ubuntu, you’ve probably heard of at least “main” and “universe”.

PPAs don’t use these components. However, a bug in Launchpad meant that, until December, PPAs were published with a number of different components. All of these components were empty and there was no way to publish anything to them.

Today, we started to remove these empty components from PPAs. The only impact we anticipate is that anyone whose sources.list referenced these components in a PPA will now see an error when performing an apt-get update or similar.

No packages are being deleted and anyone with a correctly defined sources.list will be able to carry on just as before.

(Originally from Julian.)

Read more

One of the best resources we have (that people seemingly don’t use enough) is apt.ubuntu.com. 

In the beginning we tried doing apt:// links, but that required websites and programs supporting it, so now we have a URL that you can just give to a person like http://apt.ubuntu.com/p/banshee, they clicky, and it fires up the Software Center and prompts them to install it. 

Try it:

Install Banshee

This is way handier than telling people “apt-get install blah …”, easier to use, and very handy for tweeting, etc. Well, apt.ubuntu.com is ugly, and doesn’t work on Chrome, which is a bummer for me.

Well, Marco Ceppi has been working on this, check it out:

As it ends up, mpt has an entire specification on how this is going to work, so Marco implemented it, here’s a demo: http://apt.marcoceppi.com/

Marco’s ready to take this to the next level, but he needs translations. Luckily, the project is in Launchpad, so any help you can lend a hand would be appreciated: https://translations.launchpad.net/apturlredirector

(Oh, feel free to snag the code, submit branches, etc.)

Read more
Chad Miller

Ubuntu 9.10 (“Karmic Koala”) has some neat new features, and I want to bring your attention to a few in turn.  I am not taking credit for any of them. I didn’t have much to do with any of these wonderful features, unless I mention it.

First, Ubuntu 9.10 boots fast.  Compare the speeds of the last two MSFT Windows versions to the last two Ubuntu versions. That’s interesting, eh? When you sit down to use a computer, you have something in mind you want to do. Every second of your life the system wastes is a second you might forget what you want to do. We hate wasting your time.

There’s more, too.

The booting software that ships in Ubuntu 9.10 is still safe, stable, and slow, relative to what else the Ubuntu folks have made. If you have a non-solid-state, regular old rotational disk-drive, then you can shave off maybe another 20 seconds by using a more specialized linux kernel and read-ahead implementation. How? It’s dead simple! From a terminal (Applications, Accessories) run these three commands:

sudo add-apt-repository ppa:ubuntu-boot/ppa
sudo apt-get update
sudo apt-get dist-upgrade

When that is better tested, it might be installed by default.

Read more