Canonical Voices

caribou

As many around me know, I have recently moved to a new job with Canonical.  This journey started quite funnily on April 18th by a Support Sprint event in our Montréal’s office. I say funnily because, while I now lives in France, near Versailles, I was born in northern Québec and I have lived in Montréal for three years before moving to France.

This Sprint week was a great occasion to meet with my new colleagues and to jumpstart my exposure to my new job by learning as much as I could about Natty, both on server and desktop with those responsible for providing top class support to our customers.  But most of all, it was an occasion to meet face to face, to interact with colleagues, learn to know them and understand a bit more about the context of how things are done at Canonical.

Then a little while after coming back home, I got that email from Jono Bacon, encouraging us to do more blogging. Talk about a cultural shock ! For years, I pushed myself into maintaining an internal blog at my prior job, worrying about the possibility that it would be seen as a waste of my time by some manager.  I even seen a few occurrences where very good “internal” bloggers were forced to abandon their blogging activities because of some politically incorrect statement that did not please some high ranking manager.  So being encouraged to go out on the open and talk about what I do, how it is to work here at Canonical, how I think I can help improve our customer’s experience did made my day.

So why did it took me so long to react, to start writing ? Call it paranoia, historical worries, I don’t know. I also wanted to take the opportunity to take out some of the english posts out of my french speaking blog, which required some reworking of my own personal WordPress infrastructure.  But that’s all done now. It’s time to go ahead and start sharing how it is like to “really” be a part of the Open Source effort.

Read more
caribou

crashdc : going Beta

That’s it ! crashdc is now officially beta software.

I just uploaded the 0.6-1 bits on the sf.net server which will be the official first fully functional release.

I’m also doing a presentation at Solution Linux 2010 tomorrow at 2pm.

So go ahead and download, test and bring back your results.

Read more
caribou

An example of crashdc output

Quick post to tell you that I have uploaded an example of the output that crashdc produces. It is using the newly developped BASIC mode (output from an ADVANCED mode is too big to post here).

You can view it here : http://cariblog.kamikamamak.com/crashdc-example-in-basic-mode/

Read more
caribou

crashdc : An update

It’s been a while since I haven’t posted about crashdc.  Not that nothing happened, just that I didn’t get to it.

First of all, the development tree is now public and you can see it here :

http://crashdc.svn.sourceforge.net/viewvc/crashdc/

I had to wait for my employer to allow me to contribute crashdc to the public so it took a little while to get done. During that process, I had to produce architectural diagrams that might be of some interest so I am posting them here.

For instance, this diagram shows how crashdc fits into a normal system installed and configured with kdump.  This is the automated way of using crashdc which can also be invoked from the command line.

Architectural diagram of crashdc

Architectural diagram of crashdc

In this one, you will find a high level flowchart of crashdc’s functionalities.

crashdc's flowchart

crashdc's flowchart

It highlights how crashdc works internally.

Currently, the specifics of the command lists (using built-in or custom list) is not implemented. I’m finishing testing of crashdc on all delivered kernels of RHEL5, SLES10 and SLES11 on both i386 and x86_64 before going into this.

Hopefully, I will have something close to ‘beta’ flavor by end of february if my daytime work allows for it. I’ll keep you posted.

Read more
caribou

crashdc : The architecture

So you might want to know more about how crashdc is organized.  Well here are a few pointers.

crashdc is a shell script that is called from a kdump mechanism which is triggered when a vmcore dump file gets created. The file it creates is named crash-data-{date}.txt Unfortunately (for me), this mechanism is different in all three implementation that crashdc supports :

  • RHEL5   : kdump_post
  • SLES10 : KDUMP_TRANSFER
  • SLES11 : KDUMP_POSTSCRIPT

It can also be run interactively on an existing vmcore to generate the crash-data-{date}.txt file afterward.

I will try to go in a little more details about each of the automated mechanisms. But it is important to know that crashdc itself is identical for all three distributions. The only thing that changes is the run-crashdc-{distro}.sh shell that is executed by each one of those mechanisms.

RHEL5′s kdump_post

This is the most straightforward. When uncommented, the kdump_post variable in /etc/sysconfig/kdump is setup to run the /var/crash/script/kdump_post.sh script. This script is in fact a symlink to /usr/bin/run-crashdc-rhel5.sh which takes charge of finding vmcore’s location, and to invoke crashdc.

SLES10′s KDUMP_TRANSFER
SLES10′s implementation is more complex. When KDUMP_TRANSFER is defined, the command it invokes becomes responsible for reading vmcore’s data out of /proc/vmcore and to store in in a file. Otherwise, it is the save_core shell function from /etc/init.d/kdump that does that.

So /usr/bin/run-crashdc-sles10.sh has an identical copy of the save_core function (yes I copied it) that will copy the vmcore file. The rest of the script does the same thing than for RHEL5, which is to invoke crashdc with the appropriate parameters.

SLES11′s KDUMP_POSTSCRIPT
Once again, SLES11′s implementation is different.  When invoked from witihin the kexec environment, the script pointed to by KDUMP_POSTSCRIPT runs in an environment where the / file system is memory resident, and the real disk-based / file system is mounted in /root.  So the /usr/bin/run-crashdc-sles11.sh knows all so it is still able to invoke crashdc correctly.

Those three mechanisms are all KDUMP specific handles. This mean that they run right after a kernel panic, when the KDUMP kernel is running and before a reboot has been done.

Post Reboot processing
Some system administrators might not like the idea that unnecessary processing is done in that context. This is why one last script has been developped : /etc/init.d/crashdc This script can be used when the system reboots to its regular environment to generate the crash-data-{date}.txt file.

With this method, the crash-data-{date}.txt will only get created when the system reboots on the regular kernel.  The current limitation is that it will only process the latest vmcore created in order to avoid extra processing in case of multiple kernel panics.  Yet, there should be an option to manually process all vmcores where crash-data-{date}.txt has not been found.

Right now, this script only exists in my mind so there are chances that it might evolve quite a lot before it gets done.

So I hope that this brief post is clear enough to highlight the main mechanisms used by crashdc.

Read more
caribou

crashdc

Sorry for those of you who are used to read me in french, but you will see appear here, from time to time, notes, posts and information about an opensource project that I’m working on : crashdc.

crashdc is a set of scripts to be used in conjunction with Dave Anderson’s crash tool. Without crash, crashdc is useless. With it, crashdc can automate data collection when a new kernel crash dump occurs.

Dave Anderson has kindly accepted to include crashdc in the crash package, so when crashdc will be ready, this is where you will be able to find it. I also created a sourceforge project for it called obviously crashdc, where you can find standalone bits and information about crashdc itself.

The Sf environment does not intend to ack as a parallel project, but just a place for me to host my development which itself will make its way to the crash rpm.

Right now, don’t look for the crashdc bits : they’re not available yet.  Being employed by a major computer constructor, I must first clear out the project internally so I can have the right to publicly distribute crashdc. This shoudln’t be too long. And since crashdc is mostly Alpha code right now, this would not bring you much to get it as it is.

So bear with me a little longer, and it should become available for a first round of testing soon.

Read more
caribou

Little update on LinuxCOE

Of course, nothing has happened last month since I was on vacation. But just before leaving, I was able to get my RPM working on RHEL and SLES. More testing is required, but I’m getting there.

Now a suggestion from Bryan makes a lot of sense, but will probably delay availability of the RPM packages. We should not make the RPM for SystemDesigner available unless the RPM are also available for the overlays. This is true since not much can be done without an overlay.

So I now have to figure out how to generate the O/S images from a script, instead of including them right in the RPM as it is done currently. This is needed so the RPM don’t contain binary archives and, more importantly, don’t get to heavy in size.

So I will get my good ol Camel Book out and get going.

Read more
caribou

I’m back at my desk, after two days in Grenoble with Bryan Gartner and Bruno Cornec. This is really good to be able to see them face to face. Virtual communications are great, but it always help to share some portion of reality for a little while.

We had a lot of discussion on short term goals for LinuxCOE, waystation setup and such. Got some tests packages going while they were at work setting up a waystation on eurolinux.

Now I have working packages for :

  • SystemDesigner
  • RHEL overlay
  • Fedora Overlay
  • The Documentation

I should be able to have a few more soon, since the overlays are rather easy to create, once the first one is OK.

Read more
caribou

A little LinuxCOE update.

I got the docs module working earlier today, and it looks like I got the first overlay module almost working also : the RHEL module.

Now I need to figure out what is needed to do in order to get normal setup working, document the whole thing and add it to the docs module.

Read more
caribou

Joining LinuxCOE

I recently indicated in a previous post that I was joining the LinuxCOE Open Source effort.

It has been a few weeks since (well actually almost a month now) this post and some work has already been done on creating a RPM package for SystemDesigner. We now have a roughly working package but without any of the Distro modules, it is not of much interest. The next step is to get one of the Distro module packaged and to see if I can get a working COE environment.

Now regarding this specific post, it is in english, as will probably be most of the LinuxCOE related posts, mainly because I don’t want to restrict my audience to the french speaking people. Right now, I don’t think that LinuxCOE has much of an audience in this area.

I also decided to mix these posts in my public personal blog, because I’m lazy and don’t want to go through the trouble of writing and maintaining two blogs. It is better for me to concentrate on working on LinuxCOE rather than being a full-time blog manager.

So keep posted for more LinuxCOE news.

Read more