Canonical Voices

Posts tagged with 'libreoffice'

bmichaelsen

Dear Prudence, won’t you come out to play
Dear Prudence, greet the brand new day

-- Dear Prudence, White Album, Beatles

Yesterday, I put a build of LibreOffice 3.6.7 in the according Ubuntu PPA. As LibreOffice 3.6.7 is the last minor release update for the 3.6 series this is a good time to have a look back and see how our choice of a train release model pays off. For this I had a look at the number of bugfixes and the regressions over the series. First lets look if there were any regressions in the minor series: that is, if there is something which worked in 3.6.0, but which stopped working in 3.6.7. For LibreOffice 3.6.7 we look at:

  • known issues (that is: they have to be reported e.g. at our Bug Submission Agent)
  • they have to be well-triaged to be a real problem (that is: They cant be in bug status NEEDINFO or UNCONFIRMED — which would mean that they are incomplete)
  • they have to be marked as a regression
  • they have not been there in 3.6.0, otherwise they would not be a minor release regression
  • they are not fixed in one of the 3.6.1 — 3.6.7 minor release updates.
  • they have to affect Linux

At the time of publishing 3.6.7 this query results in bugzilla replying with its infamous  “Zarro Boogs found” — so there are no known, well-triaged regressions in LibreOffice 3.6.7 vs. LibreOffice 3.6.0 at the time of release on Ubuntu.

Does this mean that LibreOffice will give you a hard guarantee on this? No, the release train model of LibreOffice is more important: It keeps everyone accountable and gives users a reliable date and time for a new release to test and use. So LibreOffice will not “stop the train”, if such a bug candidate is found. If you for example extend the above query to all platforms you will see it return 5 bugs from other platforms. It is unlikely that there are “real” minor release regressions though: those issues are just not confirmed to have been there at 3.6.0 already.

Enough talk about regressions in the 3.6 series — lets have a look at the positive things in minor releases: bug fixes. Despite LibreOffice 3.6.7 on Linux having no known, well-triaged release regressions, it has 547 bugs fixed against 3.6.0. So for Ubuntu users installing 3.6.7 this is the picture:367fixesandregressions

 

So much for the quality of minor releases, but what about major releases? Well, for major releases there is one additional important factor: timely reporting. So lets have a look at LibreOffice 4 regressions that have been filed in a timely manner: That is, between the tagging of the first alpha on November 20, 2012 and the release of LibreOffice 4.0.0 on February 7, 2013. Bugzilla shows that currently more than 95% of these are already resolved, even though LibreOffice 4 will still see a few more minor updates.

For regressions reported in the alpha and beta phase before the tagging of the first release candidate on January 9, 2013 its even more impressive: All of those regressions are resolved by now.

I assume the statistics for LibreOffice 4.1 to become even more impressive as the LibreOffice bug triage contest was hugely successful and sure will help fast tracking the triage of bugs, getting them ready to be fixed by a developer quickly. Thanks to Joel Madero, Joren De Cuyper and Robinson Tyrone for organizing this awesome event and big “Thank you!” too to all the volunteers taking part in this.

Which finally brings us to the little tune quoted at the top-right of the post: The earlier you test LibreOffice and report an issue, the better for you and for the product. With LibreOffice 4.1 approching fast, I invite you to test what will become LibreOffice 4.2 by downloading a daily build, playing with its exciting new features and get involved on #libreoffice-qa while listening to Sir Paul doing his magic on the bass over … Sir Paul doing his magic on the drums.


Read more
bmichaelsen

With a rebel yell: “more, more, more”
More, more, more.

Rebel Yell, Billy Idol

This weekend the LibreOffice community will meet again in Hamburg for the third Hackfest at this location:

335px-HHHackfest2013here is how it looked last year:

Hackers at Hackfest Hamburg 2012

Hackers on the last Hamburg Hackfest

Like last year, this years Hackfest gets kicked off with a meet and greet at the Schachcafe on Friday 20:00 o’clock local time. Think of it like the “beer event” at FOSDEM, which helps everyone warm up for the event –  except it will not be February and not freezing cold. Looking this picture, it likely wont be much like cold FOSDEM at all:

Schachcafe

All details can be found on the Hackfest 2013 wiki page. Thanks to Lanedo for sponsoring this event and also big thanks to Attraktor.org for hosting us again!


Read more
bmichaelsen

One and Only

I am the one and only nobody I’d rather be

I am the one and only you can’t take that away from me

– Chesney Hawkes, The One and Only

Just a short note: I have missed the exact date in the release madness for Ubuntu 13.04 Raring, but a few days ago something important silently happened: All supported Ubuntu releases are now shipping with LibreOffice by default, as trusty old Ubuntu 10.04 LTS (Lucid Lynx) reached its end of support for the desktop. So we now have these supported releases:

  • Ubuntu 12.04 LTS (Precise Pangolin) with LibreOffice 3.5
  • Ubuntu 12.10 (Quantal Quetzal) with LibreOffice 3.6
  • Ubuntu 13.04 (Raring Ringtail) with LibreOffice 4.0
  • and upcoming: Ubuntu 13.10 (Saucy Salamander) with LibreOffice 4.1

Also the following releases (which are not supported anymore) have been done in addition:

  • Ubuntu 11.04 (Natty Narwhal) with LibreOffice 3.3
  • Ubuntu 11.10 (Oneiric Ocelot) with LibreOffice 3.4

Looking back in time at the angstridden, not-acting-but-reacting excitement of the early days and comparing it with the way we are really pushing the envelope now, we have really come a long way, improving with every step on the way. Well worth a celebration with one of the most cheesy 1990ies hits ever – Thanks to everyone, who was and is part of this!


Read more
bmichaelsen

Sitting on a cornflake, waiting for the van to come.
Corporation t-shirt, stupid bloody Tuesday.
– I am the walrus, The Beatles

Although the “OOo does not print on Tuesdays” OpenOffice.org-bug is long fixed, OpenOffice.org never indemnified Tuesday for its loss in reputation. This is unacceptable and Tuesdays rage at the event has even passed on to its successful successor: LibreOffice.

Now, Tuesday is a weekday and as such, money does not mean that much to it. After some consultation with Tuesday, it was concluded that the only way to indemnify Tuesday was to make the other weekdays suffer the same fate. Since the set of weekdays is luckily limited, this was easily considered technically feasible, even if at some minor inconvenience for the users of LibreOffice.

Toolbar popup of the new feature

Toolbar popup of the new feature

To limit the impact for users, instead of silently failing to act when requested to print on a non-Tuesday, for the comfort of the users of LibreOffice a notification was added that explains the situation. Tuesday — after some heated discussion — gave license to this modification.

message on Non-Tuesdays

message on trying to print on non-Tuesdays

It is intended to ship this extension pre-bundled with all LibreOffice releases until Tuesdays rage is soothed. It is hoped this will happen quickly as both contributors and users of LibreOffice are known for their lack of sympathy for monopoly in any kind, shape or form — explicitly including the exclusive right of a weekday to print. For those of you excited to try out this thrilling new feature right now, the extension is available for download here:

It has to be noted that the extension is written purely in Python and is completely self-contained: it can either be treated as the oxt to be installed or as a zip file containing the source code: unpack it with the archive program of your choice, modify it to your hearts content and run the script called ‘build’ that you find in there. This will recreate a new (modified) extension.


Read more
bmichaelsen

autopkgtests for adults

“That’s not a knife — THAT’s a knife.”

– Michael J. Crocodile Dundee

I recently worked a bit to see this line showing up in my favorite editor:

ubtree0t-junit-subsequentcheck PASS

LibreOffice has multiple sets of testsuites and during the build of the package we run them all (although not yet on all platforms). However, LibreOffice depends on ~1/3 of main — so there are a lot of things that might break LibreOffice. A lot of things just break at build time and not at run time and thus prevent the such a broken package to enter the archive in the first place as we run the tests during the build already. Thats as: unless the breakage is caused by an update of a dependency of LibreOffice, therefore making the LibreOffice package in the archive FTBFS (or at least broken) in a sneaky way. Thats whats happened for the e.g. the libjpeg, boost, kdelibs examples above.

But lets keep those aside for now and concentrate on the runtime issues. Running the tests at build-time is a good early-warning already and prevents some serious breakage to enter the archive. On the other hand, these tests do not run against LibreOffice as we install it in the system from packages — it runs them against a installation set aside in the build tree. While I can not come up with an immediate example, where LibreOffice was broken when installed from the packages in a way that would have been detected by tests but missed when run against the in-tree installation, it would still be good to have the additional confidence that:

  • LibreOffice passes the tests as installed on the system
  • LibreOffice is not broken at runtime by some update of a dependency

In short: Its highly desirable to test that LibreOffice does still run and work as the ground below it keeps moving — this is even more important when Ubuntu is considering to move towards a more rolling way of releasing. And we have a means to do that: Autopkgtests.

So, what was needed to get this working for LibreOffice?

First, some parts of the testsuites are quite large and — as we run the tests during the build anyway — are already build during the build. Therefore it made sense to package these, which was done very early in the cycle (actually: during UDS).

Second, we would need to get LibreOffice to run the tests without trying to build the product. That originally wasnt as easy as it may seem. For one, LibreOffice build system was reasonably expecting that you need a product to test it and therefore would have dependencies on the product to be build. In addition, when I started considering this, we still had a lot of the old build system around — which was a pain to bend to your will. Luckily, these times are over. So, by now a patch changing some ~15 lines get us what we want.

Third, we need a config_host.mk (the output of ./configure), so that we can run the LibreOffice build. And for that, we unfortunately need the build dependencies (which are generated) of LibreOffice — otherwise we would not really test what we did build. But for a missing feature of autopkgtests, we can not reuse the existing dependencies, but have to do manual double bookkeeping there. Im not thrilled by the prospect of hunting false positives there. Some possible ways out would be:

  • to package the config_host.mk file into the package containing the other testsuite helpers, but that would make that package architecture dependendant
  • or to not really specify the dependencies at all and pragmatically and greedily request the restrictions needs-root and breaks-testbed and then — as we are root now — run this before starting the tests:
    apt-get build-dep -y libreoffice

Finally, we should be able to run:

apt-get build-dep libreoffice
apt-get install libreoffice-subsequentcheckbase
apt-get source libreoffice
cd libreoffice-*
./debian/tests/junit-subsequentcheck

and this should run the tests locally and headless — and indeed it does and the tests happily finish and report success. Great, lets quickly check if it also runs in the ‘official’ VM with:

run-adt-test

Nope, and this is why I choose the Crocodile Dundee quote below the title for this post: The VM fails before it even starts the tests — it does not even have enough discspace to copy in the LibreOffice source package. This needs to be fixed on the side of the image, there is nothing on the test side that could fix this. But to test if LibreOffice would finish if only the image could handle it, I began cannibalizing, removing one after another the directories of the icon-themes, translations and external sources from the package, each time getting a bit further: from failing to start to failing when installing the 501 additional packages and so on. With this hollowed out package, I could verify: yes, the autopkgtest would pass in the image, if only it had enough discspace.

Finally, once this is in the archive (or ppa) you will also be able to run:

apt-get build-dep libreoffice
apt-get install libreoffice-subsequentcheckbase
apt-get source libreoffice
cd libreoffice-*
libreoffice '--accept=pipe,name=blickenlights;urp'&
./debian/tests/junit-subsequentcheck 'connect:pipe,name=blinkenlights' 1

This will connect to the LibreOffice you started in the second-to-last step (which is not headless, but running in your session) and run the tests against it. The “1″ tells it not to use parallelization, but just run one suite at a time, as otherwise you have a very good chance to lock/hang your own session by compiz (or the dash or other components) being mightly confused by all the windows flashing up and closing in fast progression. With “1″ you might still get some test failures (mostly from the a11y integration) — but at least your session will survive:

ZO RELAXEN UND WATSCHEN DER BLINKENLICHTEN.

Addendum:

Preparing an adt-image with:

./bin/prepare-testbed -r raring amd64 -S12GB

seems to solve the issue. The “df -h” at the start of the test reports some 3GB of free space (with 2.6GB being needed still to create a rw-copy of the source tree after that point). So 12GB is likely the size the images on Jenkins roughly currently need (plus maybe another 1GB of wiggle room).


Read more
bmichaelsen

“Ich danke der Academy für das Erkennen von Talent.”

-- Ich danke der Akademie, Kettcar

So, I just updated the LibreOffice version for Ubuntu Raring in the LibreOffice PPA to the 4.0.1~rc2 version, that was announced as 4.0.1 final yesterday. As we dropped the old binfilter file import upstream in LibreOffice 4, LibreOffice fits again in the ppa with all localizations — previously the PPA versions were restricted to a subset of languages, as the buildds for the PPAs might run out of disc space otherwise. Oh, and a boring sidenote just for completeness: LibreOffice on Ubuntu LTS has been getting a stable release update to 3.5.7 a while ago. And now that 4.0.1 rc2 is declared final, I expect it to be sponsored soon to Ubuntu Raring proper.

I would like to take this opportunity to thank everyone who made this possible:

  • Rene Engelhard for his work on the Debian LibreOffice packaging, making all of this possible
  • Boaz Dodin and Rico Tzschichholz for their tireless work and initiative on backporting current releases
  • Christopher M. Penalver for churning through LibreOffice bugs in launchpad and on freedesktop, syncing and triaging them
  • Benjamin Drung for fixes, tweaks and contributions to the LibreOffice packaging both at Debian and Ubuntu

Will there be backports of 4.0.1 to older series? Well, an indiscreet look in Ricos PPA, suggests that it wont take long for them to end up in the LibreOffice ppa. Steadily, the LibreOffice team is growing — its exciting to see so many volunteers contribute to this!


Read more
bmichaelsen

One

“One Ring to rule them all”

– J.R.R. Tolkien, The Lord of the Rings

It has been done.

LibreOffice is now build by one instance of make that is aware of the whole dependency tree. According to my master development build (that is: a build without localization, help, extensions) yesterday, this instance of make now knows about

126.501 targets from 1.717 makefiles

and has a complete view of how they relate to each other. The memory usage of make is at 207 MiB, only slightly overshooting the initial estimations done in the early days of gbuild of 170-190MiB (counting in that the codebase changed a lot in two years, the estimate is actually really good). Given that recursive make is considered harmful and that LibreOffice — one of the biggest open source projects, with huge dependencies and doing releases on three platforms (Windows, OS X and Unix — a lot more if you separate the different Unix flavours), can do this — there is little excuse left for other projects to not follow suit.

On my machine, checking if anything needs to be rebuild in LibreOffice now takes ~28.7sec (or 37.2sec when also running the default sanity checks along that). That might sound a lot, but consider the scale! And it is a long way from the old OpenOffice.org build system that we came from: Just from my memory, it took about 5 Minutes to do that on the old build system. On Windows it took almost 30 Minutes to find out that there is nothing to do. One of my earliest talks (Slide 29) on the topic of gbuild compared the performance of partial build, if you find these numbers hard to believe. Oh, and of course you still can check for updating only a subset of LibreOffice (a “module” – e.g. Writer) and that takes only 2-3 seconds even for the biggest ones.

How gbuild spends the 37 seconds to ensure that nothing need to be rebuild: orange = reading the definition of targets (singlethreaded, CPU-bound), grey = stat'ing and checking the filesystem, blue = running sanity tests (multithreaded)

How gbuild spends the 37 seconds to ensure that nothing need to be rebuild: orange = reading/parsing the makefiles (singlethreaded), grey = stat’ing and checking the filesystem, blue = running sanity tests (multithreaded)

Does this difference in performance matter? As Linus argued to eloquently in his google tech talk on git: Yes, it does. Because it enables different ways to work, that just were not possible before. One such example is that we can have incremental build tinderboxes like the Linux-Fedora-x86_64_22-Incremental one, which comes with a turnaround of some 3-5 minutes most of the time by now and quickly reports if something was broken.

There are other things improved with the new build system too. For example, in the old build system, if you wanted to add a library, you had to touch a lot of places (at minimum: makefile.mk for building it, prj/d.lst for copying it, solenv/inc/libs.mk for others to be able to link to it, scp2 to add it to the installation and likely some other things I have forgotten), while now you have to only modify two places: one to describe what to build and one to describe where it ends up in the install. So while the old build system was like a game of jenga, we can now move more confidently and quickly.

Touching the old build system was like a game of jenga. Except that it wasnt fun. (Photo: Copyright CC BY-NC-SA 2.0 Jose Hernandez)

Then there is scalability: The old build system did not scale well beyond 4-8 jobs as it had no global notion of how make jobs where running. As we see CPU architectures become more important that have slower, but cheaper cores, this is getting increasingly relevant. Do you have a 1000 core distcc cluster you want to testdrive? LibreOffice might be the project you want to try.

Finally, the migration to gbuild is a proof of how amazing the community is that is growing around the project: While I set up the initial infrastructure for gbuild, the hard work of migrating over 200 modules (each the size of your average open source project) to it without breaking on one of three platforms or disrupting the ongoing development on features and bugfixes was mostly done by a crowd of volunteers. Looking back, I doubt the migration to gbuild would have been completed in reasonable time in an environment less inviting to volunteers and contributors — it was the distribution of the work that made this possible. So the credit for that we now can profit from the benefits of gbuild really goes to these guys. Big kudos for everyone working on this, you created something amazing!

Addendum: This post has been featured on lwn and led to a spirited discussion there.

Notes:

For estimating the number of targets, I used:

make -f Makefile -np all slowcheck|grep 'File.*update'|wc -l

For the memory usage:

pmap -d $(ps -a|grep make|cut -f1 -d\ )|egrep -o writeable/private:.[0-9]+K|cut -f 2 -d\

Read more
bmichaelsen

“Instant Karma’s gonna get you

Gonna knock you right on the head”

– John Lennon, Instant Karma

As I said a month ago:

“As I firmly believe freedom to be essential regardless of scale or context, I hereby pledge to donate 10 Euro to witness.org for every power of 2 in Euros donated to the Document Foundation in December 2012.”

So as the number are in now lets check how far we got with that:

Donations to the Document Foundation in December 2012 What I pledged to donate to witness.org for that

more than 1 €

10 €

more than 2 €

20 €

more than 4 €

30 €

more than 8 €

40 €

more than 16 €

50 €

more than 32 €

60 €

more than 64 €

70 €

more than 128 €

80 €

more than 256 €

90 €

more than 512 €

100 €

more than 1024 €

110 €

more than 2048 €

120 €

more than 4096 €

130 €

more than 8192 €

140 €

more than 16,384 €

150 €

In total, the donations to the Document Foundation in December 2012 amounted to:

21,711 €

To everyone who donated: Thank you very much!

I donated 225$ to witness.org yesterday, which more than covers my pledge. So this is completed and I am quite happy how much we can achieve together — you can see on the board-dicuss list, how these donations are intended to be spend in 2013. We will get lots of Hackers and Contributors to Hackfests and events and thereby make LibreOffice even better! To please keep the donations coming:

donate

Here is how some Hackfests look like:

Hackfest Munich 2012

Hackday Brazil 2012

LibreOffice Conference 2012, Berlin

LibreOffice Conference 2012, Berlin

Oh, and: LibreOffice will be at FOSDEM 2013 in a few weeks like every year so far!


Read more
bmichaelsen

“Instant Karma’s gonna get you

Gonna knock you right on the head”

– John Lennon, Instant Karma

Last year before Christmas, I donated 220 Euro to the Document Foundation and I hope it helped as a tiny contribution to all the awesome things that happened with the Document Foundation and LibreOffice in this year, like getting volunteers to our events, providing food and drinks at Hackfests etc..

This year I will try to do this a bit different. I think donating to the Document Foundation is as important as ever: the developments of the last year show that we are approaching a vital decision point and we are accelerating while doing so. While the Document Foundation is fighting for freedom in software and productivity, its easy to forget that to profit from the benefits of this work even more basic freedom and rights — easily taken for granted — are required. Still many have to fight hard for those.

Witness.org is one organization that helps to make human rights and freedom universal by letting no violation pass unobserved. As I firmly believe freedom to be essential regardless of scale or context, I hereby pledge to donate 10 Euro to witness.org for every power of 2 in Euros donated to the Document Foundation in December 2012. That is:

  • 1 Euro donated to the Document Foundation in December 2012 => I will donate 10 Euros to witness.org
  • 2 Euros donated to the Document Foundation in December 2012 => I will donate 20 Euros to witness.org
  • 4 Euros donated to the Document Foundation in December 2012 => I will donate 30 Euros to witness.org
  • 8 Euros donated to the Document Foundation in December 2012 => I will donate 40 Euros to witness.org
  • 16 Euros donated to the Document Foundation in December 2012 => I will donate 50 Euros to witness.org

So please: Make me pay!

donate


Read more
bmichaelsen

“Linux does endless loops in six seconds”

– allegedly Linus Torvalds, 1995 at the First Dutch International Symposium on Linux

 
The  LibreOffice 4.0 Test Marathon will start on Friday, and this is a call to arms to join in on the fun! ;)

GIn_BHS400-800

Now, while a lot of people will join the test marathon just for the good cause, for others it might need some candy to persuade them. So here is some sweet sugar:

Bibisect 4.0

Dicke Bertha had her burn-in in the last days. She compiled 146 full builds of the LibreOffice master branch from May 2012 up till now.

It took her some 25 hours churning away with a load average >32 — interrupted only twice: Once because I left two other LibreOffice compiles lying around on tmpfs before starting this and then ran out of tmpfs buildspace — whoopsie!

A second time the build run was briefly interrupted because a cppunittest loved Bertha so much it went into an endless loop. And despite Linus’ claim above, Linux even 17 years later does not do endless loops in six seconds (also: where is my flying car?).

Just for fun, here are some ccache stats from the full run:

cache hit (direct)               1404831
cache hit (preprocessed)          144192
cache miss                        677524

I combined that with the builds from the older bibisect runs into one big respository. Thus in this 4.1 GB download you have:

  • 262 full builds of LibreOffice (~16MB per install)
  • covering a range of 26365 commits since August 2011
  • thats one build every ~100 commits

Thus if you notice a regression that has been introduced at some point in the last 16 months, you can run a binary bisect and after testing 9 times in different versions of LibreOffice if the bug is there, you will have the regression pinned down to a range of ~100 commits, at which point it will be much easier to sack those responsible fix the bug quickly. So how can you help with this? In two ways:

  • read HowToBibisect and start bibisecting ;)
  • if you dont want to do that, you can still help to find bugs that are regressions and can be tested well (on Ubuntu) and mark them with “bibisectrequest” in whiteboard status

Testcase management

Thanks to the work of Yifan, Sophie, Petr and many others the Document Foundation now has a MozTrap instance — and in this marathon we will find out how its workflow integrates with the rest of LibreOffices QA and development processes. So what is MozTrap? Their own documentation webpage explains lengthly and eloquently: ” MozTrap is a test case manager.”. So we could again run to wikipedia and read that short article about that, but that would not be much fun.

If you cut past all the buzzwords, you find that ‘test case management’ means, we have a web tool that:

  • manages a set of things to test with some kind of software (in the simplest case this might be: “Does LibreOffice start?”)
  • shows testers simple instructions to perform and report back if everything works as expected
  • then allows QA people do all kinds of statistics and voodoo on this data ;)

So, it is really simple and you are invited to join in! As a bonus you can make sure that developers know now, if something broke since the last release and since the release in February is still a bit off have more time to fix it until then!

Not every bug is as beautiful as this one … (Photo: Jessica Lucia, Creative Commons license)

Ubuntu packages for alpha/beta releases

On the LibreOffice prereleases PPA you find a packaged version of LibreOffice 4.0 beta1 for Ubuntu Raring that you can use in the test marathon. There is also a version of LibreOffice 4.0 alpha1 for Ubuntu 12.10 (Quantal) there, which should you also be able to use for discovering bugs.

I have a beta1 package for Ubuntu 12.10 ready too, but it is cheating quite a bit, because I disabled Python as the beta1 requires Python3.3 which is not directly available on Ubuntu 12.10.

Ubuntu stable release updates

Just for completeness: For Ubuntu 12.04 LTS a stable release update to the final (for the series) 3.5.7 has been uploaded (actually it has even been updated while waiting in the queue). For Ubuntu 12.10, version 3.6.4 is currently in probation in the LibreOffice PPA. Should no problems turn up, it will be proposed become a update soon.


Read more
bmichaelsen

When I first came here, this was all swamp.

Everyone said I was daft to build a castle on a swamp, but I built in all the same, just to show them.

It sank into the swamp.

So I built a second one. That sank into the swamp.

So I built a third. That burned down, fell over, then sank into the swamp.

But the fourth one stayed up. And that’s what you’re going to get, Lad, the strongest castle in all of England.

– King of Swamp Castle, Monty Python and the Holy Grail

LibreOffice has a — by now mostly undeserved — reputation to be hard to build. To kill that urban myth, Peter Baumgarten made a playterm session showing what needs to be done to get a build of LibreOffice started on Ubuntu 12.10, but the same should work on all Debian derivatives. So if you once used gentoo and miss that transcendental feeling of compile commands flying by, watch this (since it is running in a small single core VM, you can even read the stuff as it scrolls by): LibreOffice Build Demo.
Or you can do the commands along, and have your first shiny new build with that!
You can also download the recorded session from bug fdo#41291 and replay it with ttyrec. Awesome work, Peter!
UPDATE:
Now also on youtube:

Read more
bmichaelsen

…the Linux philosophy is “laugh in the face of danger”. Oops. Wrong one.

Linus Torvalds

LibreOffice 4.0 alpha1 has been silently tagged in the LibreOffice repositories three days ago. Yesterday, I uploaded a build of it for Ubuntu Quantal to the libreoffice-prereleases PPA. Its a pre-release, an alpha — essentially just a named daily build — and will kill your dog and eat your children. Also, it will not cleanly install yet without removing your old version of LibreOffice first. But it has lots of nasty new features and tasty new bugs. So if you are a bug hunter and want to help to make the version of LibreOffice in Ubuntu Raring the best ever, this is your chance to test and tease this version and get it the most polished one ever in a Ubuntu release. Happy testing and bug reporting!

P.S.: At just three weeks after the UDS this is likely the earliest date of making available the upcoming LibreOffice version for the upcoming Ubuntu release. I am looking forward for this to pay off in less bugs even more stability of LibreOffice upon Ubuntu release.


Read more
bmichaelsen

Dicke Bertha online.

Oh Lord, won’t you buy me a Mercedes Benz ?
My friends all drive Porsches, I must make amends.

– Janis Joplin, Mercedes-Benz

So one cycle is over, UDS is done, blueprints are mangled — so far nothing different for me from anyone else working on Ubuntu around this time. Ah, but then there is one more thing that is special for a LibreOffice maintainer: new hardware. Shortly before UDS I consider updating my hardware for the next cycle. This time, I did — here is my new baby:

Although I name my machines usually after the element of having the same proton number as the last part of the local IP, this time it was different: She already had a name, before I even could make up my mind about IPs: Bastian and Claus from picom, who went though some extra trouble to build this beast, affectionately and unofficially called her “Big Bertha” — presumably a historical reference. Given that this “desktop” with 32 cores and 32GB RAM builds LibreOffice in 18 minutes flat (or 3 minutes 20 seconds with a warm ccache), I could not help but to keep that name.

In total this gives me this as workhorses:

  • left: Thinkpad W520 — i7-27020QM with 16GB RAM (once called ‘mobile buildserver’ by my colleagues at Canonical)
  • center: Sun X7197A screen and a Unicomp Customizer keyboard for Big Bertha
  • also center: Nexus 7 (running Ubuntu), Galaxy SII (running Android), trusty old Casio CFX-9850GB plus
  • right: Ideapad S12

The Casio is clearly the weakest computer of those, but still is more powerful than the first one I ever owned. And finally, next to the table:

  • Big Bertha
  • below the printer: my old IBM x3800 that I would fire up for emergency extra compile power or when it was getting too cold in the room (now decommissioned by Big Bertha, and on sale at ebay)
  • behind the printer: old Asus Z53 running fileserver duties
  • in front of the printer: original IBM Model M, for backup if the Unicomp should fail me
  • the trusty old Sun Ultra 24, now replaced by Big Bertha (so decommissioned, and already sold)

Not in picture: A few routers and two Pandaboards. I guess I used up all my showboating credits for a few years now and promise to keep silent for a while on this.


Read more
bmichaelsen

Dr. Soran: What if I told you I found a new truth?

Picard: The nexus?

– Star Trek: Generations

So UDS is over and you can find the results of what was discussed there showing up bit by bit on my blueprints this week. Some highlights:

Ubuntu will extend its daily testbuilds (plus testsuite runs) to three scenarios:

  • Running a master build with mostly LibreOffice internal library versions
  • Running a release build with mostly system libraries
  • Running a QEMU ARM release build

The first is intended to find upstream bugs and regressions and I intend to have it integrated it with the set of other upstream tinderboxes churning away at constantly building and testing LibreOffice. The second is more important to find regressions in the system libraries that LibreOffice depends on — as the release branch should be free of build or test failures in general given the review requirements for pushing to it. So it will test for stuff outside of LibreOffice breaking LibreOffice (like lp#1017125 or lp#745836) on Ubuntu. The third will hopefully improve the visibility of bugs in the ARM build.

Speaking of ARM: Ubuntu is easily installable on an Nexus 7 and I had to try it out obviously. Not only Ubuntu also LibreOffice is running quite nicely on it as you can see in this video (note that I even opened the document template twice in my inability to use a mouse — I prefer keyboards):

Link: LibreOffice on Ubuntu Nexus 7 and a Model M
Other stuff discussed at the UDS: Some boring packaging detail, fixing up the upstream packaging with the hope of getting more logic directly into upstream instead of fixing around it in our ./debian/rules. We also had a discussion on bibisect and if it can be applied to Ubuntu as a whole — discovering some overlap with http://snapshot.debian.org/ — still bibisect might be something to consider for example for GNOME.
I also learned that https://errors.ubuntu.com/ is shaping up nicely, and expect that to become even more awesome combined with phased updates — esp. for LibreOffice.

Finally: Valve/Steam on Ubuntu — awesome!

Steam/Valve at UDS


Read more
bmichaelsen

LibreOffice Quality Heroes

Its been a while since the LibreOffice QA team started to look for HardHacks and give them to the LibreOffice development team for their kind evaluation. While the HardHacks idea is not yet 100 days in office, it is a good idea to look how they turn out and especially give kudos to those who dared and succeeded in tackling these terrifying beasts. Here are the HardHack solvers:

It is very encouraging to see so many volunteers involved in fixing these bugs — dispelling the myth that fixes for such hard nuts to crack cannot possibly be provided by a community. Kudos to all of those who gnarled away one or more of these and a special thanks to the volunteers!

Apart from the development side, there is the bugwrangling side too: Making a good HardHack isnt easy either — it often takes some good bug triage instinct and skills to get all the information needed to have a good report for development. Luckily, there are some who are great at that kind of work too — here are the top bugwranglers in October 2012:

  • 315 bug contributions: Roman Einsle
  • 300 bug contributions: Rainer Bielefeld
  • 161 bug contributions: Julien Nabet
  • 139 bug contributions: Joel Madero
  • 109 bug contributions: bfoman
  • 67 bug contributions: Korrawit Pruegsanusak
  • 54 bug contributions: Alex Thurgood
  • 47 bug contributions: Markus Mohrhard, Michael Meeks
  • 36 bug contributions: Hashem Masoud, Michael Stahl
  • 34 bug contributions: Caolan McNamara, Kaplan Lior
  • 29 bug contributions: leighman
  • 28 bug contributions: Petr Mladek
  • 27 bug contributions: manj_k
  • 26 bug contributions: Florian Reisinger
  • 23 bug contributions: Lionel Elie Mamane, Urmas D.
  • 22 bug contributions: Horst Kaleski
  • 20 bug contributions: Jean-Baptiste Faure
  • 19 bug contributions: stfhell@googlemail.com
  • 18 bug contributions: Uwe Altmann, Andras Timar
  • 16 bug contributions: Regina Henschel
  • 15 bug contributions: Cor Nouws
  • 13 bug contributions: os2sa@yandex.ru
  • 11 bug contributions: Rob Snelders
  • 10 bug contributions: Ivan Timofeev

All in all, LibreOffice is proceeding nicely in doing away with the huge heritage of old bugs: Right now, over 58% off all bugs filed on the LibreOffice bugtracker are already resolved (*). As Florian Effenberger said at the LibreOffice conference: We have only just begin!

You are of course invited to join the developers fixing the HardHacks or the the bugwranglers helping us pinning them down for the developers. Especially, getting started in bugwrangling is easy!

(*) ignoring feature requests

UPDATE: Kohei is such a humble and honest man — he just notified me that I should read fdo#48366 and recheck whom to credit for it: Its really Markus Mohrhard, so one more fix in that provided by unaffiliated volunteer (moving Markus up to position two in that list too).


Read more
bmichaelsen

So the Ubuntu Developer Summit for Raring Ringtail (UDS-R) is starting tomorrow — although not exactly a slide-heavy conference, this is still an excellent opportunity to point out some of the new features available in LibreOffice 3.6.2 that comes with the just released Ubuntu 12.10 (Quantal Quetzal):

The PackageKit/Session Installer integration is implemented in UNO, that allow extensions and macro creators to trigger the installation of software from trusted archives in general — quite a nifty feature in itself. As we have this now in place, in the future we can also use it to complete the LibreOffice install by adding missing packages for certain actions that are not available in the default Ubuntu installation (which leaves out some parts of LibreOffice).

For the next LibreOffice release, the template installer has to be adopted to the cool work of Rafael on the template dialog. I plan to upstream both the unity and the session installer integration for the next LibreOffice release.

This series of screenshots shows all of this in action:

Click to view slideshow.
Installing the template pack from LibreOffice

Read more
bmichaelsen

You’re so vain, I’ll bet you think this song is about you.

– You’re so vain, Carly Simon

I just uploaded all my past and current slides to speakerdeck, for your pleasure and ridicule. From the just finished LibreOffice conference 2012 in Berlin, this now has:

The QA and RelEng roundtable had no slides and the same is essentially true for “LibreOffice Development Workshop” on Sunday at the UbuCon.
If you click through all the slidedecks, you will note that for the gerrit talk, I only fasttracked through the recycled “Crowdsourcing Reviews”-Talk from Paris, just to leave the stage for David and Robert to talk about the interesting and new stuff. They deserve all the credit really, as anyone having seen the talk or the stream can attest!

To the freshly volunteered canaries: I will send a you a mail on how to proceed soon, promised!

Finally, I have to highlight this awesome video by Cloph, which played right after the welcome talk. It shows how awesome, diverse and buzzing the LibreOffice community has become — One year of activity on the LibreOffice core repository:


Read more
bmichaelsen

LibreOffice rules

I love deadlines. I like the whooshing sound they make as they fly by.

– Douglas Adams

So yesterday was Ubuntu Quantals feature freeze and two important features made it in with the libreoffice-3.6.0~rc4-0ubuntu3 package:

  • the awesome work by Antonio Fernandez from Aentos (sponsored by Canonical) on the feature/unitymenus branch
  • some PackageKit integration for LibreOffice on Ubuntu (will be upstreamed for 3.7)

I’ll go into the details of both and what they mean for endusers in later blogposts as they each deserving one of their own. Also, for the current stable Ubuntu 12.04 LTS release we have in:

This allows users to be up-to-date with LibreOffice on the current stable release of Ubuntu too. Ok, so we are running a tight ship for LibreOffice on Ubuntu, but that is kind of expected, right? Well, yes — but I want to reach out to another point, which is: how we got there. To illustrate that, I want to select some random datapoints on the debian/rules file, which is the core file that makes a package out of a plain upstream build. The situation has improved since the beginning of LibreOffice (LibreOffice-3.3.0-1):

  • The debian/rules file, while probably still one of the most “impressive” of all of Ubuntu, shrank by 12% since that first LibreOffice release on Debian and is now at 3173 lines. Removing complexity here is a Good Thing(tm) and hopefully will continue.
  • Less than 3% of the lines are different between Ubuntu and Debian by now in the rules file. That is a Good Thing(tm). While there certainly are some differences between the distributions, if such vendor changes need modifications in the rules file it is usually a sign of bad design.
  • Finally, since that first release, there have been 640 commits made touching that file — 90% of those by Rene Engelhard, 10% by me. In total 1092 lines have been added and 1541 lines have been removed — meaning at least 1/3 of that file has been rewritten. Now, only 1.5 KLOC delta does not sound much, but with a turnaround time of 1.3 days on some architectures even a dedicated machine would not keep up with that for every commit. Also: I bet the rules file of a lot of other Debian/Ubuntu packages will fit in that 1.5 KLOC (or even in the 449 lines we lost since the first release).

Finally, this tasty pie chart shows that the red to yellow upstream parts of the rules file are not that big. Most of the rules file is concerned with mapping the build we want to do to both ./configure switches and dependencies(*) and splitting up the build result:

This “last mile” of getting LibreOffice on Ubuntu and Debian is often overlooked. I still think it is quite an important (although mostly invisible) job. An explicit “Thank You!” to Rene for all his hard and continuous work on this.

(*) Yes, LibreOffice has a rule to generate its own control file. Manually maintaining that would be truely painful.


Read more
bmichaelsen

So when your waiting for the next attack
You’d better stand there’s no turning back

– Iron Maiden, The Trooper

I teased in the last post, that there will be a second important change in LibreOffice QA in this week. Well, here it is: In an combined effort by contributors in QA and development, we will try to improve the way that QA and development interact. The LibreOffice EasyHacks are a well known success story of our project. Now we introduce the LibreOffice HardHacks. When I first threw around that buzzword, one initial reaction by a core developer was:

HardHacks sounds too hard to me. I would be afraid to look at such bugs :-)

This is what is hiding behind that buzzword:

  • The QA team will start in their next call to identify the 5 most critical bugs that need attention. It will repeat that from now on in each of their calls every second week.
    To qualify, these bugs have to be:

    • Among the MAB (most annoying bugs)
    • be triaged as far as possible (e.g. reproduction scenario, version the bug was introduced if it is a regression, best: bibisect for regressions)
    • so they are both important/urgent and well-prepared
  • These bugs will be handed over to the core developers
  • ESC will try to find a core developer for each bug to look at it and report back in the next week

The aim of this is to keep awareness of these critical bugs high in development and having some kind of qualified feedback in the given timeframe (ideally one week, but at least until the next QA call). “Qualified feedback” might also be: “I still need this information to get forward with this bug.” or “This cannot easily be solved because of foo” giving the QA and the reporter a hint on what is needed to push the issue forward. Hopefully this will prevent that awkward silence on some bugs where everyone thinks its on the other to push this forward and help us collecting all the needed information on the most important and urgent bugs. It will certainly also keep the most pressing quality issues present in the minds of the developers, which might otherwise strive to implement the next shiny and exciting new feature a little too early or too often. Keeping these bugs present will thus also help getting these though solved too.

Lots of bugs

Lots of bugs around – HardHacks are about those that hardened their chitin armor in unfair ways (source: wikimedia)

As you can see from the comment above these are the bugs that even the core developers have a healthy dose of respect of. But that should not scare you: If you are a developer in the LibreOffice community and already have a few patches for EasyHacks under your belt, feel free to look at the HardHacks too. The core developers will be relieved for any support they can get. Also note that these bugs are really hard nuts to crack, so failure is an option here. However, on the other hand, the benefits of solving the bug are huge: both core developers and QA will be full of utmost respect and gratitude for such an achievement.

The QA team will also try to blog about the results we got for the batch of bugs that were selected — this bit of extra fame is hopefully is another piece of motivation for the developers looking at bugs. Oh, and of course your help in cornering the bugs from both the QA and the development side is most welcome!


Read more
bmichaelsen

The truth is rarely pure and never simple.

– Oscar Wilde, The Importance of Being Earnest

Two important things happened in LibreOffice QA this week. The first one has not gone unnoticed by a lot of users: Florian Reisinger closed a lot of old and idle bugs (no activity for more than six month) in state NEEDINFO on our friendly bugzilla. As you can see from this table we have bugs in all kinds of states:

  • the “user/reporter states”: NEEDINFO, UNCONFIRMED
  • the “bugwrangler states”: NEW, REOPENED
  • the “developer states”: ASSIGNED, RESOLVED/WORKSFORME
  • the “verifier state”: RESOLVED/FIXED
  • the “janitor states”: VERIFIED, CLOSED, RESOLVED with resolutions INVALID, DUPLICATE, WONTFIX, NOTABUG, NOTOURBUG

The lifecycle of a bug ideally makes the bug go through the states UNCONFIRMED -> NEW -> ASSIGNED -> RESOLVED/FIXED -> VERIFIED -> CLOSED quickly. If it gets stuck somewhere, its up to the group named in the quotes to help this move on: For a bug in state NEEDINFO, it is expected from the user/reporter to come up with missing information. For a bug in state UNCONFIRMED, it is needed for somebody (best not the reporter, but a different user) to make sure that the bug is valid. This is not too hard, the details are described on the Bug Triage page on the wiki. LibreOffice is a community project and the best way to get your issues forward is to trade favours: Confirm the bugs of somebody else and let him confirm yours (of course, you should still check if the bugs of the other guy/gal are really valid!).

The bugwranglers then take care to make developers aware of the most important bugs in states NEW, REOPENED according to development capacities. If a bug sticks longer in here, it is usually not the bugwranglers to blame, but the fact that there are just more critical bugs to pipe to the developers at this point in time. However bugwranglers also need to make sure that a bug has all the important information on the bug before it gets to development. For example, when it is a regression: what was the first version that misbehaved, stacktraces, test case, test documents. If there are multiple bugs which are about the same severity to choose from to hint development at and one has all that information while others do not — QA will tend to go with the ones with most information on them.

Firefly (Creative Commons by Jessica Lucia via nextdoornature.org

The job of the bugwranglers is to find those bugs that stick out (Photo: Jessica Lucia, Creative Commons license)

The developers then crunch their heads on trying to solve these issues. And if they need a break and some cheering up, they will look at a WORKSFORME bug that seems to have been fixed to QA/users, but nobody claimed to have done that intentionally. If they can see that the bug was indeed solved, the bug can move to CLOSED/FIXED.  The latter is not a high priority task at all though.

If a bug is set to RESOLVED/FIXED by a developer, the fix can be verified by pretty much anyone on a daily build: while the developers are usually trustworthy folks, it does not hurt to check. A very good side effect of this is, once the daily build is installed, one can easily torture it a bit and see if the trustworthy folks of the last sentence recently broke something in a horrible way: If so they should get notice ASAP via a new bug report.

The janitor states are mostly dead: We then consider that bug finished. Now some bugs were reported more than a year ago, when the freedesktop bugzilla did not yet have a UNCONFIRMED default state — those were moved from NEW to NEEDINFO as they still needed confirmation. If no other user confirmed your bug since then and there was no activity on the bug for more than half a year, the bug would now have been moved to RESOLVED/INVALID with Florians cleanup. In such cases, find somebody to confirm that what you describe is indeed a bug and ask him to reopen that bug for you. Note you dont need to wear a QA uniform, QA hat and badges to do so, although you might earn them by doing Good Work(tm). The QA team is looking for more hands and you can best find us on the QA mailing list or on IRC and other means. Another good way to get started are the QA EasyHacks.

Oh, the second important thing happening in QA this week? Its still work in progress and this post was long enough, so I will do a post on its own about it. The buzzword will be: LibreOffice HardHacks.


Read more