Canonical Voices

Posts tagged with 'build system'

bmichaelsen

But I believe in this and it’s been tested by research
— The Clash, Death and Glory

Thanks to Norbert’s efforts, the LibreOffice project now has a Jenkins setup that not only gives us visibility on how healthy our master branch is, with the results being reported to the ESC regularly: In addition it allows everyone easily testing commits and branches on all major LibreOffice platforms (Linux, OS X, Windows) just by uploading a change to gerrit. Doing so is really easy once you are set up:

./logerrit submit                      # a little helper script in our repo
git push logerrit HEAD:refs/for/master # alternative: plain old git
git review                             # alternative: needs to install the git-review addon

Each of the above commands alone send your work for review and testbuilding to gerrit. The last one needs an additional setup, that is however really helpful and worth it for people working with gerrit from the command-line regulary.

So, what if you have a branch that you want to testbuild? Well, just pushing the branch to gerrit as suggested above still works: gerrit then will create a change for every commit, mark them as depending on each other and testbuild every commit. This is great for a small branch of a handful of commits, but will be annoying and somewhat wasteful for a branch with more than 10-15 commits. In the latter case you might not want a manual review for each commit and also not occupy our builders for each of them. So what’s the alternative, if you have a branch ${mybranch} and want to at least test the final commit to build fine everywhere?

git checkout -b ${mybranch}-ci ${mybranch} # switch to branch ${mybranch}-ci
git rebase -i remotes/logerrit/master      # rebase the branch on master interactively

Now your favourite editor comes up showing the commits of the branch. As your favourite editor will be vim, you can then type:

:2,$s/^pick/s/ | x

To squash all the commits of the branch into one commit. Then do:

git checkout -                                   # go back to whatever branch we where on before
git push logerrit ${mybranch}-ci:refs/for/master # push squashed branch to gerrit for testbuilding
git branch -D ${mybranch}-ci                     # optional: delete squashed branch locally

Now only wait for the builder on Jenkins to report back. This allowed me to find out that our compiler on OS X didnt think of this new struct as a POD-type, while our compilers on Linux and Windows where fine with it (see: “Why does C++ require a user-provided default constructor to default-construct a const object?” for the gory details). Testbuilding on gerrit allowed me to fix this before pushing something broken on a platform to master, which would have spoiled the nifty ability to test your commit before pushing for everyone else: Duly testing your commit on gerrit only to find that the master you build upon was broken by someone else on some platform is not fun.

The above allows you to ensure the end of your branch builds fine on all platforms. But what about the intermediate commits and our test-suites? Well, you can test that each and every commit passes tests quite easily locally:

git rebase -i remotes/logerrit/master --exec 'make check'

This rebases your branch on master (even if its already up to date) and builds and runs all the tests on each commit along the way. In case there is a test breakage, git stops and lets you fix things (just like with traditional troubles on rebases like changes not applying cleanly).

Note: gerrit will close the squashed branch change if you push the branch to master: The squashed commit message ends with the Change-Id of the final commit of the branch. So once that commit is pushed, the gerrit closes the review for the squashed change.

Another note: If the above git commands are too verbose for you (they are for me), consider using gitsh and aliases. Combined they help quite a lot in reducing redundant typing when working with git.


Read more
bmichaelsen

It’s so hard when it doesn’t come easy

It’s so hard when it doesn’t come fast

– So Hard, Taking The Long Way, Dixie Chicks

So, LibreOffice 4.2 is released, FOSDEM is over, was very nice and I am back home in Hamburg after a week in London. I missed the LibreOffice UX Hackfest for that, which I heard was also awesome. So without further ado, here are the slides from my quick talk at FOSDEM:

(direct link if you are watching this on a planet that does not support embedded speakerdecks: https://speakerdeck.com/sweetshark1/liberated-build-system-mission-accomplished)

and some errata for it: On slide 13 it says “the same file is also hardlinked from workdir/” — thats not true for quite a while already. LibreOffice keeps around exactly one copy of a library, unlike the confusing three copies that we had in LibreOffice 3.3. This should be a lot less confusing to the curious first time contributor.

Reviewing all these changes in toto, it became how much we simplified getting involved with LibreOffice through this. As the lyrics quoted above say: “Back when we started, we didn’t know how hard it was”.

If there is just one number to take away from all these slides, its that a noop rebuild for LibreOffice on a three year old developer notebook with the distro provided GNU make 3.81 takes just 17 seconds(*). And slide 7 shows still some possibilities to still speed things up beyond that — and while at current speeds it might not be worth it on Linux, it might be worthwhile for e.g. Windows, which is traditionally rather slow when it comes to file I/O.

On a related note, over time we improved the way new contributors can submit their changes on our instance of gerrit in many ways. Thanks a lot to David, Norbert and Robert for the work on this. One only has to look at one of daily digests generated from activity on gerrit and imagine we would still get one mail for each change, update and merge to the mailing list for manual patch tracking as we did in the early days. Thanks a lot also to Mathias Michel for his work on the script!

So if you haven’t done that yet, consider graping an EasyHack and get started!

A copy of the original .odp is also available at FOSDEM or on the LibreOffice wiki.

(*) This includes checking 1.3GB of generated c++ dependency files for some >8000 object files, which we simplify to <350MB.


Read more