Canonical Voices

Posts tagged with 'gerrit'


But I believe in this and it’s been tested by research
— The Clash, Death and Glory

Thanks to Norbert’s efforts, the LibreOffice project now has a Jenkins setup that not only gives us visibility on how healthy our master branch is, with the results being reported to the ESC regularly: In addition it allows everyone easily testing commits and branches on all major LibreOffice platforms (Linux, OS X, Windows) just by uploading a change to gerrit. Doing so is really easy once you are set up:

./logerrit submit                      # a little helper script in our repo
git push logerrit HEAD:refs/for/master # alternative: plain old git
git review                             # alternative: needs to install the git-review addon

Each of the above commands alone send your work for review and testbuilding to gerrit. The last one needs an additional setup, that is however really helpful and worth it for people working with gerrit from the command-line regulary.

So, what if you have a branch that you want to testbuild? Well, just pushing the branch to gerrit as suggested above still works: gerrit then will create a change for every commit, mark them as depending on each other and testbuild every commit. This is great for a small branch of a handful of commits, but will be annoying and somewhat wasteful for a branch with more than 10-15 commits. In the latter case you might not want a manual review for each commit and also not occupy our builders for each of them. So what’s the alternative, if you have a branch ${mybranch} and want to at least test the final commit to build fine everywhere?

git checkout -b ${mybranch}-ci ${mybranch} # switch to branch ${mybranch}-ci
git rebase -i remotes/logerrit/master      # rebase the branch on master interactively

Now your favourite editor comes up showing the commits of the branch. As your favourite editor will be vim, you can then type:

:2,$s/^pick/s/ | x

To squash all the commits of the branch into one commit. Then do:

git checkout -                                   # go back to whatever branch we where on before
git push logerrit ${mybranch}-ci:refs/for/master # push squashed branch to gerrit for testbuilding
git branch -D ${mybranch}-ci                     # optional: delete squashed branch locally

Now only wait for the builder on Jenkins to report back. This allowed me to find out that our compiler on OS X didnt think of this new struct as a POD-type, while our compilers on Linux and Windows where fine with it (see: “Why does C++ require a user-provided default constructor to default-construct a const object?” for the gory details). Testbuilding on gerrit allowed me to fix this before pushing something broken on a platform to master, which would have spoiled the nifty ability to test your commit before pushing for everyone else: Duly testing your commit on gerrit only to find that the master you build upon was broken by someone else on some platform is not fun.

The above allows you to ensure the end of your branch builds fine on all platforms. But what about the intermediate commits and our test-suites? Well, you can test that each and every commit passes tests quite easily locally:

git rebase -i remotes/logerrit/master --exec 'make check'

This rebases your branch on master (even if its already up to date) and builds and runs all the tests on each commit along the way. In case there is a test breakage, git stops and lets you fix things (just like with traditional troubles on rebases like changes not applying cleanly).

Note: gerrit will close the squashed branch change if you push the branch to master: The squashed commit message ends with the Change-Id of the final commit of the branch. So once that commit is pushed, the gerrit closes the review for the squashed change.

Another note: If the above git commands are too verbose for you (they are for me), consider using gitsh and aliases. Combined they help quite a lot in reducing redundant typing when working with git.

Read more

At Linaro we want to get some metrics for the patches we submit upstream, so we've built a system based on Patchwork that parses email messages to extract patches and categorize them by project/author. This works fine for projects that use mailing lists to keep track of patches (e.g. the kernel, qemu, etc), but it doesn't work for one project to which Linaro has already contributed and expect to contribute a lot more: the Android Open Source Project, which uses Gerrit to track changes.

For that reason we decided to pull that data directly from AOSP's Gerrit instance into our system. Unfortunately, the only public API provided by Gerrit is this one over SSH, which doesn't give a piece of data that is very important to us: the date a change was proposed.

Luckily, James pointed me to this discussion where a neat trick is suggested: watch the requests your browser sends when rendering to figure out how to use Gerrit's internal JSON-RPC API. Yes, it is not a public API (so we should not expect it to be stable), and having to watch your browser requests to learn how to use it is not the kind of documentation I'd like, but that was all we had (ok, I could check Gerrit's source, but since that's Java I'd rather watch the requests sent by the browser) so it had to do.

After experimenting a bit I was able to get a list of changes proposed by a given person as well as all the details of a given change. Here are the methods one can use for that:

Retrieving a list of changes

  endpoint: /gerrit/rpc/ChangeListService
    method: allQueryNext(query, pos, page_size)

  Return up to page_size items that match the given query whose changeSortKey is lower than pos.

  # This will get the list of changes authored by
  curl -i -X POST -H "Accept: application/json,application/jsonrequest" \
       -H "Content-Type: application/json; charset=UTF-8" \
       --data '{"jsonrpc":"2.0","method":"allQueryNext","params":["owner:","z",10],"id":1}'\

Getting the details of a change

  endpoint: gerrit/rpc/ChangeDetailService
    method: changeDetail(id)

  Return the details of the change with the given ID.

  # This will get all details of change 16615
  curl -i -X POST -H "Accept: application/json,application/jsonrequest" \
       -H "Content-Type: application/json; charset=UTF-8" \
       --data '{"jsonrpc":"2.0","method":"changeDetail","params":[{"id":16615}],"id":1}'\

Pretty simple, right? Just note that you need to specify your charset in the Content-Type header or else you'll get a server error from Gerrit, and the
JSON-RPC requires the 'id' param to correlate a response with its request, but you don't need to worry about that if you're doing things synchronously.

That was all I needed to know in order to write the code that keeps the changes authored by Linaro engineers in sync between our system and AOSP's Gerrit instance, and it should be enough for you to get started if you ever need to get data out of a Gerrit instance (assuming the API hasn't changed in the meantime ;).

Read more