Canonical Voices


About a year ago I ordered a pack of 10 atmega328p processors from China to play with. They took a while to get here, and it took even longer for me to get back to them, but a few days ago the motivation to start doing something finally appeared.

I’ve never actually played with AVRs before, and felt a bit like I was jumping a step in my electronics enthusiast progress by not diving into its architecture a bit more deeply. Also, despite the obvious advantages of ARM-based chips these days, the platform is still interesting in some perspectives, such as its widespread availability, low price in small quantities, and the ability to plug them in a breadboard and do things without pretty much any circuitry.

To get acquainted with the architecture and to depart from things I work on more frequently, the project is so far taking the shape of an assembly library of functionality relevant for developing small projects, built mainly around binutils for the AVR. I did end up cheating a bit and compiling the assembly code via avr-gcc, just to get the __do_copy_data initialization routine injected, so that I don’t have to pull up the .data section from program memory into RAM manually.

I started running the test programs with the chip itself, with the help of a Pirate Bus, to see if the whole setup was sound. Once it worked a few times, I moved on to use the simulavr simulator to make the process of running and debugging more comfortable. In addition to being able to attach gdb, and trace execution, one of the nice features of simulavr is being able to map a port from the emulated CPU and get bytes written into it sent to an arbitrary file in the outer world. That means we can easily implement a trivial println-like function in assembly:

.set    STDOUT, 0x20

loop:   ld  r17, Z+
        cpi r17, 0
        breq done

        sts STDOUT, r17
        rjmp loop

done:   ldi r17, '\n
        sts STDOUT, r17

Printing strings is only helpful if we do have strings, though, and with such a skeleton system there are no interesting ones yet. What we do have are registers, lots of them (32 in total). A good candidate for the next function would then be an itoa-like function that would put the proper bytes in memory for printing.

So, after going down that road for a bit longer, the lack of a proper way to run tests on the created code was an evident show stopper. There’s no way the created code will be sane without being able to exercise it, and write tests that can be rerun at will. Fortunately, it’s easy enough to apply traditional testing practices to such an environment, given the simulator features mentioned.

To drive those tests, a small tool named avrtest was written in Go. It takes an avrtest.list file that looks like this:

devices: atmega328


        ldi     r24, 128 ; dividend
        ldi     r22, 10  ; divisor
        call    div8u
        prnt8u  r24      ; result
        prnt8u  r22      ; divisor
        prnt8u  r20      ; remainder



cycle-limit: 400

        ldi     r24, 128
        call    itoa8u


and runs it, showing the typical test runner output:

% ./avrtest
div8u   ok  (784 cycles)
itoa8u  ok  (356 cycles)

or the typical failure, when appropriate:

div8u   failed: unexpected output

If the failure feels a bit cryptic, all of the intermediary files are kept under the ./_avrtest directory, including a detailed trace file. Here is a snippet of such a trace:

div8u.elf 0x0194: itoa8u      LDI R30, 0x0a 
div8u.elf 0x0196: itoa8u+0x1  LDI R31, 0x01 
div8u.elf 0x0198: itoa8u+0x2  PUSH R17 SP=0x8f6 0x1 
div8u.elf 0x0198: itoa8u+0x2  CPU-waitstate
div8u.elf 0x019a: itoa8u+0x3  LDI R17, 0x30 
div8u.elf 0x019c: itoa8u+0x4  LDI R22, 0x0a 
div8u.elf 0x019e: itoa8u_loop CALL 0x178 SP=0x8f5 0xd1 SP=0x8f4 0x0

Besides that, we should be able to attach gdb to any given test by running the command avrtest gdb <name>. That’s not yet there, but should be pretty soon, after the next cryptic breakage. :-)

That tooling is not organized for a proper release, but I’ll certainly push it up to a public repository as soon as I get a chance to clean up the sandbox.

Read more

As part of one of the projects we’ve been pushing at Canonical, I spent a few days researching about the possibility of extending a compiled Go application with a tiny language that would allow expressing simple procedural logic in a controlled environment. Although we’re not yet sure of the direction we’ll take, the result of this short experiment is being released as the twik language for open fiddling.

The implementation is straightforward, with under 400 lines for the parser and evaluator, and under 350 lines in the default functions provided for the language skeleton: var, func, do, if, and, or, etc.

It also comes with an interactive interpreter to play with. You can install it with:

$ go get

This is a short sample session:

> (var x 1)
> x
> (set x 2)
> x
> (set x (func (n) (+ n 1)))
> x
> (x 1)
> (func inc (n) (+ n 1))
> (inc 42)

Another one demonstrating the lexical scoping:

> (var add
.      (do
.          (var n 0)
.          (func (m) (set n (+ n m)) n)
.      )
. )
> (add 5)
> (add -1)
> n
twik source:1:1: undefined symbol: n

New functionality may be plugged in by providing Go functions. For example, here is a simple printf function:

func printf(args []interface{}) (interface{}, error) {
        if len(args) > 0 {
                if format, ok := args[0].(string); ok {
                        _, err := fmt.Printf(format, args[1:]...)
                        return nil, err
        return nil, fmt.Errorf("printf takes a format string")

func main() {
        err = scope.Create("printf", printf)

It can now greet the world:

$ cat test.twik

(func hello (name)
      (printf "Hello %s!\n" name)

(hello "world")

$ time ./twik test.twik
Hello world!
./twik test.twik  0.00s user 0.00s system 74% cpu 0.005 total

Read more

House decoration by Epson

I bought an Epson multi-functional printer last weekend, and now I have a big and expensive furniture for the house.

This is what one gets into when buying an Epson printer:

Well-priced printers: Printers tend to be subsidized, except for countries such as Brazil, where one has to shell the equivalent of 510 USD for an entry-level printer (the XP-702, for example).

Extremely expensive cartridges: The full set of cartridges for the XP-702, one cartridge with pigmented ink and four smaller cartridges with dye ink, costs the equivalent of 100 USD in Brazil, and together contain about 30ml of ink. The XP-600, that comes with the same model of cartridges (but incompatible, see below), costs 99USD on Amazon, and you get a printer for free.

Trashing of ink: Printers are extremely hungry for ink. An Epson printer, when first turned on, will eat a significant percentage of its initial cartridge just to load itself up. It’ll also trash a good amount of ink whenever one has to unclog its printing heads.

Clogging heads: Printing heads do get clogged. The first printer actually had to be replaced for a new one, because the pigmented black ink wasn’t working properly. The recommendation to avoid clogging heads is to print more often.

Official cartridges are incompatible: Every cartridge comes with a chip. Physically equivalent cartridges cannot be used across equivalent models that are branded differently, because the printer will refuse to recognize the cartridge. Physically equivalent printers have different models so that different regions are forced to use different cartridges.

Cartridges are not refillable: That same cartridge chip has an internal counter that goes down whenever the printer uses ink. When the counter gets to zero, the cartridge is not usable anymore, even if it still has ink in it! That chip also means it’s much harder to find alternative ink, because the alternative solution must have a working chip in it, made exactly to that one model of printer.

No new ink? No printer, no scanner: After the new printer has been replaced due to the clogged head, the replacement printer won’t turn on anymore, because it recognizes that the cartridges have been used before, even if they still have ink. A brand new set of cartridges are necessary to turn it on.

No color ink? No printer either: If any one of the cartridges is considered out of ink (see points above), the printer stops working altogether until the cartridge is replaced for a new one, even if you want to print that black-and-white document you want to deliver today.

That’s a very offensive way to handle consumers. Right now I have an expensive decorative piece that doesn’t work unless I feed it with gold. Instead of doing that, I’ll buy another printer. Not from Epson.

Read more

In an effort to polish the recently released draft of the strepr v1 specification, I’ve spent the last couple of days in a Go reference implementation.

The implemented algorithm is relatively simple, efficient, and consumes a conservative amount of memory. The aspect of it that deserved the most attention is the efficient encoding of a float number when it carries an integer value, as covered before. The provided tests are a useful reference as well.

The API offered by the implemented package is minimal, and matches existing conventions. For example, this simple snippet will generate a hash for the stable representation of the provided value:

value := map[string]interface{}{"a": 1, "b": []int{2, 3}}
hash := sha1.New()
fmt.Printf("%x\n", hash.Sum(nil))
// Outputs: 29a77d09441528e02a27dc498d0a757da06250a0

Along with the reference implementation comes a simple command line tool to play with the concept. It allows easily arriving at the same result obtained above by processing a JSON value instead:

$ echo '{"a": 1.0, "b": [2, 3]}' | ./strepr -in-json -out-sha1


$ cat | ./strepr -in-yaml -out-sha1                 
a: 1
   - 2
   - 3

Or even BSON, the binary format used by MongoDB:

$ bsondump dump.bson
{ "a" : 1, "b" : [ 2, 3 ] }
1 objects found
$ cat dump.bson | ./strepr -in-bson -out-sha1

In all of those cases the hash obtained is the same, despite the fact that the processed values were typed differently in some occasions. For example, due to its Javascript background, some JSON libraries may unmarshal numbers as binary floating point values, while others distinguish the value based on the formatting used. The strepr algorithm flattens out that distinction so that different platforms can easily agree on a common result.

To visualize (or debug) the stable representation defined by strepr, the reference implementation has a debug dump facility which is also exposed in the command line tool:

$ echo '{"a": 1.0, "b": [2, 3]}' | ./strepr -in-json -out-debug
map with 2 pairs (0x6d02):
   string of 1 byte (0x7301) "a" (0x61)
    => uint 1 (0x7001)
   string of 1 byte (0x7301) "b" (0x62)
    => list with 2 items (0x6c02):
          - uint 2 (0x7002)
          - uint 3 (0x7003)

Assuming a Go compiler and the go tool are available, the command line strepr tool may be installed with:

$ go get

As a result of the reference implementation work, a few clarifications and improvements were made to the specification:

  • Enforce the use of UTF-8 for Unicode strings and explain why normalization is being left out.
  • Enforce a single NaN representation for floats.
  • Explain that map key uniqueness refers to the representation.
  • Don’t claim the specification is easy to implement; floats require attention.
  • Mention reference implementation.

Read more

Here is a small programming brain teaser for the weekend:

Assume uf is an unsigned integer with 64 bits that holds the IEEE-754 representation for a binary floating point number of that size.

The questions are:

1. How to tell if uf represents an integer number?

2. How to serialize the absolute value of such an integer number in the minimum number of bytes possible, using big-endian ordering and the 8th bit as a continuation flag? For example, float64(1<<70 + 3<<21) serializes as:


The background for this problem is that the current draft of the strepr specification mentions that serialization. Some languages, such as Python and Ruby, implement transparent arbitrary precision integers, and that makes implementing the specification easier.

For example, here is a simple Python interactive session that arrives at the result provided above exploring the native integer representation.

>>> f = float((1<<70) + (3<<21))
>>> v = int(f)
>>> l = [v&0x7f]
>>> v >>= 7
>>> while v > 0:
...     l.append(0x80 | (v&0x7f))
...     v >>= 7
>>> l.reverse()
>>> "".join("%02x" % i for i in l)

Python makes the procedure simpler because it is internally converting the float into an integer of appropriate precision via standard C functions, and then offering bit operations on the resulting value.

The suggested brain teaser can be efficiently solved using just the IEEE-754 representation, though, and it’s relatively easy because the problem is being constrained to the integer space.

A link to an implementation will be provided next week.

UPDATE: The logic is now available as part of the reference implementation of strepr.

Read more

Note: This is a candidate version of the specification. This note will be removed once v1 is closed, and any changes will be described at the end. Please get in touch if you’re implementing it.



This specification defines strepr, a stable representation that enables computing hashes and cryptographic signatures out of a defined set of composite values that is commonly found across a number of languages and applications.

Although the defined representation is a serialization format, it isn’t meant to be used as a traditional one. It may not be seen entirely in memory at once, or written to disk, or sent across the network. Its role is specifically in aiding the generation of hashes and signatures for values that are serialized via other means (JSON, BSON, YAML, HTTP headers or query parameters, configuration files, etc).

The format is designed with the following principles in mind:

Understandable — The representation must be easy to understand to increase the chances of it being implemented correctly.

Portable — The defined logic works properly when the data is being transferred across different platforms and implementations, independently from the choice of protocol and serialization implementation.

Unambiguous — As a natural requirement for producing stable hashes, there is a single way to process any supported value being held in the native form of the host language.

Meaning-oriented — The stable representation holds the meaning of the data being transferred, not its type. For example, the number 7 must be represented in the same way whether it’s being held in a float64 or in an uint16.

Supported values

The following values are supported:

  • nil: the nil/null/none singleton
  • bool: the true and false singletons
  • string: raw sequence of bytes
  • integers: positive, zero, and negative integer numbers
  • floats: IEEE754 binary floating point numbers
  • list: sequence of values
  • map: associative value→value pairs


nil = 'z'

The nil/null/none singleton is represented by the single byte 'z' (0x7a).

bool = 't' / 'f'

The true and false singletons are represented by the bytes 't' (0x74) and 'f' (0x66), respectively.

unsigned integer = 'p' <value>

Positive and zero integers are represented by the byte 'p' (0x70) followed by the variable-length encoding of the number.

For example, the number 131 is always represented as {0x70, 0x81, 0x03}, independently from the type that holds it in the host language.

negative integer = 'n' <absolute value>

Negative integers are represented by the byte 'n' (0x6e) followed by the variable-length encoding of the absolute value of the number.

For example, the number -131 is always represented as {0x6e, 0x81, 0x03}, independently from the type that holds it in the host language.

string = 's' <num bytes> <bytes>

Strings are represented by the byte 's' (0x73) followed by the variable-length encoding of the number of bytes in the string, followed by the specified number of raw bytes. If the string holds a list of Unicode code points, the raw bytes must contain their UTF-8 encoding.

For example, the string hi is represented as {0x73, 0x02, 'h', 'i'}

Due to the complexity involved in Unicode normalization, it is not required for the implementation of this specification. Consequently, Unicode strings that if normalized would be equal may have different stable representations.

binary float = 'd' <binary64>

32-bit or 64-bit IEEE754 binary floating point numbers that are not holding integers are represented by the byte 'd' (0x64) followed by the big-endian 64-bit IEEE754 binary floating point encoding of the number.

There are two exceptions to that rule:

1. If the floating point value is holding a NaN, it must necessarily be encoded by the following sequence of bytes: {0x64, 0x7f, 0xf8, 0x00 0x00, 0x00, 0x00, 0x00, 0x00}. This ensures all NaN values have a single representation.

2. If the floating point value is holding an integer number it must instead be encoded as an unsigned or negative integer, as appropriate. Floating point values that hold integer numbers are defined as those where floor(v) == v && abs(v) != ∞.

For example, the value 1.1 is represented as {0x64, 0x3f, 0xf1, 0x99, 0x99, 0x99, 0x99, 0x99, 0x9a}, but the value 1.0 is represented as {0x70, 0x01}, and -0.0 is represented as {0x70, 0x00}.

This distinction means all supported numbers have a single representation, independently from the data type used by the host language and serialization format.

list = 'l' <num items> [<item> ...]

Lists of values are represented by the byte 'l' (0x6c), followed by the variable-length encoding of the number of pairs in the list, followed by the stable representation of each item in the list in the original order.

For example, the value [131, -131] is represented as {0x6c, 0x70, 0x81, 0x03, 0x6e, 0x81, 0x03, 0x65}

map = 'm' <num pairs> [<item key> <item value>  ...]

Associative maps of values are represented by the byte 'm' (0x6d) followed by the variable-length encoding of the number of pairs in the map, followed by an ordered sequence of the stable representation of each key and value in the map. The pairs must be sorted so that the stable representation of the keys is in ascending lexicographical order. A map must not have multiple keys with the same representation.

For example, the map {"a": 4, 5: "b"} is always represented as {0x6d, 0x02, 0x70, 0x05, 0x73, 0x01, 'b', 0x73, 0x01, 'a', 0x70, 0x04}.

Variable-length encoding

Integers are variable-length encoded so that they can be represented in short space and with unbounded size. In an encoded number, the last byte holds the 7 least significant bits of the unsigned value, and zero as the eight bit. If there are remaining non-zero bits, the previous byte holds the next 7 bits, and the eight bit is set on to flag the continuation to the next byte. The process continues until there are non-zero bits remaining. The most significant bits end up in the first byte of the encoded value, which must necessarily not be 0x80.

For example, the number 128 is variable-length encoded as {0x81, 0x00}.

Reference implementation

A reference implementation is available, including a test suite which should be considered when implementing the specification.


draft1 → draft2

  • Enforce the use of UTF-8 for Unicode strings and explain why normalization is being left out.
  • Enforce a single NaN representation for floats.
  • Explain that map key uniqueness refers to the representation.
  • Don’t claim the specification is easy to implement; floats require attention.
  • Mention reference implementation.

Read more

The very first time the concepts behind the juju project were presented, by then still under the prototype name of Ubuntu Pipes, was about four years ago, in July of 2009. It was a short meeting with Mark Shuttleworth, Simon Wardley, and myself, when Canonical still had an office on a tall building by the Thames. That was just the seed of a long road of meetings and presentations that eventually led to the codification of these ideas into what today is a major component of the Ubuntu strategy on servers.

Despite having covered the core concepts many times in those meetings and presentations, it recently occurred to me that they were never properly written down in any reasonable form. This is an omission that I’ll attempt to fix with this post while still holding the proper context in mind and while things haven’t changed too much.

It’s worth noting that I’ve stepped aside as the project technical lead in January, which makes more likely for some of these ideas to take a turn, but they are still of historical value, and true for the time being.


This post is long enough to deserve an index, but these sections do build up concepts incrementally, so for a full understanding sequential reading is best:

Classical deployments

In a simplistic sense, deploying an application means configuring and running a set of processes in one or more machines to compose an integrated system. This procedure includes not only configuring the processes for particular needs, but also appropriately interconnecting the processes that compose the system.

The following figure depicts a simple example of such a scenario, with two frontend machines that had the Wordpress software configured on them to serve the same content out of a single backend machine running the MySQL database.

Deploying even that simple environment already requires the administrator to deal with a variety of tasks, such as setting up physical or virtual machines, provisioning the operating system, installing the applications and the necessary dependencies, configuring web servers, configuring the database, configuring the communication across the processes including addresses and credentials, firewall rules, and so on. Then, once the system is up, the deployed system must be managed throughout its whole lifecycle, with upgrades, configuration changes, new services integrated, and more.

The lack of a good mechanism to turn all of these tasks into high-level operations that are convenient, repeatable, and extensible, is what motivated the development of juju. The next sections provide an overview of how these problems are solved.

Preparing a blank slate

Before diving into the way in which juju environments are organized, a few words must be said about what a juju environment is in the first place.

All resources managed by juju are said to be within a juju environment, and such an environment may be prepared by juju itself as long as the administrator has access to one of the supported infrastructure providers (AWS, OpenStack, MAAS, etc).

In practice, creating an environment is done by running juju’s bootstrap command:

$ juju bootstrap

This will start a machine in the configured infrastructure provider and prepare the machine for running the juju state server to control the whole environment. Once the machine and the state server are up, they’ll wait for future instructions that are provided via follow up commands or alternative user interfaces.

Service topologies

The high-level perspective that juju takes about an environment and its lifecycle is similar to the perspective that a person has about them. For instance, although the classical deployment example provided above is simple, the mental model that describes it is even simpler, and consists of just a couple of communicating services:

That’s pretty much the model that an administrator using juju has to input into the system for that deployment to be realized. This may be achieved with the following commands:

$ juju deploy cs:precise/wordpress
$ juju deploy cs:precise/mysql
$ juju add-relation wordpress mysql

These commands will communicate with the previously bootstrapped environment, and will input into the system the desired model. The commands themselves don’t actually change the current state of the deployed software, but rather inform the juju infrastructure of the state that the environment should be in. After the commands take place, the juju state server will act to transform the current state of the deployment into the desired one.

In the example described, for instance, juju starts by deploying two new machines that are able to run the service units responsible for Wordpress and MySQL, and configures the machines to run agents that manipulate the system as needed to realize the requested model. An intermediate stage of that process might conceptually be represented as:


The service units are then provided with the information necessary to configure and start the real software that is responsible for the requested workload (Wordpress and MySQL themselves, in this example), and are also provided with a mechanism that enables service units that were related together to easily exchange data such as addresses, credentials, and so on.

At this point, the service units are able to realize the requested model:


This is close to the original scenario described, except that there’s a single frontend machine running Wordpress. The next section details how to add that second frontend machine.

Scaling services horizontally

The next step to match the original scenario described is to add a second service unit that can run Wordpress, and that can be achieved by the single command:

$ juju add-unit wordpress

No further commands or information are necessary, because the juju state server understands what the model of the deployment is. That model includes both the configuration of the involved services and the fact that units of the wordpress service should talk to units of the mysql service.

This final step makes the deployed system look equivalent to the original scenario depicted:


Although that is equivalent to the classic deployment first described, as hinted by these examples an environment managed by juju isn’t static. Services may be added, removed, reconfigured, upgraded, expanded, contracted, and related together, and these actions may take place at any time during the lifetime of an environment.

The way that the service reacts to such changes isn’t enforced by the juju infrastructure. Instead, juju delegates service-specific decisions to the charm that implements the service behavior, as described in the following section.


A juju-managed environment wouldn't be nearly as interesting if all it could do was constrained by preconceived ideas that the juju developers had about what services should be supported and how they should interact among themselves and with the world.

Instead, the activities within a service deployed by juju are all orchestrated by a juju charm, which is generally named after the main software it exposes. A charm is defined by its metadata, one or more executable hooks that are called after certain events take place, and optionally some custom content.

The charm metadata contains basic declarative information, such as the name and description of the charm, relationships the charm may participate in, and configuration options that the charm is able to handle.

The charm hooks are executable files with well-defined names that may be written in any language. These hooks are run non-concurrently to inform the charm that something happened, and they give a chance for the charm to react to such events in arbitrary ways. There are hooks to inform that the service is supposed to be first installed, or started, or configured, or for when a relation was joined, departed, and so on.

This means that in the previous example the service units depicted are in fact reporting relevant events to the hooks that live within the wordpress charm, and those hooks are the ones responsible for bringing the Wordpress software and any other dependencies up.


The interface offered by juju to the charm implementation is the same, independently from which infrastructure provider is being used. As long as the charm author takes some care, one can create entire service stacks that can be moved around among different infrastructure providers.


In the examples above, the concept of service relationships was introduced naturally, because it’s indeed a common and critical aspect of any system that depends on more than a single process. Interestingly, despite it being such a foundational idea, most management systems in fact pay little attention to how the interconnections are modeled.

With juju, it’s fair to say that service relations were part of the system since inception, and have driven the whole mindset around it.

Relations in juju have three main properties: an interface, a kind, and a name.

The relation interface is simply a unique name that represents the protocol that is conventionally followed by the service units to exchange information via their respective hooks. As long as the name is the same, the charms are assumed to have been written in a compatible way, and thus the relation is allowed to be established via the user interface. Relations with different interfaces cannot be established.

The relation kind informs whether a service unit that deploys the given charm will act as a provider, a requirer, or a peer in the relation. Providers and requirers are complementary, in the sense that a service that provides an interface can only have that specific relation established with a service that requires the same interface, and vice-versa. Peer relations are automatically established internally across the units of the service that declares the relation, and enable easily clustering together these units to setup masters and slaves, rings, or any other structural organization that the underlying software supports.

The relation name uniquely identifies the given relation within the charm, and allows a single charm (and service and service units that use it) to have multiple relations with the same interface but different purposes. That identifier is then used in hook names relative to the given relation, user interfaces, and so on.

For example, the two communicating services described in examples might hold relations defined as:


When that service model is realized, juju will eventually inform all service units of the wordpress service that a relation was established with the respective service units of the mysql service. That event is communicated via hooks being called on both units, in a way resembling the following representation:


As depicted above, such an exchange might take the following form:

  1. The administrator establishes a relation between the wordpress service and the mysql service, which causes the service units of these services (wordpress/1 and mysql/0 in the example) to relate.
  2. Both service units concurrently call the relation-joined hook for the respective relation. Note that the hook is named after the local relation name for each unit. Given the conventions established for the mysql interface, the requirer side of the relation does nothing, and the provider informs the credentials and database name that should be used.
  3. The requirer side of the relation is informed that relation settings have changed via the relation-changed hook. This hook implementation may pick up the provided settings and configure the software to talk to the remote side.
  4. The Wordpress software itself is run, and establishes the required TCP connection to the configured database.

In that workflow, neither side knows for sure what service is being related to. It would be feasible (and probably welcome) to have the mysql service replaced by a mariadb service that provided a compatible mysql interface, and the wordpress charm wouldn’t have to be changed to communicate with it.

Also, although this example and many real world scenarios will have relations reflecting TCP connections, this may not always be the case. It’s reasonable to have relations conveying any kind of metadata across the related services.


Service configuration follows the same model of metadata plus executable hooks that was described above for relations. A charm can declare what configuration settings it expects in its metadata, and how to react to setting changes in an executable hook named config-changed. Then, once a valid setting is changed for a service, all of the respective service units will have that hook called to reflect the new configuration.

Changing a service setting via the command line may be as simple as:

$ juju set wordpress title="My Blog"

This will communicate with the juju state server, record the new configuration, and consequently incite the service units to realize the new configuration as described. For clarity, this process may be represented as:


Taking from here

This conceptual overview hopefully provides some insight into the original thinking that went into designing the juju project. For more in-depth information on any of the topics covered here, the following resources are good starting points:

Read more

Since relatively early in the public life of the Go language, I’ve been involved in pushing forward packages that might be used in Ubuntu, including making the compiler suite itself happier in such packaged environments. In due time, these packages were moved over to an automatic build system, so that people wouldn’t have to rely on my good will to have up-to-date packages, nor would I have to be regularly spending time maintaining those packages. Or so was the theory.

It’s well known that the real world is not so plain, though, and issues became much more regular than hoped. Some of the issues were caused by changes in the build conventions of Go, others self-inflicted due to my limited knowledge of the extensive conventions around packaging, or bugs in indirect dependencies of the process, and more recently the sub-optimal scheduling algorithm used by the build farm has driven the builds to a halt.

So, the question is how to get out of this rabbit hole, but still give people a convenient way to use Go in Ubuntu.

Enter godeb, an experiment that dynamically translates the upstream builds of Go into deb packages. In practice, it’s a simple standalone Go program that can parse the build list, fetch the requested version, and in memory translate the contents into a correct binary deb package.

Since you cannot build a Go application without a Go compiler first, there’s an x86 32-bit binary and an x86 64-bit binary of godeb available for download. After the compiler is installed, godeb may be fetched and rebuilt locally by running go get

Once the godeb binary is available, it’s easy to get up-to-date packages:

$ ./godeb install
package go_1.1.1-godeb1_amd64.deb ready
Selecting previously unselected package go.
(Reading database ... 488515 files and (...) installed.)
Unpacking go (from go_1.1.1-godeb1_amd64.deb) ...
Setting up go (1.1.1-godeb1) ...

It figures what the most recent build available is, downloads, translates, and installs it, asking for a password via sudo if necessary. Running godeb install again will fetch the latest version (or the requested one) and replace the currently installed package. Package installs default to the same architecture of godeb itself, and may be changed by setting the GOARCH environment variable to 386 or amd64, borrowing from a Go convention.

New releases of Go are immediately available, and so are the old ones:

$ ./godeb list

$ ./godeb -h
Usage: godeb <command> [<options> ...]

Available commands:

    install [<version>]
    download [<version>]

For the time being, I’m holding up maintenance of the Go PPA in Launchpad in favor of this system. Of course, you can still install the golang-* packages on Ubuntu 12.10 and 13.04 from the official repositories as usual.

Read more

This week I found some time to work on another small spin-off from the juju project at Canonical, and I’m happy to make it openly available today: the xmlpath package, which implements an efficient and strict subset of the XPath specification for the Go language.

This new package will be used in an upcoming (and long due) revision of the goamz package API, which is currently limited by the fact that once the XML result returned by Amazon is unmarshalled into a static structure, any other data that the package wasn’t prepared to deal with becomes hard to access by clients. This problem is being solved by parsing the tree into an intermediary form which can then have XPath expressions conveniently and efficiently applied to it.

Path expressions currently supported by the package are in the following format, with all components being optional:


Compatibility with the XPath specification goes to the following extent:

  • All axes are supported (“child”, “following-sibling”, etc)
  • All abbreviated forms are supported (“.”, “//”, etc)
  • All node types except for namespace are supported
  • Predicates are restricted to [N], [path], and [path=literal] forms
  • Only a single predicate is supported per path step
  • Richer expressions and namespaces are not supported

For example, consider this simple document:

  <!-- Great book. -->
  <book id="b0836217462">
    <title>Being a Dog Is a Full-Time Job</title>
    <author id="CMS">
      <name>Charles M Schulz</name>
    <character id="PP">
      <name>Peppermint Patty</name>
    <character id="Snoopy">

The following expressions can be applied to it, with the indicated result as first match:

/library/book/isbn “0836217462″
/library/*/isbn “0836217462″
/library/book/../book/./isbn “0836217462″
/library/book/character[2]/name “Snoopy”
/library/book/character[born='1950-10-04']/name “Snoopy”
/library/book//node()[@id='PP']/name “Peppermint Patty”
//*[author/@id='CMS']/name “Charles M Schulz”
/library/book/preceding::comment() ” Great book. “

The API implemented allows compiled paths to be held and re-applied any number of times, concurrently or not. For example:

path := xmlpath.MustCompile("/library/book/isbn")
root, err := xmlpath.Parse(file)
if err != nil {
if value, ok := path.String(root); ok {
        fmt.Println("Found:", value)

Result sets can also be optionally stepped over via an idiomatic iterator interface.

The performance of these operations is close to using the static unmarshaling currently implemented by Go’s encoding/xml package:

BenchmarkParse                 5000        613862 ns/op
BenchmarkSimplePathCompile     1000000     1983 ns/op
BenchmarkSimplePathString      1000000     1565 ns/op

As a reference, this is a similar encoding/xml operation, using a struct with a single nested field on the same document:

BenchmarkSimpleUnmarshal       5000        622519 ns/op

I’m hoping this will make our unavoidable XML interactions slightly less painful.

Read more

10gen, the company behind the MongoDB database, recently announced the availability of the MongoDB Backup Service. This is not a traditional backup service, though. Rather than simply sending scheduled snapshots of the data over to a remote system, the backup service has an agent sitting next to the database that monitors its operation log, and streams the individual operations over to the remote backup servers. This model enables the service to offer some non-conventional features, such as restoring the state of the database at any point in the last 24h, in addition to more traditional snapshots over longer periods.

There’s another interesting fact about how the system was developed: the backup agent is also the first software 10gen releases that is written in the Go language. Reportedly, the agent started as a Java project but, as the project matured, the team wanted to move to a language that compiled to native machine code to make it easier to install. After considering a few options, the team decided that Go was the best fit for its C-like syntax, strong standard library, first-class concurrency, and painless multi-platform support.

I’ve invited Daniel Gottlieb, the main 10gen engineer behind the service agent, to provide some high-level feedback about the use of Go and mgo, the MongoDB driver, and he kindly replied:

Programming the backup agent in Go and the mgo driver has been extremely satisfying. Between the lightweight syntax, the first-class concurrency and the well documented, idiomatic libraries such as mgo, Go has become my language of choice for writing small scripts up to large distributed applications.

The mgo driver is a real pleasure to use. The code is of high quality, the documentation is thorough, clear and detailed, and the API is a thoughtful, natural blend of idiomatic Go and Mongo.

Those are encouraging words, Daniel. It’s great to see not only 10gen making good use of the Go language for first-class services, but contributing to that community of developers by providing its support for the development of the Go driver in multiple ways. Good chance to say thanks!

Read more

Today was updated with the latest image data for Ubuntu 13.04 and all the previous releases as well. Rather than simply hardcoding the values again, though, the JavaScript code was changed so that it imports the new JSON-based feeds that Canonical has been publishing for the official Ubuntu images that are available in EC2, thanks to recent work by Scott Moser. This means the site is always up-to-date, with no manual actions.

Although the new feeds made that quite straightforward, there was a small detail to sort out: the Ubuntu Finder is visually dynamic, but it is actually a fully static web site served from S3, and the JSON feeds are served from the Canonical servers. This means the same-origin policy won’t allow that kind of cross-domain import to be easily done without further action.

The typical workaround for this kind of situation is to put a tiny proxy within the site server to load the JSON and dispatch to the browser from the same origin. Unfortunately, this isn’t an option in this case because there’s no custom server backing the data. There’s a similar option that actually works, though: deploying that tiny proxy server in some other corner and forward the JSON payload as JSONP or with cross-origin resource sharing enabled, so that browsers can bypass the same-origin restriction, and that’s what was done.

Rather than once again doing a special tiny server for that one service, though, this time around a slightly more general tool has emerged, and as an experiment it has been put live so anyone can use it. The server logic is pretty simple, and the idea is even simpler. Using the services from as an example, the following URL will serve a JSON document that can only be loaded from a page that is in a location allowed by the same-origin policy:

If one wanted to load that page from a different location, it might be transformed into a JSONP document by loading it from:

Alternatively, modern browsers that support the cross-origin resource sharing can simply load pure JSON by omitting the jsonpeercb parameter. The jsonpeer server will emit the proper header to allow the browser to load it:

This service is backed by a tiny Go server that lives in App Engine so it’s fast, secure (hopefully), and maintenance-less.

Some further details about the service:

  • Results are JSON with cross-origin resource sharing by default
  • With a query parameter jsonpeercb=<callback name>, results are JSONP
  • The callback name must consist of characters in the set [_.a-zA-Z0-9]
  • Query parameters provided to jsonpeer are used when doing the upstream request
  • HTTP headers are discarded in both directions
  • Results are cached for 5 minutes on memcache before being re-fetched
  • Upstream results must be valid JSON
  • Upstream results must have Content-Type application/json or text/plain
  • Upstream results must be under 500kb
  • Both http and https work; just tweak the URL and the path accordingly

Have fun if you need it, and please get in touch before abusing it.

UPDATE: The service and blog post were tweaked so that it defaults to returning plain JSON with CORS enabled, thanks to a suggestion by James Henstridge.

Read more

A few years ago, when I started pondering about the possibility of porting juju to the Go language, one of the first pieces of the puzzle that were put in place was goyaml: a Go package to parse and serialize a yaml document. This was just an experiment and, as a sane route to get started, a Go layer that does all the language-specific handling was written on top of the libyaml C scanner, parser, and serializer library.

This was a good initial plan, but for a number of reasons the end goal was always to have a pure Go implementation. Having a C layer in a Go program slows down builds significantly due to the time taken to build the C code, makes compiling in other platforms and cross-compiling harder, has certain runtime penalties, and also forces the application to drop the memory safety guarantees offered by Go.

For these reasons, over the last couple of weeks I took a few hours a day to port the C backend to Go. The total time, considering full time work days, would be equivalent to about a week worth of work.

The work started on the scanner and parser side of the library. This took most of the time, not only because it encompassed more than half of the code base, but also because the shared logic had to be ported too, and there was a need to understand which patterns were used in the old code and how they would be converted across in a reasonable way.

The whole scanner and parser plus header files, or around 5000 code lines of C, were ported over in a single shot without intermediate runs. To steer the process in a sane direction, gofmt was called often to reformat the converted code, and then the project was compiled every once in a while to make sure that the pieces were hanging together properly enough.

It’s worth highlighting how useful gofmt was in that process. The C code was converted in the most convenient way to type it, and then gofmt would quickly put it all together in a familiar form for analysis. Not rarely, it would also point out trivial syntactic issues. A double win.

After the scanner and parser were finally converted completely, the pre-existing Go unmarshaling logic was shifted to the new pure implementation, and the reading side of the test suite could run as-is. Naturally, though, it didn’t work out of the box.

To quickly pick up the errors in the new implementation, the C logic and the Go port were put side-by-side to run the same tests, and tracing was introduced in strategic points of the scanner and parser. With that, it was easy to spot where they diverged and pinpoint the human errors.

It took about two hours to get the full suite to run successfully, with a handful of bugs uncovered. Out of curiosity, the issues were:

  • An improperly dropped parenthesis affected the precedence of an expression
  • A slice was being iterated with copying semantics where a reference was necessary
  • A pointer arithmetic conversion missed the base where there was base+offset addressing
  • An inner scoped variable improperly shadowed the outer scope

The same process of porting and test-fixing was then repeated on the the serializing side of the project, in a much shorter time frame for the reasons cited.

The resulting code isn’t yet idiomatic Go. There are several signs in it that it was ported over from C: the name conventions, the use of custom solutions for buffering and reader/writer abstractions, the excessive copying of data due to the need of tracking data ownership so the simple deallocating destructors don’t double-free, etc. It’s also been deoptimized, due to changes such as the removal of macros and in many cases its inlining, and the direct expansion of large unions which causes some core objects to grow significantly.

At this point, though, it’s easy to gradually move the code base towards the common idiom in small increments and as time permits, and cleaning up those artifacts that were left behind.

This code will be made public over the next few days via a new goyaml release. Meanwhile, some quick facts about the process and outcome follows.

Lines of code

According to cloc, there was a total of 7070 lines of C code in .c and .h files. Of those, 6727 were ported, and 342 were 12 functions that were left unconverted as being unnecessary right now. Those 6727 lines of C became 5039 lines of Go code in a mostly one-to-one dumb translation.

That difference comes mainly from garbage collection, lack of forward declarations, standard helpers such as append, range-based for loops, first class slice type with length and capacity, internal OOM handling, and so on.

Future work code can easily increase the difference further by replacing some of the logic ported with more sensible options available in Go, such as standard abstractions for readers and writers, buffered writing support as availalbe in the standard library, etc.

Code clarity and safety

In the specific context of the work done, which is of a scanner, parser and serializer, the slice abstraction is responsible for noticeable clarity gains in the code, when compared to the equivalent logic based on pointer arithmetic. It also gives a much more comforting guarantee of correctness of the written code due to bound-checking.


While curious, this shouldn’t be taken as a performance comparison between the two languages, as it is comparing a fine tuned C implementation with something that is worse than a direct one-to-one port: not only it hasn’t seen any time at all on preventing waste, but the original logic was deoptimized due to changes such as the removal of inlining macros and the expansion of large unions. There are many obvious changes to be done for improving performance.

With that out of the way, in a simple decoding benchmark the C-backed decoder runs on about 37% of the time taken by the out-of-the-box deoptimized Go port.

Output size

The previous goyaml.a Go package file had 1463kb. The new one has 1016kb. This difference includes glue code generated for the integration.

Considering only the .c and .h files involved in the port, the C object code generated with the standard flags used by the go build tool (-g -O2) sums up to 789kb. The equivalent Go code with the standard settings compiles to 664kb. The 12 functions not ported are also part of that difference, so the difference is pretty much negligible.

Build time

Building the 8 .c files alone takes 3.6 seconds with the standard flags used by the go build tool (-g -O2). After the port, building the entire Go project with the standard settings takes 0.3 seconds.

Mechanical changes

Many of the mechanical changes were done using regular expressions. Excluding the trivial ones, about a dozen regular expressions were used to swap variable and type names, drop parenthesis, place brackets in the right locations, convert function declarations, and so on.

Read more

Last week I was part of a rant with a couple of coworkers around the fact Go handles errors for expected scenarios by returning an error value instead of using exceptions or a similar mechanism. This is a rather controversial topic because people have grown used to having errors out of their way via exceptions, and Go brings back an improved version of a well known pattern previously adopted by a number of languages — including C — where errors are communicated via return values. This means that errors are in the programmer’s face and have to be dealt with all the time. In addition, the controversy extends towards the fact that, in languages with exceptions, every unadorned error comes with a full traceback of what happened and where, which in some cases is convenient.

All this convenience has a cost, though, which is rather simple to summarize:

Exceptions teach developers to not care about errors.

A sad corollary is that this is relevant even if you are a brilliant developer, as you’ll be affected by the world around you being lenient towards error handling. The problem will show up in the libraries that you import, in the applications that are sitting in your desktop, and in the servers that back your data as well.

Raymond Chen described the issue back in 2004 as:

Writing correct code in the exception-throwing model is in a sense harder than in an error-code model, since anything can fail, and you have to be ready for it. In an error-code model, it’s obvious when you have to check for errors: When you get an error code. In an exception model, you just have to know that errors can occur anywhere.

In other words, in an error-code model, it is obvious when somebody failed to handle an error: They didn’t check the error code. But in an exception-throwing model, it is not obvious from looking at the code whether somebody handled the error, since the error is not explicit.
When you’re writing code, do you think about what the consequences of an exception would be if it were raised by each line of code? You have to do this if you intend to write correct code.

That’s exactly right. Every line that may raise an exception holds a hidden “else” branch for the error scenario that is very easy to forget about. Even if it sounds like a pointless repetitive task to be entering that error handling code, the exercise of writing it down forces developers to keep the alternative scenario in mind, and pretty often it doesn’t end up empty.

It isn’t the first time I write about that, and given the controversy that surrounds these claims, I generally try to find one or two examples that bring the issue home. So here is the best example I could find today, within the pty module of Python’s 3.3 standard library:

def spawn(argv, master_read=_read, stdin_read=_read):
    """Create a spawned process."""
    if type(argv) == type(''):
        argv = (argv,)
    pid, master_fd = fork()
    if pid == CHILD:
        os.execlp(argv[0], *argv)

Every time someone calls this logic with an improper executable in argv there will be a new Python process lying around, uncollected, and unknown to the application, because execlp will fail, and the process just forked will be disregarded. It doesn’t matter if a client of that module catches that exception or not. It’s too late. The local duty wasn’t done. Of course, the bug is trivial to fix by adding a try/except within the spawn function itself. The problem, though, is that this logic looked fine for everybody that ever looked at that function since 1994 when Guido van Rossum first committed it!

Here is another interesting one:

$ make clean
Sorry, command-not-found has crashed! Please file a bug report at:

Please include the following information with the report:

command-not-found version: 0.3
Python version: 3.2.3 final 0
Distributor ID: Ubuntu
Description:    Ubuntu 13.04
Release:        13.04
Codename:       raring
Exception information:

unsupported locale setting
Traceback (most recent call last):
  File "/.../CommandNotFound/", line 24, in crash_guard
  File "/usr/lib/command-not-found", line 69, in main
  File "/usr/lib/command-not-found", line 40, in enable_i18n
    locale.setlocale(locale.LC_ALL, '')
  File "/usr/lib/python3.2/", line 541, in setlocale
    return _setlocale(category, locale)
locale.Error: unsupported locale setting

That’s a pretty harsh crash for the lack of locale data in a system-level application that is, ironically, supposed to tell users what packages to install when commands are missing. Note that at the top of the stack there’s a reference to crash_guard. This function has the intent of catching all exceptions right at the edge of the call stack, and displaying a detailed system specification and traceback to aid in fixing the problem.

Such “parachute catching” is a fairly common pattern in exception-oriented programming and tends to give developers the false sense of having good error handling within the application. Rather than actually guarding the application, though, it’s just a useful way to crash. The proper thing to have done in the case above would be to print a warning, if at all, and then let the program run as usual. This would have been achieved by simply wrapping that one line as in:

    locale.setlocale(locale.LC_ALL, '')
except Exception as e:
    print("Cannot change locale:", e)

Clearly, it was easy to handle that one. The problem, again, is that it was very natural to not do it in the first place. In fact, it’s more than natural: it actually feels good to not be looking at the error path. It’s less code, more linear, and what’s left is the most desired outcome.

The consequence, unfortunately, is that we’re immersing ourselves in a world of brittle software and pretty whales. Although more verbose, the error result style builds the correct mindset: does that function or method have a possible error outcome? How is it being handled? Is that system-interacting function not returning an error? What is being done with the problem that, of course, can happen?

A surprising number of crashes and plain misbehavior is a result of such unconscious negligence.

Read more

This weekend the proper environment settled out for sorting a pet peeve that shows up every once in a while when coding: writing logic that interacts with other applications in the system via their stdin and stdout streams is often more involved than it should be, which seems pretty ironic when sitting in front of a Unix-like system.

Rather than going over the trouble of setting up pipes and hooking them up in a custom way, often applications end up just delegating the job to /bin/sh, which is not ideal for a number of reasons: argument formatting isn’t straightforward, injecting custom application-defined logic is hard, which means even simple tasks that might be easily achieved by the language end up shelling out to further external applications, and so on.

In an attempt to address that, I’ve spent some time working on an experimental Go package that is being released today: pipe.

I hope you like it as well, and please drop me a note if you find any issues.

Read more

There are a number of common misconceptions in software development surrounding the idea of concurrency. This has been coming for decades, and some of the issues have just been reinforced one more time in an otherwise interesting post in LinkedIn’s engineering blog that recommends their development framework.

Such issues may be observed throughout the post, but can be elucidated via this short paragraph:

As we saw with the Scala and JavaScript examples above, for very simple cases, the Evented (asynchronous) code is generally more complicated than Threaded (synchronous) code. However, in most real-world scenarios, you’ll have to make several I/O calls, and to make them fast, you’ll need to do them in parallel.

At a glance, this may look like a sane proposition. There’s agreement that an asynchronous API or framework is one that does not block the flow of execution when faced with a task that has a long or non-predictable deadline, and this coding style is harder for human beings to get right. For example, if you see code such as:

data = read(filename)

There’s less brain work to process and build on it than so called asynchronous logic such as:

read(filename, callback)

It’s also true that there are important interfaces that follow the asynchronous style to prevent resource waste. Some of these exist in the kernel I/O API.

So what’s the issue, then?

There are a few. The first one is the statement that to make I/O scale you have to do it in parallel. That’s clearly not true. Scalable I/O requires your program to not waste an irresponsible amount of memory and CPU per operation. This may be achieved with simple concurrent techniques, and concurrency is not parallelism.

This drives to the next point, which is the strong association between synchronous programming and threads. You can have synchronous programming, and its simplified mental model, without operating system threads. This can be done by having a compiler and runtime that is mindful about performance and resource consumption, building on the efficient interfaces to implement its abstractions.

These ideas have also been covered in this paper from 2003, including benchmark results that debunk the performance myth. What seems most interesting about this paper is that it theorizes such a compiler and runtime that would allow “overcom[ing] limitations in current threads packages and improv[ing] safety, programmer productivity, and performance”, by using techniques such as dynamic stack growth, stack moving, cheaper synchronization, and compile-time data race detection.

That exact mix, including all of the properties described in the paper, are available today in the Go language. You can have synchronous programming, concurrency, parallelism, and performance. We live in the future.

Read more

12 years ago

These ancient entries were taken from my old Advogato diary, written in my early twenties, a year after I joined the development team of Conectiva Linux. I’m copying them for historic purposes, with the content untouched. It’s curious to look back and have such details of what was going on at the time, things that feel good, and things that feel awkward such as the “Dear Diary, …” style of writing, and the amount of exclamations!!

6 Feb 2002

Wow!! It has been a long time since my last diary entry.

I’ve left Linuxconf development team coordination in favor of the Conectiva Linux port to the S390 platform coordination (ok, I’m mostly coordinating myself now :-) . Most of the work is done. I have developed an acceptable installer (in Python!) and most of the packages are ported. We had some problems with IBM OCO modules (ick!), but we are already workarounding them (we gave up on some of our kernel patches, and I patched insmod to recognize OCO modules). Anyway, more information about this later (if I don’t disappear for another year.. ;-)

In the process of porting Conectiva Linux to S390 and PPC (Harald Welve started the PPC port, and I’m keeping it up to date and working on missing stuff) we are learning some lessons. We are trying to use those lessons to build a defacto package building system using the Python language. Unfortunately, we don’t have enough people here to develop it quickly, so we are trying a more realistic and evolutive approach this time. The first part of it is almost done. While devloping it, I’ve studied a little bit about process groups and extended python with a missing killpg() system call. I’ve also discovered that when python spawns a new thread, it blocks all signals. With this information in mind, I have also extended it with a new execv() syscall, that besides doing the usual work, unblocks every signal before the real call to execv(). I hope this project becomes real someday.

I’ve also been playing with Python optimization lately. There’s a big opportunity for somebody wanting to study and implement some concepts there. I’ve read some documentation about Stack Machine Optimization and made some tries (basically, optimizations around the inner loop and the Big Switch, stack caching, and other flavors of this joy). Today I found a paper from Skip Montanaro documenting some of the tries I’ve made (reading it first would save me a lot of time, but this knowledge will be useful anyway). You should have a look at his paper if you have any interest in the topic. Oh, don’t forget to get yourself a copy of Lemburg’s pybench to have a general idea of what you’re doing (don’t trust too much on it, it’s just a benchmark). I’ve written Skip a mail to discuss a little bit about what could be integrated into the interpreter. Let’s see where we get.

Oh… I must not forget to update the people I’ve certified in the past to reflect what they’ve been doing.

27 Jan 2001

I’ve just tested the patch floppy_cs on kernel 2.2.17 and it works just fine!!! I had no problems applying it. Now my Libretto 50CT has a floppy drive. Maybe Conectiva Linux can ship with this patch. The only drawback is that floppy support must be modular.

In the few last days, I’ve implemented support for Inputgrid into gnome-linuxconf. It allows one to define sensitive areas into a drawing area. Gurus are already working with it.

I’ve also created two new commands for Linuxconf’s gui protocol: Splash and Hidesplash. Linuxconf will send them while it is starting, and the graphic frontend is suposed to show a nice splash screen. Support in gnome-linuxconf has also been implemented with an image designed by Everaldo (thanks!!!).

Btw, yesterday I’ve fixed a bug in Python’s bsddb module. It was handling DB_RECNO databases with string keys. As the documentation says, these databases must use, in key’s “data” field, a pointer to a memory location holding a recno_t type.

My parents have arrived yesterday!! They’ll stay here until monday and then will go to Minas Gerais, visiting Raul and Lulude.

22 Jan 2001

Yesterday I’ve implemented a message signing module for Mailman. Darian (aka dmalloc), from has asked if I could do such module to use on (I said yes… ;-) . I’m going to ask the people here at Conectiva if they want to use this module in some of our lists.

Pybot has a few new features: CTCP handling/sending, timer module, unhandled messages hook, and something else I probably forgot.

About Linuxconf, I have spent the last days fixing a few simple bugs introduced in 1.24r2 and in the last modules developed by Conectiva. I hope to release a Linuxconf update to Conectiva 6.0 tomorrow.

gnome-linuxconf has won a home with screenshots and everything else at I’ve also published it at freshmeat and put files to download at SourceForge.

Oh, good news I forgot to tell: for those of you that are using wxxt-linuxconf, Jack (Jacques Gelinas) has implemented the Treemenu icons into this frontend as well.

16 Jan 2001

Today I’ve fixed a bug in the Pythonmod module of Linuxconf. Linuxconf has a default handler for the SIGCHLD signal that controls all of its child termination. This method has a few disadvantages. Before calling any external processes without using default Linuxconf methods, you must block this handler, otherwise Linuxconf will get on your way. Because of this, If a Pythonmod module tried to fork external processes, they were failing. Now Pythonmod is setting the SIGCHLD signal to SIG_DFL (POSIX doesn’t allow us to SIG_IGN it) before calling python code, and after returning from a few Linuxconf API functions that set the handler back. When the python code returns, popen_initsignal() is called, putting the Linuxconf handler back in place.

On the gnome-linuxconf side, I’ve implemented the drawing context command Defpen. Now we have colored lines and primitives!! (ok… not that good… ;-)

I’ve also spent a few hours in the last two days backing up and restoring data in my colocated machine. Now my personal emails are back online and the server has an updated kernel. I hope it doesn’t bother me for a long time.

Unfortunately, the server stuff didn’t let me work on Pybot, but I had time to implement dynamically loading, unloading and reloading of modules, before I started on the server. This will help a lot in the development, since I don’t have to reboot the bot everytime something is wrong. Anyway, now that the server is ok (I hope so), I’m planning to spend some of my spare time on the bot (yes, I still have some… ;-) .

11 Jan 2001

Today I’ve added the ability of using icons while in the Treemenu mode of Linuxconf. I have just changed some functions to pass the icon name around until it got into the treemenu module and then sent it to the GUI front-end. A little hack on gnome-linuxconf did the work at the front-end side. Following this line of improvements, I’m planning to add a splash screen or something like that soon. Icons would also be welcome in the web interface.

Besides that, I’m also playing with a Python IRC bot. It’s not meant to be a war or a channel control bot. I’m planning to implement useful modules to help making IRC even more useful as an information media (no it won’t be just another infobot clone). The core and a few modules are ready. I’ll post more information later… for now, I’ll just tell that it is a multi-channel, multi-server bot, and that I’m trying to make its commands with natural language (eg. forward messages from #blah on servername to #bloh on servername).

Happy birthday Diogo!!

16 Aug 2000

Created advogato account.

Read more

A small and fun experiment is out:

Read more

Ethics for code reviewers

In the previous post, I explored a bit how ephemeral most of the artifacts of software development processes are. One of these processes is code reviewing, which is arguably a major player in code quality, knowledge acquisition, and even team dynamics.

Even being so important, the outcome of the code review process — the review itself — tends to reach a very limited audience and have a short life time. It’ll be hard to change that picture given the nature of reviews: they are conversational, and address specific issues for the integration of a change in the project. At the same time, even if code reviews are not generally useful as permanent documentation, we can increase their value as reference material by improving the quality of those conversations. Having a good conversation has many other great side effects, of course.

As a small step in that direction, what follows are personal guidelines that I have been evolving empirically over the years as a software developer and code reviewer. They may not bring you fortune and fame, and are not always easy to apply, but hopefully they will help improving your experience as a member of your team and the value of those reviews.

Explain why

Unless the change is about an extremely obvious mistake, explain why you’re suggesting it. If the reasoning was natural to the author, he’d have done it in the first place. Good explanations also help avoiding the same mistake over and over again, and are much more rewarding to the listener. They also become a target for future references.

If you don’t have enough time to justify it and would rather provide the review sooner than later, one approach is to just recommend the change and invite the author for a conversation later if that would be helpful. That said, try to have that conversation over a media that may be shared with the rest of the team, or recollected whenever necessary.

Be respectful

Always keep in mind that there’s a person on the other side of the wire, not a machine, and that it’s hard to understand written words with little context. Avoid letting anger and frustration leak into the review, even if you feel it is justified. There’s no good outcome in those situations.

It doesn’t matter who broke it, or who coded that silly piece of code. If there is broken code, and the project has reviews, multiple people were in the pipeline for that result, and they were trying to get it right. Take shared ownership of the problem, and look for the solution and for how to avoid such issues in the future.

Praise the good work

Reviews carry some low energy feel by their very nature. No matter how positive you are about them, and how much the whole team understands and agrees it is for the best, you are in fact looking for places to put your finger in someone else’s work. For that reason, it is very helpful to take every chance you can of praising logic, design, code organization, or whatever else that you honestly felt was well done. It won’t ever balance it out, but it will at least remind the author that the contributions are welcome.

Suggestions are appreciated

Perhaps a longer variable name would be helpful, or that constant could have a more descriptive name? In many circumstances, the change is indeed subjective, and the gain is pretty marginal. In those cases, if you really can’t resist the urge to say something, a good approach is a suggestion that may be exercised or not at the author’s discretion. Ideally, suggest several options that would feel better to you, so that your point is better understood and agreement is easier. That said, read on.

Avoid trivialities

When reviewing that very simple point, think to yourself: all things considered, does it actually matter? Is the cost of the author’s time, and the potential debate, really worth it? You surely have your opinion about whether to spell “min <= count” or “count >= min“, but so does everybody else. When it’s purely a matter of preference, the author is entitled to have one after all.

Small branches win

Code reviews are useful for a number of secondary reasons, but the primary goal of the code review is to analyze a proposed change, to fix it for inclusion, or to reject it. It’s often tempting to recommend further changes to be bundled onto the same review, but it’s important to keep some focus. Are these additional changes tightly related to the original idea, or would they rather be more appropriate on a future branch?

Also keep an eye on large review submissions. It’s quite rare to see changes of a thousand lines or more that are really an indivisible unit. More often, it ends up like that organically, as a result of the workflow followed by the author. These branches may be very frustrating, both for the author and for reviewers. For the reviewer, it’s hard to keep the necessary level of attention and enthusiasm for the problem over expanded periods of time. For the author, it will be equally problematic to run over a large review. In some extreme cases, it may be worth going back and breaking down the change into more change sets.

Overall, fast iterations on small branches are much more rewarding to work with.

Work with inline comments

This is about tooling, but doing anything else should be considered unethical really. If you don’t have a system that allows the change diff to be seen within the rest of the content, and comments to be made inlined right where you see the issue, implement one right now. Moving to such a system was the most dramatic change in productivity I had as a reviewer in the past several years, and makes the whole experience a lot more bearable for everyone.


Make sure you’re enjoying what you do, and appreciate what your code reviews are achieving. There’s little point in playing the role of an intelligent computer over extended periods of time if you are unhappy about it. Get yourself your preferred slow-drinking beverage (chimarrão?), perhaps some snacks, a comfortable chair, and relax.

Read more

Lately I’ve been considering the amount of waste we produce during software development, and how to increase the amount of recycled content. I’m not talking about actual trash, though, but rather about software development artifacts.

Over the years, we’ve learned about and put in practice several means for improving the quality and success rate of projects we create or contribute to. We have practices such as sprints to get people together with high communication bandwidth; we have code reviews for sharing knowledge and improving project quality; we’ve got technical leadership roles to mentor developers and guide the progress of projects; we’ve created kanban boards and burndown charts to help people visualize what they’re going through; and so on.

While all of that seems to have helped tremendously, there’s a sad fact about where we stand: the artifacts of most of these processes are local to their context, and very sensitive to time. That burndown chart is meaningless after it’s burned, and a kanban has no relevant history. Our technical leads indeed guide their teams, but their wisdom stays with the few people that had the chance to interact with them, and subjectively so. That brilliant code review from our best developers has a very limited audience, and rarely carries any meaning just days after it has been accomplished.

That last one is specially interesting. The process of reviewing code is an intense task, very expensive, and that takes a significant portion of the life of an active developer, and even then very little is carried forward as the outcome of that process. We have no effective means or even culture of sharing the generated wisdom to other teams. In fact, we rarely share these details even within the team itself. Why was that line changed like this? Why an interface like that is a bad idea? Who will instruct the new guy next week, and where did we record a bit of the wisdom of the brilliant guy that has left the company recently?

Unfortunately there’s probably no easy solution for this problem. At this point, I mainly recognize that most of the efforts I’ve lead to improve software development for the past several years had a very limited scope. The software itself became immediately better as a result of my efforts, its design became more sensible, and hopefully I contributed a bit to the growth of people around me, but at a company or even community-wide scope, all of these code reviews, sprints, and IRC conversations are buried for very rare revives.

I want to start doing something about this, though. There must be a way to shape these conversations in a more reusable format; in a way that knowledge and agreement can be more proactively preserved and scattered. Perhaps it’s more about how than it is about what. Perhaps we just need to write more posts like this, and cover more topics related to daily development findings. Not sure. I’ll be thinking…

Read more

Our son Otávio was born recently. Right in the first few days, we decided to keep tight control on the feeding times for a while, as it is an intense routine pretty unlike anything else, and obviously critical for the health of the baby. I imagined that it wouldn’t be hard to find an Android app that would do that in a reasonable way, and indeed there are quite a few. We went with Baby Care, as it has a polished interface and more features than we’ll ever use. The app also includes some basic statistics, but not enough for our needs. Luckily, though, it is able to export the data as a CSV file, and post-processing that file with the R language is easy, and allows extracting some fun facts about what the routine of a healthy baby can look like in the first month, as shown below.


The first thing to do is to import the raw data from the CSV file. It is a one-liner in R:

> info = read.csv("baby-care.csv", header=TRUE)

Then, this file actually comes with other events that won’t be processed now, so we’ll slice it and grab only the rows and columns of interest:

> feeding <- info[info$Event.type == "Breast",
        c("Event.subType", "Start.Time", "End.Time", "Duration")]

This is how it looks like:

> feeding[100:103,]
    Event.subType       Start.Time         End.Time Duration
129          Left 2013/01/04 13:45 2013/01/04 14:01    00:16
132          Left 2013/01/04 16:21 2013/01/04 16:30    00:09
134         Right 2013/01/04 17:46 2013/01/04 17:54    00:08

Now things get more interesting. Let’s extract that duration column into a more useful vector, and do some basic analysis:

> duration <- as.difftime(as.vector(feeding$Duration), "%H:%M")

> length(duration)
[1] 365

> total = sum(duration)
> units(total) = "hours"
> total
Time difference of 63.71667 hours

> mean(duration)
Time difference of 10.47397 mins
> sd(duration)
[1] 5.937172

A total of 63 hours surprised me, but the mean time of around 10 minutes per feeding is within the recommendation, and the standard deviation looks reasonable. It may be more conveniently pictured as a histogram:

> hist(as.numeric(duration), breaks="FD",
    col="blue", main="", xlab="Minutes")

Duration histogram

Another point we were interested on is if both sides are properly balanced:

> sides <- c("  Right", "  Left")
> tapply(duration, feeding$Event.subType, mean)[sides]
   Right     Left 
10.72283 10.22099

Looks good.

All of the analysis so far goes over the whole period, but how has the daily intake changed over time? We’ll need an additional vector to compute this and visualize in a chart:

> day <- format(strptime(feeding$Start.Time, "%Y/%m/%d %H:%M"),
> perday <- tapply(duration, day, sum)
> mean(perday)
[1] 136.5357
> sd(perday)
[1] 53.72735
> sd(perday[8:length(perday)])
[1] 17.49735

> plot(perday, type="h", col="blue", xlab="Day", ylab="Minutes")

Daily duration

The mean looks good, with about two hours every day. The standard deviation looks high on a first look, but it’s actually not that bad if we take off the first few days. Looking at the graph shows why: the slope on the left-hand side, which is expected as there’s less milk and the baby has more trouble right after birth.

The chart shows a red flag, though: one day seems well below the mean. This is something to be careful about, as babies can get into a loop where they sleep too much and miss being hungry, the lack of feeding causes hypoglycemia, which causes more sleep, and it doesn’t end up well. A rule of thumb is to wake the baby up every two hours in the first few days, and at most every four hours once he stabilizes for the following weeks.

So, this was another point of interest: what are the intervals between feedings?

> start = strptime(feeding$Start.Time, "%Y/%m/%d %H:%M")
> end = strptime(feeding$End.Time, "%Y/%m/%d %H:%M")
> interval <- start[-1]-end[-length(end)]

> hist(as.numeric(interval), breaks="FD", col="blue",
       main="", xlab="Minutes")

Interval histogram

Seems great, with most feedings well under two hours. There's a worrying outlier, though, of more than 6 hours. Unsurprisingly, it happened over night:

> feeding$End.Time[interval > 300]
[1] 2013/01/07 00:52

It wasn't a significant issue, but we don't want that happening often while his body isn't yet ready to hold enough energy for a full night of sleep. That's the kind of reason we've been monitoring him, and is important because our bodies are eager to get full nights of sleep, which opens the door for unintended slack. As a reward for that kind of control, we've got the chance to enjoy not only his health, but also an admirable mood.

Love, Dad.

Read more