On style checkers and warnings as errors

The quest for software quality has given us lots of new tools: new compiler warnings, making the compiler treat all warnings as errors, style checkers, static analyzers and the like. These are all good things to have. Sooner or later in a project someone will decide to make them mandatory. This is fine as well, and the reason we have continuous integration servers.

Still, some people might, at times, propose MRs that cause CI to emit errors. The issues are fixed, some time is lost when doing this but on the whole it is no big deal. But then someone comes to the conclusion that these checks should be mandatory on every build so the errors never get to the CI server. Having all builds pristine all the time is great, the reasoning goes, because then errors are found as soon as possible. This is as per the agile manifesto and universally a good thing to have.

Except that it is not. It is terrible! It is a massive drain on productivity and the kind of thing that makes people hate their job and all things related to it.

This is a strong and somewhat counter-intuitive statement. Let’s explore it with an example. Suppose we have this simple snippet of code.

  x = this_very_long_function_that_does_something(foo, bar, baz10, foofoofoo);

Now let’s suppose we have a bug somewhere. As part of the debugging cycle we would like to check what would happen if x had the value 3 instead of whatever value the function returns. The simple way to check is to change the code like this.

  x = this_very_long_function_that_does_something(foo, bar, baz10, foofoofoo);
  x = 3;

This does not give you the result you want. Instead you get a compile/style/whatever checker error. Why? Because you assign to variable x twice without using the first value for anything. This is called a dead assignment. It may cause latent bugs, so the checker issues an error halting the build.

Fair enough, let’s do this then.

  this_very_long_function_that_does_something(foo, bar, baz10, foofoofoo);
  x = 3;

This won’t work either. The code is ignoring the return value of the function, which is an error (in certain circumstances but not others).

Grumble, grumble. On to iteration 3.

  //x = this_very_long_function_that_does_something(foo, bar, baz10, foofoofoo);
  x = 3;

This will also fail. The line with the comment is over 80 characters wide and this is not tolerated by many style guides, presumably because said code bases are being worked on by people who only have access to text consoles. On to attempt 4.

//x = this_very_long_function_that_does_something(foo, bar, baz10, foofoofoo);
  x = 3;

This won’t work either for two different reasons. Case 1: the variables used as arguments might not be used at all and will therefore trigger unused variable warnings. Case 2: if any argument is passed by reference or pointer, their state might not be properly updated.

The latter case is the worst, because no static checker can detect it. Requiring the code to conform to a cosmetic requirement caused it to grow an actual bug.

Getting this kind of test code through all the testers is a lot of work. Sometimes more than the actual debug work. Most importantly, it is completely useless work. Making this kind of exploratory code fully polished is useless because it will never, ever enter any kind of production. If it does, your process has bigger issues than code style. Any time spent working around a style checker is demotivational wasted effort.

But wait, it gets worse!

Type checkers are usually slow. They can take over ten times longer to do their checks than just plain compiling the source code. Which means that developer productivity with these tools is, correspondingly, several times lower than it could be. Programmers are expensive, having them sitting around watching preprocessor checker text scroll slowly by in a terminal is not a very good use of their time.

Fortunately there is a correct solution for this. Make the style checks a part of the unit test suite, possibly as part of an optional suite of slow tests. Run said tests on every CI merge. This allows developers to be fast and productive most of the time but precise and polished when required.