I recently read this book: Debug It! – by Paul Butcher. This describes the “Scientific method of debugging”, and much more than that.
One could criticize it like: “All of theses advice are, you know, common sense.” I wish they were common practices. Anyway, even if this criticism were true, it’s good to see how these “common sense” techniques make a coherent method.
Let’s see a few examples!
Intuition or systematic search?
Butcher spends an entire chapter to define what the goal of debugging is. No, it isn’t only “making the bug go away”.
Then, he spends four more chapters on how to apply this “Scientific Method”. Which is, using the software itself to tell us what’s going on under the hood. As a first step, he suggests to reproduce the bug. (Yes, this should be obvious. Unfortunately, it isn’t for everyone.)
Then, as the book goes, we should make a series of experiments to learn more about the software. The most deliberating thought about it is that there are no hints in this book on what things to try first. It just says, experiments. For each possible code change we can try if it made the things better or worse. Which means that if you have a hunch like “we need a null-check here”, or “maybe the obfuscation of the different modules are incompatible”, then you can go ahead and give it a try – maybe they’ll save you some time. If not, then we can fall back to any systematic search that works well on our platform.
A note on intuition
Intuition is often misleading. E.g. simply adding a null-check usually is not a good idea. Anyway, the book puts a great emphasis on making things better, not worse. This means, among other things, that making a bug more difficult to catch is not an acceptable fix.
Many things can go wrong during bug fixing:
- we can break other parts of the system
- instead of fixing the real thing we can make it look like a new but unrelated problem
- we can make the code difficult to understand
Fortunately, there are tons of tools to help us with these problems. Butcher enumerates the usual agile tool set: automatic testing and continuous integration. He gives a good reasoning for each of these tools. Like, an automatic test for a new bug can speed things up when looking for the root cause of the bug. Then, a good test suite can prevent us from causing regression. Finally, he says, a good test suite enables refactoring.
A note on refactoring:
There are two things that I’m disagreeing with the statements of ‘Debug it’. One of those things is that he suggests not to refactor when there are no tests in place. I think we should, only as little as necessary. But, what’s necessary? Well, Michael Feathers wrote a great book about that. Feathers’s answer is: refactor only as much as you need to write unit tests. Which is more like a real-life answer to me.
Hide and seek.
There are two chapters in this book that can make us have more bug reports. Well, this is one of the thought-provoking ideas you can find in this book. What this actually means is that we need to be better at uncovering bugs.
One approach is to make the users easy to report a problem. I.e. your system could have an internal reporting dialog for unexpected behavior.
The other approach needs more grunt work: we can document our assumptions with assertions so they’ll always be true. If not, the system fails fast with something like an AssertionError. Besides the assertions, we can add meaningful debug logging so we’ll have enough information if something goes wrong.
A note on assertions
Here is the other thing I don’t agree with Butcher. He suggests to turn off the assertions for the production build. He suggests to have a meaningful shortcut that deals with broken assumptions.
On the contrary, I think that if we can deal with them meaningfully then we just coded a too restrictive assumption. However, if we really cannot deal with a given input or state, then we stop processing it ASAP.
I think it’s a great book. It’s not so new, it was written in 2009. Fortunately, most of the book is language-agnostic. Thus it provides the sort of knowledge that just doesn’t go out of fashion.
Moreover, this book is short. I mean, distilled, bullshit-free. I strongly recommend to read it.