Who would have thought that an entire book devoted to the bias-variance tradeoff would make it to the NY Times business best seller list? The book is the recently-published Noise, by Daniel Kahneman, Olivier Sibony and Cass R. Sunstein, and, as of the beginning of August, it was #2 on the list. Kahneman, who wrote the influential book, Thinking, Fast and Slow(2011), is a master at relating sometimes abstract statistical concepts to lay audiences and real world scenarios. With the bias-variance tradeoff he has set himself and his co-authors a worthy challenge.

Bias in human estimation and judgment was the key topic in Thinking, Fast and Slow. Now, a decade later, it is only natural that Kahneman would turn his attention to variance, or, as he terms it, noise.

Noise was inspired by extensive evidence gathered on the variability in criminal sentencing. In studies of both real and hypothetical criminal cases, individual judges differed by orders of magnitude in the lengths of sentences assigned to defendants. The resulting inconsistency seems manifestly unfair.

The authors also recount how different underwriters provide widely differing insurance risk assessments. These assessments are encapsulated in price quotes that are the core of the insurance business: too low, and you make no profit, or maybe even incur a loss. Too high, and you might lose the business.

Kahneman, Sibony and Sunstein make several points concerning these judgment “errors:”

●Bias, the error of being consistently off-target, is more easily visualized and receives more popular attention than variance.
●Error resulting from variance can be just as costly as that from bias, often more so.
●Error from variance can often be quantified in situations where error from bias cannot, since the “true” value, needed to calculate bias, is not fully known (such as the risk of loss for a new insurance client or the right sentence for a crime).

They go on to show how “noise audits” can quantify the loss from noise and reduce or mitigate it. Here, their audience is mainly business, where people are supremely concerned with quantifying value in a uniformly accepted unit (money), reasonably able to act as a unified organization, and less tangled up in politics than those active in, say, criminal justice.

As with Kahneman’s earlier work, Noise does a valuable service in explaining statistical concepts to a lay audience. More importantly, it shows data scientists and statisticians where their work fits in the scheme of practical decision making, an all too rare skill in those professions. All the same, even the most dedicated aficionados of statistics in practice will likely find their attention flagging before reaching the midpoint in this tome. Although important, variability as a concept for the lay audience needs only the first third of the book. Overall, unfortunately, Noise is lacking another key statistical quality: parsimony.