“Our topic is human error. Bias and noise – systematic deviation and random scatter -are different components of error.”
What?
The authors are eminent academics. This is a long book at over 400 pages. They start by asking you to imagine four teams of five shooting at a target. Team A has shots which cluster tightly around the bullseye. Team B are similarly tightly clustered but this time consistently to the left and below the bullseye; they exhibit bias. Team C’s shots are scattered randomly around the target; they’re unpredictable, follow no pattern and are therefore noisy. Team D have shots which re widely scattered and systematically off target to the left; both biased and noisy.
This shooting range is described as a metaphor for what can go wrong in human judgement, especially in those decisions made on behalf of others or of organisations. Everyone is attempting to achieve the same things but ending up in different places.
So What?
“Although it is often ignored, noise is a large source of malfunction in society. In a 1981 study, for example, 208 federal judges were asked to determine the appropriate sentences for the same 16 cases… The average difference between the sentences that two randomly chosen judges gave for the same crime was more than 3.5 years. Considering that the mean sentence was seven years, that was a disconcerting amount of noise.”
The authors describe how we are all affected by the judgments of others, both professional and personal. Eliminating bias from judgments does not eliminate all error, as we see in the quote above. Much of the book concerns itself with system noise – the randomness in judgments made by teams of surgeons, underwriters, child custody managers, asylum courts, interview panels.
Now What?
Take the concept of noise seriously. Do what you can to reduce it.
“Once you become aware of noise, you can look for ways to reduce it. For instance, independent judgments from a number of people can be averaged (a frequent practice in forecasting). Guidelines, such as those often used in medicine, can help professionals reach better and more uniform decisions. As studies of hiring practices have consistently shown, imposing structure and discipline in interviews and other forms of assessment tends to improve judgments of job candidates.”
The authors say that ‘noise’ is rarely discussed, which given its importance is a curiosity in itself. They say bias has an ‘explanatory charisma’ which means it is the default explanation for poor judgments. One of the best ways to overcome noise it to use a decision observer to help apply decision hygiene: accuracy, not individual expression, is the goal.