If you realise that there's more variability than you expected, you have several possibilities:
<ul>
<li>If the judgement is relatively simple, consider if you actually need human judgment or if you can replace judgment with some rule or algorithm. For example, the Apgar score, how to decide whether infants are healthy, is a rule.</li>
<li>In more complex cases, try to discipline judgment with "decision hygiene". It is a disciplined process where people making judgments follow the same thought process and likely reach similar conclusions.</li>
</ul>

What to do after the noise audit

You can do a noise edit if you have several employees performing an interchangeable function, like different federal judges, doctors,  or different underwriters in an insurance company. In a noise audit, people are given a realistic problem that they could encounter on their job. Then look at the variability of judgments. If the judgements are variable, then errors are variable.

Organisations can address noise with a "noise audit"

Wherever there is judgment, there will be noise. When trying to reduce noise, you can expect some pushback as people perceive what is happening as limiting their options and as a bureaucratic infringement on their role. Employees who are affected by this have to see this as something that helps them do their job, rather than something that replaces them or constrains them too much.  The purpose is not to eliminate intuition, but to delay it - to engage in a disciplined thought process before they form an intuition or global judgment.

Noise can be reduced, not eliminated

Reducing noise is primarily for organizations that make judgments and decisions, and it wants to do that with one voice. Noise is unwanted variability. In many other situations, diversity is actually interesting and valuable, for example, film reviewers and creative processes. There are also situations where a team with diverse expertise make partial judgments that you don't want to be identical. You want them to reflect different characteristics of the problem.

Situations where noise reduction doesn't apply

Bias in an average error. It is also a psychological mechanism that causes systematic errors in people's judgments. It explains why an individual will make one mistake over another. Noise is the phenomenon of individual differences. It's variability across individuals.

Bias vs noise

Decision hygiene is intended to reduce noise, but it is also applicable to single decisions where noise is invisible. When facing a decision with multiple options, treat options like job candidates. Break up the problem and evaluate various aspects of the option, just like you would break up various traits of candidates. Do that while keeping the specific judgements:<ul><li>fact-based as much as possible</li><li>as independent of each other as possible</li></ul>

Decisions hygiene is a disciplined process

When it comes to accuracy in decision-making, there are two types of error: <ul><li>Bias, which is an average error</li><li>Noise is the variability of error.</li></ul> For example:In a noisy system, judges give vastly different sentences to defendants who committed the same crime. Some judges give a one-month sentence, others one year, others seven years, and others somewhere in between.In a biased system, judges might consistently give sentences too high for certain crimes. In many situations, noise is a more severe source of inaccuracy and error than bias.

Two types of error that influence decision-making

Lessons in Decision-Making from Nobel Laureate Daniel Kahneman

books

Books

problemsolving

Problem Solving

communication

Communication

personaldevelopment

Personal Development

Understand yourself and those around you.

psychology

Psychology

The Art of Decision-Making

Noise is unwanted variability in judgments. Noise covers another way we make systematic errors in decision-making.

Ten years on from "Thinking Fast and Slow", Kahneman is back with a new book, Noise, that will again have you questioning what you thought you knew about making decisions.