Real experts know what they don’t know and we should value it | Aeon Essays

When communicating scientific knowledge to policymakers and the public, there are three levels of questions:

• Level-one questions: Anyone with even modest expertise or access to a search engine can answer these questions. For example, 'Will price controls cause shortages?'
• Level-two questions: Only the most qualified experts, within existing paradigms of scholarly knowledge, have something to say. For example, 'Can we design algorithms to assign medical residents to programmes in an effective way?'
• Level-three questions: Even the experts don't know the answers, such as what interest rates will be in two years.

Knowing which questions fall into which category requires expertise. Politicians and executives might be experts in the area of decision-making, but they are seldom experts in the areas where they make decisions.

When there are decisions that require an expert perspective, the real experts might admit that they don't know, making the decision-maker vulnerable to uninformed experts who convince them that they have the answers.

• When experts and pundits don't acknowledge that they don't know, it can lead to bad decisions.
• When uncertainty is admitted with a level-two problem, qualified experts' advice can get lost in the noise, or decision-makers might ask the wrong experts.
• When the research frontier doesn't offer definitive answers or gives the wrong answers, it can result in bad policy.
• A false confidence in understanding important questions will delay the discovery of real improvements.

True expertise means knowing the limits of one's knowledge.

Institutions can be designed to encourage real experts to admit uncertainty. The key lies in whether the question was answerable in the first place. Good experts should be willing to say they don't know because it is an unanswerable question. Once the competent experts are saying they don't know, incompetent experts may follow suit.

• Ask different experts different questions. Rather than asking experts: "Will this drug work?', ask some of them, "Is there good evidence about whether the drug will work?" Qualified experts won't always know the answer to the first problem, but will know the answer to the last question.
• Don't just broadcast the most extreme and confident views. The most confident might not be knowledgeable enough to realise they shouldn't be confident.
• Listen to conversations between experts, as they might be more honest about their level of confidence in this context.
• Qualified and knowledgeable experts may be driven to overstate their certainty. To overcome this, the decision-maker should ask better questions and be willing to listen to the less confident experts.