Real experts know what they don’t know and we should value it | Aeon Essays

The rise of social media means that experts willing to share their knowledge are more accessible to the public. One might think that communication between experts and decision-makers should be very good. But this is not the case.

Outlets are flooded with self-appointed 'experts' who lack real expertise. In every domain where decision-makers need experts or specialized knowledge, they will compete with those who don't have relevant knowledge.

Real experts are often confident in their claims, but in the private market, the opposite can be more common.

• Self-appointed experts want airtime and overstate their conclusions.
• Actual experts want to draw attention to their work. They may not have all the answers and may admit it.
• Consultants want future business. They have to try to pretend they have complete certainty about uncertain things.

Mixing the information of the pundit, scholar, and consultant creates information noise that makes it difficult for decision-makers to know what to do.

When communicating scientific knowledge to policymakers and the public, there are three levels of questions:

• Level-one questions: Anyone with even modest expertise or access to a search engine can answer these questions. For example, 'Will price controls cause shortages?'
• Level-two questions: Only the most qualified experts, within existing paradigms of scholarly knowledge, have something to say. For example, 'Can we design algorithms to assign medical residents to programmes in an effective way?'
• Level-three questions: Even the experts don't know the answers, such as what interest rates will be in two years.

Knowing which questions fall into which category requires expertise. Politicians and executives might be experts in the area of decision-making, but they are seldom experts in the areas where they make decisions.

When there are decisions that require an expert perspective, the real experts might admit that they don't know, making the decision-maker vulnerable to uninformed experts who convince them that they have the answers.

• When experts and pundits don't acknowledge that they don't know, it can lead to bad decisions.
• When uncertainty is admitted with a level-two problem, qualified experts' advice can get lost in the noise, or decision-makers might ask the wrong experts.
• When the research frontier doesn't offer definitive answers or gives the wrong answers, it can result in bad policy.
• A false confidence in understanding important questions will delay the discovery of real improvements.

True expertise means knowing the limits of one's knowledge.

Institutions can be designed to encourage real experts to admit uncertainty. The key lies in whether the question was answerable in the first place. Good experts should be willing to say they don't know because it is an unanswerable question. Once the competent experts are saying they don't know, incompetent experts may follow suit.

• Ask different experts different questions. Rather than asking experts: "Will this drug work?', ask some of them, "Is there good evidence about whether the drug will work?" Qualified experts won't always know the answer to the first problem, but will know the answer to the last question.
• Don't just broadcast the most extreme and confident views. The most confident might not be knowledgeable enough to realise they shouldn't be confident.
• Listen to conversations between experts, as they might be more honest about their level of confidence in this context.
• Qualified and knowledgeable experts may be driven to overstate their certainty. To overcome this, the decision-maker should ask better questions and be willing to listen to the less confident experts.