We may feel that we are not smart enough to do what we do. One explanation for this is the Dunning-Kruger effect: when you are incompetent, you can't know you're incompetent and can't properly assess your own competence. Research shows that incompetent students think they are much better than they are and top students underestimate themselves.

Feeling insecure about what you know

When you start on a learning path, you may feel confident in learning, but the better you get, the more self-conscious you become of what you can't do or understand. When you delve deeper, you'll find some ideas that get sharpened and others that get more complicated and nuanced. It can cause you to start doubting yourself. This feeling of self-doubt is a feature of learning and arise from two causes:<ul><li>The structure of knowledge and skills</li><li>Social comparisons with experts</li></ul>

Delving into knowledge

Knowledge structures. The more knowledge you learn, the more subtle patterns appear. Consider learning English. The word "good" may be sufficient until you know words like "fine", "excellent", "stupendous", sublime."
Social comparisons. When you start, you may compare yourself to your peers. But the more you learn, the more you compare your quality to experts. Unfortunately, this is not always helpful.

Knowledge structures and social comparisons

There are two approaches:<ul><li>"Leave it to the experts." While it is true that the most informed speak the clearest, it is also true that fields of expertise can develop closed intellectual cultures where certain assumptions get ingrained.  The best answers come from open and deliberate processes.</li><li>Try to be honest but humble. Do your best, and be open to changing your mind if some people who know more than you disagree with you.</li></ul>

Dealing with intellectual insecurity

I often don’t feel smart enough to do what I do.I don’t say this out of false modesty or as an attempt to be more relatable. I spent the last decade taking on rather-immodest learning challenges. I’ve always been confident in my ability to learn new things.Instead, the insecurity comes from recognizing that, no matter how hard I try, there are tons of people who know way more than me. Perhaps they, not I, should be the ones to write these essays?I was reminded of this recently while reading Scott Alexander’s newsletter. <a href="https://astralcodexten.substack.com/p/epistemic-minor-leagues" dir="ltr">He writes</a>:<BLOCKQUOTE>I often find myself trying to justify my existence; how can I write about science when I’m not a professional scientist, or philosophy when I’m not a professional philosopher, or politics when I’m not a professional policy wonk? When I’m in a good mood, I like to think it’s because I have something helpful to say about these topics. But when I’m in a bad mood, I think the best apology I can give for myself is that the discovery drive is part of what it is to be human, and I’m handling it more gracefully than some.</BLOCKQUOTE>Alexander is extremely smart. He’s written some of my favorite essays. He’s able to quickly summarize deep research, reveal hidden patterns in our debates, and manages to be funny at the same time. As a writer, I’m envious.Why should Alexander feel the need to apologize for his efforts to assemble new ideas? Why should you or I not feel smart enough to do the work we want to do, and what should we do about it?<h2>Dunning-Kruger Effect?</h2><div dir="ltr">
Listen to this article 
</div>The popular explanation of this is <a href="https://en.wikipedia.org/wiki/Dunning_Kruger" dir="ltr">the Dunning-Kruger effect</a>. Those who are incompetent also cannot accurately self-assess their competence. As David Dunning explains:<BLOCKQUOTE>“If you’re incompetent, you can’t know you’re incompetent … The skills you need to produce a right answer are exactly the skills you need to recognize what a right answer is.”</BLOCKQUOTE>Bertrand Russell expressed a similar sentiment nearly a century ago: “The fundamental cause of the trouble is that in the modern world the stupid are cocksure while the intelligent are full of doubt.”So that’s the problem? Smart people are insecure about their knowledge, and the ignorant are oblivious to their incompetence?Except that’s not what the research actually shows. The study found that incompetent students overestimated their class rank, and top students underestimated theirs. However, incompetent students didn’t think they were better than good students.<figure><img width="745" height="373" src="https://www.scotthyoung.com/blog/wp-content/uploads/2021/11/Smart-enough.jpg" alt="" srcset="https://www.scotthyoung.com/blog/wp-content/uploads/2021/11/Smart-enough.jpg 800w, https://www.scotthyoung.com/blog/wp-content/uploads/2021/11/Smart-enough-300x150.jpg 300w, https://www.scotthyoung.com/blog/wp-content/uploads/2021/11/Smart-enough-768x384.jpg 768w"></figure><BLOCKQUOTE>[T]he bias is definitively not that incompetent people think they’re better than competent people. Rather, it’s that incompetent people think they’re much better than they actually are. But they typically still don’t think they’re quite as good as people who, you know, actually are good.</BLOCKQUOTE>Did all those people who casually cited the Dunning-Kruger effect to mean “stupid people are cocksure” just fall victim to Dunning-Kruger? Or is something else going on here?<h2>Digging Into Knowledge</h2>My own experience with learning things tends to involve the following cycle:<OL><LI>Read claim X.</LI><LI>Read a bit more about claim X. It turns out the authors who proposed X didn’t actually say X, but X’. Also, some people think Y and Z with good reason. Nobody thinks Q.</LI><LI>Actually, some people think Q. And X’ is contradicted by work done in a different field, except there it’s called P. Maybe X is right all along?</LI></OL>The expectation is that as you learn more and more, you’ll eventually hit a bedrock of irrefutable scientific fact. Except usually, the bottom of one’s investigation is muck. Some parts of the original idea get sharpened, others blur as more complications and nuance are introduced.This experience doesn’t just apply to researching facts. When I started learning languages, I was more than happy to carry on a decent conversation. But, once I achieved it, I became self-conscious of all the things I couldn’t do well. I’d struggle to follow movies or read literature. Group conversations could be surprisingly challenging. Occasionally I’d bump into a seemingly basic situation that I couldn’t grasp. I’d think: “Wait… Am I actually terrible at this?”Given my modest talents, it’s impossible to assert that this pattern continues forever. Perhaps there truly is a vaunted place of excellence on the other side, where doubts cease to exist and one is supremely self-confident. If such a place exists, I certainly haven’t found it.Instead, I suspect that this squirming feeling of self-doubt is a persistent feature of learning. Speculating further, I’d suggest it arises from at least two principle causes: one inherent in the cognitive structure of knowledge and skills, and the other in the social comparison with expert performers.<h2>Knowledge Structures and Social Comparisons</h2>The first cause of this trouble seems to be in how knowledge works. Ability appears to result from accumulating increasingly nuanced patterns of discrimination, motor skill, and reasoning.Consider learning to speak English. You first learn the word “good,” which applies to a whole range of situations. Later you learn shades of intensity: “okay,” “fine,” and “excellent.” As you gain more knowledge, you learn “stupendous,” “sublime,” and “stellar.”Each word occupies a slightly different meaning. A space of discrimination previously described by “good” now gets subdivided into many overlapping sections. Blanks exist at each stage of the learning process—areas where you don’t have a good word to describe the situation. As you get more precise, the size of blanks appears to grow. Gaps that seemed negligible begin to look like chasms.From a relative perspective, there is consistent progress. You always know more than you knew before. But it often doesn’t feel that way. Instead, learning more seems to make you feel more ignorant about everything else you don’t know.The other effect of increasing knowledge is changing who you compare yourself to. When I started writing, I mostly read other advice writers and compared my work to theirs. Now that I spend most of my days reading original research, I’m comparing the quality of my thinking to a more rarefied intellectual strata. That comparison is not always favorable.<h2>Dealing with Intellectual Insecurity</h2>There seem to be two strategies for dealing with this kind of intellectual insecurity.The first is to “leave it to the experts.” The relatively smart often apply this one to the relatively uninformed as a kind of browbeating. In other cases, it’s intended to try to remove noise from a discussion so those with the most experience can speak the loudest.While I understand wanting to let the most-informed speak clearest, I have serious doubts about the consequences if this strategy were to be applied consistently.Fields of expertise often develop insular intellectual cultures where questionable assumptions can get entrenched. While a failure to heed expertise is blamed for many of our current woes, an overconfidence in expertise may have produced some of the <a href="https://www.nytimes.com/2021/09/20/opinion/afghanistan-experts-expertise.html" dir="ltr">woes of the past</a>. It’s not obvious, to me at least, that our overall credence in authorities is set too high or too low.I am persuaded by Hugo Mercier and Dan Sperber’s <a href="https://www.scotthyoung.com/blog/2019/02/01/reason/" dir="ltr">account of reasoning</a>. They argue that the best answers arrive not from individual rationality, but from open, deliberative processes. Reason, in this sense, functions more like a conversation than a solitary mental act. While individual voices may be wrong, the collective takeaway from the discussion is fairly good.Still, there’s an obvious, opposite danger of intellectual egalitarianism: considering everybody’s opinions as equal, even if some people have spent years in careful study and others have not.The second strategy, which I prefer, is to try to be honest yet humble. Do your best, given what you know. Be willing to change your mind. If many people who know more than you disagree, it’s a sign to stop and listen.Perhaps it is a tad self-serving in light of my career as a writer of non-expert opinions, but my gut tells me that the right way to use one’s insecurity is as a drive to improve, not to halt inquiry. Be honest and do the best work you can, but don’t ever stop learning.

How do you cope with feelings of intellectual inadequacy? Are the stupid really cocksure while the smartest are full of doubt?

Stashed ideas

I often don’t feel smart enough to do what I do.

I don’t say this out of false modesty or as an attempt to be more relatable. I spent the last decade taking on rather-immodest learning challenges. I’ve always been confident in my ability to learn new things.

Instead, the insecurity comes from recognizing that, no matter how hard I try, there are tons of people who know way more than me. Perhaps they, not I, should be the ones to write these essays?

I was reminded of this recently while reading Scott Alexander’s newsletter. <a href="https://astralcodexten.substack.com/p/epistemic-minor-leagues">He writes </a>:

I often find myself trying to justify my existence; how can I write about science when I’m not a professional scientist, or philosophy when I’m not a professional philosopher, or politics when I’m not a professional policy wonk? When I’m in a good mood, I like to think it’s because I have something helpful to say about these topics. But when I’m in a bad mood, I think the best apology I can give for myself is that the discovery drive is part of what it is to be human, and I’m handling it more gracefully than some.

Alexander is extremely smart. He’s written some of my favorite essays. He’s able to quickly summarize deep research, reveal hidden patterns in our debates, and manages to be funny at the same time. As a writer, I’m envious.

Why should Alexander feel the need to apologize for his efforts to assemble new ideas? Why should you or I not feel smart enough to do the work we want to do, and what should we do about it?

The popular explanation of this is <a href="https://en.wikipedia.org/wiki/Dunning_Kruger">the Dunning-Kruger effect </a>. Those who are incompetent also cannot accurately self-assess their competence. As David Dunning explains:

“If you’re incompetent, you can’t know you’re incompetent … The skills you need to produce a right answer are exactly the skills you need to recognize what a right answer is.”

Bertrand Russell expressed a similar sentiment nearly a century ago: “The fundamental cause of the trouble is that in the modern world the stupid are cocksure while the intelligent are full of doubt.”

So that’s the problem? Smart people are insecure about their knowledge, and the ignorant are oblivious to their incompetence?

Except that’s not what the research actually shows. The study found that incompetent students overestimated their class rank, and top students underestimated theirs. However, incompetent students didn’t think they were better than good students.

[T]he bias is definitively not that incompetent people think they’re better than competent people. Rather, it’s that incompetent people think they’re much better than they actually are. But they typically still don’t think they’re quite as good as people who, you know, actually are good.

Did all those people who casually cited the Dunning-Kruger effect to mean “stupid people are cocksure” just fall victim to Dunning-Kruger? Or is something else going on here?

My own experience with learning things tends to involve the following cycle:

<ol><li>Read claim X. </li><li>Read a bit more about claim X. It turns out the authors who proposed X didn’t actually say X, but X’. Also, some people think Y and Z with good reason. Nobody thinks Q. </li><li>Actually, some people think Q. And X’ is contradicted by work done in a different field, except there it’s called P. Maybe X is right all along? </li> </ol>

The expectation is that as you learn more and more, you’ll eventually hit a bedrock of irrefutable scientific fact. Except usually, the bottom of one’s investigation is muck. Some parts of the original idea get sharpened, others blur as more complications and nuance are introduced.

This experience doesn’t just apply to researching facts. When I started learning languages, I was more than happy to carry on a decent conversation. But, once I achieved it, I became self-conscious of all the things I couldn’t do well. I’d struggle to follow movies or read literature. Group conversations could be surprisingly challenging. Occasionally I’d bump into a seemingly basic situation that I couldn’t grasp. I’d think: “Wait… Am I actually terrible at this?”

Given my modest talents, it’s impossible to assert that this pattern continues forever. Perhaps there truly is a vaunted place of excellence on the other side, where doubts cease to exist and one is supremely self-confident. If such a place exists, I certainly haven’t found it.

Instead, I suspect that this squirming feeling of self-doubt is a persistent feature of learning. Speculating further, I’d suggest it arises from at least two principle causes: one inherent in the cognitive structure of knowledge and skills, and the other in the social comparison with expert performers.

Knowledge Structures and Social Comparisons

The first cause of this trouble seems to be in how knowledge works. Ability appears to result from accumulating increasingly nuanced patterns of discrimination, motor skill, and reasoning.

Consider learning to speak English. You first learn the word “good,” which applies to a whole range of situations. Later you learn shades of intensity: “okay,” “fine,” and “excellent.” As you gain more knowledge, you learn “stupendous,” “sublime,” and “stellar.”

Each word occupies a slightly different meaning. A space of discrimination previously described by “good” now gets subdivided into many overlapping sections. Blanks exist at each stage of the learning process—areas where you don’t have a good word to describe the situation. As you get more precise, the size of blanks appears to grow. Gaps that seemed negligible begin to look like chasms.

From a relative perspective, there is consistent progress. You always know more than you knew before. But it often doesn’t feel that way. Instead, learning more seems to make you feel more ignorant about everything else you don’t know.

The other effect of increasing knowledge is changing who you compare yourself to. When I started writing, I mostly read other advice writers and compared my work to theirs. Now that I spend most of my days reading original research, I’m comparing the quality of my thinking to a more rarefied intellectual strata. That comparison is not always favorable.

There seem to be two strategies for dealing with this kind of intellectual insecurity.

The first is to “leave it to the experts.” The relatively smart often apply this one to the relatively uninformed as a kind of browbeating. In other cases, it’s intended to try to remove noise from a discussion so those with the most experience can speak the loudest.

While I understand wanting to let the most-informed speak clearest, I have serious doubts about the consequences if this strategy were to be applied consistently.

Fields of expertise often develop insular intellectual cultures where questionable assumptions can get entrenched. While a failure to heed expertise is blamed for many of our current woes, an overconfidence in expertise may have produced some of the <a href="https://www.nytimes.com/2021/09/20/opinion/afghanistan-experts-expertise.html">woes of the past </a>. It’s not obvious, to me at least, that our overall credence in authorities is set too high or too low.

I am persuaded by Hugo Mercier and Dan Sperber’s <a href="https://www.scotthyoung.com/blog/2019/02/01/reason/">account of reasoning </a>. They argue that the best answers arrive not from individual rationality, but from open, deliberative processes. Reason, in this sense, functions more like a conversation than a solitary mental act. While individual voices may be wrong, the collective takeaway from the discussion is fairly good.

Still, there’s an obvious, opposite danger of intellectual egalitarianism: considering everybody’s opinions as equal, even if some people have spent years in careful study and others have not.

The second strategy, which I prefer, is to try to be honest yet humble. Do your best, given what you know. Be willing to change your mind. If many people who know more than you disagree, it’s a sign to stop and listen.

Perhaps it is a tad self-serving in light of my career as a writer of non-expert opinions, but my gut tells me that the right way to use one’s insecurity is as a drive to improve, not to halt inquiry. Be honest and do the best work you can, but don’t ever stop learning.

Bite-sized knowledge
to upgrade
your career

What if You Don't Feel Smart Enough?

Feeling insecure about what you know

Delving into knowledge

Knowledge structures and social comparisons

Dealing with intellectual insecurity

IDEAS ABOUT

MORE LIKE THIS

The Dunning-Kruger effect: you don't know what you don't know

The Dunning-Kruger Effect: The Paradox of Our Own Ignorance

Dunning-Kruger Effect: Why Incompetent People Think They Are Superior

It's time to

READ

LIKE

A PRO!

Bite-sized knowledgeto upgradeyour career

What if You Don't Feel Smart Enough?

Feeling insecure about what you know

Delving into knowledge

Knowledge structures and social comparisons

Dealing with intellectual insecurity

IDEAS ABOUT

MORE LIKE THIS

The Dunning-Kruger effect: you don't know what you don't know

The Dunning-Kruger Effect: The Paradox of Our Own Ignorance

Dunning-Kruger Effect: Why Incompetent People Think They Are Superior

It's time to

READ

LIKE

A PRO!

Bite-sized knowledge
to upgrade
your career