The 'illusion of knowledge' that makes people overconfident

It’s easy to think you’re a font of knowledge. And while you may have plenty of skills and expertise, it’s quite likely you know less than you think.

f you consider yourself reasonably intelligent and educated, you might assume that you have a fair grasp on the core ways the world works – knowledge about the familiar inventions and natural phenomena that surround us.

Now, think about the following questions: How are rainbows formed? Why can sunny days be colder than cloudy days? How does a helicopter fly? How does a toilet flush?

Next, ask yourself: could you give a detailed response to any or all these questions? Or do you have only the vaguest gist of what’s happening in each case?

If you are like many of the participants in psychological studies, you may have initially expected to perform very well. However, when they are asked to offer a nuanced answer to each question, most people are completely stumped – just as you may be, too.

This bias is known as an “illusion of knowledge”. You may think that these specific examples are trivial – they’re the kinds of questions, after all, that an inquisitive child might ask you, where the worst consequence may be a red face in front of your family. But illusions of knowledge can afflict our judgement in many domains. In the workplace, for example, it can lead us to overclaim our knowledge in an interview, overlook the contributions of our colleagues and take on jobs we may be wholly unable to perform.

Many of us go through life completely oblivious to this intellectual arrogance and its consequences. The good news is that some psychologists suggest there may be some disarmingly simple ways to avoid this pervasive thinking trap.

Passive observation can increase people’s confidence in their abilities to perform complex life-or-death tasks, such as landing a plane (Credit: Getty Images)

Unknown unknowns

The illusion of knowledge – also called the “illusion of explanatory depth” – first came to light in 2002. In a series of studies, Leonid Rozenblit and Frank Keil at Yale University first provided participants with example explanations of scientific phenomena and technological mechanisms, which were scored on a scale of 1 (very vague) to 7 (very thorough). This ensured all participants were on the same page when it came to judging what comprised a “vague” or “thorough” understanding of a topic.

Next came the test. When presented with further science and tech questions, the participants had to rate how well they thought they would be able to answer each one, using that same scale, before writing out their explanation in as much detail as possible.

Rozenblit and Keil found that the participants’ initial appraisals of their understanding were often dramatically optimistic. They assumed they could write paragraphs on the subject, but often failed to offer more than the barest gist of an answer – and afterwards, many expressed surprise at how little they knew.

The researchers suspected that the overconfidence arose from the participants’ ability to visualise the concepts in question; it’s not hard to picture the flight of a helicopter, for example, and the ease with which that mental film came to mind led the participants to feel more confident about explaining the mechanics of its movements.

Since this seminal paper, psychologists have unveiled illusions of knowledge in many different contexts. For example, Matthew Fisher, an assistant professor in marketing at Southern Methodist University, Texas, has found that many university graduates vastly overestimate their grasp of their college major, once they have left their studies.

Much like the first experiment, the participants were asked to rate their understanding of different concepts before providing a detailed explanation of what they meant. This time, however, the questions came from the subject they had studied years before. (A physics graduate might have attempted to explain the laws of thermodynamics, for example.) Thanks to the natural attrition of their memories, the participants seemed to have forgotten many of the important details, but they hadn’t noticed how much knowledge they had lost – leading them to be overconfident in their initial predictions. When judging their understanding, they assumed that they knew just as much as when they were completely steeped in their subject.

Many of us overestimate how much we can learn by observing others – resulting in an ‘illusion of skill acquisition’

Further research has shown that having online resources at our fingertips may feed our overconfidence, as we mistake the wealth of knowledge on the internet for our own memories. Fisher asked one group of participants to answer questions – such as “how does a zipper work?” – with the aid of a search engine, while another group were simply asked to rate their understanding of the topic without using any additional sources. Afterwards, both groups went through the original test of the illusion of knowledge for four additional questions – such as “how do tornadoes form?” and “why are cloudy nights warmer?”. He found that the people who had used the internet in their initial question demonstrated greater overconfidence in the subsequent task.

The illusion of skill acquisition

Perhaps most seriously, many of us overestimate how much we can learn by observing others – resulting in an “illusion of skill acquisition”.

Michael Kardas, a post-doctoral fellow in management and marketing at Northwestern University, US, asked participants to watch repeated videos of various skills, such as throwing darts or doing the moonwalk dance, up to 20 times. They then had to estimate their abilities, before trying the task for themselves. Most participants assumed that simply observing the film clips would have helped them to learn the skills. And the more they watched the films, the greater their initial confidence.

The reality, however, was distinctly disappointing. “People thought they’d score a greater number of points if they watched the video 20 times compared to if they’d watched it once,” says Kardas. “But their actual performance did not show any evidence of learning.”

Quite astonishingly, passive observation can even increase people’s confidence in their abilities to perform complex life-or-death tasks, such as landing a plane. Kayla Jordan, a PhD student at the University of Waikato, New Zealand, who led this study, was directly inspired by Kardas’s research. “We wanted to test the limits of the phenomenon – whether it could apply for really expert skills.” She points out that piloting requires hundreds of hours of training and a deep understanding of physics, meteorology and engineering, which people are unable to pick up through a short video.

The participants were first told to “imagine you are on a small commuter plane. Due to an emergency, the pilot is incapacitated, and you are the only person left to land the plane”. Half were then shown a four-minute video of a pilot landing a plane, while the rest did not see the clip.

Crucially, the film did not even show what the pilot’s hands were doing during the procedure – it could not have been of any instructional use. Many of the people who had seen the clip, however, became much more optimistic about their capacity to safely land a plane themselves. “They were about 30% more confident, relative to people who didn’t watch that video,” says Jordan.

Overconfidence about your knowledge can seep into the workplace, even making people arrogant (Credit: Getty Images)

Real-life dilemmas

These illusions of knowledge can have important consequences. Overconfidence in your knowledge may mean that you prepare less for an interview or presentation, for example, leaving you embarrassed when you are pressed to demonstrate your expertise.

Overconfidence may be a particular problem when you are aiming for promotion. When observing people from afar, you might assume you know what the job takes, and that you have already absorbed the necessary skills. Once you have started the job, however, you may discover that there was much more to the role than met the eye.

It may also lead us to undervalue our colleagues. In much the same way we mistake Googled knowledge for our own, we may not realise how much we are relying on the skills and abilities of the people around us. “When seeing others’ skills and knowledge base – people can sometimes mistake that as an extension of what they know themselves,” says Jordan.

If we start to claim our colleagues’ knowledge as our own, we may be less likely to remember and show gratitude for their contributions – a form of arrogance that is a common bugbear in the office. Overestimating our knowledge, and forgetting the support we have received from others, could also create serious problems when we attempt to go it alone with a solo project.

What can people do to avoid these traps? One solution is simple: test yourself. If you are appraising your capacity to perform an unfamiliar task, for instance, don’t just rely on a vague, gist-like idea of what it would involve. Instead, take a bit more time to think carefully through the steps that you would have to take to achieve the goal. You may find that there are huge gaps in your knowledge that you need to fill before you put yourself forward.

Even better, you might approach an expert and ask them what they are doing – a conversation that should check any arrogant assumptions you might be carrying.

Given the potential of technological crutches to inflate confidence in your knowledge, you could also check your online habits. Fisher suggests that you briefly pause and try your hardest to remember a fact before resorting to an internet search. By consciously recognising your mental blank, you may begin to form a more realistic appraisal of your memory and its limits. “It requires a willingness to be stumped,” he says. “You have to feel your ignorance, which can be uncomfortable.”

The aim, with all of this, is to cultivate a little more humility – one of the classic “intellectual virtues” celebrated by philosophers. By recognising our illusions of knowledge and admitting the limits of our understanding, we may all sidestep some unfortunate thinking traps to enjoy wiser thinking and decision making.

Source