The False Consensus Effect

In July 2011, Netflix told its American subscribers that their plan was changing. For $9.99 a month, they had been getting unlimited streaming plus DVDs by post. From now on, streaming and DVDs would be separate plans at $7.99 each. Anyone who wanted to keep both would pay $15.98, a rise of 60%.

Customers were furious, and two months later the company's co-founder and chief executive, Reed Hastings, wrote a blog post that began with an apology for how the change had been handled. In the same post, he announced that the DVD business would be split off under a new name, Qwikster, with its own website. Customers who wanted both services would now need two accounts, two sets of ratings and two bills. Many people found this even worse than the price rise. Within a few weeks, Qwikster was cancelled before it had launched. Netflix lost around 800,000 American subscribers in a single quarter, and its share price fell by roughly three quarters between July and the end of the year.

Hastings was right about the big picture. Streaming was the future, and Netflix recovered spectacularly. What he misjudged was how his customers felt about the present. He saw the DVD business as a fading part of the company. Millions of subscribers still loved it, and many of them saw no reason to pay more or split their account to help Netflix move on. In his apology he wrote that he had slid into arrogance based on past success. Years later, he told another part of the story. Several of his senior executives had serious doubts about Qwikster at the time, but didn't push back with any force. Each assumed Hastings must see something they didn't, and none of them realised the others felt the same.

So the room believed it agreed, and its leader believed his customers would come round to his view. That second belief has a name. The false consensus effect is our tendency to overestimate how many other people share our own opinions, preferences and habits. What I think, want and do feels normal, so I assume it's common.

With this post, the series turns from flawed arguments to the biases in our own heads, starting with the ones that shape the people who design, build and decide. The false consensus effect felt like the right place to begin, because it's the bias that most quietly turns a team's taste into a product's assumptions.

The name comes from a 1977 paper by the Stanford psychologists Lee Ross, David Greene and Pamela House. In one study, they asked students whether they would be willing to walk around campus for half an hour wearing a large sandwich board with a short slogan on it, such as "Eat at Joe's". They were free to refuse. Then each student was asked to estimate what percentage of their fellow students would agree.

The students who agreed to wear the board thought that, on average, about 62% of their peers would agree too. The students who refused thought only about 33% would agree, which is to say that most people would refuse, just as they had. Both groups saw their own choice as the common one. They also judged the people who would choose differently more harshly, drawing stronger conclusions about their personalities. If you would refuse and someone else agrees, there must be something unusual about them.

The same paper found the pattern in written questionnaires too. Students read short scenarios, such as whether they would pay a speeding fine or contest it in court, chose what they would do, and estimated what others would choose. Again, whichever option people picked, they thought it was the more popular one.

Unlike some famous findings in psychology, this one has held up well. In 1985, Brian Mullen and his colleagues combined 115 tests of it in a meta-analysis and found the effect consistently, though often of moderate size. In 2018, a large replication project called Many Labs 2 repeated two of the original questionnaire scenarios with thousands of participants in labs around the world, and found the effect again. The more interesting debate has been about how "false" it really is, which I'll come back to.

The first reason is simply what comes to mind. My own opinion is the one I know best and think about most, so when I try to imagine what others think, it's the first answer that arrives. The psychologists Nicholas Epley, Boaz Keysar, Leaf Van Boven and Thomas Gilovich showed in 2004 that people take other perspectives by starting from their own and then adjusting, and that they tend to stop adjusting too soon, especially when they're rushed.

The second reason is the company we keep. Sociologists have long documented that people tend to spend time with others similar to them in age, education, class, language and views. When most of the people I talk to share my habits, my sample of humanity is already tilted towards me. A product team is an extreme version of this: people of similar age and background, often in the same city, using the same phones on the same fast connections, hired partly because they share a way of thinking.

Most of the time, this shortcut serves us well. In everyday life we mostly deal with people who are like us in many ways, so starting from ourselves is a quick and usually decent first guess. The trouble starts when we design for people who aren't. There's comfort in it too. When someone disagrees, it's easier to see them as the odd one out, as Ross's students did, than to suspect that I might be in the minority.

For years, most Indian consumer apps were built English-first. The teams building them worked in English, studied in English and used the internet in English, and many products treated other Indian languages as a later translation project rather than the starting point. It's hard to prove what was in anyone's head, but the assumption is easy to read in the products: the user is someone like us.

In 2017, a report by Google and KPMG, Indian Languages: Defining India's Internet, put numbers on how wrong that picture had become. By the end of 2016, it estimated, about 234 million Indians used the internet mainly in an Indian language, compared with about 175 million who used it in English, and most of the next wave of users would come from the first group.

Some of the products that did well started from the other end. ShareChat, a social app founded in 2015 by three graduates of IIT Kanpur, launched in Indian languages and deliberately left English out entirely. To a team that lived online in English, that looked like a strange decision. To the people it was built for, it was the obvious one. The opportunity had been sitting outside the office for years. The false consensus effect is one reason so many makers walked past it: when everyone you work with reads English, it feels as though everyone does.

Picture a team that builds an app for booking intercity bus tickets. Every ticket is sent two ways: inside the app, and as a text message with a link. Maintaining the text messages costs money and engineering time, and in a roadmap review someone suggests dropping them. "Honestly, who opens a ticket from an SMS? Everyone just uses the app." Around the table, people nod. They all open their tickets in the app. So do their friends.

Before deciding, the design lead tries a small exercise. Everyone writes down two things on a sticky note: how they personally open their tickets, and what share of customers they think open tickets from the text message. Every person in the room uses the app. Their guesses for the text-message share range from 2% to 10%.

Then someone pulls the data. Around a third of tickets are opened from the text link, and the share is far higher on bookings made for someone else: a son booking for his parents, a woman booking for her aunt, an office booking for a driver. Those travellers often don't have the app at all. Some share a phone with the rest of the family. The people in the room hadn't imagined them, because nobody in the room travels that way.

The better version doesn't depend on the data happening to exist. The team separates "what we do" from "what our customers do" in its language, and treats the first as a sample of a few people from one corner of the market. It looks at the data split by segment, not just in total. And when the data can't answer a question, it goes and finds the people who aren't in the room, rather than asking the room to imagine them. The team keeps the text messages, because a feature they thought was a leftover turned out to be the front door for a large group of travellers.

Using yourself as a guide to others isn't always a mistake. In 1989, the psychologist Robyn Dawes pointed out that if you know nothing else about a group, your own answer is a genuine piece of evidence about it: you are one member of the population, after all. Someone who prefers tea and guesses that most people prefer tea is making a reasonable inference from a sample of one. Joachim Krueger and Russell Clement later showed that people still lean on their own answer more than this kind of reasoning justifies, which is why the effect is called false. But the starting instinct isn't irrational.

It also helps when you really are like the people you're predicting. The marketing researcher Stephen Hoch found in 1987 that projecting from your own preferences can make predictions more accurate when you share a lot with the people you're judging. That's why a designer building a tool for designers, or a parent building something for parents, often has good instincts. The problem for most product teams is that they are not much like their users, and the differences are exactly the ones they can't see from the inside.

The opposite error exists too. Sometimes we underestimate how many people agree with us, as Netflix's executives did about their own doubts. Psychologists call this pluralistic ignorance. In a well-known 1993 study at Princeton, Deborah Prentice and Dale Miller found that many students privately felt less comfortable with campus drinking than they believed their peers were. Both errors come from the same place: guessing what others think instead of finding out.

The first habit is to write my guess down before I look at any data. What percentage of users do I think will use this, prefer that, or notice the other? Then I check, and I keep a rough record of how far off I was. My misses aren't random. They lean towards people like me, and seeing that pattern in my own handwriting is more persuasive than any study.

The second is to watch my language. "I'd never use that" and "nobody uses that" are different claims. When I catch myself saying the second, I ask what it's based on beyond my own habits and those of the people around me.

The third is borrowed from what Netflix did afterwards. Hastings has described how, before big decisions, the company began asking people to write down their own view privately, on a scale from strongly for to strongly against, with their reasons, so that real disagreement could surface before anyone assumed it didn't exist. It works in design reviews too. The room's consensus is only worth something if it survived a chance to be revealed as false.

The Qwikster episode lasted barely a month, and Netflix survived it. But it still seems to me a perfect picture of this bias: a thoughtful leader, a confident room and a customer base that never agreed. In the next post I'll look at a close relative, the bias that makes us search for evidence that confirms what we already believe: confirmation bias.

Further reading: Lee Ross, David Greene and Pamela House, "The 'False Consensus Effect': An Egocentric Bias in Social Perception and Attribution Processes" (1977) · Brian Mullen and colleagues, "The False Consensus Effect: A Meta-Analysis of 115 Hypothesis Tests" (1985) · Robyn Dawes, "Statistical Criteria for Establishing a Truly False Consensus Effect" (1989) · Reed Hastings and Erin Meyer, No Rules Rules: Netflix and the Culture of Reinvention (2020) · Google and KPMG, Indian Languages: Defining India's Internet (2017)