Illusory Correlation

We tend to see relationships between things that aren't actually related. When two events happen close together once or twice, our brains quietly file them away as connected, even when the link is pure coincidence. We then start expecting that connection to keep showing up, and sure enough, we notice the hits but ignore the misses.

But first, let's hear from this week's sponsor...

Now, back to the Illusory Correlation ⏬

Have you ever watched your team make a confident product decision based on a pattern that, when you actually look at the data, barely exists?

I spent weeks convinced an engineering lead had it out for my design team.

👍 We'd share specs 👎 He'd push back
👍 We'd share research 👎 He'd push back
👍 We'd share a prototype 👎 He'd push back

So my brain did what brains do…

It built a story:

"He's got it out for us."
"He doesn't trust design."
"He thinks UX is a waste of time."

Then, in a retro, a PM praised him for being thorough.

He pushed back on everyone: stakeholders, product, design, and even his own team.

He wasn't singling design out. That was just how he worked.

I'd taken two things that showed up together: his skepticism and my team's work, and my brain decided they were linked.

That's illusory correlation. It's the bias that turns coincidence into conviction and makes you see patterns that aren't there.

And it doesn't just show up in how we read people. It shows up in:

⇢ How we interpret a feature's performance after launch
⇢ How we build personas from a handful of interviews
⇢ How our teams develop superstitions about what causes what
⇢ How reputations get built on a couple of bad moments

We remember the hits. We forget the misses.

And we make real decisions on patterns that don’t exist.

The fix isn't to distrust every observation. Instead, ask one boring but powerful question: How often did the thing happen without the other thing?

​

Would you rather listen? Follow the Cognition Catalog on Spotify!

Illusory Correlation: The Bi... Jun 9 · Beyond UX Design 15:53   ​

---

The term "illusory correlation" was coined by psychologist Loren J. Chapman in 1967, but the concept itself is easier to understand than the name suggests. Chapman noticed something strange happening inside clinical psychology: trained professionals were reporting connections between patients' behavior on projective tests, e.g., Rorschach and ink blot tests, and their diagnoses. These were connections that, statistically, didn't actually exist.

In one of the most cited early experiments, Chapman and his wife and colleague Jean Chapman asked clinical psychologists to review drawings made by patients as part of a "Draw-A-Person" test. Some clinicians reported seeing a strong link between patients who drew people with large eyes and those diagnosed with paranoia. When the researchers analyzed the actual data, no link actually existed. The clinicians had completely invented the pattern.

What explains that? Two things, mostly.

First, our brains are wired to notice what's unusual. When something rare or distinctive happens at the same time as something else rare or distinctive, it feels significant, as if there must be a reason for it. This is sometimes called the distinctiveness effect. That co-occurrence grabs our attention in a way that hundreds of unremarkable pairings never would.

Second, we tend to remember the hits and forget the misses. Once we form a belief that two things are connected, we unconsciously start cataloging the moments that confirm it. But all those times that didn’t fit, we conveniently let those times slip by.

In 1976, David Hamilton and Robert Gifford took this a step further. They ran a series of experiments showing participants descriptions of behaviors attributed to two fictional groups. One group was larger, the other group was smaller. The ratio of positive to negative behaviors was identical across both groups. But because negative behaviors were rarer overall, and the smaller group was already less common, participants consistently overestimated how often the smaller group behaved negatively. The two rare things got mentally linked, even though the data showed no actual difference.

This is worth taking a second to think about. Hamilton and Gifford didn’t find that the bias was driven by prejudice or conscious intention. It came from the way human memory processes frequency and importance. Rare things get stored differently because they feel more important. And when two rare things show up together, the brain files them away as connected.

The famous duo Daniel Kahneman and Amos Tversky’s work in the 1970s on cognitive heuristics added another interesting layer. Their research on the availability heuristic, that is, the idea that we judge probability based on how easily examples come to mind, helps explain why illusory correlations stick. The co-occurrence that felt notable is the one we can recall most easily, which makes it seem even more common and meaningful than it is.

The result is a human brain that's constantly connecting dots, even when those dots belong to completely different pictures.

Illusory correlation is the bias most likely to send a product team in the wrong direction without anybody noticing. And that’s because it can feel like insight. It can sound like a signal. But it’s almost always built on a handful of vivid cases plus the human brain’s urge to connect them.

Start with how teams read their own metrics. A feature ships on Monday, weekly active users tick up on Wednesday, and by Thursday, the team starts referring to the launch as a “win.” But marketing ran a campaign on Tuesday, a competitor’s website was down on Wednesday, and traffic always lifts midweek.

The launch is the most available explanation, so it gets the credit. Future roadmap arguments now lean on a causal claim that the data never supported to begin with.

A/B testing makes the problem mechanical. Ron Kohavi, who ran experimentation at Microsoft and Amazon and basically wrote the book on A/B testing, has found that roughly one in four “statistically significant winners” at big tech companies fail to replicate. Teams that peek early, run underpowered tests, or try many variants without correction will see patterns that are pure noise. The dashboards look convincing because of the clustering illusion, the random data’s tendency to clump together in ways that look meaningful. If you want to go deeper on that one, we’ve got a whole episode on it. But the short version is that clustering looks identical to a real signal until you actually check the math.

The bias also distorts how teams think about people. Three angry support tickets from healthcare customers, and suddenly, “enterprise healthcare is a tough segment” becomes received wisdom. The 200 quiet, satisfied healthcare customers don’t generate tickets, so they don’t enter the team’s mental sample. That’s Hamilton-Gifford’s findings in production: rare group plus rare event, encoded with too much weight, hardening into a stereotype that drives prioritization.

User research gets the same treatment. Six interviews with power users, two mentions of keyboard shortcuts, and a persona narrative crystallizes. From there, every quote that fits the story gets remembered; quotes that contradict it slip out of the synthesis deck. The Chapmans showed this happens even when the underlying pairings are random, so it will definitely happen when the sample is only six.

Engineering teams build their own folk correlations, too. The “Tuesday curse.” The intern who “broke prod.” The exec demo that “always” triggers an outage. Two distinct events occur concurrently: the team encodes a vivid memory, and a superstition becomes a working assumption that shapes deployment schedules, code review intensity, and how junior engineers are treated.

The team-dynamics damage compounds. Decisions feel data-driven because someone can name a specific anecdote. Disagreement gets framed as “not paying attention to what’s obvious.” Quiet skeptics who ask for the base rate get labeled obstructionists. The team loses calibration without losing confidence, which is the worst combination.

The fix isn’t more data, but better contingency thinking. How often does the outcome happen when the suspected cause is absent? How often does the suspected cause happen with no outcome? Without that habit, teams will keep mistaking pattern recognition for understanding.

The fix isn’t to distrust every observation. It’s to ask a boring but powerful question before acting on a perceived pattern: how often did the thing happen without the other thing? That’s the data point illusory correlation conveniently forgets.

---

🎯 Here are some key takeaways

1️⃣ Count the misses, not just the hits: Illusory correlation survives because we remember the times two things happened together and quietly forget the times they didn't. When you catch yourself believing "X always leads to Y," go looking for the counterexamples on purpose. How many times did the Friday deploy go fine? How often did that "difficult" client cooperate? The pattern usually shrinks fast once you account for the cases that didn't make it into your memory because they weren't dramatic enough to stick.

2️⃣ Be suspicious of vivid evidence: The more distinctive a moment is, the more weight your brain gives it, regardless of how representative it actually was. One furious customer, one spectacular outage, one heroic save. These linger because they stand out, not because they're typical. Before you let a memorable event define a person, a tool, or a strategy, ask whether you're reacting to a real trend or just to the thing that was easiest to remember. Distinctive isn't the same as common.

3️⃣ Watch how reputations get built: People who are newer, quieter, or less visible tend to get tagged with their rare mistakes more than people who are constantly around. The distinctive combination of "unfamiliar person" plus "noticeable error" forms a stronger false link than the same error from a familiar face. When you hear a confident claim about how someone "always" does something, check whether it's based on a pattern or on one or two sticky moments that happened to involve the wrong person at the wrong time.

4️⃣ Treat team superstitions as hypotheses, not facts: Every team accumulates beliefs about what causes what. Some are real. Many are illusory correlations dressed up as wisdom. The healthy move is to write the belief down and actually check it. Pull the deploy data. Count the meeting lengths. Look at the launch numbers. If the pattern holds, great, now you know. If it doesn't, you just freed your team from making decisions around a coincidence that everyone treated as a settled truth.

5️⃣ Make "compared to what?" a normal question: The single most useful habit against this bias is asking for the base rate. How often does this happen overall, not just in the cases you remember? When someone says a feature is failing or an approach never works, that's your cue to ask what the full picture looks like, including the unremarkable cases nobody bothered to mention. It feels pedantic in the moment. It's also the difference between responding to reality and responding to whatever was most memorable.

Learn more about Illusory Correlation!

---

---

---

---