Akshay DineshSuccess leaves records. The company that lasts for decades gives interviews, the fund that remains open still has a performance page, and the aircraft that returns from a mission can be inspected. Failures are more likely to vanish. When we judge a process only from the cases still visible, the evidence can look far more encouraging than it really is. That distortion is called survivorship bias.
The word “survivor” does not have to mean a person who stayed alive. It can be any case that passed through a filter: a business that stayed open, a student who completed a course, a product that reached the market, or a study participant who remained until the final visit. The danger begins when the surviving group is treated as though it represents everyone who started.
What survivorship bias leaves out
Suppose someone studies the habits of successful entrepreneurs by interviewing founders of thriving companies. The interviews may reveal real patterns, such as persistence, careful hiring, or close attention to customers. Yet founders whose companies closed may have practiced the same habits. Without comparing both groups, it is impossible to know whether a celebrated habit caused success, merely accompanied it, or was common among failures too.
This is a form of selection bias. The selection rule is survival through some process, and that rule changes who remains available to observe. Ordinary sampling bias often begins when a sample is gathered, such as a phone survey that misses people who rarely answer calls. Survivorship bias can develop later, after a broader group has already entered the picture. Cases disappear through bankruptcy, withdrawal, breakdown, death, graduation requirements, or another outcome related to the question being studied.
That last point matters. Missing cases do not always create a serious distortion. If records vanish randomly, the remaining group may still resemble the original one. Bias becomes likely when disappearance is connected to performance, risk, health, cost, or some other variable being measured. A database of currently operating restaurants, for example, will systematically omit many restaurants whose decisions ended badly. The missingness carries information.
The bomber problem and the planes that did not return
One of the clearest historical examples comes from Abraham Wald, a statistician working with Columbia University’s Statistical Research Group during World War II. Military analysts recorded bullet damage on aircraft that returned from combat. A tempting response was to add armor where those surviving planes showed the most holes. Wald’s 1943 work on aircraft vulnerability asked a sharper question: what could be learned from the aircraft that were absent from the inspection?
A plane with damage to a wing or another less critical area might still return, which made holes in those places easy to count. Aircraft struck in vital areas were less likely to come back at all. Their damage patterns were missing from the hangar because the selection process had removed them. The heavily marked regions on returning planes could therefore indicate where an aircraft was able to absorb hits, while relatively unmarked critical regions could be the places most worth protecting.
The familiar retelling is sometimes compressed into a neat story about simply armoring every blank spot. Wald’s actual memoranda used a more careful probability model. He considered the distribution of hits, the chance that damage in different regions would destroy an aircraft, and the fact that only survivors were observed. The lasting insight is not a wartime trick. It is a way of reading absence as evidence.
How visible winners distort everyday evidence
Survivorship bias appears whenever visibility depends on success. Old buildings can seem better made than modern ones because the sturdy or valued examples remain, while ordinary structures that decayed or were demolished are no longer part of the comparison. A playlist of admired music from decades ago can make an earlier era seem unusually brilliant because forgettable releases have fallen out of circulation. The surviving examples are real; the mistake is assuming they are typical.
Investment data provide a measurable case. A list of mutual funds operating today excludes funds that closed or merged, often after weak performance. If an analyst calculates past returns using only current funds, the historical record can be tilted upward. In a 1997 Journal of Finance study, Mark Carhart built a large database designed to include funds that had disappeared. His survivor-free analysis found that costs and common market factors explained most apparent persistence in fund performance, while the strongest reliable pattern was continued underperformance among poor performers.
Health research faces a related problem. A study of risk factors among older adults can include only people who survived long enough to enroll. If an exposure also affects early mortality, the older participants with that exposure may be an unusually resilient subset. Researchers call this differential survival, and they must account for it before generalizing from the observed group to everyone who had the exposure.
Even school data can be affected. Imagine comparing final exam scores among students who completed an optional advanced course. Those scores do not describe everyone who enrolled if struggling students were more likely to withdraw. The completion requirement has filtered the group. A claim that the course works well for all starters needs enrollment, withdrawal, and completion data, not only the grades of students who remained.
A small example shows how the bias grows
Consider a hypothetical directory containing 100 new investment funds. After ten years, 60 remain open and their average annual return over the period is 7 percent. The other 40 have closed. If the directory deletes a closed fund’s history, a researcher looking backward sees only the 60 survivors and may report 7 percent as the typical experience.
Now suppose the closed funds averaged 1 percent before disappearing. Treating all 100 original funds equally gives a combined average of 4.6 percent: (60 x 7 + 40 x 1) / 100. The survivor-only result is not wrong about the funds that lasted. It answers a narrower question than readers may realize. It describes survivors rather than the original opportunity set.
Real analyses are more complicated because funds open and close at different times, returns compound, and mergers may preserve only part of a record. Still, the arithmetic captures the central problem. Removing weak cases can improve the average without improving any individual case. The dataset changes, so the story changes.
How to look for the evidence that disappeared
A useful defense begins with one question: What had to happen for this case to become visible to me? A founder in a business magazine had to build a company that attracted attention. A product review may cover only items that reached stores. A clinical study may require participants to survive, consent, and return for follow-up. Naming the filter makes the missing group easier to imagine.
Then identify the original starting group, sometimes called the cohort or denominator. For a graduation rate, begin with everyone who entered. For a product’s reliability, include units that failed, were returned, or were replaced. For investment performance, use a database that retains dead or merged funds. The goal is not to distrust every success story, but to match the evidence to the claim.
Several practical checks help:
- Ask whether failures, withdrawals, closures, and dropouts remain in the records.
- Compare the traits of cases that stayed with those that disappeared.
- Check whether the time window begins before or after the selection process.
- Look for results based on an original cohort rather than a current snapshot.
- Treat advice from winners as a source of ideas, not proof that their habits caused the outcome.
Survivorship bias is persuasive because the visible evidence is often vivid and accurate. The returning planes really did have bullet holes. The successful founders really did work long hours. The surviving funds really did post their returns. Better reasoning begins by noticing that a true description of the winners may still be a misleading description of the whole field. Sometimes the most important part of a dataset is the part that is no longer there.
