Survivorship Bias and Other Tricks of the Trade

“It’s easy to lie with statistics, but it’s hard to tell the truth without them.”
― Charles Wheelan, Naked Statistics: Stripping the Dread from the Data

“Past performance is not a guarantee of future results.”

How many times have you read this disclaimer when making a decision to invest in a mutual fund? In theory, it makes perfect sense: A fund that performed well in the past might not perform the same way after you buy it. And yet, when people research which mutual funds to buy, they often still make decisions based on how well the fund has performed historically.

In this week’s blog, I want to talk about something called “survivorship bias.” Survivorship bias occurs when mutual fund companies drop poor-performing funds, or merge the assets into higher performing funds, hiding the poor-performance statistics and boosting the profile of funds that happened to outperform their benchmarks. Investment companies know that investors tend to prioritize their investment choices by concentrating on past performance. By eliminating the under-performers, mutual fund companies present, like the Siren’s irresistible call, the funds that out-performed their peers, appealing to investors’ more base instincts.

Let’s take an example: Acme Mutual Fund Company starts with 100 mutual funds. But within five years, 25 of those funds had performed poorly and Acme closed them, or merged them with the other more successful funds. The 75 funds that remain – the “survivors” – were the statistical out-performers. They appear to have a much better track record, and Acme may now boast that its funds “outperform their peer groups over the past five years.” Wouldn’t you find it hard to resist buying a fund that outperformed its peer group over the past five years? Seems reasonable to expect that if it did so over the past five years that it might continue to do so in the future, right? Wrong!

One of the reasons we turn to evidence-based investing is to shepherd us past the misguided strategies that can otherwise cause an investor’s expected returns to be devoured by that big bad wolf of Wall Street. How do we determine which of it comes from sound science and which may steer you wrong?

Survivorship bias is just one trick of the trade we need to watch for when accepting or rejecting a performance analysis. Let’s examine this a bit further.

How often do funds disappear?

In the competitive capital markets in which we operate, fund managers launch new products and discontinue existing ones all the time. Individual funds probably disappear far more frequently than you might think.

  • A recent S&P Dow Jones Indices analysis found that, for the five-year period ending in December 2015, “nearly 23 percent of domestic equity funds, 22 percent of global/international equity funds, and 17 percent of fixed income funds have been merged or liquidated.”
  • As might be expected, the longer the timeframe, the higher the death rate. A January 2013 Vanguard analysis of survivorship bias looked at a 15-year, 1997–2011 sample of funds identified by Morningstar. The analysis found that 46 percent “were either liquidated or merged, in some cases more than once.”
  • A May 2015 Pensions & Investments (P&I) article reported that Exchange-Traded Products (including ETFs) weren’t immune from the phenomenon either, having just reached the milestone of 500 products closed. According to the “ETF Deathwatch” cited source, this represented a mortality rate of just under 23 percent.
  • The same P&I piece cited Dimensional Fund Advisors and Vanguard analyses that estimated 15-year mortality rates for traditional U.S. mutual funds in the range of a 50/50 coin flip, or worse.
  • A November 2015 article by financial columnist Scott Burns found similar survival rates for the 15-year period ending in 2014. “At the beginning of the period, there were 2,711 funds,” he reported. “At the end of the period, there were 1,139. Only 42 percent of the starting funds had survived.”

Why Does Survivorship Bias Matter?

Why should you care about the returns of funds that no longer exist?

The funds that disappear from view are usually the ones that have underperformed their peers. The aforementioned Vanguard analysis found that, whether a fund was liquidated or merged out of existence, underperformance was the common denominator prior to closure.

If these disregarded data points were athletes on a professional sports team, they’d be the ones bringing down their team’s averages. Imagine if a professional football team could throw out the games lost during the season.

Survivorship bias is an act of revisionist history. A statistical lie.

Better Investment Experience

An analysis marred by survivorship bias is highly likely to report overly optimistic outcomes for the group being considered. In essence, it appears that the group of survivors were “better” funds, and the statistical trickery causes investors to believe that these funds will continue to be better in the future. While a degree of optimism can be admirable in many walks of life, basing your investment decisions on this statistical trickery is more likely to set you up for future disappointment than to position you for realistic, long-term success. This is the rub. If you follow those survivors for another five years, and learn the truth that statistics presents, you find that the pool of survivors continues to shrink, with the winners eventually becoming losers even while new fund “winners” are added to the mix.

Survivorship bias is only one of a number of faults that can weaken seemingly otherwise solid investment strategy.

One way in which we strive to add value to investors’ evidence-based investment experience is to help them separate robust data analysis from misleading data trickery.

AMDG Financial is proud to work with Dimensional Fund Advisors (DFA) – a company with a science-based approach to investing. If you’d like to learn more about survivorship bias, DFA offers two terrific resources in the form of a white paper.

And, as always, if we can assist you with your strategies and investment selections, please give us a call or fill out our contact form. We’d love to help you cut through the clutter of statistically misleading claims and help you achieve your financial goals.

Click here to view previous news releases from AMDG Financial.

Subscribe to our blog!