The Folly of Peer Group Analysis

This article is a guest contribution by John West, Research Affiliates.

Email this article | Print this article

In August 2000, more than 52 million Americans tuned in to watch the season finale of “Survivor,” the television program where contestants were marooned in faraway locales and pitted against the elements and one another. Indeed the slogan for the show was “Outwit, Outplay, Outlast.” For better or worse, this episode kicked off a 10-year deluge of “reality television” providing the obnoxious and the marginally interesting with their 15 minutes of fame and then some—but it was never truly “reality.”

The investment management industry has its own form of “reality TV.” In this issue we explore the recent impact of the changing composition of peer groups and revisit some of the long festering issues of peer group comparisons. In our view, the manner in which aggregate active manager performance is displayed, vis-à-vis index funds in commonly used peer groups, is not reality.

The S&P 500 In the Bottom Decile of Active Managers… Really?!?

As we noted in the past few issues of Fundamentals, the S&P 500 Capitalization-Weighted Index lost 1% per annum during the “lost decade” of the 2000s. Ugly. But it gets worse on second glance when we consider the alternatives—active managers. Compared to U.S. large-cap core equity managers, the S&P 500 lands in the 90th percentile—that is, the S&P 500 beat only 10 percent of its large-cap core active rivals for the decade!

Our antennas immediately went up. We are firm believers in Jack Bogle’s “cost matters hypothesis,” which states that, as a group, active managers perform the same as the market less costs.1 After all, if indexes track the market, then removing the index funds leaves the self-same portfolio, which is the universe held by all manner of active investors. So in aggregate, active manager performance will equal the market, but take out their expensive cost structures and the average active fund must trail the market.

Clearly, the “reality” of peer group comparisons does not align with our views of how the market should work, so we examined the numbers with a bit more joie de vivre.

Batten Down the Hatches and Close Products!

Faced with its most existential crisis ever, the asset management industry responded like “Survivor” contestants dropped on a tropical island. Basic needs ascended to the top of the priority list and, for managers, that meant resources must be concentrated on revenues and profits. Non-essential strategies became a Gucci handbag in the rainforest… almost worthless. Managers began to kill off their “weakest links.”

In 2009, 13% of all large-cap core managers shut down! This followed a dropout rate of 8% in 2008. These figures are well above the typical 5% annual attrition rate of mutual funds (Fung and Hsieh, 2000).2 All told, of the 393 managers running large-cap money at the beginning of 2006, 116 (or 30%) are now gone!! One-third of the active managers that investors relied on four years ago have gone the way of the dodo bird.

Active manager peer groups—a reflection of this evolving opportunity set—naturally were impacted by this crisis-induced turnover. As Table 1 shows, the group of non-survivors ranked in the bottom third of the peer group before disappearing. If underperforming strategies were the ones that closed down during the recent crisis, doesn’t this artificially push up the median performance of active managers? With the dead weight gone, the S&P 500—or any indexing strategy—looks less compelling against the “average” large-cap core manager.

Digging Deeper into Survivorship Bias

We decided to dig deeper to see if survivorship bias among active managers explains the reality gap. Previous survivorship bias studies have found evidence of survivorship bias, finding an average upward bias in the peer group returns of between 0.10–1.50% per year when portfolios aren’t included in the peer group after they disappear.3 Our analysis found a 0.58% return difference between the median performance of the survivor and survivor-free peer groups for the 10-year period through 2009—well within the range of previous studies. When we eliminate survivorship bias from the peer group analysis, the S&P 500 jumps more than a decile higher and now beats 25% of active institutional managers. Better, but not good enough to explain the reality gap that we see, so we continued to dig.

Most investment strategies are incubated with “seed” capital of a few million dollars to test whether the strategy performs well with live money. If it does, this “new” strategy is added to databases, including its historical track record. Unfortunately, subsequent performance is rarely as good as the initial performance, which leads to another bias in the peer group data. This is known as backfill bias, and studies have found including the backfilled data artificially inflates peer group returns by 1.4% per year.4 To control for backfill bias, we conservatively “netted” down the peer group by 0.50% per year.

A final source of bias relates to fee differentials. Institutions report performance gross of management fees to facilitate apples-to-apples performance comparisons. But, as we outlined earlier, fees matter and are guaranteed to eat into investment returns. Separate account fees for eVestment Alliance’s large-cap core universe range from 70 bps for $1 million to 50 bps for $100 million accounts. Using the middle of the range (60 bps) for active managers and 5 bps for passive management gets us to a 55 bps cost advantage for index funds.

These three adjustments—survivorship bias, backfill bias, and fees—bring the median active peer group 10-year annualized returns down to –0.15%, much closer to the S&P 500 return as seen in Figure 1. And, as believers in the Fundamental Index® concept know, passive investors can achieve even higher returns by breaking the link between index weights and price. In this case, the advantage is substantial!

Implications for Individual Investors

If the past 10 years is representative, institutional investors face prospective coin-toss odds for active large company management versus passive management. In order to win, they must believe that they can overcome three successively higher hurdles:

• Markets are inefficient, therefore there are ways to beat the market, even though the inefficiencies are presumably constantly changing, with some arbitraged away, only to be replaced by new inefficiencies.
• Some managers have the skill to identify these constantly changing inefficiencies in advance, even though their successes must be funded by failing managers’ mistakes.
• The ability to identify these superior managers, in advance, even though successes must be funded by failing investors’ underperforming manager choices.

We must surmount all three hurdles, in order to win. It may sound like we believe the active management game is a fool’s sport. Far from it. But, we do believe it’s a very tough game, which most investors cannot win.

The average individual investor should be so lucky. Without scale to negotiate directly with managers like plan sponsors, mutual fund fees are twice as high. In fact, studies of mutual fund peer groups—which are reported net of all costs—tell a completely different story for the S&P 500 during the 2000s. Even without adjusting for survivorship bias, the S&P 500 ranks in the 60th percentile in the Lipper peer group and in the 50th percentile for the Morningstar peer group. This is 30–40 percentile ranks better than the gross-of-fees institutional peer group! Adjusting for survivorship bias in the mutual fund peer groups would place the S&P 500 squarely better than the average active fund for the 2000s. And, because many individual accounts are taxable, the results would only get worse after we take into account after-tax returns due to active management’s propensity for higher turnover (Arnott, Berkin, and Ye, 2000).5

Timing Matters

This type of analysis may not convince active management advocates. But, if ever there was a time period when the stars were aligned and active management should have done very well relative to passive management, it was the decade of the 2000s. In fact, we termed it the “naughties” because of how poorly cap-weighted indexes performed. Entering the decade at epic valuations, the S&P 500 had a 44 times price-to-earnings multiple—nearly three times its long-term average—in 2000.6 The S&P 500 proceeded to lose half its value from 2000–2002, managed to climb back above its high watermark by 2007—but not net of inflation!—and then proceeded to lose half its value again in 2008–2009. After that rollercoaster ride for the S&P 500 and a nice tailwind from active managers’ small-cap bias, the S&P 500 still managed to beat over half its actively managed mutual fund peers, while the average institutional manager barely beat the S&P 500.7

Conclusion

Peer groups are an important part of evaluating the relative merits of any active management strategy. They provide another layer of context, and when used in combination with appropriate benchmarks and time horizons, effectively allow fiduciaries to make well-informed decisions. But, plan sponsors need to understand how peer group rankings are constructed before they use them to evaluate their active managers relative to indexing alternatives. If you control for the effect of survivorship and backfill bias, and net out management costs, the peer group doesn’t beat the S&P 500 by a whopping margin for 10 years. Furthermore, surviving managers—whether at the upper or lower end of the distribution—likely did significantly better than indicated once we take into account many of the failing products that were part of the opportunity set at the time of hire.

There are times when looking at the returns of passive indexes and active peer groups a priori leads to an assumption that active managers are doing a laudable job against the unmanaged indexes—and today is one of those times. The global financial crisis has led to a significant remake of the active manager opportunity set, but don’t let the ever-shifting sands of survivorship and backfill biased peer group returns fool you. Indexing is a smart bet. Importantly, if you want to be a “survivor,” remember the biases of peer groups because what may look like a smart active manager “alliance” could turn out to be a vote off the island of investment success…caveat emptor!

[AA] Relevant TSX Listed ETFs [AA]

Claymore International Fundamental Index ETF (CIE)
Claymore Japan Fundamental Index ETF (CJP)
Claymore US Fundamental Index ETF - CAD Hedged (CLU)
Claymore US Fundamental Index ETF - Non-Hedged (CLU.B)
Claymore Canadian Fundamental Index ETF (CRQ)

[AA] Relevant Canadian Mutual Funds [AA]

Pro-Financial Asset Management - Pro-Index Funds

Endnotes

1. Credit to John Bogle of Vanguard for the “cost matters hypothesis.”

2. Reporting the work of mutual fund studies before them, William Fung and David Hsieh, 2000, “Performance Characteristics of Hedge Funds and Commodity Funds: Natural vs. Spurious Biases,” Journal of Financial and Quantitative Analysis, vol. 35, no. 3 (September):291–307

3. Mark Grinblatt and Sheridan Titman, 1989, “Mutual Fund Performance: An Analysis of Quarterly Portfolio Holdings,” Journal of Business, vol. 62, no. 3 (July):393–416; Stephen Brown and William Goetzmann, 1995, “Performance Persistence, Journal of Finance, vol. 50, no. 2 (June):679–689; Burton Malkiel, 1995, “Returns from Investing in Equity Mutual Funds 1971 to 1991,” Journal of Finance, vol. 50, no. 2 (June): 549–572; Edwin J. Elton, Martin J. Gruber, and Christopher R. Blake, 1996, “Survivorship Bias and Mutual Fund Performance,” Review of Financial Studies, vol. 9, no. 4 (Winter):1097–1120; Mark Carhart, Jennifer Carpenter, Anthony Lynch, and David Musto, 2002, “Mutual Fund Survivorship,” Review of Financial Studies, vol. 15:(1439–1463); Robert Arnott, 1993, “Is Your Alpha Big Enough to Cover Its Taxes?” Journal of Portfolio Management, vol. 19, no. 3 (Spring):15–25; and Robert Arnott, Andrew Berkin, and Jia Ye, 2000, “How Well Have Taxable Investors Been Served in the 1980s and 1990s?” Journal of Portfolio Management, vol. 26, no. 4 (Summer):84–93.

4. Fung and Hsieh (2000). Backfill bias is not a problem in mutual fund peer groups because the SEC does not allow past hypothetical returns. But in hedge fund and institutional manager databases, it is a real problem that can be difficult to calculate.

5. As shown in Arnott, Berkin, and Ye (2000), on average during the 20-year study period, only 5% to 16% of funds beat the S&P 500 after capital gains and dividend taxes. With Bush’s capital gains and dividend tax cuts set to expire at the end of this year, tax management for investment strategies will be more crucial than ever going forward.

6. Long-term average is 16 times. Based on Shiller’s 10-year normalized price/earnings ratio (www.econ.yale.edu/~shiller/data.htm).

7. The Russell 2000 beat the S&P 500 by 4.5% per year during the 2000s. We found a significant negative relationship between manager excess returns and small-cap premium. That is, when small-caps outperform, so do active managers. This makes sense as during our 20- year period of the study, only between 6 and 24 stocks were larger than the weighted average of the S&P 500. It would be tough for the group of managers to be that concentrated in mega-cap stocks.

Download PDF Version

Copyright 2010 Research Affiliates LLC

www.rallc.com