Bold, Confident & WRONG: Why You Should Ignore Expert Forecasts
by Adam Butler, Michael Philbrick, Rodrigo Gordillo, ReSolve Asset Management, via GestaltU.com
If you read the paper, watch the news, and listen to investment experts you are doing it all wrong. There are no market wizards; the emperors have no clothes; most people are ‘swimming naked’. The following paragraphs offer abundant and incontrovertible evidence condemning expert judgment for the great sham it really is. We also offer some practical ways to cope with the terrifying reality that no one is in control.
Turn on any media conduit nowadays and you’re likely to find an expert offering some kind of opinion on the future. The problem is that the experts you are most likely to see are least likely to know what they’re talking about. They may know a great deal about their subject matter, but this domain expertise will not translate into better forecasts of future events. You see, no matter how much knowledge or experience these experts have at their disposal, their crystal ball is just as foggy as yours.
Now, the fact that no one can predict the future may seem obvious. You don’t really believe in crystal balls, fortune telling, astronomy, or phrenology, right? But odds are you will tune-in to your favorite media source to hear what their experts have to say. Admit it – when an expert recognized by a respected media source brings to bear a mosaic of knowledge, insight and logic to offer an opinion about what will happen in the future, you pay attention. After all, if they don’t know what will happen in their domain of expertise, who does?
Unfortunately, no one.
That’s right, no one knows. And no amount of knowledge, logic, or insight will change this fact. But don’t take our word for it – take the word of Dr. Philip Tetlock.
In 1985, disillusioned by his experience taking notes at political intelligence committees in the early 1980s, Philip Tetlock set out to discover whether experts could predict future events. Over a span of almost 20 years, he interviewed 284 experts about their level of confidence that a certain outcome would come to pass. Forecasts were solicited across a wide variety of domains, including economics, politics, climate, military strategy, financial markets, legal opinions, and other complex fields with uncertain outcomes. In all, Tetlock accumulated an astounding 28,000 forecasts.
Tetlock was interested specifically in measuring forecast calibration; that is, how experts’ confidence in a particular forecast calibrated with the actual percentage of times that their forecasts came to pass. If experts were well calibrated, when they assigned a 60% probability to forecasts, those forecasts should prove correct about 60% of the time. Unfortunately, when Tetlock measured the actual realized calibration of expert forecasts, individually and in aggregate, he discovered that experts’ confidence in forecast outcomes exhibited virtually no relationship with actual results.
In fact, his results represent an unequivocal condemnation of the global forecasting business:
- On average, experts delivered forecasts, and confidence in their forecasts, that were less well calibrated than one might expect from random guessing.
- Not one expert distinguished him/herself with better-than-random calibration.
- Experts expressed more extreme forecasts, with greater confidence, and were thus less well calibrated in their own field of expertise than when making forecasts outside their own domain.
- Experts who more regularly appeared and were cited in media were less well calibrated than those who labored in obscurity.
- Experts who typically expressed less confident forecasts exhibited higher calibration, but were still worse than random.
- The average of all forecasts were better calibrated than any individual experts.
- Simple algorithms like continuation of trend in the short term, or reversion to the mean in the long term, outperformed all the experts, and delivered better than random forecasts.
In summary, Tetlock discovered that the primary mechanism that most people rely on to make decisions every day – expert judgment – is irrevocably flawed. This has profound implications for decision-making in every dimension of life, but it implies a complete overhaul in how people think about their investments.
No Market Wizards
If you work in the financial industry your livelihood probably depends on ignoring Tetlock’s conclusions, and most analysts, fund managers, strategists etc. will do just that. This is natural – few people can operate for long with such a high level of cognitive dissonance. But as advisors, we have a rare opportunity to look truth in the eye, and deal with it. And the truth is pretty grim.
Consider these sobering facts. CXO Advisory has been tracking and publishing gurus’ forecasts of market direction since 1998. Recently, CXO published a review of all 6,459 forecasts from all of the market ‘gurus’ that they tracked from 1998 – 2012. Specifically, the gurus were graded on their ability to call the direction of the market, but were not penalized for missing the magnitude of the move.
Over 14 years, CXO concluded that the average guru’s accuracy in calling the direction of the market has been about 47%, or slightly worse than a coin toss. The following chart shows how the accuracy of forecasts has stabilized over time around the 47% mark as the sample size expanded over time. In other words, the experts were less reliable than flipping coins.
Chart 1. Cumulative Accuracy of S&P 500 Market Timers
Source: CXO Advisory
The evidence doesn’t end there. The following charts build a formidable case against expert forecasts in every facet of the global investment business. From earnings to interest rates to market outcomes, financial experts universally fail to provide useful guidance about the future.
Chart 2. Consensus forecasts for 10-year Treasury yields versus realized yields, 2000 – 2017.
Source: Bank of America, Thompson Reuters, Datastream, Consensus Economics
Chart 3. Consensus S&P500 aggregate forecast earnings vs. realized earnings, 1985 – 2005
Source: Montier, J. Behavioural Investing (Wiley, 2007)
Do any experts get it right? What about the experts at the Federal Reserve who are in charge of setting interest rates? Can they predict the magnitude or direction of interest rates just six months hence?
A working paper entitled “History of the Forecasters: An Assessment of the Semi-Annual U.S. Treasury Bond Yield Forecast Survey” (Brooks & Gray, 2003) studied the Federal Reserve economists from 1982 – 2002, including Alan Greenspan, to discover whether the group of experts that sets interest rates is able to effectively forecast their trajectory through time.
Chart 4. Mean rate prediction and subsequent realization.
Source: (Brooks & Gray, 2003)
Again we see a strong talent for extrapolating what has just happened, but no talent whatsoever for predicting what will happen next. But it’s difficult to see from Chart 4. just how magnificently wrong these forecasts actually were. When forecast returns are plotted against actual returns 6-months in the future, it’s clear that Fed economists didn’t just miss the magnitude of the change in rates, they also consistently missed the direction of the move.
Chart 5. Actual percentage change in yield compared with forecast percentage change.
Source: (Brooks & Gray, 2003)
Tetlock demonstrated that experts can’t forecast the future across a wide variety of domains. The charts above make the same case against financial experts. That is, financial experts systematically produce forecasts of market direction, bond yields, earnings, and short-term interest rates that are worse than what might be expected from random guesses. Why would anyone pay attention to these experts? Moreover, why would any investor use their forecasts to inform their investment portfolio?
Investing under uncertainty
“That’s what diversification is for. It’s an explicit recognition of ignorance.” – Peter Bernstein
In our business, we embrace uncertainty head-on by adopting systematic strategies founded on the principle that we can’t know the future. That means focusing on diversification. Strategies like the Global Market Portfolio, global risk parity, and diversified risk premia strategies are all rational ways to maximize diversification against an uncertain future.
Investors in the Global Market Portfolio (GMP) hold all liquid global assets in proportion to market capitalization. As such, they express the belief that the optimal asset allocation reflects the average bets of all market participants. On the other hand, investors who allocate to a global risk parity (GRP) strategy express the reasonable belief that assets should produce long-term returns in proportion to their risks. And this is not just good theory; long-term asset returns validate this relationship between risk and return, as shown in Chart 6. Importantly, investors in the GMP and GRP eschew forecasts of future asset class returns altogether.
Chart 6. Long-term asset class returns and risks.
Source: Bridgewater, JP Morgan
Alternative risk premia strategies like Adaptive Asset Allocation capitalize on the market’s ‘willing losers’. A large portion of investors sacrifice wealth to express alternative preferences, such as benchmarking, home bias, or return chasing. These investors leave residual returns on the table for wealth-maximizing investors to harvest through systematic factors like momentum, value and low beta. Again, these approaches require no forecasts about how markets will evolve. Rather, they express the belief that investors will continue to act on non-wealth-maximizing preferences as they have since the dawn of markets.
In contrast to the forecast-free approaches above, most investor portfolios reflect strong forecasts about expected economic outcomes and capital market assumptions. Importantly, these forecasts are mathematically implied in every portfolio whether they were made explicitly or not. For example, if U.S. stocks and bonds mirror their historical risk and correlation in the future, and bonds can be expected to yield 2% over the next 10 years, a 60/40 balanced portfolio implies stock returns of 13.5%[1]. If forecasts at the beginning of the investment process are even mildly off the mark, these portfolios are incredibly inefficient. In other words, they are likely to produce results that deviate profoundly from their original objectives, and underperform less biased portfolios by a substantial margin.
A thoughtful approach to markets starts with deep introspection about how we believe markets work. For advisors, consultants, and CIOs, these beliefs should inform all of our choices about how to allocate our clients’ hard-earned savings. Tetlock’s study involved over 28,000 observations. The results are astonishingly statistically significant and incontrovertible.
Voltaire said that “Doubt is not a pleasant condition, but certainty is absurd.” While it is natural to seek comfort by putting faith in expert judgment, the fact is there is no wizard behind the curtain. As a result, the cornerstone of any successful long-term investment plan is learning how to deal with ambiguity. This means embracing real global diversification in ways you probably haven’t contemplated before, and perhaps introducing alternative sources of return that don’t rely on forecasts. To learn more about the Global Market Portfolio, Global Risk Parity, and Adaptive Asset Allocation, please visit investresolve.com.
[1] Assumes 60/40 is maximum Sharpe ratio portfolio with cash at 0% current yield, and historical covariance between stocks and Treasury bonds.
