How to Spot a Bad Forecaster

by Ben Carlson, A Wealth of Common Sense

“One might wonder if forecasters eventually get bored with being utterly wrong and would like to give up guessing the future.” – James Montier

In The Little Book of Behavioral Investing, James Montier shares some research from Philip Tetlock, the foremost expert on, well, experts. Tetlock has performed exhaustive studies on the forecasting abilities of experts in a wide variety of fields from politics to economics to the financial markets.

His studies show that even the foremost experts in most fields are no better than a flip of the coin at predicting what’s going to happen in their particular area of expertise. In fact, in one study those with 80% confidence in their forecasts were only right 45% of the time, so even worse than a coin flip.

At this point most people know that forecasting the future of the markets is extremely difficult, but what bothers me about most soothsayers is the fact that the majority of them never admit when they’re wrong, even when there’s clear evidence in place that proves it. Tetlock discovered that there are five excuses that are used by these so-called experts on a consistent basis (with commentary from Montier):

1. The If Only Defense: “If only the Fed had raised rates, then it would have been true.” Effectively, the experts claim that they would have been correct if only their advice would have been followed.

2. Ceteris Paribus: Something outside of the model of analysis occurred; therefore it isn’t my fault.

3. I Was Almost Right: “Although the predicted outcome didn’t occur, it almost did.”

4. It Just Hasn’t Happened Yet: “I wasn’t wrong, it just hasn’t occurred yet.” This is my personal favorite.

5. The Single Prediction: “You can’t judge me by the performance of a single forecast.”

Like most level-headed investors, I find little use in basing an entire investment process around my ability to predict the future. The folly of forecasting is becoming more mainstream as we’ve seen over the past few years that no one really has any clue what’s going to happen next. The oil crash, lack of rising rates and continued stock market strength are all cases in point.

But while I think it’s good advice to ignore the noise, it’s basically impossible these days since information is so abundant. The dirty little secret is that although everyone says you should ignore forecasts, they all still look. We can’t help peaking to see what the forecasters think is going to happen, even though everyone knows most will be wrong. So if you’re going to read a bunch of 2015 market forecasts, you have to at least know how to spot the terrible ones from the not-so-terrible ones.

Tetlock got the ball rolling for me by pointing out that the worst forecasters consistently make up excuses when they’re wrong. It doesn’t even bother me that much when so many market predictions turn out to be wrong. It’s that so many make excuses after the fact and aren’t honest about it. I wish more people would just be honest about the fact that predicting the future is hard. If you want to spot a bad forecaster, someone that consistently blames forces outside of their control for being wrong is a good place to start.

Here’s some more negative knowledge if you want to act like a poor forecaster at all times:

• Argue your case with anyone and everyone that will hear you out.
• Use words like never, always and guaranteed.
• Make sure to have an opinion on how to fix everything — the Fed, tax policies, fiscal & monetary policies, the Royal Family, the season finale of Homeland, the NBA draft lottery system, etc.
• Have a complete lack of self-awareness.
• Refuse to see both sides of any argument. Your’s is the only one that matters.
• Make outlandish predictions that have an extremely small chance of occurring. If by chance it does happen, take credit. If not, no worries, it was just a guess.
• Use the term black swan as much as possible.
• Constantly predict market crashes or melt-ups. These things happen so you’re bound to be right once and a while.
• Act with complete and utter certainty at all times. Why concern yourself with probabilistic outcomes when you know exactly how things will work out. Assume you’re going to be right no matter what.
• Be as precise as possible. Take you estimates out to at least 2 decimal points. Ranges are for weaklings.
• Don’t change your mind when new facts come to light.
• Use the phrase “we’ve seen this movie before and it ends badly.”
• Never say “I don’t know.”

Source:
The Little Book of Behavioral Investing

Now here’s the best of what I’ve been reading in between holiday meals and opening presents this week:

• It’s hard to fire yourself, but easier to blame someone else (Alpha Architect)
• “Assume fraud until genius is proven” (Simon Lack)
• Good investment advice doesn’t need to be updated all that often (Abnormal Returns)
• Paul Graham: “When experts are wrong, it’s often because they’re experts on an earlier version of the world.” (Paul Graham)
• A dozen business and life lessons from comedians (25iq)
• Debunking the “thought leadership” narrative (Tom Brakke)
• What is this intrinsic value you speak of? (Philosophical Economics)
• 2015’s Twitter must-follows, including my underrated picks (MarketWatch)
• Strategies can beat the market but you can’t (Irrelevant Investor)
• The New Year is a perfect time to get back to the basics of investing (Washington Post)
• 3 lessons learned in 2014 from Cullen Roche (Prag Cap) and James Osbourne (Bason)
• The Jimmy Fallon episode with Seinfeld was my favorite of the season (Comedians in Cars Getting Coffee)

Subscribe to receive email updates and my monthly newsletter by clicking here.

Follow me on Twitter: @awealthofcs

Copyright © Ben Carlson, A Wealth of Common Sense