by Olaf Storbeck, Economics Intelligence
A striking new paper by two economists with Brandeis University and Oxford fundamentally questions the informational content of credit ratings. How “incomprehensible and irrational” are the assessments of Standard & Poor’s, Moody’s and Fitch?
Try your luck at the big rating gamble. (c) Klaus Meinhardt, 2012
Christian Noyer is usually a calm and sober-minded central banker. Last December, however, the president of the Banque de France was unusually outspoken and indulged himself in a vitriolic rant against credit rating agencies. According to Noyer, their assessments had become nothing less than “incomprehensible and irrational”. Some decisions “do not seem to be justified on the basis of economic fundamentals,” Noyer complained.
A new study by two economists now provides empirical evidence for this drastic view. Jens Hilscher (Brandeis University International Business School) and Mungo Wilson( Saïd Business School, University of Oxford) come to the conclusion that credit ratings do not contain much information about actual default probabilities. “The informational value of credit ratings is surprisingly low,” says Hilscher in summarising the key findings of the paper entitled “Credit Ratings and Credit Risk.”
More often than not a rather simple analysis of publicly available data easily beats the assessments of the credit rating agencies and gives a much better picture about the actual default risks, the researchers observe. “The judgements of the credit rating agencies are significantly over-rated ,“ concludes Hilscher.
These results are directly challenging the pivotal role credit agencies currently play on world financial markets. The agencies largely enjoy their dominant position thanks to the decisions of financial regulators. For instance, a lot of investment funds are only allowed to hold assets with a certain minimum rating. The equity buffers that banks have to put aside to protect themselves against potential losses in the current regulatory framework also hinge on the assessment of S&P, Moody’s and Fitch.
Since the financial crisis broke out, these privileges have been under attack. Critical researchers take the new paper by Hilscher and Wilson as a confirmation of their views. “My personal view is that credit ratings have many problems, and should not be used for regulatory policy,“ says Robert Jarrow, Professor of Investment Management at the Johnson Graduate School of Management, Cornell University. Jarrow feels vindicated by the Hilscher/Wilson paper. „I think the paper’s results are correct and should be taken very seriously.“
John Campbell, professor of economics at Harvard, also has a high opinion about the paper: “I do think Jens and Mungo’s work is convincing and methodologically sound.”
At the core of the paper are corporate bond ratings by Standard & Poor’s. Hilscher and Wilson look at all assessments that have been given between 1986 and 2008. Additionally, they constructed an alternative indicator that is meant to gauge the default risk of the bond issuers. The economists only use publicly available information for this “failure score”, mainly balance sheet data like profitability, leverage and cash holdings.
Hilscher and Wilson compare the S&P ratings and their own failure score with actual defaults and come to a straightforward conclusion: Their simple indicator delivers much better predictions about expected defaults than the verdicts of the rating agencies. According to the paper, the failure score is almost twice as reliable:
“We find that this measure (…) is substantially more accurate than rating at predicting failure at horizons of 1 to 10 years. The higher accuracy in predicting the cumulative failure probability is driven by a much higher ability of failure score at predicting marginal default probabilities at horizons of up to 2 years and the fact that credit rating adds little information to marginal default prediction at horizons up to 5 years.”
This leads Hilscher and Wilson to a straightforward conlusion that is rather inconvenient for the suppliers and users of credit ratings: “ratings are in fact a poor predictor of corporate failure.”
As Jens Hilscher explains, those findings probably also apply to sovereign bonds. As a piece of evidence, Hilscher refers to a paper co-authored with Yves Nosbusch in 2010. Both economists analysed the bond spreads of emerging market sovereigns and show that publicly available data about the macroeconomic situation of the borrowers can explain them much better than the verdicts of rating agencies.
The rating agencies rebuff the findings of Hilscher and Wilson. “The research is criticising ratings for missing goals that we do not try to achieve,” asserts Albert Metz, Managing Director Credit Policy Research with Moody’s:
“The objective of our ratings is not to predict absolute default probabilities of individual firms. Such ratings would be more volatile and highly cyclical. The purpose of Moody’s ratings is to produce a stable indicator of relative credit risk.”
At a given point in time, two companies that have a similar rating should have a similar expected loss profile. “That’s our aspiration,” says Metz.
However, according to the paper by Hilscher and Wilson, even this goal is frequently missed: According to their findings, the ratings for different firms prove to be quite inconsistent at a given point in time:
“Firms with the same rating often have widely different default probabilities and firms with very different ratings often have very similar default probabilities. (…) In fact, the 75th percentile AA-rated issuer (two notches below AAA) is more likely to default than the median issuer rated BBB-, the last rating before reaching junk status.”
S&P fundamentally questions the design and the approach of the paper. “If the intention of this study is to assess the track record of credit ratings, then its methodology is flawed and many of the assumptions it makes about the meaning and basis of ratings are wrong”, says Martin Winn, Vice President Communication with S&P.
Winn asserts that a close correlation between ratings and defaults in fact exits. As an example, he refers to the track record of the company since 1980. Only around 1% of corporates that were rated investment grade (BBB- or above) defaulted within 5 years, compared with around 20% of companies rated speculative grade (BB+ or below).
Hilscher and Wilson reply that they do not claim that ratings have no informational content whatsoever. Having a S&P rating was certainly better than no information at all, stresses Hilscher.
However, according to their findings ratings only prove meaningful with regard to the question how a company will fare in bad economic conditions. Higher rated firms are more likely to survive an economic downturn, the researchers observe: “Ratings reflect sensitivity of credit risk to bad times and, therefore, (..) rating may at least partly capture systematic risk of default.”
However, this does not change the basic findings that a simple failure score based on publicly available information easily beats the ratings when it comes to the assessment of absolute and relative default risks of borrowers. The key question according to Hilscher is: “Why are their measures not more accurate if more accurate information is easily available?”
Olaf Storbeck is the International Economics Correspondent with Handelsblatt, Germany's business daily. Based in London, he is writing about current economic research.
Olaf wrote the award-winning book "Economics 2.0 - What the best minds in Economics can teach you about Business and Life" (jointly with Norbert Häring).
