And Bad Mistakes, I've Made a Few
When I first heard about the Reinhart/Rogoff Excel fiasco my first thoughts were not, "How might they have changed the course of European history?" They probably should have been, but they weren't. My first thoughts were, "I'd better go back and check my equations on that model of the unemployment rate so I don't get made fun of as well." It's healthy for smart people to have their thoughts challenged, because in the end it makes us trust science, and math, and statistics. (Just kidding, no one trusts statistics.) Today we'll take a look at a report that was in the news for all the wrong reasons. Twice.In 2010, Carmen Reinhart and Kenneth Rogoff (aka RR, both of Harvard) wrote a paper (non-peer reviewed) in which they looked at historical data for developed countries' GDP and National Debt, and determined that:
A Debt/GDP ratio greater than 90% was associated with Negative GDP Growth.
(I don't know if I need to mention it but negative GDP growth is bad. Two or more quarters of it is called a recession.) This finding was important because in 2010 the US was nearing that magical 90% mark, and several European countries were already well over it. The paper got lots of media coverage and was cited everywhere from NPR's Planet Money to the Paul Ryan Budget. The US had already set it's stimulus in motion, but the EU used the study to justify linking bailout money to austerity measures with the ultimate goal of bringing down European debt/GDP ratios. The implication (sound byte) of the paper was that this 90% cutoff marked an unstable tipping point after which borrowing became difficult, resulting in inflation that decreased real GDP growth.
Economic Warfare
Flash forward to last month when Thomas Herndon, Michael Ash, and Robert Pollin (aka HAP, U Mass) came out with a paper refuting RR on three different points. The most hilarious of these, and the one which made it on to twitter and the Colbert Report is the typo in the Excel spreadsheet that removed a chunk of data from an "=AVERAGE( )" that conveniently contradicted their theory. This week RR released a correction to account for their typo. This stuff happens. As a researcher it is your worst nightmare that something you do by accident could invalidate your results. I am not going to model their typos… it offends my aesthetic. (Just kidding, I’m not doing it because it’s been done before so it feels like cheating.)
Instead, let’s forget the typos (I know, how is that possible?) and focus on the methodology critiques and the questions they were trying to answer. The first methodology critique is quite small; they didn’t include all their data. They claim the data wasn’t available when they did their analysis (which is reasonable) and with over a thousand data points you’d think missing a handful wouldn’t matter. The problem is that while there are a ton of data overall, there are only about a hundred points where a country went above 90% debt/GDP for the year, so when the few you miss are “high debt – high growth” ones, it will skew your results. We now have all that data (provided by HAP).
Less forgivable is their “irregular” way of taking averages. RR had many years of data for some countries ("many-year-countries"), and only one year of data for others ("one-year-countries"). How would you tackle this problem? You could:
If you chose anything but D, congratulations, you can justify your findings and submit your paper for peer review.
As you might guess, RR chose D.
In the HAP rebuttal, they use method B, and find that instead of a 0.1% decrease in GDP, countries with >90% debt actually had 2.2% growth. Seeing as we can't trust anyone else's Excel spreadsheets any more, lets make our own. If you download their data, you get 5 columns:
(Follow my work.)
A = Country
B = Year
C = Debt/GDP ratio (%)
D = Category (< 30, 30-60, 60-90, > 90%)
E = Change in Real GDP (%)
Lets not bother ourselves with "in what country" or "when" our data happened, and lets not prejudice our findings with the RR/HAP categories. Delete columns A, B, and D (Shift cells left so you have two columns A and B). We are going simple here. We just want to know if the GDP growth for the "> 90" is different than the "< 90", and is it a negative or positive GDP growth. Make four new columns to separate the data (typing the [In Brackets] part):
This has actual policy implications if you are a country right at that 90% debt/GDP cusp (like we were), and are hoping to stimulate your economy (like we did).
1. Compare to all countries: What happens at the 90% debt/GDP ratio? Is there no effect? Does the curve flatten out? Is it an inflection point at which we hit a death spiral of ever-decreasing GDP growth?
Right around 1973 we bottomed out our debt/GDP ratio, while the Aussies kept working to lower theirs. All other things being equal, if our debt choices had a significant effect on GDP growth you would expect Australia to be clobbering us. Instead, plotting our % GDP growth gives us this:
The first thing that jumps out at me is how much GDP growth jumps around. In the 40s it goes from dangerously high (> 15%) to depressingly low (< -10%) in just a few years. Also, you can see that the worst time for perpetual recessions in the late 1800s and early 1900s, when debt levels were actually quite low. Lets do a simple 10 year moving average for GDP growth (like I've done before for U3) and compare that:
Once again, I'm not seeing the correlation based on our own history.
By the way, I know I didn't include too much modeling in this post. I'm working on an interactive model for next week, though, where we'll try to answer the question, "If RR or HAP are right, can we still grow our way out of debt?" Stay tuned.
-----
My humor may be low, but I'm trying to keep the quality of my posts high. That means updating only once a week, or whenever I get a really good idea. This, in combination with the imminent demise of google reader, has led people to ask if I could email them when the blog is updated. A compromise that I think will work well is a subscription to this google group (an email list). Send an email to that link (overly-complicated-excel+subscribe@googlegroups.com) and you will be added to the list. The blog will email that list whenever it posts are published. If you want your money back you can always unsubscribe the same way.
Instead, let’s forget the typos (I know, how is that possible?) and focus on the methodology critiques and the questions they were trying to answer. The first methodology critique is quite small; they didn’t include all their data. They claim the data wasn’t available when they did their analysis (which is reasonable) and with over a thousand data points you’d think missing a handful wouldn’t matter. The problem is that while there are a ton of data overall, there are only about a hundred points where a country went above 90% debt/GDP for the year, so when the few you miss are “high debt – high growth” ones, it will skew your results. We now have all that data (provided by HAP).
Less forgivable is their “irregular” way of taking averages. RR had many years of data for some countries ("many-year-countries"), and only one year of data for others ("one-year-countries"). How would you tackle this problem? You could:
A. Ignore the “one-year-countries”, or treat them as a separate group. (Too lazy?)
B. Treat each year from each country as a separate data point. (Too simplified?)
C. Assume the “many-year-countries” have phases of high debt and low debt. Determine when these phases are and treat each phase as a data point to plot with the "one-year-countries". (Did I just write your thesis premis for you?)
D. Average all your years of data for each "many-year-country" into one data point. Weight each country equally and take an average these data points within your arbitrary categories. (Take a nap. Have a nightmare about averaging your average of averages.)
If you chose anything but D, congratulations, you can justify your findings and submit your paper for peer review.
As you might guess, RR chose D.
In the HAP rebuttal, they use method B, and find that instead of a 0.1% decrease in GDP, countries with >90% debt actually had 2.2% growth. Seeing as we can't trust anyone else's Excel spreadsheets any more, lets make our own. If you download their data, you get 5 columns:
(Follow my work.)
A = Country
B = Year
C = Debt/GDP ratio (%)
D = Category (< 30, 30-60, 60-90, > 90%)
E = Change in Real GDP (%)
Lets not bother ourselves with "in what country" or "when" our data happened, and lets not prejudice our findings with the RR/HAP categories. Delete columns A, B, and D (Shift cells left so you have two columns A and B). We are going simple here. We just want to know if the GDP growth for the "> 90" is different than the "< 90", and is it a negative or positive GDP growth. Make four new columns to separate the data (typing the [In Brackets] part):
Debt < 90: (Cell C2) [=IF(A2<90,A2," ")]
GDP Growth for Debt < 90: (Cell D2) [=IF(A2<90,B2," ")]
Debt >= 90: (Cell E2) [=IF(A2<90," ",A2)]
GDP Growth for Debt <=90: (Cell F2) [=IF(A2<90," ",B2)]
Complete the columns. Find the averages [=AVERAGE(D2:D1276)], [=AVERAGE(F2:F1276)] and you can see that indeed there is a lower growth for the >90% debt years (2.15%) than the <90% debt years (3.66%), but also the standard deviation [=STDEV(D2:D1276)], [=STDEV(F2:F1276)] for both (4.3 for low debt, 3.5 for high debt) is huge! On average, low debt is associated with better growth, but in no way does it ensure prosperity. On the flip side, 2.15% growth for the high debt countries is not fantastic (it barely keeps up with population growth), but it won't feel like a recession (-0.1%) that RR predict. It feels like... well, now.
Asking the Wrong Questions
Conceptually, the bigger problem I have is that the answer RR found (even if it was wrong) was to a meaningless question. Are there countries that suddenly jump from < 90% debt to > 90% debt? Certainly! But if a country does pass that threshold, I'm sure there is a year that they are at 83% or 88% and a following year they are at 92%, so the interesting question is, "What happens to these countries?"This has actual policy implications if you are a country right at that 90% debt/GDP cusp (like we were), and are hoping to stimulate your economy (like we did).
Stimulating Discussion
The idea behind stimulus is that a government can borrow money (adding to debt) and spend it right away on improving the economy (adding it to GDP). Because it is affecting both the numerator and the denominator of debt/GDP it has a dampened effect on the ratio. The bonus is that all that money you paid this year to build bridges, give tax refunds, or even landscape the national mall goes to people who can spend it again, increasing GDP next year. The hope is that you can "grow your way out of debt" with the debt/GDP denominator gradually increasing and the debt becoming less burdensome as inflation eats away at it. (In fact, over the last few years the US government is borrowing at a Negative inflation adjusted interest rate. People are paying the government for the ability to lend them money!) But will it work? If GDP growth is slowed enough by our debt load, you might lose out on all that you have gained from the stimulus. That's what we should be testing. In addition to the >90% vs <90% average growth, there are three things I'd like to know if I were, say, the US government trying to get out of a recession.1. Compare to all countries: What happens at the 90% debt/GDP ratio? Is there no effect? Does the curve flatten out? Is it an inflection point at which we hit a death spiral of ever-decreasing GDP growth?
2. Compare to a case study: Is there a country that has made a similar set of debt choices as the US, but after a certain point made different ones? Did this affect their GDP?
3. Compare to our own history: When have our recessions come, and might it have been our debt load that caused them?
Aaaaand, go.
Thinking Linearly
I absolutely know that the best fit will not be linear, but as this is Overly Complicated Excel (and not PRISM or GraphPad) we'll go the simple route. We'll take our data from before, copy cells C2:F1276 and paste them "special: values" into a new sheet. Sort reverse-alphabetically by column C and we now have continuous data for plotting. Plot it!
Not great R-squared's. Its almost like GDP growth is caused by other factors than debt! Anyways, the slope of the < 90% debt decreases faster than that of the > 90% debt. This means that while being at 90% is bad, going up to 95% is not that much worse. HAP find similar results with their fancy plotting program:
A Case Study from Down Under
We share a lot with our Australian friends: language, cultural values, and up until the mid 70s, our debt policy. You can see this by downloading HAP's data again and plotting just the US and Australia debt/GDP ratios:Right around 1973 we bottomed out our debt/GDP ratio, while the Aussies kept working to lower theirs. All other things being equal, if our debt choices had a significant effect on GDP growth you would expect Australia to be clobbering us. Instead, plotting our % GDP growth gives us this:
From the mid 70s on, we had more debt but roughly the same GDP growth. Now, I'd be the first to point out that a case study is not proof, but this is one more piece of evidence that there must be other factors influencing GDP growth that play a much larger role.
All About US
Historically, the United States has had periods with no debt, and periods with huge debt. We also have had many recessions. If the correlation held true, we'd expect that those recessions might have come during our periods of high debt. Lets see if that is the case. RR/HAP actually had a lot of data for the US, so lets use all of it to go back in time and look at our historical debt/GDP ratio and our % GDP growth:
Once again, I'm not seeing the correlation based on our own history.
Conclusions
Overall it looks like there is some correlation between debt load and GDP growth, but it isn't strong, and it certainly isn't evident the US-based examples I've shown. Maybe we're different, though. We have a very strong and trusted currency, and perhaps investors overlook our debt because they don't envision a world in which the US actually stops paying its debts. Even for other countries though, the repercussions (a 1% drop in our GDP growth) of a transiently high debt load might not be that bad. Thats not to say that countries should load up on debt without careful consideration. I certainly want to see the US debt edge back down over time. But if you are trying to jump-start the economy with a debt fueled stimulus, a few years of > 90% debt/GDP might be an ok stepping stone.By the way, I know I didn't include too much modeling in this post. I'm working on an interactive model for next week, though, where we'll try to answer the question, "If RR or HAP are right, can we still grow our way out of debt?" Stay tuned.
-----
My humor may be low, but I'm trying to keep the quality of my posts high. That means updating only once a week, or whenever I get a really good idea. This, in combination with the imminent demise of google reader, has led people to ask if I could email them when the blog is updated. A compromise that I think will work well is a subscription to this google group (an email list). Send an email to that link (overly-complicated-excel+subscribe@googlegroups.com) and you will be added to the list. The blog will email that list whenever it posts are published. If you want your money back you can always unsubscribe the same way.
No comments:
Post a Comment