Earlier this week Ezra Klein’s Wonkblog in the Washington Post published an article titled “The Romney campaign says stimulus doesn’t work. Here are the studies they left out.” The article criticizes a Romney campaign paper by Glenn Hubbard, Greg Mankiw, John Taylor and Kevin Hassett for its selective references of empirical studies of the Obama Administration’s American Recovery and Reinvestment Act (ARRA). The Wonkblog article, posted by Dylan Matthews, provides a list of 15 papers (most of which have not yet been peer reviewed) and claims that 12 of them find that the ARRA “worked.”
I decided to read a few of these studies to see how much empirical support there was, in the academic community, for the idea that the ARRA was effective. I read four of the papers, and so far, I am underwhelmed by the empirical evidence.
I read three of the papers listed by Wonkblog as providing evidence that the Obama stimulus worked. The first, by James Feyrer and Bruce Sacerdote estimated the impact of the ARRA on jobs at the state and local level. They find that the ARRA had some impact on jobs but estimated that the cost of each job created was between $170,000 and $400,000. In addition, they find that different types of spending have different effects on jobs. For example they report that spending “to fund teachers and police have if anything a negative impact.” In general, their results are supportive of the study by John Cogan and John Taylor that found that state and local governments used Federal stimulus funding to reduce borrowing rather than increase hiring.
Feyrer and Sacerdote cleverly control for the statistical problem that stimulus spending is generally larger in places where the economy is worse through a technique called “instrumental variables.” Their study validates what many ARRA critics have asserted: the ARRA was in large part pork barrel spending. They find that states where members of Congress have longer tenure in Washington received more stimulus spending. Their study uses this variation in pork barrel spending to estimate the effectiveness of the ARRA.
A second paper listed by Wonkblog under the “stimulus works” category is “Measuring Tax Multipliers: The Narrative Method in Fiscal VARs” by Carlo Favero and Francesco Giavazzi. This paper has nothing to say about the effectiveness of the ARRA; it relies on data through the first quarter of 2007. (The purpose of the article was to reconcile two different methods for estimating the impact of tax cuts or tax increases on economic activity.)
A third paper listed as supportive of the ARRA is “Fiscal Policy Multipliers on Subnational Government Spending” by Jeffrey Clemens and Stephen Miran. Again this paper has nothing to say about the effectiveness of the ARRA; it relies on data through 2004. It is fair to say, however, that the authors find low multiplier effects of government spending programs, perhaps due to crowding out of private expenditures. They state:
“our relatively low multiplier estimates (based on deficit-financed government spending) can be reconciled with the larger estimates in several recent studies (based on windfall-financed government spending) if government debt crowds out current private consumption and investment. We view these contrasting, but not contradictory, results as evidence of an important role for neoclassical considerations.”
Finally “The American Recovery and Reinvestment Act: Public Sector Jobs Saved, Private Sector Jobs Forestalled” by Timothy Conley and Bill Dupor finds that the ARRA had a slight positive effect on public sector employment and a slight negative effect on private sector employment. They report that the largest declines in private sector employment were in health, education, and private business (HELP) services. In particular they report that government employment increased by between zero and 900,000 jobs and private HELP service employment decreased by 166,000 to 1,378,000 jobs. They report that ARRA had a negative, but insignificant, effect on private sector jobs in goods-producing industries. Taken together, Conley and Dupor find the ARRA had no statistically significant positive impact on overall employment, and may have actually reduced employment.
Dylan Matthews and the editors at Wonkblog apparently don’t understand what it means for a study to find no significant effect of a policy. The Wonkblog article states “The biggest problem with the Conley and Dupor study is that their estimates are not statistically significant.” What is true is that any study of the ARRA will be fairly imprecise and have limited statistical power because of a small sample size and the methods required so that estimates don’t obviously suffer from endogeneity bias. Given the statistical precision of their study, Conley and Dupor could have detected a statistically significant increase in jobs in service-producing industries as small as 1.5%, if such an increase had occurred. Conley and Dupor most likely found no positive effect of the ARRA on private sector employment because the stimulus was either ineffective or had such a small effect that it could not reliably be measured.
I have more reading to do, but my initial reaction is: two studies had nothing to say about the effectiveness of ARRA, one study found no positive effect on employment at all, and the other gave mixed results. Even the study with mixed results found that the ARRA cost hundreds of thousands of dollars per job created and that the stimulus resembled political pork: spending was higher where Representatives had more tenure in Congress. My primary conclusion is that Wonkblog has helped the Romney campaign by calling more attention to academic studies questioning the efficacy of the Obama Administration’s economic policies.