Wednesday, November 15, 2017

The "cackling cartoon villain" defense of DSGE


I've sworn off macro-bashing. I said what I had to say. And I'm seeing lots of young macro people doing good stuff. And the task of macro-method-criticizing has been taken over by people who are better at it than I am, and who have much better credentials. My macro-bashing days are done.

But sometimes I just have to offer macro folks some marketing advice.

The new defense of DSGE by Christiano, Eichenbaum, and Trabandt is pretty cringe-inducing. Check this out:
People who don’t like dynamic stochastic general equilibrium (DSGE) models are dilettantes. By this we mean they aren’t serious about policy analysis. Why do we say this? Macroeconomic policy questions involve trade-offs between competing forces in the economy. The problem is how to assess the strength of those forces for the particular policy question at hand. One strategy is to perform experiments on actual economies. This strategy is not available to social scientists. As Lucas (1980) pointed out roughly forty years ago, the only place that we can do experiments is in our models. No amount of a priori theorizing or regressions on micro data can be a substitute for those experiments. Dilettantes who only point to the existence of competing forces at work – and informally judge their relative importance via implicit thought experiments – can never give serious policy advice.
That reads like a line from a cackling cartoon villain. "Buahahaha, you pitiful fools" kind of stuff. It's so silly that I almost suspect Christiano et al. of staging a false-flag operation to get more people to hate DSGE modelers.

First, calling DSGE critics "dilettantes" was a bad move. By far the best recent critique of DSGE (in my opinion) was written by Anton Korinek of Johns Hopkins. Korinek is a DSGE macroeconomist. He makes DSGE models for a living. But according to Christiano et al., the fact that he thinks his own field has problems makes him a "dilettante."

OK, but let's be generous and suppose Christiano et al. didn't know about Korinek (or Ricardo Caballero, or Paul Romer, or Paul Pfleiderer, etc.). Let's suppose they were only talking about Joseph Stiglitz, who really is something of a dilettante these days. Or about bloggers like Yours Truly (who are actual dilettantes). Or about the INET folks. Even if so, this sort of dismissive snorting is still a bad look.

Why? Because declaring that outsiders are never qualified to criticize your field makes you look insular and arrogant. Every economist knows about regulatory capture. It's not much of a leap to think that researchers can be captured too -- that if the only people who are allowed to criticize X are people who make a living doing X, then all the potential critics will have a vested interest in preserving the status quo.

In other words, Christiano et al.'s essay looks like a demand for outsiders to shut up and keep mailing the checks.

Second of all, Christiano et al. give ammo to the "econ isn't a science" crowd by using the word "experiments" to refer to model simulations. Brad DeLong already wrote about this unfortunate terminology. Everyone knows that thought experiments aren't experiments, of course - Christiano et al. aren't actually confusing the two. But obstinately insisting on using this word just makes econ look like a pseudoscience to outside observers. It's bad marketing.

Third, Christiano et al. are just incorrect. Their defense of DSGE is, basically, that it's the only game in town - the only way to make quantitative predictions about the effects of policy changes.

That's wrong. There are at least two other approaches that are in common use - sVARs and SEMs. sVARs are often used for policy analysis in academic papers. SEMs are used by central banks to inform policy decisions. Both sVARs and SEMs claim to be structural. Lots of people laugh at those claims. But then again, lots of people laugh at DSGE too.

In fact, you don't always even need a structural model to make quantitative predictions about policy; often, you can do it in reduced form. When policy changes can be treated like natural experiments, their effects - including general equilibrium effects! - can be measured directly instead of inferred from a structural model.

As Justin Wolfers pointed out on Twitter, at least one of questions that Christiano, et al. claim is only answerable by DSGE simulations can actually be answered in reduced form:
Does an increase in unemployment benefits increase unemployment? On the one hand, conventional wisdom argues that higher benefits lead to higher wages and more unemployment. On the other hand, if the nominal interest rate is relatively insensitive to economic conditions, then the rise in wages raises inflation. The resulting decline in the real interest rate leads to higher aggregate demand, a rise in economic activity and lower unemployment. Which of these effects is stronger?
A 2015 paper by John Coglianese addresses this question without using a DSGE model:
I analyze a natural experiment created by a federal UI extension enacted in the United States during the Great Recession and measure the effect on state-level employment. I exploit a feature of this UI extension whereby random sampling error in a national survey altered the duration of unemployment insurance in several states, resulting in random variation in the number of weeks of unemployment insurance available at the state level. 
Christiano et al. totally ignore the existence of natural experiments. They claim that in the absence of laboratory experiments, model simulations are the best we've got. The rapidly rising popularity of the natural experiment approach in economics doesn't even register on their radar screens. That's not a good look.

Finally, Christiano et al. strike a tone of dismissive arrogance, at a time when the world (including the rest of the econ profession) is rightly calling for greater humility from macroeconomists. The most prominent, common DSGE models - the type created by Christiano and Eichenbaum themselves - failed pretty spectacularly in 2008-12. That's not a record to be arrogant about - it's something to apologize for. Now the profession has patched those models up, adding finance, a zero lower bound, nonlinearity, etc. It remains to be seen how well the new crop of models will do out of sample. Hopefully they'll do better.

But the burden of proof is on the DSGE-makers, not on the critics. Christiano et al. should look around and realize that people outside their small circle of the world aren't buying it. Central banks still use SEMs, human judgment, and lots of other tools. Finance industry people don't use DSGEs at all. Even in academia, use of DSGE models is probably trending downward:


In other words, Christiano et al. and other DSGE champions are still getting paid nice salaries to make DSGE models, but they're not winning the intellectual battle in the wider world. Dismissive rhetoric like this essay will not help their case. Even Miles Kimball, who spent his career making DSGE models, and who made crucial contributions to the models for which Christiano and Eichenbaum got famous, was put off by this essay. 

Look. There are good defenses of modern macro, and of DSGE, to be made. Ricardo Reis made a really good defense earlier this year. Fabio Ghironi made another good one. Their essays are humble and smart. They acknowledge the key importance of empirical evidence and of a diversity of approaches. They also acknowledge that macroeconomists need to do better, and that this will take some time. They focus on the young people doing good work, striving to improve things, and striking out in new directions. These are the macro defenses the profession needs.

The idea of DSGE models is not a bad one. Working on DSGE models isn't necessarily wasted effort. Nor are most DSGE modelers the dismissive, chest-thumping caricature that Christiano et al.'s essay paints them as. People are out there doing good work, trying to improve the performance of macro models. But rhetoric like this ends up hurting, rather than helping, their task. 

25 comments:

  1. Anonymous6:23 PM

    A question I often ask myself is "What would Feynman think of modern (macro)economics?" Cargo cult science seems not too inaccurate for macro at least.

    ReplyDelete
  2. Anonymous9:43 PM

    DSGE models are based on a GE framework. GE models firstly don't deal with aggregate variables, they deal with individual equilibrium in every market in the model. Secondly, they are models based on optimization/maximization subject to an income constraint. That is they assume away what it is they purport to study, the level of income (and employment).

    Henry Rech

    ReplyDelete
    Replies
    1. Anonymous9:01 AM

      >That is they assume away what it is they purport to study, the level of income (and employment).

      if you don't understand why this makes no sense, then you should stop acting like you're an informed commenter.

      Delete
    2. Anonymous12:04 PM

      "if you don't understand why this makes no sense, then you should stop acting like you're an informed commenter."

      Why don't you explain yourself then?

      Delete
    3. Anonymous8:22 PM

      Anonymous, I am disappointed you have not responded. I would genuinely like to see what you have to say.

      One thing about the GE part of DSGE. It seems to me that these models are not of the Walrasian/ADM general equilibrium variety, that is those models which characterize equilibrium in multiple individual markets simultaneously. Given there is generally one consumer, one firm, they look more like Marshallian partial equilibrium models. Perhaps having "general" tacked on to the name yields them a greater cachet.



      Delete
  3. Anonymous3:20 AM

    Very good points raised but I'm not convinced that natural experiments make a perfect substitute for computer simulation like you seem to say implicitly. The possibility to control for any omitted variables and endogeneity bias is the backbone of modern econometrics, but some NEs happen so rarely that it's almost impossible to account for all these factors due to small sample size. Second, natural scientists like cosmologists or climatologists rely a lot on simulatons too, so is it fair to ditch the method as unscientific? I bet you wouldn't want to wait a couple of decades to see how the climate change really affects when you can have data set of millions of simulations available already.

    Tommi Salenius

    ReplyDelete
    Replies
    1. Simulation is useful to the extent that we trust the dynamic equations that we plug into the system. Then we are only using the simulation to work out the macro consequences of these trusted fundamental laws, which we're not smart enough to do with pen and paper. The dynamic equations we feed into DSGEs don't have the same trusted status as the Navier-Stokes equations or general relativity (which are presumably the inputs into climate or cosmological simulations), to put it mildly. If we did possess trusted laws describing the behavior of individual economic agents, and the only remaining problem was to figure out the consequence of all their interactions, simulations would be a great idea. (These still wouldn't have to be DSGEs - they could be VARs, SEMs, ABMs, etc. depending on the nature of those individual laws). We don't, though, and I doubt we ever will, so simulation must have at best a different role to play in economics, relative to cosmology or climatology.

      Delete
  4. Given that the example showing up in the links is the FRB/US model, what does "SEM" stand for?

    ReplyDelete
    Replies
    1. https://noahpinionblog.blogspot.co.uk/2014/04/the-foxy-fed_7.html

      and

      https://mainlymacro.blogspot.co.uk/2014/04/the-feds-macroeconomic-model.html

      SEM stands for Structural Econometric Model

      Delete
    2. Anonymous1:56 PM

      Simultaneous equation models

      Delete
    3. Got it, thanks. Yes, those old dinosaurs still stalk the halls of most central banks pretty seriously.

      Delete
  5. Barkley "SEM" stands for "Structural Econometric Model". It means the kind of model macroeconomists used before Lucas critiqued them. The SEM approach was pionered by Jan Tinbergen in Holland and by Jacob Marschak at the University of Chicago. Also the critique (Lucas cited Marschak who had said it all).

    SEM models could also be called old Keynesian macro models. They will have something like an IS curve, something like a Phillips curve and (back in the day) something like an LM curve. Now they would have a Taylor rule not an LM curve.

    They are totally different from New Keynesian models which have an Euler equation (something like an LM curve) a new Keynesian Phillips curve and a Taylor rule. Huh uh what uh so why are they different ? Well the *new* Keynesian models start with sound fundamentals like the Calvo fairy and derive the old Keynesian equations through sophisticated math and heroic linearization assumptions. So they are much better. Now it is true that NK models have firm implications on questions where old K models had only economists' guesses. The problem is that all of the implications are contradicted by the data, so one fiddle factor has to be added for every moment matched.

    But it was a famous victory.

    A paragraph in, I thought I might have a useful comment to make. Now I know I don't, but I had been formulating it while reading.

    As Noah pointed out, Christiano, Eichebaum and Trabandt conflate the set of models with the set of DSGE models. The argument that those who dismiss DSGE models are dilettantes is exactly the argument that those who refuse to use models are dilettantes. There is no reference to general equilibrium (however defined) in their argument.

    As Noah notes, they don't critique structural econometric models, because they don't admit that such models exist (you will guess I am less thrilled about VAR s and otherwise but at least I admit they exist).


    There is one part of the quoted passage which I found particularly contemptible.

    They appeal to the authority of Lucas. This is not a good strategy if one is claiming to be a scientist. Biologists do not appeal to the authority of Darwin (and dismiss much of what he wrote or, rather, don't bother reading it). Physicists don't appeal to the authority of Einstein (generally rejected) or Feynman (sorry anonymous if you are a physicist).

    In fact, Lucas's argument is entirely without merit. Lucas claimed that, if one didn't assume rational expectations, one could fit anything and therefore forecast nothing. He is correct. The same is true if you do assume rational expectations (the proof is trivial). The edifice is built on a contradiction -- two claims which are opposite -- not only can't they both be true but they can't both be false.

    Noah explained this over 2 years ago
    http://noahpinionblog.blogspot.it/2015/03/the-pincer-attack-on-macro-models.html
    noting I had explained it over 5 years ago
    http://rjwaldmann.blogspot.it/2012/03/modern-macroeconomic-methodology-modern.html

    Anyone who appeals to Lucas and the Lucas critique must answer the Smith critique of their selective use of the Lucas critique. None has, because non can, because there is no answer, because their argument has no merit whatsoever.

    The response to the argument (which was made very clearly in 1978 by Solow so my "more than 5 years ago" is a silly boast) was something along the lines of "we are the cutting edge of academic economics and if you want to get published in top journals you have to accept our dicta even if they contradict each other" (usually put slightly more politely). Now, as Noah notes, they just don't recognise that they don't have the standing in the profession necessary to dismiss arguments to which they can't respond. It's almost sad, but, hey, they have tenure so it's all right.

    ReplyDelete
    Replies
    1. Robert,

      Thanks. I know your long speech, indeed have given long before you did here. Just was not sure what the acronym was, and, now that I see what it is, when it became sort of standard for referring to these models.

      Delete
    2. "As Noah notes, they don't critique structural econometric models, because they don't admit that such models exist"

      That denial was well expressed in the statement "According to Fischer (2017), DSGE models play a central, though not exclusive, role in this process." In fact, Fischer highlighted FRB/US model and explained it in detail in his talk, whereas he treated DSGE models only in a footnote.

      Delete
  6. The problem with DSGE is fundamental. GE is a monumental mistake. The problem is not modelling as such, it is the crazy models that are used.

    reason

    ReplyDelete
  7. If you don't understand my last comment, let me put it this way, there is absolutely no reason to think that the world was ever at or near GE, or that if it happened to be at GE it would stay there, or that it in general makes movements towards GE. We need to model how the world moves, not how a supposed (and unknowable) GE moves.

    ReplyDelete
  8. If you all look at Noah's link to central banks using SEMs you will see that I have laid it out at length, well beyond Robert's comments, including a bit on what goes on in central bank decision making, which indeed amounts usually to looking at SEMs, DSGEs, VARs, and then a lot of intuitive input from those making the decisions based on what they are hearing from influential sources in the business and policy world. In the end, the front room, as Jim Bullard calls it, dominates the back room (or "basement" as some denizens of it at the Fed used to call it, although literally at the Bd of Govs in Washington their shop is to the rear of the main building, not below it).

    ReplyDelete
  9. Anonymous11:25 AM

    Dilettante Joseph Stiglitz:

    "Stiglitz says Bernie Sanders would have beaten Trump. “I see the election of 2016 as an election of protest. Bernie represented a return to the old values: a middle-class lifestyle, a home, a secure retirement, education for your children, healthcare. Jeremy Corbyn is saying the same thing in the UK.”"

    https://www.theguardian.com/business/2017/nov/16/joseph-stiglitz-trump-fascist-globalisation-bernie-sanders

    ReplyDelete
  10. Both small (IS-LM) and large (1980s Fed, DSGE) are useful for answering policy questions that people care about. They are not good for forecasting or for anticipating structural collapse. Indeed, we have tried - with limited success - to forecast crises for over 20 years at the IMF. Sometimes it works sometimes not. The big problem is forecasting a crisis and thereby causing it. Also agents behavior is changed by the most recent crisis, eg, KFG spec attack worked well for Mexico until we had used it and so the governments shifted to selling crappy debt to the banks. These guys are our former students - some pretty good.

    ReplyDelete
  11. "Every economist knows about regulatory capture" - I'm not sure this assumption holds.

    ReplyDelete
  12. I think Stiglitz is not a dilletante but, in Isaiah Berlin's terminology, a fox. Deductive lovers do not like them, but as Tetlock shows, the discipline needs many, many more foxes.

    ReplyDelete
  13. Dilettantes at the end of the coal-pit
    Comment on Noah Smith’s ‘The “cackling cartoon villain” defense of DSGE’

    As Hume said, “... when the road ends at a coal-pit, he [the traveler] doesn’t need much judgment to know that he has gone wrong, and perhaps to find out what has led him astray.”

    With DSGE, Walrasian economics has, after 140+ years, reached the end of the coal-pit. Lacking sound scientific judgment, though, Christiano/Eichenbaum/Trabandt maintain: “People who don’t like dynamic stochastic general equilibrium (DSGE) models are dilettantes. By this we mean they aren’t serious about policy analysis…”

    Science is NOT about like/dislike but about true/false. Fact is that DSGE is provably false. Because of this, all policy proposals that have ever been derived from DSGE models lack sound scientific foundations.

    Science is about the true theory. The characteristic of science is the insistence on consistency: “Research is in fact a continuous discussion of the consistency of theories: formal consistency insofar as the discussion relates to the logical cohesion of what is asserted in joint theories; material consistency insofar as the agreement of observations with theories is concerned.” (Klant)

    Economics pretends to be a science but is what Feynman called a cargo cult science “They’re doing everything right. The form is perfect. ... But it doesn’t work. ... So I call these things cargo cult science because they follow all the apparent precepts and forms of scientific investigation, but they’re missing something essential.”

    Economists lack genuine scientific instinct/ambition: “The highest ambition an economist can entertain who believes in the scientific character of economics would be fulfilled as soon as he succeeded in constructing a simple model displaying all the essential features of the economic process by means of a reasonably small number of equations connecting a reasonably small number of variables. (Schumpeter, 1946)

    Theory construction starts since 2000+ years with clearly stated premises#1: “To Senior belongs the signal honor of having been the first to make the attempt to state, consciously and explicitly, the postulates that are necessary and sufficient in order to build up … that little analytic apparatus commonly known as economic theory, or to put it differently, to provide for it an axiomatic basis.” (Schumpeter)#2

    Not only DSGE has failed at constructing the Simple Ur-Model. The four main approaches ― Walrasianism, Keynesianism, Marxianism, Austrianism ― are mutually contradictory, axiomatically false, materially/formally inconsistent and all got the foundational economic concept profit wrong.

    At the end of the coal-pit, the lethal methodological blunder of DSGE is quite obvious: microfoundations are false since Jevons/Walras/Menger. And Keynes’ attempt to move from microfoundations to macrofoundations failed.#3

    The methodologically correct action in the given situation is the paradigm shift. False Walrasian microfoundations and false Keynesian macrofoundations have to be replaced by true macrofoundations.

    Economics is a failed/fake science. At the end of the coal-pit, it is now quite obvious that scientific dilettantism leads the representative orthodox/heterodox economist astray since 200+ years.

    Egmont Kakarot-Handtke

    #1 “When the premises are certain, true, and primary, and the conclusion formally follows from them, this is demonstration, and produces scientific knowledge of a thing.” (Aristotle)
    #2 Microfoundations are given with this verbalized axiom set: “HC1 economic agents have preferences over outcomes; HC2 agents individually optimize subject to constraints; HC3 agent choice is manifest in interrelated markets; HC4 agents have full relevant knowledge; HC5 observable outcomes are coordinated, and must be discussed with reference to equilibrium states.” (Weintraub)
    #3 How Keynes got macro wrong and Allais got it right
    https://axecorg.blogspot.de/2016/09/how-keynes-got-macro-wrong-and-allais.html

    ReplyDelete
    Replies
    1. Sorry, Egmont, but science is probably ultimately more about data and empirical testing than it is about theory, important as theory is.

      Delete
  14. Anonymous4:40 AM

    Eco o metric Ian here. Actually a structural SVAR is just a type of Structural Econometric Model (SEM) or Simultaneous Equations model, these models can be linear or not, in the case of linear dynamic model the classical example is the SVAR. From the econometric perspective a SDGE is a somewhat different type of SEM, but not very different (this is more clear if you consider the Euler equations of all decisors). So, the key differences are unrelated to econometric framework, they lay in the assumptions about how economic agents take decisions, In which one is more realistic and detailed but yet it can be handed. Experiments and behavioral economics can be very useful here.

    ReplyDelete
  15. "DSGE" covers a lot of ground. One interpretation of what Christiano et al. are saying is that, if you're objecting to the use of any quantitative stuctural modeling approach to giving policy advice. And that would be very limiting, I think - essentially throwing information away. I think if you talked to Larry Christiano, you'll find that he has no objection to VAR methodology (indeed, that's a key part of his approach to fitting a model) or natural experiments.

    But... no policy maker should take, for example, Christiano, Eichenbaum, and Evans off the shelf and use that model to determine what the target for the Fed's policy rate should be at the next meeting. There's nothing wrong with treating such a model as a "laboratory" to "run experiments," and we can learn a lot from doing that. But for people who actually do policy, everything should be taken with a grain of salt, and the theory and measurement are simply not good enough yet to literally use such models directly for giving advice.

    ReplyDelete