Thursday, January 07, 2016

The Data Revolution goes mainstream


My Bloomberg View colleague Justin Fox has written an excellent post publicizing the econ profession's shift from theory to empirics. Fox puts the Hamermesh data on publication percentages into a lovely Bloombergy chart:


By now this is an old story to people within the profession. But I think it's an important story to keep telling to the public, in order to increase people's trust of economists. 

People instinctively know that empirical sciences - fields where theories have to be tested by data in order to gain currency - are more reliable than ones based on pure theory. Seeing that economists are now checking their ideas carefully will make the public more confident that prevailing ideas were not simply accepted because of ideology or intellectual whimsy. This is also why I think colleges should start teaching empirics in Econ 101.

Of course, there's bound to be resistance to the popular narrative of the Data Revolution. In an earlier post I tossed out some possible reasons for resistance within the econ profession. Some of those are good reasons, and some are less good, but the people in the public discussion will probably have a much simpler objective function. Ideological free-market types believe that simple Econ 101 type theory supports their ideas. Empirical results, because they deal with an inevitably non-101 reality, threaten that nice clean little intellectual world that the ideological free-marketers have built.

So I predict that we'll see most of the public disapproval and skepticism of the Data Revolution - on Twitter, in the media, etc. - coming from folks with a strong free-market ideological bent. Though a bit will come from lefty types annoyed that it was data rather than Marxist hand-waving that ended up transforming economics.

What form will the public criticism take? At first, I expect to see a lot of people saying stuff like "without theory, data is useless," or "data doesn't speak for itself." This, of course, is trivially true. But the Data Revolution isn't actually about replacing theory with data, it's about checking theory with data. Empiricists aren't out there running batch-file regressions - they're doing local tests of theories. (In fact, they're probably doing too much structural modeling, even when it's only related to their study by the most tenuous of threads!)

The battle of ideas is really between A) those who think that theories should have to have empirical support before we believe them, and B) those who want to believe theories until proven otherwise. It's about the strength of priors, in some sense - science vs. theoryderp. The Data Revolution really is just as important as Justin Fox writes. It represents a true paradigm shift in econ.

Eventually, the simplistic rhetorical criticism will run out of steam, and people will start gunning for some famous empirical results. The idea will be to discredit one or two top studies, and imply that empirical econ in general is unreliable, like nutrition science or social psychology. Naturally, there will be some targets available for this sort of attack, since A) scientists make mistakes, and B) conditions in the economy change, so old results sometimes no longer hold. 

The most sophisticated attack, I predict, will be based on replication and publication bias. That criticism will be the most effective, since it's almost certainly true. Econ has terrible data management practices, and is just as subject to publication bias, p-hacking, and data mining as any field. 

But I think that in the end, empirical econ will benefit from these public attacks. Data management practices will improve, and meta-analyses will sort the reliable results from the ephemeral. Randomization and control procedures will get better, and statistical analysis will become more sophisticated. Credibility will increase and increase.

In the end, I predict that the Data Revolution will not be a fad or a flash in the pan. Humans like believing in wanky theory when data isn't available, but when data is available, they want to have some confirmation that theories actually work. Most people are scientists at heart. And now that econ has tools that are more like the tools of science, I predict that the changes in the field are irrevocable.


Update

Here is a list of my previous prolix posts pompously pontificating on this particular point...





15 comments:

  1. The public should have as much trust in the new data driven economics as they do in the newest nutrition research. It's gonna be aso conclusive.

    ReplyDelete
  2. In general, the problem is that economists don't know how to extract information from data. Not because they are dumb, it's because they have every incentive not to know.

    ReplyDelete
  3. "Most people are scientists at heart."

    Your course load does still entail teaching actual students, right?

    I just find the above comment absurd in its naive optimism, afloat as it is in the blood dimmed tide of presumptive and assertive idiocy that is the modern Internet...

    ReplyDelete
  4. "In general, the problem is that economists don't know how to extract information from data."

    I beg to differ, try taking a modern econometrics course. Economists are all to aware of the limitations of sample extrapolation, spurious regressions, robustustness etc... because economic data and models based on that data is scrutinized far more than anything else.

    ReplyDelete
    Replies
    1. Sorry, that was rude. The very fact that Noah here offers disquisitions how the new data driven econ will show us the true complex truth is testimony to pretty committed cluelessness. With all due respect.

      My argument is that the commitment is not an accident. Economists have to do something to justify getting paid.

      Delete
    2. I am Responding to an Idiot12:24 PM

      Good thing you're here to tell us how it is. But I wonder what it is that makes you think you're right. Can't be intelligence, because you don't have any of that. Must be stupidity.

      Delete
    3. Hmm, are you an economist who is so proud of the new powerful tools? Easy to check how powerful they are. Where there is a strong conviction, there is always a trade. I trust you should be smart enough to translate your convictions into a trade, then mortgage your house and enjoy the pleasures of your insight.
      You'll show idiots like me who's boss.

      Delete
  5. Anonymous10:57 PM

    Data enters economics as a determinative factor - about 100 years late. Why next, we might start thinking of testing hypotheses!
    Dr. Deaton recently got the Nobel prize for, among other things, for his Almost Ideal Demand System wherein a consumer's demand does not depend linearly on income but decreases incrementally as income rises. Stop the presses!
    Seriously, it's good that progress is being made.

    ReplyDelete
  6. Look dude, nobody find Lars Syll more annoying than I do, but a "hand-waving Marxist" he is not, no matter how much you want him to be. Is that really the best data you could find to support your contention that such exist? Then your argument would have been stronger without out it. I guess you must have a pretty strong prior.

    ReplyDelete
    Replies
    1. I can't but agree, Phil. I've been called a lot during my long academic career, but "hand-waving Marxist" was certainly something new. Absolutely gobsmacking!

      Delete
  7. "(In fact, they're probably doing too much structural modeling, even when it's only related to their study by the most tenuous of threads!)"

    Your hyperlink here links to David Giles' arguing there is Not Enough structural modeling. Huh?

    Otherwise: Keep on banging this drum, my man \m/

    ReplyDelete
    Replies
    1. Giles is bemoaning the weak link between the structural section and the empirical section. He thinks these folks should either do structural estimation, or leave out the structural section entirely. I agree totally. We talked about this on Twitter, actually!

      Angrist doesn't like strucutral estimation, but I think it has its place.

      Delete
  8. I'm very curious to know if there is a list of particular things supported by "theory" that are inaccurate according to "data."

    Something tells me that any new data approach will further illustrate that there are exceptions to what are still the same rules.

    ReplyDelete

  9. Am here to testify what this great spell caster done for me. i never believe in spell casting, until when i was was tempted to try it. i and my husband have been having a lot of problem living together, he will always not make me happy because he have fallen in love with another lady outside our relationship, i tried my best to make sure that my husband leave this woman but the more i talk to him the more he makes me fell sad, so my marriage is now leading to divorce because he no longer gives me attention. so with all this pain and agony, i decided to contact this spell caster to see if things can work out between me and my husband again. this spell caster who was a woman told me that my husband is really under a great spell that he have been charm by some magic, so she told me that she was going to make all things normal back. she did the spell on my husband and after 5 days my husband changed completely he even apologize with the way he treated me that he was not him self, i really thank this woman her name is Dr Aluta she have bring back my husband back to me i want you all to contact her who are having any problem related to marriage issue and relationship problem she will solve it for you. her email is traditionalspellhospital@gmail.com she is a woman and she is great. wish you good time.
    He cast spells for different purposes like
    (1) If you want your ex back.
    (2) if you always have bad dream
    (3) You want to be promoted in your office.
    (4) You want women/men to run after you.
    (5) If you want a child.
    (6) You want to be rich.
    (7) You want to tie your husband/wife to be yours forever.
    (8) If you need financial assistance.
    (9) HIV/AIDS CURE
    (10) is the only answer to that your problem of winning the lottery

    Contact him today on: traditionalspellhospital@gmail.com

    ReplyDelete