Originally published 13 December 2016 and then re-published following the Great Server Error of 2016,
Answering research questions is one of the hardest and most valuable skills we can impart to students. But we often do this by teaching good research design–demonstrating the ideal, illustrating it with examples of designs that approximate the ideal, and then inviting students to imagine their own ideal research designs.
There are many pedagogical benefits to this method, but it overlooks a central fact: good research design is rare for good reasons.
People, including students and faculty, live in a world in which truth-claims are supported by research designs that wouldn’t pass muster with even the most generous reviewers. We can do a much better service by pointing out how to make a bad research design.
I present one here: The Starbucks Curse.
Imagine that you’re a columnist for a major national newspaper (say, The New York Times-Picayune). Imagine further that you have lots of conversations with taxi drivers. And imagine that you meet a taxi driver who tells you not that your Lexus is flat but that Starbucks causes murder.
Intriguing, you say.
You get back to the office and you pull up a list of the most dangerous cities in the United States. The first one you find is from WorldAtlas, which seems respectable (who lies on the Internet, of all things?). You find that the most violent US city is Detroit; and, lo, there are Starbucks in Detroit! One point for the taxi driver, eh? There are also Starbuckses in Memphis, St. Louis, Stockton, Indianapolis, and even our nation’s capital, Washington, D.C.
Alarmed, you look outside your building. There’s a Starbucks nearby.
You phone your agent. The world must be told. He agrees, and calls back later to announce that a Large Publishing Group has agreed to give you an advance–seven figures, of course, since you’re a leading expert consulted by business groups the world ’round.
You chase down the hypothesis. You investigate homicide cases and you find that many victims and even murderers have had Starbucks. You learn that academics have written articles with words like “imperialism” in articles about coffee--that will make for a great, erudite introductory parable. You have a drink–ironically, you will write, a coffee–with a fair-trade activist who is willing to agree with you that, yes, buying more fair trade coffee probably will drive down the homicide rate. You find that folks in Utah are unusually wise in banning coffee–and they have such low homicide rates! And you carefully note depictions of characters in noir movies and Quentin Tarantino films doing things like having coffee before committing murders.
Finally, after days of intensive writing (fueled, your acknowledgments will wryly admit, by coffee–but thankfully you haven’t killed anyone!), you release the book to widespread acclaim: Economist covers, Lunch with the FT, and, of course, a splashy spread in your newspaper’s magazine.
Anderson Cooper tells you he’s terrified, thanks to your book, when he interviews you. On a coffee plantation. In Ethiopia.
One day, you get an email from a professor of coffee studies, who earnestly tells you that there is no link between coffee and murder. But what does she know, anyhow? The marketplace of ideas has endorsed your theory. And besides, she can’t write all that engagingly about things like hot and crowded olive trees….