The Cult of Statistical Significance

Skimming over the recent book by McCloskey's latest book, The Cult of Statistical Significance: How the Standard Error Costs Us Jobs, Justice, and Lives, came across the following, p.169;

On the Truck and Barter blog Kevin Brancato of the Rand Corporation looked into the matter with some care, starting from a position sympathetic with Hoover and Siegler. But of their blast he concluded, “I must say that I'm disappointed … I don't think Hoover and Siegler have much new to say other than [that] the problem is not as bad as Ziliak and McCloskey claim. However, this is an empirical question, that in my mind, Hoover and Siegler fail to address thoroughly -- in fact, not even in a cursory fashion… In sum: A majority of papers in the AER in the 1980's and 1990's did not distinguish economic and statistical significance, although trends in the share are not yet determinable.” And a bit later the distinguished economist Thomas Mayer, who has for a long time been making the same point about econometric practice as we make here, remarked: “I don't understand the relevance of the table with the additional papers. Isn't the main issue whether authors pay attention to the size of the coefficients? And that is not in the table. What am I missing?” Professor Mayer is not missing anything: the main issues is, as Mayer suggests, not whether the Ziliak and McCloskey survey as originally published contained “the entire population” but whether, on the evidence of any reasonable sample of statistical practice in the American Economic Review (100 percent or an as-it-happens random sample of less than 100 percent), the authors pay attention to the size of the coeffiecients. A large majority- over 80 percent- do not. About this error, our critics Hoover and Stigler, and it appears the rest of the econometric profession, have nothing to say. “Not even,” as Brancato points out, “in a cursory fashion."

It is heartening to see that discussions on economics blogs are going mainstream in to books.