Chopped liver

In 1999, Voros McCracken discovered what has come to be known as the “DIPS” theory – that pitchers have only a small influence on whether a ball in play becomes a hit. Since then, and in the standard fashion of scientific enquiry, there have been frequent studies testing McCracken’s hypothesis. The consensus, seven years later, after countless hours of research, analysis, and dialogue, is that the theory is generally true – pitchers have much less control over batted balls than previously believed, and any material deviation from league average is almost always just luck.

But to J.C. Bradbury, that research might as well not exist.

As he writes, all that research was done only by the “analytical baseball community” – not by reputable Ph.D.s in economics. It has not undergone “formal peer review.” Further, “it has not been tested with sufficient statistical rigor” and “has undergone very little formal scrutiny.” It does not use “proper econometric techniques.”

And so Dr. Bradbury sets out to correct this. How? Not by reviewing the existing research, and validating it academically. Not by finding those studies which have “insufficient statistical rigor” and analyzing them statistically. Not by summarizing what’s already out there and criticizing it.

No, Dr. Bradbury’s paper ignores it. Completely. He mentions none of it in his article, not even in the bibliography. Instead, Dr. Bradbury’s explains his own study as if it’s the first and only test of the DIPS hypothesis.

This happens all the time in academic studies involving baseball. Years of sabermetric advances, as valid as anything in the journals, are dismissed out of hand because of a kind of academic credentialism, an assumption that only formal academic treatment entitles a body of knowledge to be considered, and the presumption that only the kinds of methods that econometricians use are worthy of acknowledgement.

The truth is, there’s pretty decent statistical rigor in some of what us amateurs have done. In “Solving DIPS,” a bunch of really smart people, statistically literate, probably no less intelligent than academic economists and much better versed in sabermetrics, do an awesome and groundbreaking job of determining the causes of variation in the results of balls in play. There’s Tom Tippett’s famous study that showed that power pitchers performed better than projected by DIPS -- the same conclusion that Dr. Bradbury reaches. (Tom didn’t do any significance testing, though, which I guess makes his excellent analysis unworthy of citation.) The May, 2001 issue of “By the Numbers” had three articles on DIPS, one of which (mine) did roughly the same kind of regressions (although admittedly much less thorough) as Dr. Bradbury does in his own paper.

Dr. Bradbury knows about our previous work. In this article from last year, he mentions the Tippett article and a few others. He knows what’s out there. He must know that all this research has been peer reviewed, albeit informally, in hundreds, or even thousands, of blog postings and message boards. He must understand that the network of sabermetricians is perfectly respectful of the scientific method and understanding of the way theories must be tested and revised in the face of contradictory evidence. He’s got to know that our method of peer review, while informal, is much, much more likely to expose flaws in the theory than submitting a paper to a couple of economics referees who know little about sabermetrics.

And he’s got to be aware that in the field of sabermetrics, the achievements of non-academics, starting with Bill James and Pete Palmer, have been orders of magnitude above anything that’s come out of academia. Think of all the discoveries that have changed the way we look at baseball – DIPS, runs created, linear weights, the clutch hitting myth, catcher ERA, major league equivalencies, and so forth. Of the most important principles of sabermetrics, how many of those ideas were developed in academia? If the answer isn’t zero, it’s pretty close.

I feel bad singling out Dr. Bradbury. He does some good work on his blog, and, outside of this paper, seems very supportive and appreciative of the work we do. He seems like a nice guy. And ignoring non-academic research is pandemic when professors venture into sabermetrics – “The Wages of Wins” being the most prominent recent example.

So perhaps I’m being unfair. Maybe it’s one of the realities of academic life that to get an article published, you have to ignore non-academic work. Maybe the professorial culture requires that you presume no knowledge is real until it’s been published in peer-reviewed journals. Maybe Dr. Bradbury is right that all the previous results are questionable unless he uses the exact technique that he does. And maybe I’m just overreacting to a couple of throwaway sentences intended only to get his paper past the referees.

But still, there is a basic tradition among scientists of acknowledging that we stand on the shoulders of giants. When it comes to sabermetrics, academia repeatedly pretends that it hoisted itself up on its own bootstraps.

And that’s an insult to all of us.