An interesting editorial on research practices came out in PLOS Medicine yesterday. It’s good to hear about reproducibility and reforms we need to see in science from a fellow statistician, John Ioannidis over at Stanford. Each discipline has its own quirks and accepted practices, but statistics is a common factor in every study. I believe we statisticians have a unique perspective on the problem: we get to play the role of data advisor on other peoples’ studies and the PIs on our own.
Ioannidis cites examples of things that work in several fields, including reproducibility practices, data registration, and stricter statistical analyses. Then he proposes a new “structure of scientific careers” that doesn’t just favor old men with fancy titles and big grants. In this framework,
Resources and power are seen as opportunities, and researchers need to match their output to the opportunities that they have been offered—the more opportunities, the more the expected (replicated and, hopefully, even translated) output. Academic ranks have no value in this model and may even be eliminated: researchers simply have to maintain a non-negative balance of output versus opportunities. In this deliberately provocative scenario, investigators would be loath to obtain grants or become powerful (in the current sense), because this would be seen as a burden.
I got to this part of the article and thought, “Wait, this sounds crazy?” It almost seems like there would be no incentive to work hard, like any award would come with some negative consequences and you’d be punished if your work didn’t produce results. Isn’t that exactly what research reforms are trying to get around? Maybe a greater emphasis on sharing negative results would get around this problem, but I digress.
After reading this the first time and feeling my knee-jerk disagreement, I took a step back and realized that my negative response is precisely due to my being immersed in the current culture of “publish or perish” and academic hierarchies. I’m so entrenched in this way of thought that it’s hard to see other models for scientific careers. However, I’m on Ioannidis’s side and I believe we need to seriously rethink the way research is done in order to have more high quality results.
Frankly my commentary on the subject is pretty useless because it’s a hard question and I’m no expert. You should just go read the article here.