The phrase “conflict of interest” often occurs in the context of talking about scientific studies. Childhood Obesity News discussed Tainted Truth: The Manipulation of Fact in America, the landmark work by Cynthia Crossen. The public has been trained to accept the validity of studies issued by the “research industry,” especially those with prominent names attached, and those emanating from prestigious institutions.
Crossen pointed out how information is never neutral, and wrote in favor of increased public awareness, as well as tighter controls on research practices. Publishers Weekly said,
Particularly disturbing, the author notes, is that scientific and academic research, which has traditionally represented the disinterested pursuit of knowledge, is increasingly underwritten by corporate sponsors seeking to manipulate the results.
Twenty years after the publication of Tainted Truth, Joseph Salerno reminded readers of the Mises Economics Blog that the peer review process was still alive and well for a very important reason:
The number of papers that an academic publishes in peer-reviewed journals and the number of times his or her articles are cited in other peer-reviewed articles are the main factors determining whether or not he or she is promoted and awarded tenure.
Salerno reiterates that the peer review process is “supposed to be the unimpeachable guarantee” that whatever is published in the journals complies with the highest scientific and academic standards. Yet two decades after Crossen sounded the alarm, the process was still not only susceptible to monkey business, but “gravely flawed and easily abused.”
For example, after exposing the existence of a “peer review ring”, one journal had to retract 60 published articles. Read it and weep:
As best as can be determined, the ring posted up to 130 fabricated names and fake email addresses on an online reviewing system… These bogus identities were used by the members of the ring to write favorable reviews of one another’s submissions…
Apparently, one bold scientist even wrote and published a review — favorable, of course — of his own work, under an alias. After the scandal receded, the editor-in-chief, who had opened the investigation, felt obliged to retire from the editorship.
The United Kingdom’s Royal Society of Medicine is about as venerable an institution as can be found anywhere, and its publication, the British Medical Journal, is highly respected. Former editor Richard Smith described the traditional peer review mechanism as “little better than tossing a coin.” Because the system works on trust, fraud can go completely undetected.
Curious to see how much actual chicanery could be accomplished, he and others on the staff donned their Sherlock Holmes hats:
At the BMJ we did several studies where we inserted major errors into papers that we then sent to many reviewers. Nobody ever spotted all of the errors. Some reviewers did not spot any, and most reviewers spotted only about a quarter.
The painful truth is, no one who was not present can really know what occurred in any study. The peer review is less about the nuts and bolts of the science itself, and more about whether the project is being explained clearly to the interested colleagues.
Ideally, an experiment reported by one lab can be replicated by any similarly equipped lab following the same protocols. A peer reviewer is unlikely to delve into things to that extent, and cannot realistically be expected to. In the world of science journalism, the peer reviewer courteously assumes that everything is present and correct.
Your responses and feedback are welcome!