Tenth in a series on the edges of science
Science often presents itself—or rather, scientists present themselves—as objective truth. “This is what we know.” Yet what we “know” is often based on research papers published in peer-reviewed journals.
For nearly twenty years, I was an associate editor of Systems Engineering, the journal of the International Council on Systems Engineering (INCOSE). Some years earlier, as INCOSE President, I had a strong hand in initiating that journal. I am quite familiar with the peer review process.
The primary goal of any such journal is to disseminate the knowledge gained by research. Unfortunately, this primary goal is conflated with other goals: journal reputation, sales and distribution, editors’ reputations, and the ubiquitous “publish or perish” mantra of academia. Scientists gain standing by publication; the more respected the journal, the more status gained.
I have seen poor-quality papers submitted repeatedly, responding to critique with modifications, until the journal editors tired of sending it back (or were in need of a paper to fill a hole). I have seen papers published with conclusions unwarranted by the research questions or methods. I have seen interpersonal, inter-organizational, and international politics drive the acceptance process. In at least one research project, when an assistant pointed out a fatal inconsistency, the primary researcher responded, “Well, I have to continue because this research provides funding.”
The peer review process is based on people, with all of our flaws.
The Wall Street Journal last week published an in-depth article titled “Anti-Abortion Lawsuits Leaned on Discredited, Disputed Research.” The headline seems to say it all: the research basis must have been flawed. The article points out in the lead sentence, “Recent efforts to roll back access to abortion have relied in part on certain scientific studies that have been disputed, disavowed, or subsequently retracted.” Based on the rest of the article, this inflammatory summary is only partially true. The article cites only three studies. All three have indeed been disputed, largely on political grounds. Only one was retracted.
(In summary. One paper found evidence that use of the abortion pill correlated with a higher incidence of emergency room visits. A second claimed women who had an abortion were 81% more likely to experience anxiety and depression. The third found that women who had an abortion were at higher risk of severe disorders such as PTSD, bipolar, and substance abuse.)
The opponents question methodology, “poor quality” (a rather subjective term), and inability to replicate the results. (See my prior essay on reproducibility.) The one paper that was retracted by the publishing journal was taken back because the lead author did not disclose a conflict of interest—his position with an institute that opposes abortion rights. Yet the opponents of these papers have identical conflicts of interest, often funded by organizations defending abortion rights.
In every case, the authors vehemently defend their work, and they continue to publish results that advance and support the work. Their funding sources—anti-abortion agencies—demand they do so.
The opponents also do research, and they publish papers that contradict the authors. Their funding sources—pro-abortion agencies—also make demands.
Who can untangle this web of accusation and counter-accusation? Perhaps we need King Solomon as an arbiter of science.
A similar controversy revolved around the widely-quoted claim of a few years ago that 97% of “climate experts” believe climate change to be anthropomorphic, i.e. caused by humankind. The original research establishing this figure surveyed over 900 papers that cited the phrase “global climate change.” The researchers did not look for proof of cause-and-effect linkages. Instead, they looked for inferences and assumptions in the papers that indicated the original author’s belief
structure about climate change. In actuality, only about 25% of the papers explicitly supported the anthropomorphic cause. An additional 50% were interpreted (by the survey researchers) to have implicitly endorsed the cause. Even allowing these interpretations, the results still only came up to about 75%. Yet the researcher published that his work “demonstrates that any remaining professional dissent is now exceedingly minor.”
My purpose in this essay is not to argue pro- or anti-abortion, nor to argue for or against anthropomorphic climate change. Rather, my point is the human, political nature of peer review in publication.
Scientific papers are the bedrock of science. By publishing, scientists claim and document the truth of their work. In many cases, that truth then becomes cause for argument among dissenting scientists, flat-earthers versus round-earthers, keto dieticians versus vegetarians, particles versus waves. People are people, and we have different opinions. Science tries to discover facts, but it is often obscured by the belief structures of the scientists themselves.
As I write about possible futures, I am constantly faced with today’s assumptions about science. How many of them exist only because they have been widely accepted? (Like the 97% number) Which ones have actually been proven, and to what extent? (Has anyone seen a Big Bang? How much do we really know about the limit of lightspeed, when humankind has yet to create any macro-object going faster than 0.05%c?)
The political nature of science publication often calls into question the reality of science’s claims.
Doc Honour
June 2024
Doc Honour Books 