You cannot 'restore' high scientific standards if they are already in place

President Trump’s executive order “Restoring Gold Standard Science” provides a directive to restore a higher standard for scientific research and discovery. Yet despite the concerns it raises, the very standards that it describes already exist and are widely applied.

Section one of the order describes why the administration believes that it is needed.  Some of it focuses on the responses to the COVID pandemic.  Many now agree that many facets of the response were off-base.  The reason for this is much more than the biases noted in this section.

Scientific discovery often takes thoughtful reflection, which can be time consuming, fraught with missteps that guide and refocus the research process. The challenge during the COVID pandemic is that real-time policies could not be evaluated as quickly as the public health needs demanded. The net effect was less than ideal solutions and outcomes. With 20-20 hindsight, it is easy now to throw those who were offering guidance “under the bus.”

A parallel to this situation can be found when US Airways flight 1549 lost both its engines upon takeoff from NYC’s LaGuardia Airport due to a bird strike. Captain Chesley “Sully” Sullenberger was forced to use his best judgment and experience to safely land the airplane in the Hudson River. An investigation by the National Transportation Safety Board initially found that the airplane could have landed safely at LaGuardia or Teterboro airports. However, when a 35 second delay was inserted into the decision-making process, paralleling the real-life situation, testing on flight simulators demonstrated that Captain Sullenberger’s decision was indeed the right one, saving countless lives.

Much of what the executive order espouses is already embedded in the scientific method, a systematic approach to discovering new knowledge. Whether this knowledge is acquired through data collection and analysis, by physical experimentation, or using mathematical analysis based on axiomatic principles, the result is new understandings and insights.

Built into the scientific method is the need for reproducibility. This means that if one group of researchers makes a discovery following principles using the scientific method, then another group of researchers who follow the same principles should be able to make the same discovery.

The scientific method also allows for incremental discovery, whereby researchers can build upon known results to obtain new insights or stronger conclusions.

Yet new knowledge can only provide benefit if it can be widely disseminated, ideally in archival journals. This is where the peer review process comes into play. There are estimated to be more than 30,000 academic journals in existence, not all of which provide equal value.  The quality of the editorial board and the integrity of the review processes are critical to assess the novelty, correctness, and value of the research being reported.

There are obstacles to such research-archiving. Predatory journals provide rapid dissemination with thin review processes. Their goal is to collect publication fees from unsuspecting researchers who wish to disseminate their findings quickly and with little resistance. Often, researchers early in their careers, or more seasoned researchers whose work is not meeting the peer-review standard of well-respected journals, fall prey to these.

The internet also provides an avenue for dissemination. For example, ArXiv allows researchers to post their unpublished manuscripts prior to peer review, allowing them to lay claim to their ideas. It also permits unscrupulous researchers to take others' ideas and publish them as their own in peer-reviewed journals, creating a wild west environment for research dissemination.

The executive order also cites a growing number of publication retractions. In 2023, more than 10,000 papers were retracted from research journals. But a deeper dive into these numbers suggests that the problem in the U.S. is not as severe as the executive order suggests. Of the 10,000 papers retracted, 80 percent were from a single publisher.

Although the absolute number of papers retracted appears large, the retraction rate is under 0.25 percent, or less than one in 400 papers. 

Then there is the issue of where the authors of the retracted papers reside.  The U.S. is not in the top eight, with Saudi Arabia, Pakistan, Russia, and China topping the list.  Even then, the retraction rate in each of these countries was below one in 330 papers.

Scientific research is messy. The scientific method provides guardrails around it. There is nothing wrong with what the executive order espouses. Yet much of what it includes is already imbedded in the scientific method and the research integrity principals that are widely adhered to in academia, industry and government. 

However, what is “good for the goose must also be good for the gander.”  That means any reports issued by the administration — such as the recent MAHA report by the Department of Health and Human Services — should adhere to the same standards. 

This places a high standard for any policies put forth by the administration that affect health, economics, technology, and science. It also opens the door for greater scrutiny of such policies and encourages ample feedback from the academic community, who are under currently attack by the administration.

What the Restoring Gold Standard Science executive order actually does is give academic, industry, and government researchers the opportunity to take a victory lap, given that much of it outlines the ideal that we all already aspire to.

Sheldon H. Jacobson, Ph.D., is a professor of computer science in the Grainger College of Engineering at the University of Illinois Urbana-Champaign. He applies his expertise in data-driven risk-based decision-making to evaluate and inform public policy.