Inability to Reproduce Results: A Crisis

 Its the scientific method.   I have seen many examples where you could not recreate the context involved, but the results were accepted as proof, because they followed some narrative that was generally accepted, or not accepted.  Bias of many types.   Dangerous crisis. Good article, but deeper than is stated.  Especially if investment and resulting risk is high.

An Inability to Reproduce   By Samuel Greengard 
Communications of the ACM, September 2019, Vol. 62 No. 9, Pages 13-15
10.1145/3344289

Science has always hinged on the idea that researchers must be able to prove and reproduce the results of their research. Simply put, that is what makes science...science. Yet in recent years, as computing power has increased, the cloud has taken shape, and data sets have grown, a problem has appeared: it has becoming increasingly difficult to generate the same results consistently—even when researchers include the same dataset.

"One basic requirement of scientific results is reproducibility: shake an apple tree, and apples will fall downwards each and every time," observes Kai Zhang, an associate professor in the department of statistics and operations research at The University of North Carolina, Chapel Hill. "The problem today is that in many cases, researchers cannot replicate existing findings in the literature and they cannot produce the same conclusions. This is undermining the credibility of scientists and science. It is producing a crisis."

The problem is so widespread that it is now attracting attention at conferences and in academic papers, and even is garnering attention in the mainstream press. While a number of factors contribute to the problem—including experimental errors, publication bias, the improper use of statistical methods, and subpar machine learning techniques—the common denominator is that researchers are finding patterns in data that have no relationship to the real world. As Zhang puts it, "The chance of picking up spurious signals is higher as the nature of data and data analysis changes."

At a time when anti-science sentiment is growing and junk science is flourishing, the repercussions are potentially enormous. If results cannot be trusted, then the entire nature of research and science comes into question, experts say. What is more, all of this is taking place at a time when machine learning is emerging at the forefront of research. A lack of certainty about the validity of results could also lead people to question the value of machine learning and artificial intelligence.

Methods Matter

A simple but disturbing fact is at the center of this problem. Researchers are increasingly starting with no hypothesis and then searching—some might say grasping—for meaningful correlations in data. If the data universe is large enough—and this is frequently the case—there are reasonably good odds that by sheer chance, a valid p-value will appear. Consider: if a person tosses a coin eight times and it lands on heads every time, this is noteworthy; however, if a person tosses a coin 8,000 times and, at some point, the coin lands on heads eight consecutive times, what might appear to be a significant discovery is merely a random event.

The idea that scientific outcomes may be inaccurate or useless is not new. In 2005, John Ioannidis, a professor of health research and policy at Stanford University, published an academic paper titled Why Most Published Findings Are False, in the journal PLOS Medicine. It put the topic of reproducibility of results on the radar of the scientific community. Ioannidis took direct aim at methodologies, study design flaws, and biases. "Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true," he wrote in that paper.

Others took notice. In 2011, Glenn Begley, then head of the oncology division at biopharmaceutical firm Amgen, decided to see if he could reproduce results for 53 foundational papers in oncology that appeared between 2001 and 2011. In the end, he found he could replicate results for only six papers, despite using datasets identical to the originals. That same year, a study by German pharmaceutical firm Bayer found only 25% of studies were reproducible.  .... "