In a time when we have to rely on clinical trials for COVID-19 drugs and vaccines, a new study in the Proceedings of the National Academy of Sciences (PNAS) brings good news about the credibility of registered clinical trials.
The authors are two Bocconi Professors of Economics, Jerome Adda and Marco Ottaviani, and a former MSc student of theirs, Christian Decker, now a Ph.D. candidate at the University of Zurich.
In a clinical trial, statistical significance is a key prerequisite for marketing approval of new drugs. Under a certain threshold of significance, the results of the trial could, in fact, be due to chance and not to the efficacy of the drug. Given the research costs involved and the lure of large potential profits by the pharmaceutical companies sponsoring the trial, investigators may be pressured to beautify data, a standard argument goes. Previous studies of results of statistical tests reported in scientific journals across a number of disciplines did actually detect an anomalous concentration of significance values immediately above the significance threshold, raising suspicions of selective reporting as well as manipulation.
Adda, Ottaviani and Decker concentrated their attention on the trials registered in ClinicalTrials.gov, the largest registry in the world, and focused on the differences in the results of phase II and phase III trials of the same drug. In phase II a drug’s efficacy is initially established in a small sample of people (usually in the low hundreds); in phase III safety and efficacy are then confirmed in a larger group of volunteers (usually in the low thousands). Reasonable expectations are that the statistical significance of phase II will be confirmed in phase III and that trials that record a significance level under the threshold in phase II will be suspended.