Peer Review Congress 2022

September 11, 2022

Chicago, United States

Assessment and Comparison of Preprints and Peer-Reviewed Publications of Reporting Characteristics of Randomized Clinical Trials of Pharmacologic Treatment for COVID-19


data sharing and access

pandemic science

quality of reporting

Objective Due to the pandemic, preprint servers contained up to 25% more trials, whereas some medical journals accelerated their editorial processes to ensure the rapid dissemination of findings.1 Concerns regarding quality and transparency rose.2 This meta-study assessed the transparency, completeness, and consistency of COVID-19 reports and whether there was an improvement after journal peer review.

Design The Cochrane COVID-19 Study Register and L·OVE COVID-19 platform were searched to identify all reports (preprints or peer-reviewed publications) of randomized clinical trials (RCTs) assessing pharmacologic interventions for the treatment of COVID-19, up to May 31, 2021. A standardized, online data-extraction form was developed. Data extraction covered general trial characteristics, transparency indicators (eg, trial registration and data sharing statement), completeness of reporting (eg, 10 of the most important Consolidated Standards of Reporting Trials CONSORT 2010 items3 ), and changed outcomes between report and registry (ie, switched outcomes). Two reviewers were trained and assessed 20 trials separately, with an agreement of 96.6% and a κ coefficient of 0.87. A third person resolved disagreements, when needed. A single reviewer assessed all remaining trials. For all trials published first as a preprint, a systematic search was performed for a subsequent publication in peer-reviewed journals, up to October 7, 2021. In a second step, the peer-reviewed journal publication was assessed and compared with the matched preprint.

Results A total of 251 trial reports were identified: 121 peer-reviewed journal publications (48%) and 130 preprints (52%). Approximately half of the trials were prospectively registered (140 56%); 38% (n = 95) made their full protocols or statistical analysis plan available, and 29% (n = 72) provided access to their statistical analysis plan report. A data sharing statement was available in 68% of the reports (n = 170); 91% stated their willingness to share. Only 32% of the trials (n = 81) completely defined the prespecified primary outcome measures; 57% (n = 143) reported the process of allocation concealment. Overall, 51% (n = 127) adequately reported results for the primary outcomes, whereas only 14% of the trials (n = 36) adequately described harms. Primary outcome(s) reported in trial registries and published reports were inconsistent in 49% of the trials (n = 104); only 15% (n = 16) disclosed outcome switching in the report. There were no major differences between preprints and peer-reviewed publications. Of the 130 RCTs published as a preprint, 78 were then published in a peer-reviewed journal (median delay to journal publication, 94 days; IQR, 55-168 days). There was no major improvement after the peer review process.

Conclusions Lack of transparency, completeness, and consistency of reporting are important barriers to trust, interpretation, and synthesis in COVID-19 clinical trials. Peer-reviewed publications did not report the items assessed better than preprints. The comparison of paired reports published as a preprint and as a peer-reviewed publication did not indicate major improvements.


  1. Else H. How a torrent of COVID science changed research publishing—in 7 charts. Nature. 2020;588(7839):553. doi:10.1038/d41586-020-03564-y
  2. Besançon L, Peiffer-Smadja N, Segalas C, et al. Open science saves lives: lessons from the COVID-19 pandemic. BMC Med Res Methodol. 2021;221(1):117. doi:10.1186/ s12874-021-01304-y
  3. Chauvin A, Ravaud P, Moher D, et al. Accuracy in detecting inadequate research reporting by early career peer reviewers using an online CONSORT-based peer-review tool (COBPeer) versus the usual peer-review process: a cross-sectional diagnostic study. BMC Med. 2019;17(1):205. doi:10.1186/ s12916-019-1436-0

Conflict of Interest Disclosures Isabelle Boutron is a member of the Peer Review Congress Advisory Board but was not involved in the editorial review or decision for this abstract. No other disclosures were reported.

Funding/Support This study was part of the COVID-NMA initiative, which received funding from Université de Paris, Assistance Publique Hôpitaux de Paris, INSERM, Cochrane France (Ministry of Health), the French Ministry of Higher Education and Research, Agence Nationale de la Recherche, and the World Health Organization.

Next from Peer Review Congress 2022

Day and Time of Submissions of Manuscripts to the Journal of Paediatrics and Child Health Before and During the COVID-19 Pandemic

Day and Time of Submissions of Manuscripts to the Journal of Paediatrics and Child Health Before and During the COVID-19 Pandemic

Peer Review Congress 2022

Richard G McGee

11 September 2022

Similar lecture

Remodeling Peer Review In Light Of Preprints
technical paper

Remodeling Peer Review In Light Of Preprints


Jessica Polka
Jessica Polka

01 October 2020

Stay up to date with the latest Underline news!

Select topic of interest (you can select more than one)


  • All Lectures
  • For Librarians
  • Resource Center
  • Free Trial
Underline Science, Inc.
1216 Broadway, 2nd Floor, New York, NY 10001, USA

© 2023 Underline - All rights reserved