technical paper

Peer Review Congress 2022

September 08, 2022

Chicago, United States

An Analysis of the History, Content, and Spin of Abstracts of COVID-19-Related Randomized Clinical Trials Posted as Preprints and Subsequently Published in Peer-Reviewed Journals or Unpublished


pandemic science

quality of reporting


Objective Preprint servers have gained traction in many academic fields. Most preprints are unlikely to adversely affect public health. However, during a pandemic, when there is an urgent need for data, preprints may cause harm by disseminating incomplete, incorrect, or misleading information. 1,2 The aim of this study was to characterize and compare the characteristics, completeness, and spin of the abstracts of all randomized clinical trials (RCTs) related to COVID-19 posted to medRxiv from March 13, 2020, to December 31, 2021. An additional aim was to identify all corresponding published versions of these abstracts and perform a similar qualitative comparison to examine the impact of the peer review process.

Design An experienced librarian identified all COVID-19– related RCT preprints posted to medRxiv and all versions subsequently published in peer-reviewed journals as of June 1, 2022. An assistant created identically formatted Word documents of all abstracts. After training and the confirmation of adequate interrater reliability, 3 blinded reviewers scored individual abstracts presented in random order for completeness using items from CONSORT for Abstracts. They then evaluated blinded medRxiv/published abstract pairs for differences in content and spin, using criteria modified from Boutron et al. 3 Last, the abstracts of all unpublished preprints, along with an equal-sized sample of subsequently published preprints and their published counterparts, were assessed for extent of spin. Analysis was descriptive.

Results Two hundred ninety-one preprints were initially identified; 236 were confirmed as RCTs. Of these, 161 (68%) were found to have associated publications, which were published a median of 126 (IQR, 78-185; range, 0-654) days after medRxiv posting. The 75 unpublished preprints were posted a median of 344 (IQR, 235-429; range, 158-782) days prior to the final search. For most items, abstract completeness was higher in preprints that were subsequently published and was modestly higher still in published form (Table 9). The extent of spin was higher in unpublished preprints than in preprints that were subsequently published (Table 9). Last, of 161 published-preprint abstract pairs studied, 25% had more spin in the preprint version, 8% had more spin in the published version, and 66% had no difference in spin. Conversely, 12% had more consensible preprints, 42% had more consensible published versions, and 45% had no difference.

Conclusions At this time, almost one-third of study medRxiv COVID-19–related RCTs have not been published, although roughly half were posted more than 1 year ago. This subset of unpublished preprints had lower CONSORT compliance and more spin than the medRxiv preprints that went on to be published. This study’s comparison of published-preprint pairs is consistent with other literature that shows that peer review and the publication process improves—but does not eliminate—incomplete reporting and spin. Limitations of this study include the restriction of the search to a single preprint server and the focus on abstracts alone.

References 1. Fraser N, Brierley L, Dey G, et al. The evolving role of preprints in the dissemination of COVID-19 research and their impact on the science communication landscape. PLOS Biology. 2021;19(4):e3000959. doi:10.1371/journal. pbio.3000959

2. Kodvanj I, Homolak J, Virag D, et al. Publishing of COVID-19 preprints in peer-reviewed journals, preprinting trends, public discussion and quality issues. Scientometrics. 2022;127(3):1339-1352. doi:10.1007/s11192-021-04249-7

3. Boutron I, Dutton S, Ravaud P, Altman DG. Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA. 2010;303(20):2058-2064. doi:10.1001/jama.2010.651

Conflict of Interest Disclosures None reported. David L. Schriger is a member of the Peer Review Congress Advisory Board but was not involved in the review or decision for this abstract.

Funding/Support David L. Schriger was supported in part by an unrestricted research grant from the Korein Foundation.


SlidesTranscript English (automatic)

Next from Peer Review Congress 2022

Barriers to Using Research: Reducing Flawed, Inappropriate, and Poorly Reported Research, Paul Glasziou

Barriers to Using Research: Reducing Flawed, Inappropriate, and Poorly Reported Research, Paul Glasziou

Peer Review Congress 2022

Paul Glasziou
Paul Glasziou

08 September 2022

Similar lecture

From amazing work to I beg to differ: analysis of bioRxiv preprints that received one public comment till September 2019
technical paper

From amazing work to I beg to differ: analysis of bioRxiv preprints that received one public comment till September 2019


+1Juan Pablo AlperinMario Malicki
Mario Malicki and 3 other authors

30 September 2020

Stay up to date with the latest Underline news!

Select topic of interest (you can select more than one)


  • All Lectures
  • For Librarians
  • Resource Center
  • Free Trial
Underline Science, Inc.
1216 Broadway, 2nd Floor, New York, NY 10001, USA

© 2023 Underline - All rights reserved