Objective This study aimed to describe and explore the content of comments received by preprints posted in 2020 to bioRxiv and medRxiv. It extended a previous study1 thatassessed comments on bioRxiv until 2019, considering the increased attention received by preprints after the beginning of the COVID-19 pandemic.2

Design This was an observational qualitative study. The list of preprints posted to bioRxiv and medRxiv and the number of comments received by them were obtained using each platform’s application program interface. After excluding preprints with no comments and those with more than 20 comments (for feasibility and avoiding overrepresentation of individual preprints), eligible preprints were randomly sampled and their comments were assigned to 3 reviewers. Content analysis of comments was based on a structured form, with predefined categories based on previous research on preprint comments1 and on a scoping review of peer review.3 If 2 of 3 reviewers agreed, the classification was considered final; in case all 3 disagreed, a fourth reviewer made the final call. Neither manuscripts nor the comment appropriateness or validity in regard to the manuscript text were assessed.

Results A total of 1037 preprints were assessed, with a total of 1921 comments (mean SD, 1.8 1.9 comments per preprint); 439 comments (22.8%) were replies to previous comments, leaving 1482 comments available for classification based on our prespecified categories (Table 19). A total of 14.6% (95% CI, 12.7%-16.5%) of reader comments were considered not to be about the content of the preprints (n = 192). Comments from preprint authors (n = 165) accounted for 11.1% (95% CI, 9.5%-12.7%) of the remaining comments, and mostly addressed publication status (n = 89); 38% (95% CI, 35.2%-40.9%) of reader (nonauthor) comments included compliments (n = 428), 61.7% (95% CI, 58.8%- 64.5%) included criticisms, corrections, or suggestions (n = 694), and 34.9% (95% CI, 32.1%-37.7%) included questions (n = 393). Although compliments were mostly general (n = 279), criticisms, corrections, or suggestions largely addressed specific points, such as interpretation (n = 286), methodologic design (n = 267), materials/data collection methods (n = 238), and analyses (n = 228). Most questions were about information not provided in the preprint (n = 170) or about materials/data collection (n = 166).



Conclusions As shown before,1 only a small percentage of preprints received comments on their respective preprint platforms, but this study found that the content of these comments showed similarities with the type of content typically expected from journal-elicited peer review as previously described.3 These results may help generate hypotheses to inform future research on preprints and peer review.

References 1. Malički M, Costello J, Alperin JP, Maggio LA. Analysis of single comments left for bioRxiv preprints till September 2019. Biochem Med (Zagreb). Published online April 15, 2021. doi:10.11613/BM.2021.020201

2. Fraser N, Brierley L, Dey G, et al. The evolving role of preprints in the dissemination of COVID-19 research and their impact on the science communication landscape. PLoS Biol. Published online April 2, 2021. doi:10.1371/journal. pbio.3000959

3. Glonti K, Cauchi D, Cobo E, Boutron I, Moher D, Hren D. A scoping review on the roles and tasks of peer reviewers in the manuscript review process in biomedical journals. BMC Med. Published online June 20, 2019. doi:10.1186/s12916-019- 1347-0

Conflict of Interest Disclosures None reported.

Funding/Support Clarissa França Dias Carneiro received a scholarship from the Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq). Olavo B. Amaral received funding from FAPERJ (E-26/200.824/2021), CNPq (310813/2021-2), and the Serrapilheira Institute.

Role of the Funder/Sponsor The funders had no direct role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the abstract; and decision to submit the abstract for presentation.

Acknowledgment The authors thank Richard Abdill for his contributions to data collection.

Additional Information Mario Malički and Olavo B. Amaral are co–corresponding authors.