Objective One of the changes to the peer review process introduced by open science is the participation of reviewers who were not selected by editors or not indicated by authors in the evaluation of the manuscripts submitted for publication. Previous studies characterizing open peer review and its traits have called this phenomenon “open participation,”1 “crowdsourced peer review,”2 and “public peer review.”3 This study aimed to identify and describe from the literature what peers, concepts, and models of peer review are associated with open participation in scientific journals.

Design An integrative review was conducted. The search strategy was used with Scopus, Web of Science, and PubMed. The literature was retrieved in March 2021 and was limited to the languages English, Portuguese, and Spanish but with no time limitations. All data were collected from the final sample. Models of open peer review with open participation were considered as those that contemplated the participation of the public. These models were characterized according to the terminology of open peer review established by Ross- Hellauer.1 Self-appointed reviewers were regarded as a new type of reviewer. The concepts around open peer review models with open participation were understood to be the foundation for its implementation.

Results A total of 562 studies were retrieved. However, 407 remained after removing duplicates, and 20 met the inclusion criteria. These studies were published in English from 1998 to 2018. Of these 20 studies, 9 (45%) presented implemented models, 7 (35%) discussed public participation and comments, 2 (10%) proposed a model, 1 (5%) approached the quality of the manuscript selection process, and 1 (5%) analyzed the reliability of peer review in open participation. Six open peer review models with open participation were found. These models were totally open or combined with the traditional peer review to discuss manuscripts publicly. These models kept the steps of the traditional peer review process or were divided into 2 or more stages (Table 71). The peer reviewers were the readers of the journal, reviewers invited by the editor or author, or patients. The studies that focused on public participation and on comments were questioning the expertise of this audience to evaluate manuscripts, the incentives to comment, and the potential effect these models may have on the careers of researchers. Public comments may help to improve the quality of the publication and of the manuscript selection process, but they might not increase interrater reliability among reviewers. Open access, crowdsourcing, interaction, and transparency were the main concepts of the models of open peer review with open participation.

Conclusions Open peer review models with open participation have different degrees of openness, are expanding the idea of who a peer reviewer can be, and are bringing new challenges to the peer review process. Furthermore, open identities are a sensitive aspect of these models, for which the concepts are aligned with open science values.

References 1. Ross-Hellauer T. What is open peer review? a systematic review. F1000Res. 2017;6:588. doi:10.12688/ f1000research.11369.2

2. Ford E. Defining and characterizing open peer review: a review of the literature. J Scholarly Publishing. 2013;44(4):311-326. doi:10.3138/jsp.44-4-001

3. Pöschl U. Multi-stage open peer review: scientific evaluation integrating the strengths of traditional peer review with the virtues of transparency and self-regulation. Front Comput Neurosci. 2012;6:33. doi:10.3389/ fncom.2012.00033

Conflict of Interest Disclosures None reported.