Objective In open transparent peer review, the authors and reviewers know each other’s identities, and published articles are accompanied by the signed reviewer comments. Annals of Internal Medicine (Annals) uses single-blind review— reviewers know authors’ identities, but reviewers’ identities are not disclosed to authors and reviews are not published. The objective of this survey was to examine whether moving to more open peer review could affect the willingness of Annals peer reviewers to review for the journal or the nature of their comments.

Design The authors conducted an online survey of 5977 persons who reviewed for the journal from January 1, 2019, to January 1, 2022. In addition to information on age, specialty, and professional setting and experience, the survey asked how likely (on a 5-point scale) respondents would be to continue to review for Annals if they were asked to sign their comments for authors, if the published article identified the reviewers, and if the reviews accompanied published articles. In addition, the survey asked whether and how open review would influence the nature of their comments.

Results Of 1421 respondents (24% response rate), 71% were more than 10 years past completing training, 48% reported social media, in policy documents, or by news articles2); however, their association with reviewer scores has not been explored.3 It is unclear whether reviewer ratings more closely reflect scientific impact (ie, citations) or social impact (altmetric scores), and which metric is more valid to reflect reviewers’ priority ratings.

Design This was a longitudinal bibliometric cohort study that followed 451 original research articles published in 2018 in the Journal of Medical Internet Research, a health services research and medical informatics journal (impact being researchers or clinician researchers, 61% peer reviewed at least 5 times in a typical year, and 66% had authored more than 20 peer-reviewed publications. Approximately one-third reported that they would be unlikely to review if Annals adopted an open review policy (Table 45). Whereas 42% reported that this model would not affect their comments, 10% reported that their reviews would be less detailed and 28% reported that their reviews would be less critical. However, 17% reported that their comments would be more detailed, 3% would be more critical of the article, and 20% responded with personal answers to the question.

Table 45. Willingness of Respondents to Continue to Review With an Open Review Model

Conclusions An open review model could adversely affect the willingness of current Annals peer reviewers to continue to review and could alter the nature of reviewer comments. 1Annals of Internal Medicine, American College of Physicians, Philadelphia, PA, USA, jjackson@acponline.org

Conflict of Interest Disclosures Christine Laine is a member of the Peer Review Congress advisory board but was not involved in the review or decision of this abstract. No other disclosures reported.