technical paper
Online Training in Scholarly Peer Review: A Systematic Review | VIDEO
keywords:
peer review
Objective To perform a systematic review of available online
training for scholarly peer review of biomedical journal
articles.
Design A search strategy was developed and reviewed using
the PRESS (Peer Review of Electronic Search Strategies)
checklist by a medical librarian. A database search of
MEDLINE, PsycINFO, Embase, ERIC, and Web of Science
was conducted. Additional supplementary searches were
done of preprint servers, Google, YouTube, university library
websites, publisher websites, and peer review–related events
and groups. All English or French training documents for
scholarly peer review of biomedical manuscripts freely
accessible online between January 1, 2012, and the date of the
search (September 13, 2021) were included. January 1, 2012,
was used as the earliest cutoff because this was the year
Publons was launched. A Preferred Reporting Items for
Systematic Reviews and Meta-Analyses (PRISMA) flow
diagram with full exclusion criteria is shown in Figure 11.
Screening was done in duplicate by 2 independent reviewers
in 2 stages, with conflicts resolved by a third party. Data
extraction and risk of bias were done by 1 reviewer and then
verified by a second. As no current risk of bias tool could be
found for evaluating training material, one was created,
which was pilot tested for feasibility.
Figure 11. PRISMA Flow Diagram
Results Of 1244 records screened, 45 online training
documents were identified for peer review. Barriers such as
paywalls and membership requirements limited access to just
more than half of these documents (23 of 45 51%); thus,
they were excluded from data analysis because they were not
freely available online. The included documents were mostly
websites (13 of 22 59%) and videos (6 of 22 27%) offered
exclusively in English (19 of 22 86%). Many of the
documents did not report a creation year (10 of 22 45%),
author information (10 of 22 45%), or funding sources (19
of 22 86%). Countries that developed the greatest amount
of training were the US (8 of 22 36%), United Kingdom (4
of 22 18%), and Germany (3 of 22 14%). The main training
formats were online modules (13 of 22 59%) and webinars
(5 of 22 23%) and took less than 1 hour to complete (15 of
22 68%). Topics that were frequently included were an
overview of the peer review process (18 of 22 82%),
synthesis of a peer review report (20 of 22 91%), and critical
appraisal of data (18 of 22 82%). Conversely, critical
appraisal of clinical trials (4 of 22 18%), statistics (4 of 18
3%), and reporting guidelines (9 of 22 41%) were less
commonly included.
Conclusions This systematic review identified a
comprehensive list of available online training material for
scholarly peer review of biomedical journals and an analysis
of their characteristics.
Conflict of Interest Disclosures David Moher is an associate
director of the Peer Review Congress but was not involved in the
review or decision of this abstract.