**Objective** Reproducibility is a central tenet of research.
Explicit reproducibility checks are made across different
disciplines trying to assess the replicability of previously
published studies. This scoping review aimed to synthesize
the literature on reproducibility; describe its epidemiological
characteristics, including how reproducibility is defined and
assessed; and determine and compare estimates for
reproducibility across different fields. 
 **Design** All English-language quantitative replication studies
within the fields of economics, education, psychology, health
sciences, and biomedicine published in 2018 or 2019 were
included, as were studies that were explicitly self-described as
a replication or a reproducibility study in which a previously
published quantitative study is referred to and conducted
again. Conference proceedings, commentaries, narrative
reviews, systematic reviews, and clinical case studies were
excluded. MEDLINE, Embase, PsycINFO, Cumulative Index
of Nursing and Allied Health Literature (CINAHL), Education
Source via EBSCOHost, ERIC, EconPapers, International
Bibliography of the Social Sciences, and EconLit were
searched. Retrieved documents were screened in duplicate
against our inclusion criteria. The year of publication, number
of authors, country of affiliation of the corresponding author,
and funding were extracted. For individual replication
studies, whether a registered protocol was used, whether
there was contact between the replicating team and the
original authors, what study design was used, and what the
primary outcome was were extracted from each replication
study. Finally, how replication was defined by the authors and
whether the assessed study or studies successfully replicated
a previous study based on this definition were also extracted.
Extraction was done by 2 reviewers. 
 **Results** The search identified 11,224 unique documents, of
which 47 were included (**Figure 14**). Most studies were
related to either psychology (48.6%) or health sciences
(23.7%). Among these 47 documents, 36 described a single
replication study, while the remaining 11 reported at least 2
replications in the same paper. Less than half of the studies
referred to a registered protocol. There was variability in the
definitions of replication success, with studies related to
psychology and health sciences tending to use a comparison
of effect sizes to define replication success. Based on the
definition used by the authors of each study, 95 of 177 studies
(53.7%) achieved replication success. 
 
![](https://assets.underline.io/uploads/markdown_image/1/image/b5026168df52908f433fcadb0182173c.png) 
 **Conclusions** This study gives an overview of research across
5 disciplines that explicitly set out to replicate previous
research. While estimates of reproducibility vary across fields
in this modest sample, so too do norms in definitions used to
define reproducibility. 
 **Conflict of Interest Disclosures** David Moher is a member of
the Peer Review Congress Advisory Board but was not involved in
the review or decision of this abstract. No other disclosures were
reported. 
 **Additional Information** Alixe Ménard is a patient author. 
 
![](https://assets.underline.io/uploads/markdown_image/1/image/13a99ff23c015e263b9246b81fe3ea56.png)

Characteristics of Studies of Research Reproducibility in Economics, Education, Psychology, Health Sciences, and Biomedicine: A Scoping Review

Open Science, Reproducibility, and Postpublication Peer Review

technical paper

Our aim is to encourage research into the quality and credibility of peer review and scientific publication, to establish the evidence base on which scientists can improve the conduct, reporting, and dissemination of scientific research.
 
 **Welcome!**
 
 The JAMA Network, The BMJ, and METRICS welcome you to the Ninth International Congress on Peer Review and Scientific Publication. We have continued our efforts to broaden the scope of the Congress to all aspects of peer review and publication—from funding to postpublication—and to all sciences.
 
 We will have 3 days for presentations of new research into peer review and all aspects of scientific publication, bias, quality of reporting, and information access and dissemination. There are 50 plenary session research presentations and 5 plenary invited talks. Each plenary session research presentation will be followed by equal time for discussion and questions from the audience. In addition, there are 125 posters. All posters, including virtual posters, are available to view throughout the meeting. In-person posters will also be presented at designated times on September 9 and 10. If you are attending virtually, please visit the individual poster pages and ask your questions in the chat box. See the link to Posters on the left.
 
 We hope you will take an active part in the program, as we depend on your participation in the discussion sessions to make the Congress a success.

 [Peer Review Congress Organizers, Directors, and Advisory Board](https://peerreviewcongress.org/organizers-and-advisory-board/)
 
 [Health and Safety Policies](https://web.cvent.com/event/1b505794-6a0b-428c-a44c-6c46c4c50e99/websitePage:152962bb-4567-4561-9d81-21ebfa97f487)

![](https://assets.underline.io/uploads/markdown_image/1/image/7c9f0143dee383780cc4670269ade132.png)

A PDF of the schedule is available [here](https://drive.google.com/file/d/1-Jd191W-nxbWcN1prdX5o9nlSfeo9_-H/view?usp=sharing) and a flipbook version of the complete program and abstracts is included below. See also the link to the interactive schedule to the left.

<iframe width="700px" height="425px" src="https://online.fliphtml5.com/ebtyf/nlfn/#p=1" frameborder="0" allowfullscreen allowtransparency></iframe>

Registration is required to attend this event, please use the URL linked below to secure your ticket. Registration will close on Friday, September 2 at 5:00 PM CT. 

[**Register Here**]( https://web.cvent.com/event/1b505794-6a0b-428c-a44c-6c46c4c50e99/summary)

Peer Review Congress 2022

Our aim is to encourage research into the quality and credibility of peer review and scientific publication, to establish the evidence base on which scientists can improve the conduct, reporting, and dissemination of scientific research.



David Moher

1

2

Presentations

Characteristics of Studies of Research Reproducibility in Economics, Education, Psychology, Health Sciences, and Biomedicine: A Scoping Review

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES