United States

&lt;p&gt;Olmo R. van den Akker,&lt;sup&gt;1,2&lt;/sup&gt; Marcel A. L. M. van Assen,&lt;sup&gt;1&lt;/sup&gt; Marjan Bakker,&lt;sup&gt;1&lt;/sup&gt; Jelte M. Wicherts&lt;sup&gt;1&lt;/sup&gt;&lt;/p&gt;
&lt;h4&gt;Objective &lt;/h4&gt;
&lt;p&gt;Preregistration,&lt;sup&gt;1&lt;/sup&gt; the practice in which researchers publish their hypotheses, study design, and/or analysis plan before collecting or analyzing their data, has become more and more established in the scientific ecosystem even though the evidence that it prevents bias has been lacking. This set of studies was conducted to assess the effectiveness of preregistration to reduce bias in psychology.&lt;/p&gt;
&lt;h4&gt;Design &lt;/h4&gt;
&lt;p&gt;In the first study, conducted from 2021 to 2024, we assessed the consistency of preregistrations and their corresponding publications (published in 40 psychology journals). These preregistered publications were published and awarded a preregistration badge by a journal or a preregistration challenge prize by the Center for Open Science from 2017 to 2020. We also assessed whether any inconsistencies were disclosed by the authors. In the second study, we compared a subset of preregistration-publication pairs for which a preregistered statistical result could be extracted with a set of nonpreregistered studies (from 37 journals) that were selected for comparability via the Web of Science related records function. Using multilevel regressions, we examined whether preregistration was associated with &lt;i&gt;P&lt;/i&gt; hacking, operationalized as a lower proportion of statistical results and smaller effect sizes. We also looked at associations with other research characteristics, such as the number of citations, sample sizes, and the frequency of statistical errors (assessed via Statcheck&lt;sup&gt;2&lt;/sup&gt;).&lt;/p&gt;
&lt;h4&gt;Results &lt;/h4&gt;
&lt;p&gt;The sample included 300 preregistration-publication pairs as well as a subset of 193 preregistration-publications pairs matched with 193 nonpreregistered studies. We found inconsistencies for 6 important study elements (operationalizations of measured, manipulated, and dependent variables, data collection procedures, statistical model, and inference criteria [&lt;b&gt;Table 25-0947&lt;/b&gt;]). The comparison between preregistered and nonpreregistered studies did not indicate an association between preregistration and statistical significance (β = 0.01; 99% CI, −0.56 to 0.59; &lt;i&gt;P&lt;/i&gt; = .96), effect sizes (β = −0.04; 99% CI, −0.12 to 0.04; &lt;i&gt;P&lt;/i&gt; = .18), and statistical errors (β = −1.19; 95% CI, −2.51 to 0.13; &lt;i&gt;P&lt;/i&gt; = .08). However, preregistered studies typically had higher sample sizes (959.6 vs 536.6; β = 0.45; 99% CI, 0.14 to 0.76; &lt;i&gt;P&lt;/i&gt; &amp;lt; .001) and more citations (18.3 vs 15.1; β = 0.20; 95% CI, 0.01 to 0.40; &lt;i&gt;P&lt;/i&gt; = .04).&lt;/p&gt;

![](https://assets.underline.io/markdown_image/1/image/f985f58c82af47fcce202e540e6513e4.png)

&lt;h4&gt;Conclusions &lt;/h4&gt;
&lt;p&gt;Ideally, publications would be concordant with their preregistrations and any deviations would be reported and explained. In these studies, this ideal was often not met. Preregistration was associated with higher statistical power and more subsequent citations, but it did not appear to prevent &lt;i&gt;P&lt;/i&gt; hacking. The studies were limited by their focus on psychology, and generalizations to other fields should therefore be avoided (although economics and political science had comparable results&lt;sup&gt;3&lt;/sup&gt;).&lt;/p&gt;
&lt;h4&gt;References&lt;/h4&gt;
&lt;p&gt;1. Rice DB, Moher D. Curtailing the use of preregistration: a misused term. &lt;i&gt;Perspect Psychol Sci&lt;/i&gt;. 2019;14(6):1105-1108. doi:10.1177/1745691619858427&lt;/p&gt;
&lt;p&gt;2. Nuijten MB, Hartgerink CH, Van Assen MA, Epskamp S, Wicherts JM. The prevalence of statistical reporting errors in psychology (1985-2013). &lt;i&gt;Behav Res Methods&lt;/i&gt;. 2016;48:1205-1226. doi:10.3758/s13428-015-0664-2&lt;/p&gt;
&lt;p&gt;3. Ofosu GK, Posner DN. Pre-analysis plans: an early stocktaking. &lt;i&gt;Perspect Polit&lt;/i&gt;. 2023;21(1):174-190. doi:10.1017/S1537592721000931&lt;/p&gt;
&lt;sup&gt;1&lt;/sup&gt;Tilburg University, Tilburg, The Netherlands, ovdakker@gmail.com; &lt;sup&gt;2&lt;/sup&gt;QUEST Center for Responsible Research, Berlin Institute of Health at Charité, Berlin, Germany.&lt;/p&gt;
&lt;h4&gt;Conflict of Interest Disclosures&lt;/h4&gt;
&lt;p&gt; None reported.&lt;/p&gt;
&lt;h4&gt;Funding/Support&lt;/h4&gt;
&lt;p&gt; The work in this presentation was supported by a Consolidator Grant (IMPROVE) from the European Research Council (grant number 726361).&lt;/p&gt;
&lt;h4&gt;Additional Information&lt;/h4&gt;
&lt;p&gt; Both studies presented in this abstract were preregistered: the assessment of preregistration effectiveness at &lt;a href=&quot;https://osf.io/qbhsv)&quot;&gt;&lt;span class=&quot;Hyperlink CharOverride-6&quot;&gt;https://osf.io/qbhsv&lt;/span&gt;&lt;/a&gt;, and the comparison of preregistered with nonpreregistered studies at &lt;a href=&quot;https://osf.io/7w5g2&quot;&gt;&lt;span class=&quot;Hyperlink CharOverride-6&quot;&gt;https://osf.io/7w5g2&lt;/span&gt;&lt;/a&gt;.&lt;/p&gt;


The 10th Peer Review Congress

Effectiveness of Preregistration in Psychology

preregistration of studies

quality of reporting

bias

The Tenth Peer Review Congress featured 3 days for presentations of new research into peer review and all aspects of scientific publication, bias, quality of reporting, and information access and dissemination.

This year’s meeting was hybrid, and all plenary sessions were livestreamed with opportunities to view presentations and engage with participants in person and virtually during and after the meeting. All recordings are now available on Underline for viewing.

<div style="position:relative;padding-top:max(60%,324px);width:100%;height:0;"><iframe style="position:absolute;border:none;width:100%;height:100%;left:0;top:0;" src="https://online.fliphtml5.com/ebtyf/mget/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe></div>


#### Why? 
Our aim is to encourage research into the quality and credibility of peer review and scientific publication, to establish the evidence base on which scientists can improve the conduct, reporting, and dissemination of scientific research.

#### Who? 
Participants include editors and publishers of scientific peer-reviewed journals, researchers, funders, bibliometric and informatics experts, information innovators, librarians, journalists, policymakers, ethicists, scientific information disseminators, and anyone interested in the progress of the scientific information enterprise and the quality of scientific evidence.

The Congress embraces a wide range of disciplines, including biomedicine, health and life sciences, applied sciences, basic sciences, physical and chemical sciences, mathematics, computer sciences, engineering, economics, and social sciences as well as new and emerging disciplines.

#### What? 
The Congress provides a forum for the presentation and discussion of new research into the quality and credibility of editorial and funding peer review, scientific and scholarly publication, research methods and reporting practices, identification and management of biases, transparency, reproducibility, access, publication and dissemination metrics, identification and management of threats to the integrity of science, and advances in information exchange.

**Follow us**

[![](https://assets.underline.io/markdown_image/1/image/662a22766b9788ed3766cb23cd156e32.png)](https://x.com/peerrevcongress)  [![](https://assets.underline.io/markdown_image/1/image/bb105afb1773a01fa169aa97ca1af49a.png)](https://www.youtube.com/channel/UCsG_0nKi3_Zfyv0yRPdxjWQ)  [![](https://assets.underline.io/markdown_image/1/image/034e1e70a194a9a4178fdf470ef796db.png)](https://bsky.app/profile/peerreviewcongress.bsky.social)



You need to log in with the email address you registered with. 



Please log in to explore this event.

Registration is required to attend this event, please [here](https://peerreviewcongress.org/registration-information/) to secure your ticket. Please check your spam/promotional inbox too.



Please register!

The Tenth Peer Review Congress features 3 days for presentations of new research into peer review and all aspects of scientific publication, bias, quality of reporting, and information access and dissemination.

<p>Olmo R. van den Akker,<sup>1,2</sup> Marcel A. L. M. van Assen,<sup>1</sup> Marjan Bakker,<sup>1</sup> Jelte M. Wicherts<sup>1</sup></p>
<h4>Objective </h4>
<p>Preregistration,<sup>1</sup> the practice in which researchers publish their hypotheses, study design, and/or analysis plan before collecting or analyzing their data, has become more and more established in the scientific ecosystem even though the evidence that it prevents bias has been lacking. This set of studies was conducted to assess the effectiveness of preregistration to reduce bias in psychology.</p>
<h4>Design </h4>
<p>In the first study, conducted from 2021 to 2024, we assessed the consistency of preregistrations and their corresponding publications (published in 40 psychology journals). These preregistered publications were published and awarded a preregistration badge by a journal or a preregistration challenge prize by the Center for Open Science from 2017 to 2020. We also assessed whether any inconsistencies were disclosed by the authors. In the second study, we compared a subset of preregistration-publication pairs for which a preregistered statistical result could be extracted with a set of nonpreregistered studies (from 37 journals) that were selected for comparability via the Web of Science related records function. Using multilevel regressions, we examined whether preregistration was associated with <i>P</i> hacking, operationalized as a lower proportion of statistical results and smaller effect sizes. We also looked at associations with other research characteristics, such as the number of citations, sample sizes, and the frequency of statistical errors (assessed via Statcheck<sup>2</sup>).</p>
<h4>Results </h4>
<p>The sample included 300 preregistration-publication pairs as well as a subset of 193 preregistration-publications pairs matched with 193 nonpreregistered studies. We found inconsistencies for 6 important study elements (operationalizations of measured, manipulated, and dependent variables, data collection procedures, statistical model, and inference criteria [<b>Table 25-0947</b>]). The comparison between preregistered and nonpreregistered studies did not indicate an association between preregistration and statistical significance (β = 0.01; 99% CI, −0.56 to 0.59; <i>P</i> = .96), effect sizes (β = −0.04; 99% CI, −0.12 to 0.04; <i>P</i> = .18), and statistical errors (β = −1.19; 95% CI, −2.51 to 0.13; <i>P</i> = .08). However, preregistered studies typically had higher sample sizes (959.6 vs 536.6; β = 0.45; 99% CI, 0.14 to 0.76; <i>P</i> &lt; .001) and more citations (18.3 vs 15.1; β = 0.20; 95% CI, 0.01 to 0.40; <i>P</i> = .04).</p>

![](https://assets.underline.io/markdown_image/1/image/f985f58c82af47fcce202e540e6513e4.png)

<h4>Conclusions </h4>
<p>Ideally, publications would be concordant with their preregistrations and any deviations would be reported and explained. In these studies, this ideal was often not met. Preregistration was associated with higher statistical power and more subsequent citations, but it did not appear to prevent <i>P</i> hacking. The studies were limited by their focus on psychology, and generalizations to other fields should therefore be avoided (although economics and political science had comparable results<sup>3</sup>).</p>
<h4>References</h4>
<p>1. Rice DB, Moher D. Curtailing the use of preregistration: a misused term. <i>Perspect Psychol Sci</i>. 2019;14(6):1105-1108. doi:10.1177/1745691619858427</p>
<p>2. Nuijten MB, Hartgerink CH, Van Assen MA, Epskamp S, Wicherts JM. The prevalence of statistical reporting errors in psychology (1985-2013). <i>Behav Res Methods</i>. 2016;48:1205-1226. doi:10.3758/s13428-015-0664-2</p>
<p>3. Ofosu GK, Posner DN. Pre-analysis plans: an early stocktaking. <i>Perspect Polit</i>. 2023;21(1):174-190. doi:10.1017/S1537592721000931</p>
<sup>1</sup>Tilburg University, Tilburg, The Netherlands, ovdakker@gmail.com; <sup>2</sup>QUEST Center for Responsible Research, Berlin Institute of Health at Charité, Berlin, Germany.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> None reported.</p>
<h4>Funding/Support</h4>
<p> The work in this presentation was supported by a Consolidator Grant (IMPROVE) from the European Research Council (grant number 726361).</p>
<h4>Additional Information</h4>
<p> Both studies presented in this abstract were preregistered: the assessment of preregistration effectiveness at <a href="https://osf.io/qbhsv)"><span class="Hyperlink CharOverride-6">https://osf.io/qbhsv</span></a>, and the comparison of preregistered with nonpreregistered studies at <a href="https://osf.io/7w5g2"><span class="Hyperlink CharOverride-6">https://osf.io/7w5g2</span></a>.</p>


poster

<p>Cecilie Jespersen,<sup>1,2</sup> Zexing Song,<sup>3</sup> An-Wen Chan,<sup>3,4</sup> Asbjørn Hróbjartsson<sup>1,2</sup></p>
<h4>Objective </h4>
<p>Observational studies of interventions use causal inference to assess the impact of interventions on health-related outcomes.<sup>1</sup> Despite concerns about reporting bias, observational studies are not subject to the same registration requirements as clinical trials.<sup>2,3</sup> We aimed to determine the prevalence of registration among published observational studies of interventions, assess the association between registration and study characteristics, analyze journal registration policies, and explore authors’ and editors’ attitudes about registration.</p>
<h4>Design </h4>
<p>We conducted a meta-epidemiologic cross-sectional study triangulating data from 4 sources. First, we searched PubMed for observational studies published in 2023. Eligible studies were cohort or case-control studies with a control group that assessed causal effects of health interventions. Corresponding registration information was collected. Second, authors of included studies were surveyed to explore reasons for and barriers to registration. Third, editorial policies were sampled from 40 journals: 20 sample-representative journals and the journals ranked in the top 20 in Journal Citation Reports by 2023 Journal Impact Factor across 8 specialty categories. Fourth, 1 editor per journal was invited to share their perspectives on registration. Primary outcomes were the prevalence of registered observational studies of interventions published in 2023 and the estimated association between registration and study characteristics, assessed by multivariable logistic regression. Sample size was estimated based on an expected 15% registration rate.</p>
<h4>Results </h4>
<p>Among 1100 screened studies, 200 were included: 69 and 128 cohort studies with prospective and retrospective data collection, respectively, and 3 case-control studies. In total, 28 (14%) were registered, and 17 of these (61%) were prospectively registered (&lt;1 month of their start date) (<b>Table 25-1069</b>). Prospective design and protocol availability were positively associated with registration (retrospective vs prospective cohort: odds ratio [OR], 0.19 [95% CI, 0.07-0.54]; <i>P</i> = .002; no public protocol vs public protocol: OR, 0.04 [95% CI, 0.01-0.23]; <i>P</i> &lt; .001). The survey response rate was 23% (46 responses); 60% of authors supported registration, although many only when registration was deemed relevant. Identified barriers included lack of journal requirements for registration (56%) and limited resources (62%). None of the journal policies explicitly required registration of observational studies of interventions, while 12 (30%) encouraged it. Journals that encouraged registration had a higher 2023 Journal Impact Factor and more frequently encouraged public protocols. Editors had divergent opinions on registration. While some considered it to be worthwhile, just as many questioned the added value.</p>

![](https://assets.underline.io/markdown_image/1/image/99ddb5f59ea342d2117443673974b4db.png)

<h4>Conclusions </h4>
<p>Only 1 in 7 contemporary observational studies of interventions were registered, although more often in cohort studies with prospective data collection and studies with a publicly available protocol. Authors identified the lack of journal requirements to registration as a key registration barrier, and only one-third of journals had supportive policies. Clearer guidance and journal policies on registration relevance (discriminating hypothesis-testing and hypothesis-generating studies) may reduce the risk of reporting biases in observational studies of interventions.</p>
<h4>References</h4>
<p>1. Hernán MA, Wang W, Leaf DE. Target trial emulation: a framework for causal inference from observational data. <i>JAMA</i>. 2022;328(24):2446-2447. doi:10.1001/jama.2022.21383</p>
<p>2. Williams RJ, Tse T, Harlan WR, Zarin DA. Registration of observational studies: is it time? <i>CMAJ</i>. 2010;182(15):1638-1642. doi:10.1503/cmaj.09225</p>
<p>3. Leducq S, Zaki F, Hollestein LM, et al. The majority of observational studies in leading peer-reviewed medicine journals are not registered and do not have a publicly accessible protocol: a scoping review. <i>J Clin Epidemiol</i><span lang="da-DK">. 2024;170:111341. doi:10.1016/j.jclinepi.2024.111341</span></p>
<sup>1</sup>Cochrane Denmark &amp; Centre for Evidence-Based Medicine Odense (CEBMO), University of Southern Denmark, Odense, Denmark, ceciliejespersen@health.sdu.dk; <sup>2</sup>Open Patient data Explorative Network (OPEN), Odense University Hospital, Odense, Denmark; <sup>3</sup>Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada; <sup>4</sup>Women’s College Research Institute, Dept. of Medicine, University of Toronto, Toronto, Ontario, Canada.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> An-Wen Chan is a member of the Peer Review Congress Advisory Board but was not involved in the review or decision for this abstract. No other disclosures were reported.</p>
<h4>Acknowledgments</h4>
<p> We thank all researchers who participated in the author survey for their valuable contribution to the findings of this study.</p>


Registration of Observational Studies of Interventions: Prevalence, Characteristics, and Journal Policies

<p>Gunther Eysenbach<sup>1</sup></p>
<h4>Objective</h4>
<p> Registered Reports (RRs) refer to the publication of a study that is published in 2 stages: a protocol (RR stage 1 or RR1) and a results paper (RR2). Some journals have adopted the RR system and guarantee acceptance of subsequent results articles published in the same journal after the protocol is peer reviewed. However, in a distributed open science ecosystem, protocols may already be peer reviewed and published elsewhere, but no standardized system exists to link protocols to subsequent results articles and vice versa. We implemented RRs across the publisher portfolio, with 1 dedicated journal to peer review and publish protocols, and the publisher guaranteeing acceptance of RR2s in 1 of its other journals independently based on whether the results were negative or positive. We propose a machine- and human-readable mechanism to link RR2s with RR1s to assist in peer review and to enhance transparency, accountability, and reproducibility.</p>
<h4>Design</h4>
<p> In 2018, we proposed and implemented a cross-journal, DOI-based, persistent identifier called an International Registered Report Identifier (IRRID) published in article abstracts. An RR2 references an RR1 using an identifier that is based on the DOI of the protocol. For example, RR2-10.2196/24264 indicates that the publication is a results article for a protocol that was previously published under the DOI 10.2196/24264.<sup>1</sup> Protocols that are peer reviewed contain the IRRIDs in the format [DE|P]RR1-[DOI], where [DOI] is the DOI of the protocol itself, and DE or P qualifiers indicate whether the protocol was written before [P] or after [DE] data collection. On submission of a protocol, authors were asked if the protocol was submitted before or after data were collected. Authors were incentivized to register their protocol by being offered a 20% discount on the Article Processing Charge on subsequent results articles.</p>
<h4>Results</h4>
<p> A total of 3995 articles were published with IRRIDs between 2018 and February 13, 2025, of which 3240 (81%) were protocols (RR1). Among the protocols, 2151 (66%) were published when data already existed (DERR1), and 917 (28%) were published before data were collected (PRR1). A total of 732 results articles had an RR2 identifier indicating previous protocol publication, although these protocols were not always peer reviewed (eg, OSF or <i>BMJ Open</i>). A third-party audit found that outcome switching and undeclared deviations from the protocol still remain a problem.<sup>2</sup></p>
<h4>Conclusions</h4>
<p> We propose that other journals adopt IRRIDs to help identify protocol and results article pairs that together form RRs across journals (<a href="https://irridregistry.org/"><span class="Hyperlink CharOverride-6">https://irridregistry.org/</span></a>).</p>
<h4>References</h4>
<p>1. What is an International Registered Report Identifier (IRRID)? JMIR Publications Knowledge Base and Help Center. Accessed February 13, 2025. <a href="https://support.jmir.org/hc/en-us/articles/360003797672-What-is-an-International-Registered-Report-Identifier-IRRID"><span class="Hyperlink CharOverride-6">https://support.jmir.org/hc/en-us/articles/360003797672-What-is-an-International-Registered-Report-Identifier-IRRID</span></a></p>
<p>2. Anthony N, Tisseaux A, Naudet F. Published registered reports are rare, limited to one journal group, and inadequate for randomized controlled trials in the clinical field. <i>J Clin Epidemiol</i>. 2023;160:61-70. doi:10.1016/j.jclinepi.2023.05.016</p>
<sup>1</sup>JMIR Publications, Toronto ON, Canada, geysenba@gmail.com.</p>
<h4>Conflict of Interest Disclosure </h4>
<p>Gunther Eysenbach has an equity stake in and receives a salary from JMIR Publications.</p>


International Registered Reports Identifiers (IRRIDs): 7 Years of Experiences

<p>Megan Curtin,<sup>1</sup> Allisun Wiltshire,<sup>1</sup> Brix Kowalski,<sup>2</sup> Maximilian J. Siebert<sup>3</sup></p>
<h4>Objective</h4>
<p> Antimicrobial resistance (AMR) is one of the leading causes of death globally. Timely and complete reporting of clinical trials involving antimicrobial agents (AMAs) is essential to evaluate the safety and efficacy of potential therapies for public use. This study investigates the quality of results reporting of applicable clinical trials (ACTs) and probable ACTs (pACTs) for AMAs. ACTs are interventional studies (excluding phase 1) regulated by the US Food and Drug Administration with at least one site based in the United States. pACTs adhere to the same criteria; however, these studies were initiated prior to January 18, 2017, when the Final Rule came into effect. The rule strengthened reporting obligations by requiring a designated responsible party for results submission to ClinicalTrials.gov and provided clear ACT designation criteria with stricter compliance requirements. We assess how this regulatory change affected the reporting of trials containing AMAs, focusing on timeliness and study characteristics.</p>
<h4>Design</h4>
<p> We extracted data from ClinicalTrials.gov for trials involving AMAs with primary completion dates between May 1, 2013, and May 1, 2023. We analyzed the time from primary completion to results reporting and estimated the hazard ratio to compare timeliness between ACTs and pACTs. Additionally, we assessed delays in reporting across different study types and funding sources.</p>
<h4>Results </h4>
<p>Our search resulted in 2629 trials. We excluded 104 because they only included agents with nonantimicrobial properties. We found 1796 pACTs (71.1%; 95% CI, 69.3%-72.9%) and 756 ACTs (29.9%; 95% CI, 28.2%-31.8%). Among the 2525 trials sampled, 2249 trials (89.1%; 95% CI, 87.8%-90.2%) were reported on ClinicalTrials.gov. There was a median (IQR) of 0.5 years (0.2-1.5) between 1 year after the primary completion date and the results submissions across all studies. The median (IQR) time lag for late reporting for ACTs was 0.3 years (0.1-0.9), while it was 0.6 years (0.2-1.8) for pACTs. Overall, 81.3% (95% CI, 79.7%-82.3%) of trials were reported late (75.0% of ACTs vs 83.6% of pACTs). Our analysis showed that ACTs were more likely to report results earlier than pACTs, with a hazard ratio of 1.4 (95% CI, 1.3-1.5) (<b>Figure 24-0810</b>). Regarding funding sources, trials supported by philanthropic foundations or private donors had the slowest reporting (median [IQR] delay, 0.7 [0.3-1.5] years), whereas government-funded trials had the most timely reporting (median [IQR] delay, 0.4 [0.2-1.0] years).</p>

![](https://assets.underline.io/markdown_image/1/image/cb279f741ce20b24431d3c94a3e825f6.png)

<h4>Conclusions</h4>
<p> Studies including AMAs designated as ACTs demonstrated higher rates of reporting compliance and shorter delays in the reporting of overdue results. While this analysis provides initial insights, limitations related to timeline and sample scope suggest that broader investigations are needed to fully evaluate the policy’s impact.</p>
<sup>1</sup>University of California, Berkeley, CA, US; <sup>2</sup>University of Santa Cruz, Santa Cruz, CA, US; <sup>3</sup>Harvard-MIT Center for Regulatory Science, Harvard Medical School, Boston, MA, US, maximiliansiebert91@gmail.com.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> None reported.</p>
<h4>Additional Information</h4>
<p> The protocol for this study was registered at <a href="https://osf.io/zp8ug/"><span class="Hyperlink CharOverride-6">https://osf.io/zp8ug/</span></a>.</p>


Assessing the Quality and Timeliness of Results Reporting for Clinical Trials on Antimicrobial Agents

<p>Soo Young Kim,<sup>1</sup> Sue Kim,<sup>2</sup> Hyun Jung Yi<sup>3</sup></p>
<h4>Objective</h4>
<p> This study evaluated whether study designs were reported in papers published in Korean medical journals and whether the reported study designs were appropriate. Additionally, factors influencing such reporting and appropriateness were investigated.</p>
<h4>Design</h4>
<p> A cross-sectional study was conducted in accordance with the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement. A total of 600 articles, representing 5% of the 12,000 articles indexed in KoreaMed (<a href="http://www.koreamed.org/"><span class="Hyperlink CharOverride-6">www.koreamed.org</span></a>, a comprehensive search portal managed by the Korean Association of Medical Journal Editors that includes 288 member journals in the fields of medicine, dentistry, nursing, nutrition, and veterinary medicine) were randomly selected for analysis between January and December 2023. The inclusion criteria specified human participant studies published in either Korean or English. Articles classified as reviews, letters, editorials, or case reports were excluded. These exclusions were applied during the initial random selection process. The primary outcomes assessed were (1) whether the research design was stated in the title, abstract, or methods section; (2) whether the research design was appropriate, as evaluated using the Design Algorithm for Medical Literature on Intervention<sup>1</sup>; and (3) whether the journal submission guidelines referenced reporting guidelines, including mention of the EQUATOR Network.<sup>2</sup> Two independent reviewers evaluated the full texts of the selected articles. Any discrepancies in assessment were resolved through discussion until a consensus was achieved. A χ² test was used to examine associations between the reporting study design and (1) the inclusion of reporting guidelines in journal submission policies and (2) references to the EQUATOR Network.</p>
<h4>Results</h4>
<p> Of the 600 articles obtained through a 5% random sampling, a total of 233 articles were included. Among them, 149 (63.9%) reported the study design in at least 1 section: the title (n = 64 [27.5%]), abstract (n = 92 [39.5%]), or methods (n = 129 [55.4%]). However, only 58 of 149 (38.9%) appropriately reported their study design. The reasons for inappropriate study design reporting were (1) mislabeling of nondesign elements as study designs (eg, subgroup analysis, retrospective study [n= 37]), (2) incorrect classification of study design (eg, describing a cohort study as a case-control study [n = 36]), and (3) insufficient detail in describing the study design (eg, labeling a nonrandomized study as a phase trial, labeling a cohort study as a longitudinal study [n = 18]). Reporting guidelines in submission policies, including reference to the EQUATOR Network, showed no significant association with either the reporting or appropriateness of study designs.</p>
<h4>Conclusions</h4>
<p> More than half of the articles in Korean medical journals reported their study designs, yet only small fractions were deemed appropriate. Enhanced education for researchers on study design is necessary to improve reporting quality.</p>
<h4>References</h4>
<p>1. Seo HJ, Kim SY, Lee YJ, et al. A newly developed tool for classifying study designs in systematic reviews of interventions and exposures showed substantial reliability and validity. <i>J Clin Epidemiol</i>. 2016;70:200-205. doi:10.1016/j.jclinepi.2015.09.013</p>
<p>2. The EQUATOR Network. Accessed January 13, 2025. <a href="https://www.equator-network.org"><span class="Hyperlink CharOverride-6">https://www.equator-network.org</span></a></p>
<p><sup>1</sup>Department of Family Medicine, Kangdong Sacred Heart Hospital, Hallym University College of Medicine, Seoul, Korea, hallymfm@gmail.com; <sup>2</sup>College of Nursing, Yonsei University, Seoul, South Korea; <sup>3</sup>Medical Library, Hanyang University Guri Hospital, Guri, Korea.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> None reported.</p>


Reporting Study Design in Korean Medical Journal Articles

<p>Nina Vitlov,<sup>1</sup> Nensi Bralić,<sup>1</sup> Tina Poklepović Peričić,<sup>2</sup> Daniel Garcia-Costa,<sup>3</sup> Emilia López-Iñesta,<sup>4</sup> Elena Álvarez-García,<sup>3</sup> Francisco Grimaldo,<sup>3</sup> Ana Marušić<sup>1</sup></p>
<h4>Objective</h4>
<p> To explore how GRADE and Cochrane experts, clinicians, and health care consumers interact with Summary of Findings (SoF) tables from systematic reviews of evidence when answering questions about table content.</p>
<h4>Design</h4>
<p> A quasi-experimental study was conducted from February to June 2025, using the Read&amp;Learn platform, an online tool designed for monitoring the interaction with a word problem or a table.<sup>1,2</sup> The sample size was calculated at 25 participants per group to detect a mean (SD) of 4 (5) points’ difference in the number of correct answers (range, 0-16), with 80% power and 5% α level. The study intervention was blurring of SoF table cells, which the participants were required to navigate and open in order to answer questions about 4 different SoF tables, alternating between the question screen and table cells. All necessary data for answering the questions were contained within the target cells, requiring no additional calculations. We measured the number of correct answers to the questions about SoF tables, total number of table cells visited, number of target table cells visited, number of nontarget table cells visited, total time spent on the quiz from beginning to end, time spent on initial reading, total time spent reading table cells, time spent on reading target table cells, and time spent on reading nontarget table cells. The study was approved by the Ethics Committee of the University of Split School of Medicine, and all participants gave informed consent.</p>
<h4>Results</h4>
<p> We collected the data from 40 experts, 40 clinicians, and 40 health care consumers (<b>Table 25-0884</b>). There were no significant differences between experts, clinicians, and health care consumers in terms of quiz performance, with all groups achieving a similar number of correct answers. Additionally, total time spent on the quiz, initial reading, and quiz questions did not differ between the groups. Significant differences were observed in their interaction with the SoF tables. Experts visited fewer numbers of table cells in total and fewer nontarget segments and spent the least amount of time reading the nontarget segments, compared with clinicians and health care consumers. They also spent the least amount of time reading the question statement.</p>

![](https://assets.underline.io/markdown_image/1/image/341c5dad8e21fa52c9cd92a2a6c0f0c1.png)

<h4>Conclusions</h4>
<p> Experts in evidence synthesis, Cochrane systematic review authors, and GRADE experts showed different interaction patterns with the SoF table, possibly reflecting their greater familiarity with the content. Future studies, including qualitative approaches, should further explore how different users approach and use the information presented in SoF tables.</p>
<h4>References</h4>
<p>1. Matas J, Tokalić R, García-Costa D, et al. Tool to assess recognition and understanding of elements in Summary of Findings table for health evidence synthesis: a cross-sectional study. <i>Sci Rep</i>. 2023;13:18044. doi:10.1038/s41598-023-45359-x</p>
<p>2. Sanz MT, López-Iñesta E, García-Costa D, Grimaldo F. Measuring arithmetic word problem complexity through reading comprehension and learning analytics. <i>Mathematics</i>. 2020;8(9):1556. doi:10.3390/math8091556</p>
<sup>1</sup>Department of Research in Biomedicine and Health, Center for Evidence-based Medicine, University of Split School of Medicine, Split, Croatia, nina.vitlov@mefst.hr; <sup>2</sup>Department of Prosthodontics, School of Dental Medicine, University of Split School of Medicine, Split, Croatia; <sup>3</sup>Department of Computer Science, Universitat de València, València, Spain; <sup>4</sup>Department of Didactics of Mathematics, Universitat de València, València, Spain.</p>
<h4>Conflict of Interest Disclosures </h4>
<p>Nina Vitlov is funded by the Croatian Science Foundation under the Programme for Career Development of Early Career Researchers-Training of New Doctoral Students, NPOO (C3 2 R2-I1). Ana Marušić is a member of the Peer Review Congress Advisory Board but was not involved in the review or decision for this abstract.</p>


Comparative Analysis of Expert, Clinician, and Consumer Interactions With Summary of Findings Tables: A Quasi-Experimental Study

<p>Sina Rashedi,<sup>1</sup> Farbod Zahedi Tajrishi,<sup>2</sup> Ashkan Hashemi,<sup>3</sup> Isaac Dreyfus,<sup>4</sup> Nicholas Varunok,<sup>5</sup> John Burton,<sup>6</sup> Seng Chan You,<sup>7</sup> Bjorn Redfors,<sup>8,9</sup> Gregory Piazza,<sup>1,10</sup> Joshua D. Wallach,<sup>11</sup> Lesley Curtis,<sup>12</sup> Sanjay Kaul,<sup>13</sup> David J. Cohen,<sup>14,15</sup> Roxana Mehran,<sup>16</sup> Mitchell S. V. Elkind,<sup>17</sup> Flavia Geraldes,<sup>18</sup> Joseph S. Ross,<sup>19,20</sup> Jane A. Leopold,<sup>10</sup> Harlan M. Krumholz,<sup>19,21</sup> Gregg W. Stone,<sup>16</sup> Behnood Bikdeli<sup>1,10,19</sup></p>
<h4>Objective </h4>
<p>A large number of cardiovascular randomized clinical trials (RCTs) do not meet their primary outcome (neutral or negative trials). Some of these negative/neutral RCTs show positive secondary findings. We aimed to assess the proportion of cardiovascular disease– or stroke-related RCTs with negative or neutral primary results but positive secondary findings among all negative/neutral cardiovascular RCTs published in the highest-impact medical journals and to evaluate whether these findings were pursued in subsequent confirmatory RCTs or influenced clinical practice recommendations or regulatory decisions.</p>
<h4>Design</h4>
<p> We searched PubMed to identify cardiovascular RCTs published in 3 major clinical journals (<i>NEJM</i>, <i>The Lancet</i>, and <i>JAMA</i>) between 2014 and 2019. Negative/neutral RCTs with positive secondary findings were defined as those that did not meet their (co)primary outcome(s) (<i>P </i>&gt; .05) but showed positive effects in prespecified subgroup analyses for the primary outcome and/or for prespecified secondary outcomes among the entire study population. Secondary outcomes and subgroup analyses were considered if they were reported as prespecified in the trial publications. We investigated whether these positive secondary findings were examined in subsequent confirmatory RCTs via PubMed searches or directly resulted in changes in either professional society recommendations or regulatory authority endorsements in the US or Europe through December 2024. Two investigators independently performed the literature search and data extraction, with discrepancies resolved by a third investigator.</p>
<h4>Results </h4>
<p>Overall, 435 cardiovascular RCTs were identified; the median time from the index trial to December 2024 was 7.5 (IQR, 5.5-9.5) years. Of these 435 RCTs, the primary outcome(s) were not met in 174 trials (40.0%), of which 93 (53.4%) had at least 1 positive secondary finding. Among these trials, 51 (54.8%) had only positive secondary outcomes, 20 (21.5%) had only positive findings in subgroup analyses, and 22 (23.7%) had positive findings in both secondary outcomes and subgroup analyses. Among the 93 negative/neutral RCTs with positive secondary findings, 14 (15.1%) were consequential: subsequent RCTs were performed due to positive secondary findings for 2 trials; the secondary results from 12 RCTs contributed to changes in professional society recommendations; and 2 trials contributed to changes in labeled indications (<b>Figure 25-0928</b>).</p>

![](https://assets.underline.io/markdown_image/1/image/b487af44493837ae64ec3a334989da10.png)

<h4>Conclusions</h4>
<p> Although nearly half of the negative/neutral cardiovascular trials published in the highest-impact journals report positive secondary findings, only a small proportion directly influence practice or policy, and very few are followed by subsequent confirmatory RCTs. Our findings caution against overemphasizing positive secondary results from cardiovascular RCTs and underscore the need for subsequent confirmatory RCTs to evaluate positive secondary findings from otherwise negative/neutral trials. Further research is planned to assess the generalizability of these findings in major specialty cardiovascular journals and to identify trial- and outcome-level characteristics that influence the selection of positive secondary findings for replication in subsequent trials.</p>
<sup>1</sup>Thrombosis Research Group, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, US, srashedi@bwh.harvard.edu; <sup>2</sup>Tulane University School of Medicine, New Orleans, LA, US; <sup>3</sup>Program for the Care and Study of the Aging Heart, Department of Medicine, Weill Cornell Medicine, New York, NY, US; <sup>4</sup>Division of Cardiology, Department of Medicine, University of California Los Angeles, Los Angeles, CA, US; <sup>5</sup>Vanderbilt University Medical Center, Nashville, TN, US; <sup>6</sup>University of Southern California, Los Angeles, CA, US; <sup>7</sup>Department of Biomedical Systems Informatics, Yonsei University College of Medicine, Seoul, Korea; <sup>8</sup>Department of Cardiology, Sahlgrenska University Hospital, Gothenburg, Sweden; <sup>9</sup>Cardiovascular Research Foundation, New York, NY, US; <sup>10</sup>Division of Cardiovascular Medicine, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, US; <sup>11</sup>Department of Epidemiology, Rollins School of Public Health, Emory University, Atlanta, GA, US; <sup>12</sup>Duke University School of Medicine, Department of Population Health Sciences, Durham, NC, US; <sup>13</sup><span lang="pt-PT">Department of Cardiology, Cedars-Sinai Medical Center, Los Angeles, CA, US; </span><sup>14</sup>Clinical Trials Center, Cardiovascular Research Foundation, New York, NY, US; <sup>15</sup>St Francis Hospital and Heart Center, Roslyn, NY, US; <sup>16</sup>Icahn School of Medicine at Mount Sinai, New York, NY, US; <sup>17</sup>Department of Neurology, Columbia University Irving Medical Center, New York, NY, US; <sup>18</sup><i>The Lancet</i>, London, United Kingdom; <sup>19</sup>YNHH/Yale Center for Outcomes Research and Evaluation (CORE), New Haven, CT, US; <sup>20</sup>Department of General Internal Medicine, Yale School of Medicine, New Haven, CT, US; <sup>21</sup>Section of Cardiovascular Medicine, Yale School of Medicine, New Haven, CT, US.</p>
<h4>Conflict of Interest Disclosures </h4>
<p>Outside the submitted work, Behnood Bikdeli reported receiving the following support: a Career Development Award from the American Heart Association and VIVA Physicians (#938814), the Scott Schoen and Nancy Adams IGNITE Award, the Mary Ann Tynan Research Scientist Award from the Mary Horrigan Connors Center for Women’s Health and Gender Biology at Brigham and Women’s Hospital, and the Heart and Vascular Center Junior Faculty Award from Brigham and Women’s Hospital; serving as consulting expert on behalf of the plaintiff for litigation related to 2 specific brand models of IVC filters (although he has not been involved in the litigation in 2022-2025, nor has he received any compensation in 2022-2025); serving as a member of the Medical Advisory Board for the North American Thrombosis Forum (now VascuLearn Network) and on the Data Safety and Monitoring Board of the NAIL-IT trial funded by the National Heart, Lung, and Blood Institute and Translational Sciences; serving as a collaborating consultant with the International Consulting Associates and the US Food and Drug Administration in a study to generate knowledge about the utilization, predictors, retrieval, and safety of IVC filters; receiving compensation as an associate editor for <i>NEJM Journal Watch Cardiology</i>, as an associate editor for <i>Thrombosis Research</i>, and as an executive associate editor for JACC; and serving as a section editor for <i>Thrombosis and Haemostasis</i> without compensation.</p>
<h4>Additional Information</h4>
<p> Behnood Bikdeli is a co–corresponding author (behnood.bikdeli@yale.edu).</p>


Replication and Impact of Positive Secondary Findings in Negative or Neutral Cardiovascular Trials

<p>Christof Manuel Schönenberger,<sup>1</sup>Malena Chiaborelli,<sup>1</sup> Ala Taji Heravi,<sup>1</sup> Lukas Kübler,<sup>2</sup> Pooja Gandhi,<sup>3</sup> Zsuzsanna Kontar,<sup>4</sup>Julia Hüllstrung,<sup>1</sup> Mona Elalfy,<sup>1</sup> Jan Glasstetter,<sup>1</sup> Dmitry Gryaznov,<sup>5</sup> Belinda von Niederhäusern,<sup>6</sup> Anette Blümle,<sup>7</sup> Jason W. Busse,<sup>8</sup> Szimonetta Lohner,<sup>4</sup>Sally Hopewell,<sup>9</sup> Alexandra Griessbach,<sup>1</sup>Matthias Briel,<sup>1,10</sup> Benjamin Speich<sup>1</sup></p>
<h4>Objective</h4>
<p> Making protocols of randomized clinical trials (RCTs) publicly available is important for the trustworthiness and quality of medical research.<sup>1</sup> In a previous study, only 36% of RCTs that received ethical approval in 2012 had a publicly available protocol.<sup>2</sup> Repeating this study with RCTs approved in 2016, we aimed to generate current evidence on the availability of RCT protocols, sources where they are shared, and changes over time.</p>
<h4>Design</h4>
<p> Using a sample of RCTs receiving ethical approval in 2016 in Switzerland, Canada, Germany, or the UK,<sup>3</sup> we investigated the number of available protocols. For all RCTs, we checked if a result publication was available by searching PubMed, Google Scholar, trial registries, and Google. We extracted baseline characteristics for all RCTs. Up to June 2024, we systematically searched for (1) protocols available as peer-reviewed publications, (2) protocols attached to trial registries, and (3) protocols shared with result publications of RCTs. Multiple sources per protocol were possible. All searches and data extraction were performed in duplicate. We used multivariable logistic regression to examine the association of protocol availability with trial characteristics such as sample size, drug vs nondrug interventions, multicenter vs single-center status, and RCT approval in 2016 vs 2012.</p>
<h4>Results</h4>
<p> Of 347 included RCTs, 228 (65.7%) had an available protocol (<b>Table 25-0957</b>); 150 protocols (43.2%) were available as files on trial registries, 91 (26.2%) as supplementary material to a result publication, and 81 (23.3%) as peer-reviewed publications. Protocol availability improved over time (65.7% in 2016 vs 36.2% in 2012), particularly in industry trials (83.4% in 2016 vs 34.6% in 2012), but improved little in nonindustry trials (46.4% 2016 vs 38.1% 2012). In a regression model with pooled data from trials approved in 2012 and approved in 2016, trials with a medium sample size (100-500 participants) (odds ratio [OR], 2.12; 95% CI, 1.32-3.43; <i>P</i> = .001), trials with a large sample size (&gt;500 participants) (OR, 6.12; 95% CI, 3.45-11.07; <i>P</i> &lt; .001), and multicenter trials (vs single-center trials) (OR, 2.66; 95% CI, 1.59-4.52; <i>P</i> &lt; .001) showed significantly higher odds of having an available protocol. A significant interaction term indicated dependence of sponsorship and year (OR, 0.13; 95% CI 0.06-0.27; <i>P</i> &lt; .001).</p>

![](https://assets.underline.io/markdown_image/1/image/b7e71ef2066ff55aa9831845fed69144.png)

<h4>Conclusions </h4>
<p>The availability of protocols increased in RCTs approved in 2016 compared with RCTs approved in 2012. This was mainly driven by industry-sponsored trials and may be explained by updated US regulations. Since 2017, all trials investigating drugs or devices that are or are intended to be approved, licensed, or cleared by the US Food and Drug Administration must have a publicly available protocol. Efforts to further improve protocol availability should be continued, especially in nonindustry-sponsored RCTs.</p>
<h4>References</h4>
<p>1. Chan AW, Song F, Vickers A, et al. Increasing value and reducing waste: addressing inaccessible research. <i>Lancet</i>. 2014;383(9913):257-266. doi:10.1016/S0140-6736(13)62296-5</p>
<p>2. Schönenberger CM, Griessbach A, Taji Heravi A, et al. A meta-research study of randomized controlled trials found infrequent and delayed availability of protocols. <i>J Clin Epidemiol</i>. 2022;149:45-52. doi:10.1016/j.jclinepi.2022.05.014</p>
<p>3. Gryaznov D, Odutayo A, von Niederhäusern B, et al. Rationale and design of repeated cross-sectional studies to evaluate the reporting quality of trial protocols: the Adherence to Spirit Recommendations (ASPIRE) study and associated projects. <i>Trials</i>. 2020;21(1):896. doi:10.1186/s13063-020-04808-y</p>
<sup>1</sup>Division of Clinical Epidemiology, Department of Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland, christofmanuel.schoenenberger@usb.ch; <sup>2</sup>Department of Biomedicine, University Hospital Basel, University of Basel, Basel, Switzerland; <sup>3</sup>University Health Network, Toronto, Ontario, Canada; <sup>4</sup>MTA-PTE Lendület “Momentum” Evidence in Medicine Research Group, Department of Public Health Medicine, Medical School, University of Pécs, Pécs, Hungary; <sup>5</sup><span lang="de-DE">Viatris Innovation GMBH, Allschwil, Switzerland; </span><sup>6</sup>Roche Pharma AG, Grenzach-Wyhlen, Germany; </span><sup>7</sup>Clinical Trials Unit, Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany; <sup>8</sup>Department of Anesthesia, McMaster University, Hamilton, Ontario, Canada; <sup>9</sup>Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK; <sup>10</sup>Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> <span lang="de-DE">None reported.</span></p>
<h4>Funding/Support</h4>
<p> <span lang="de-DE">This project was supported by the Swiss Federal Office of Public Health (to Matthias Briel). Christof Manuel Schönenberger was funded by the Janggen Pöhn Foundation and the Swiss National Science Foundation (MD-PhD grant number 323530_221860).</span></p>
<h4>Role of the Funder/Sponsor</h4>
<p> The funders had no role in the conduct of the study, the analysis of the data, or the writing of the abstract.</p>
<h4>Acknowledgment</h4>
<p> <span lang="de-DE">We thank all participating research ethics committees from Germany (Freiburg), Switzerland (Basel, Bellinzona, Bern, Geneva, Lausanne, St Gallen, Frauenfeld, and Zurich), Canada (Hamilton), and the UK (National Health Service Health Research Authority) for their support and cooperation.</span></p>


Public availability of Randomized Clinical Trial Protocols: A Repeated Cross-Sectional Study

<p>Carolina Grana,<sup>1,2,3</sup> Lina Ghosn,<sup>1,2,3</sup> Carolina Riveros,<sup>1,2,3</sup> Philippe Ravaud,<sup>1,2,3</sup> Isabelle Boutron<sup>1,2,3</sup></p>
<h4>Objective</h4>
<p> The World Health Organization (WHO) requests that “key outcomes are to be made publicly available within 12 months of study completion by posting to the results section of the primary clinical trial registry.”<sup>1</sup> In some regions, this requirement is also mandated by law<sup>2</sup>; however, evidence indicates low adherence.<sup>3</sup> The extent to which journal editors support this request remains unclear. This study aimed to assess adherence to the WHO request among randomized clinical trials (RCTs) published in high-impact journals and to examine whether journal editors encourage this practice in their recommendations to authors.</p>
<h4>Design</h4>
<p> We conducted a cross-sectional study and reported our findings according to the STROBE guideline. On April 18, 2025, we searched PubMed using the terms <i>date-create: 01/01/2024</i> <i>to 01/01/2025</i> <i>AND journal: [Name of the journal]</i>, applying the “randomized controlled trial” filter. We selected the 5 journals with the highest impact factors in the “medicine, general &amp; internal”<i> </i>category and in selected medical specialties based on the 2023 Clarivate Journal Citation Reports (Web of Science). We included RCTs evaluating pharmacologic, biologic, or medical device interventions. Data were extracted in duplicate from each trial publication and corresponding primary registry entry. We recorded the publication date, the trial’s primary completion date, and results status in the registry. The primary outcome was the proportion of RCTs that at the time of publication had results posted in the primary registry within 12 months of trial completion. The analysis was primarily descriptive, comparing the timing of trial completion, results posting, and publication.</p>
<h4>Results</h4>
<p> Preliminary results (publications up to June 2024) identified 185 RCTs published across 18 journals. Among these, 142 (77%) evaluated pharmacologic interventions and 108 (58%) were multinational. Trial sites included at least 1 European Union country in 102 trials (55%), the US in 94 (51%), and the UK in 74 (40%). All 185 trials were registered, and 163 (88%) registrations were prospective. A total of 100 trials (54%) were published more than 12 months after their primary completion date. Among these, at the time of publication, only 23 (23%) had adhered to WHO requirements by having results posted in the trial registry within 12 months of completion. Conversely, 83 trials (45%) were published within 12 months of completion, and 45 of these (54%) had results posted in the registry when the trial report was published. Completion date was not reported for 2 trials (1%). None of the 18 journals assessed requested public posting of trial results in registries. </p>
<h4>Conclusions</h4>
<p> Preliminary findings from trials included in this study, all published in high–impact factor journals, revealed low adherence to the WHO call for timely posting of results to the registries.</p>
<h4>References</h4>
<p>1. WHO statement on public disclosure of clinical trial results. World Health Organization. Accessed January 10, 2025. <a href="https://www.who.int/news/item/09-04-2015-japan-primary-registries-network"><span class="Hyperlink CharOverride-6">https://www.who.int/news/item/09-04-2015-japan-primary-registries-network</span></a></p>
<p>2. Zarin DA, Tse T, Williams RJ, Califf RM, Ide NC. The ClinicalTrials.gov results database—update and key issues. <i>N Engl J Med</i>. 2011;364(9):852-860. doi:10.1056/NEJMsa1012065</p>
<p>3. DeVito NJ, Bacon S, Goldacre B. Compliance with legal requirement to report clinical trial results on ClinicalTrials.gov: a cohort study. <i>Lancet</i>. 2020;395(10221):361-369. doi:10.1016/S0140-6736(19)33220-9</p>
<sup>1</sup>Université Paris Cité and Université Sorbonne Paris Nord, Inserm, INRAe, Centre for Research in Epidemiology and Statistics (CRESS), Paris, France, isabelle.boutron@aphp.fr; <sup>2</sup>Centre d’Épidémiologie Clinique, AP-HP, Hôpital Hôtel-Dieu, Paris, France; <sup>3</sup>Cochrane France, Paris, France.</p>
<h4>Conflict of Interest Disclosures </h4>
<p>Isabelle Boutron is a member of the Peer Review Congress Advisory Board but was not involved in the review or decision for this abstract. No other disclosures were reported.</p>
<h4><a id="_idTextAnchor108"></a>Additional Information</h4>
<p> Carolina Grana (carolinaestela.granapossamai@aphp.fr) and Lina Ghosn (lina.elchall@aphp.fr) are co–corresponding authors.</p>


Adherence to the WHO Statement on Public Disclosure of Clinical Trial Results by Trials Published in High Impact Factor Journals

<p>Ioana Alina Cristea,<sup>1,2</sup> Florian Naudet,<sup>2,3,4</sup> Guillaume Cabanac,<sup>4,5</sup> John P. A. Ioannidis<sup>2,6,7,8</sup></p>
<h4>Objective </h4>
<p>An older, field-wide analysis<sup>1</sup> of studies published between 2005 and 2017 estimated the prevalence of prospectively registered randomized controlled trials (RCTs) at 21%. Recent estimates indicated higher rates, eg, approximately 59% for rheumatology RCTs published between 2009 and 2022 in 5 International Committee of Medical Journal Editors (ICMJE) journals.<sup>2</sup> There are no recent field-wide estimates of the prevalence of prospective registration across clinical specialties and journals and of the related outcome reporting bias (ie, discrepancies between registered and published outcomes). We report these across a randomly selected sample of recently published RCTs.</p>
<h4>Design </h4>
<p>We conducted a retrospective cohort study, reported following the STROBE guidelines. We searched PubMed with the Cochrane Highly Sensitive Search Strategy<sup>3</sup> on January 15, 2025. RCTs evaluating health-related interventions and outcomes (per the ICMJE definition) published in 2024 were included. Search results were randomized using Entrez Programming Utilities and the Linux shuf command, and a sample of 1720 records was selected. One researcher (I.A.C.) manually checked titles and abstracts to select RCTs and extracted trial registration information from publications reporting on primary outcomes (POs). Prospective registration was defined as submission date to registry preceding recruitment or study start or postdating by less than 30 days. We favored actual vs estimated study start dates when available in publications or registries. For prospectively registered trials, 1 researcher (I.A.C.) extracted all POs registered and declared in publications, and tabulated all discrepancies regarding outcome domain, measurement, or time point.</p>
<h4>Results </h4>
<p>Of 324,177 records identified, 1720 were screened, leading to 136 eligible RCTs. Six trials were excluded (inaccessible full text [n = 3]; Chinese language [n = 3]). Eighty-two of 130 publications (63%) reported study start dates, with a median (IQR) of 2020 (2) (range, 2013-2023). Sixty-nine of 130 RCTs (53% [95% CI, 44%-62%]) were prospectively registered (67 in ICMJE-compliant registries), including 4 registered while recruiting. Sixty-one of 130 RCTs (47% [95% CI, 38%-56%]) were declared nonregistered (n = 4), were registered retrospectively (n = 27), indicated a registration number for another trial (n = 2), or did not mention registration (n = 28). Registration delays were a median (IQR) of 632 (533) (range, 49-1524) days. For 24 of 60 nonprospectively registered trials, journal instructions to authors explicitly required prospective registration or compliance with ICMJE or Declaration of Helsinki registration requirements. Of 69 prospectively registered trials, 6 did not declare POs in publications and 2 insufficiently specified POs in the registries, while 37 (54% [95% CI, 41%-66%]) had no substantive discrepancies between registry and publication. The remaining 24 of 69 RCTs (35% [95% CI, 24%-47%]) included 1 (n = 17), 2 (n = 6), or 3 (n = 1) types of discrepancies (<b>Table 25-1102</b>).</p>
<a href="/wp-content/uploads/2025_conference/tables/prc25-1102t1.png"> <img loading="lazy" src="/wp-content/uploads/2025_conference/tables/prc25-1102t1.png"> </a>
<h4>Conclusions </h4>
<p>In a random sample of RCTs published in 2024 across clinical specialties, approximately half were prospectively registered and approximately one-third of these contained substantial changes between registered and reported primary outcomes.</p>
<h4>References</h4>
<p>1. Trinquart L, Dunn AG, Bourgeois FT. Registration of published randomized trials: a systematic review and meta-analysis. <i>BMC Med</i>. 2018;16(1):173. doi:10.1186/s12916-018-1168-6</p>
<p>2. Mongin D, Buitrago-Garcia D, Capderou S, et al. Prospective registration of trials: where we are, why, and how we could get better. <i>J Clin Epidemiol</i>. 2024;176. doi:10.1016/j.jclinepi.2024.111586</p>
<p>3. Lefebvre C, Glanville J, Briscoe S, et al. Searching for and selecting studies. In: Higgins JPT, Thomas J, Chandler J, et al, eds. <i>Cochrane Handbook for Systematic Reviews of Interventions</i>, Version 6.5. Cochrane; 2024</p>
<sup>1</sup>Department of General Psychology, University of Padova, Padova, Italy, ioanaalina.cristea@unipd.it; <sup>2</sup>Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, US; <sup>3</sup>Université Rennes, CHU Rennes, Inserm, Centre d’investigation clinique de Rennes (CIC1414), service de pharmacologie clinique, Institut de recherche en santé, environnement et travail (Irset), UMR S 1085, EHESP, 35000, Rennes, France; <sup>4</sup>Institut Universitaire de France (IUF), Paris, France; <sup>5</sup>Université de Toulouse, IRIT (UMR 5505 CNRS), Toulouse, France; <sup>6</sup>Stanford Prevention Research Center, Department of Medicine, Stanford University School of Medicine, Stanford, CA, US; <sup>7</sup>Department of Epidemiology &amp; Population Health, Stanford University School of Medicine, Stanford, CA, US; <sup>8</sup>Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, CA, US.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> John P. A. Ioannidis is a member of the Peer Review Congress Advisory Board but was not involved in the review or decision for this abstract.</p>
<h4>Funding/Support</h4>
<p> Ioana Alina Cristea is supported by a European Research Council (ERC) Starting Grant (grant agreement: 101042701; <a href="https://cordis.europa.eu/project/id/101042701"><span class="Hyperlink CharOverride-6">https://cordis.europa.eu/project/id/101042701</span></a>), funded by the European Union (EU). Florian Naudet received funding from the French National Research Agency, the French Ministry of Health, and the French Ministry of Research and is a work package leader in the OSIRIS project (Open Science to Increase Reproducibility in Science) and for the doctoral network MSCA-DN SHARE-CTD (HORIZON-MSCA-2022-DN-01 101120360), funded by the EU. Guillaume Cabanac received funding from the Institut Universitaire de France. John P. A. Ioannidis is supported by an unrestricted gift from Sue and Bob O’Donnell to Stanford University.</p>
<h4>Role of the Funder/Sponsor </h4>
<p>The funders had no role in the study design, data collection and analysis, decision to publish, or preparation of the abstract.</p>
<h4>Additional Information </h4>
<p>John P. A. Ioannidis is a co–corresponding author (jioannid@stanford.edu).</p>


Prevalence of Prospective Registration and Primary Outcome Discrepancies in Recently Published Randomized Controlled Trials

<p>Christopher C. Winchester,<sup>1</sup> Mark J. Rolfe,<sup>1</sup> William T. Gattrell,<sup>2</sup> Patricia Logullo,<sup>3</sup> Keith Goldman,<sup>4</sup> Amy Price,<sup>5</sup> Paul Blazey,<sup>6</sup> Esther J. van Zuuren,<sup>7</sup> Niall Harrison<sup>8</sup></p>
<h4>Objective </h4>
<p>The ACcurate COnsensus Reporting Document (ACCORD) guideline supports the reporting of biomedical studies involving any consensus method.<sup>1</sup> We evaluated its uptake by bibliometric analysis and qualitative and quantitative assessment of implementation activities 1 year post publication.</p>
<h4>Design</h4>
<p> An implementation plan<sup>2</sup> was established by the ACCORD Steering Committee before guideline publication, including developing an explanation and elaboration (E&amp;E) document, writing journal editorials, presenting at congresses, and encouraging journals and publishers to include ACCORD in author instructions. Guideline views and downloads (via <i>PLOS Medicine</i>,<sup>1</sup> the publishing journal) and citations (Dimensions, plus additional references from the Steering Committee’s reference libraries) were obtained for the period from January 23, 2024, to January 22, 2025. Likely journal and publisher beneficiaries of ACCORD were identified by a literature search to determine the 10 journals that published the most consensus studies (Web of Science for 2020-2023) and other publications at the discretion of the Steering Committee. Relevant journal and publisher stakeholders were contacted directly or indirectly (eg, via journal websites). Activities were evaluated against the implementation<i> </i>plan<sup>2</sup>; journal and publisher responses were described.</p>
<h4>Results</h4>
<p> In the 1 year following publication, the ACCORD guideline was viewed 13,752 times, downloaded (full-text PDF) 3790 times, and cited 105 times (5 citations [4.8%] were in articles by ACCORD authors, including the ACCORD E&amp;E<sup>3</sup>). Sources of guideline citations included consensus studies (67.6% [71 of 105]), reviews (11.4% [12 of 105]), study protocols (8.6% [9 of 105]), other reporting standards and guidelines (6.7% [7 of 105]), and editorials (3.8% [4 of 105]). Of the journals contacted by ACCORD Steering Committee members (between August 24, 2023 [preprint publication], and January 22, 2025), <i>BMJ Open</i>, <i>British Journal of Dermatology</i>, and <i>Journal of Clinical Epidemiology </i>mandated the use of ACCORD for consensus studies, while the <i>British Journal of Sports Medicine</i>,<i> Internal Medicine Journal</i>, and <i>Pragmatic and Observational Research </i>encouraged its use. <i>PLOS </i>has confirmed its intention to include the guideline in their author instructions. Although a further 4 journals have expressed interest in ACCORD and 1 publisher has offered to raise awareness via a webinar, they are yet to commit to its inclusion in author instructions; 2 journals and publishers have not responded to inquiries.</p>
<h4>Conclusions </h4>
<p>One year after publication, the ACCORD guideline has been cited and used to inform the design and reporting of consensus-based research. However, journal and publisher adoption has been limited, which may affect long-term uptake. Given the resources invested in developing reporting guidelines and their potential to improve reporting, the role of biomedical journals and publishers in their adoption warrants wider discussion.</p>
<h4>References</h4>
<p>1. Gattrell WT, Logullo P, van Zuuren EJ, et al. ACCORD (ACcurate COnsensus Reporting Document): a reporting guideline for consensus methods in biomedicine developed via a modified Delphi. <i>PLOS Med</i>. 2024;21:e1004326. doi:10.1371/journal.pmed.1004326</p>
<p>2. Gattrell WT, Hungin AP, Price A, et al. ACCORD guideline for reporting consensus-based methods in biomedical research and clinical practice: a study protocol. <i>Res Integr Peer Rev</i><span lang="de-DE">. 2022;7:3. doi:10.1186/s41073-022-00122-0</span></p>
<p><span lang="de-DE">3. Logullo P, van Zuuren EJ, Winchester CC, et al. </span>ACcurate COnsensus Reporting Document (ACCORD) explanation and elaboration: guidance and examples to support reporting consensus methods. <i>PLOS Med</i>. 2024;21:e1004390. doi:10.1371/journal.pmed.1004390</p>
<sup>1</sup>Oxford PharmaGenesis, Oxford, UK, chris.winchester@pharmagenesis.com; <sup>2</sup>Independent Medical Communications Professional, Oxfordshire, UK; <sup>3</sup>Bodleian Libraries, University of Oxford, Oxford, UK; <sup>4</sup>Medical Affairs + Health Impact, AbbVie, North Chicago, IL, US; <sup>5</sup>Dartmouth Institute for Health Policy &amp; Clinical Practice (TDI), Geisel School of Medicine, Dartmouth College, Hanover, NH, US; <sup>6</sup>School of Kinesiology, Department of Medicine, University of British Columbia, Vancouver, British Columbia, Canada; <sup>7</sup>Leiden University Medical Centre, Leiden, the Netherlands; <sup>8</sup>OPEN Health Communications, London, UK.</p>
<h4>Conflict of Interest Disclosures </h4>
<p>Christopher C. Winchester is an employee, director, and shareholder of Oxford PharmaGenesis; a director of Oxford Health Policy Forum CIC; a trustee of the Friends of the National Library of Medicine; and an associate fellow of Green Templeton College, University of Oxford. Mark J. Rolfe is an employee of Oxford PharmaGenesis. William T. Gattrell was an independent medical communications professional at the time of this study and is currently an employee of Bristol Myers Squibb. Keith Goldman is an employee of AbbVie. Niall Harrison is an employee of OPEN Health Communications.</p>
<h4>Funding/Support </h4>
<p>Medical writing, editorial, and project management support for this abstract were provided by Oxford PharmaGenesis.</p>
<h4>Acknowledgments</h4>
<p> Medical writing support was provided by Alison Chisholm, with editorial support from Jenny Thorp, and administrative support was provided by Mehraj Ahmed, Ryan Gamble, and Jessica Miller.</p>
<h4>Additional Information </h4>
<p>No authors were reimbursed for participating in the initiative.</p>


Next from The 10th Peer Review Congress

Registration of Observational Studies of Interventions: Prevalence, Characteristics, and Journal Policies

Similar lecture

Automated Interpretation of Statistical Tables to Assess Reporting Errors and Associations With Open Science Policies in Economic Journals

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES