technical paper
Analysis of Reporting Consistency Between Clinical Trials Presented at Major Medical Conferences, Their Corresponding Publications, and Press Releases
keywords:
quality of reporting
dissemination of information
publication
bias
Objective Clinical trial integrity is compromised when
investigators selectively report or misreport results, which
leads to inaccurate claims of benefit and/or extrapolations
from incomplete data.1-3 This study examined the extent to
which trials presented at major international medical
conferences in 2016 consistently reported their study design,
end points, and results across conference abstracts, published
article abstracts, and press releases.
Design A cross-sectional analysis of trials presented at 12
conferences in the US in 2016 was conducted. Conferences
were identified from a list prepared by the Healthcare
Convention and Exhibitors Association and were included if
abstracts were publicly reported. From these conferences, all
late-breaker trials were included and other trials were
randomly selected, bringing the total sample to 25 abstracts
per conference. First, it was determined whether trials were
registered and reported results in an International Committee
of Medical Journal Editors–approved trial registry. Second, it
was determined whether trial results were published in a
peer-reviewed journal. Finally, information on trial media
coverage and press releases was collected using LexisNexis.
For all published trials, the consistency of reporting of the
following characteristics was examined, through comparison
of the trials’ conference and publication abstracts: primary
efficacy end point, safety end point, sample size, follow-up
period, primary end point effect size, and characterization of
results (comparisons were made for this characteristic across
press releases too, if any). Authors determined consistency of
reporting when identical information was presented across
abstracts (and press releases). Primary analyses were
descriptive; secondary analyses included χ2 tests and multiple
logistic regression.
Results The sample comprised 240 trials presented at 12
conferences. Of these, 208 trials (86.7%) were registered, 95
(39.6%) reported summary results in a registry, and 177
(73.8%) were published; 82 trials (34.2%) were covered by
the media, and 68 (28.3%) had press releases. Among the 177
published trials (Table 14), 171 (96.6%) reported consistent
primary efficacy end points across abstracts, whereas 96 of
128 trials (75.0%) reported consistent safety outcomes. There
were 107 of 172 trials (62.2%) with consistent sample sizes
across abstracts, 101 of 137 trials (73.7%) that reported their
follow-up periods consistently, 92 of 175 trials (52.6%) that
described their effect sizes consistently, and 157 of 175 trials
(89.7%) that characterized their results consistently. Among
the trials that were published and had press releases, 32 of 32
(100%) characterized their results consistently across
conference and publication abstracts and press releases. No
trial characteristics were associated with reporting primary
efficacy end points consistently.
Conclusions This study demonstrates that trials are
consistently reporting primary efficacy end points and results
characterization. Lower consistency rates for other
characteristics indicate that trial presentations are less likely
to report safety end points and that authors could be
presenting preliminary data at conferences with shorter
follow-up periods and smaller sample sizes, owing to
incomplete patient recruitment, and consequently, varying
effect sizes. However, this does not rule out the possibility of
misreporting at conferences or publications.
References
1. Boutron I. Reporting and interpretation of randomized
controlled trials with statistically nonsignificant results for
primary outcomes. JAMA. 2010;303:2058-2064.
doi:10.1001/jama.2010.651
2. Chiu K, Grundy Q, Bero L. “Spin” in published biomedical
literature: a methodological systematic review. PLoS Biol.
2017;15:e2002173. doi:10.1371/journal.pbio.2002173
3. Diong J, Butler AA, Gandevia SC, Héroux ME. Poor
statistical reporting, inadequate data presentation and spin
persist despite editorial advice. PLoS One. 2018;13:e0202121.
doi:10.1371/journal.pone.0202121
Conflict of Interest Disclosures The Laura and John Arnold
Foundation funds the Restoring Invisible & Abandoned Trials
Support Center, which supports the salaries of Anisa Rowhani-
Farid and Kyungwan Hong. Kyungwan Hong was supported by
the Food and Drug Administration (FDA) of the US Department of
Health and Human Services (HHS) as part of a financial assistance
award U01FD005946, unrelated to this manuscript, totaling $5000
with 100% funded by the FDA/HHS. Audrey D. Zhang currently
receives research support from the National Institute on Aging
through the Duke Creating Alzheimer’s Disease and Related
Dementias Researchers for the Next Generation—Stimulating
Access to Research in Residency (CARiNG-StARR) program
(R38AG065762). Joshua D. Wallach is supported by the FDA and the
National Institute on Alcohol Abuse and Alcoholism of the National
Institutes of Health (NIH) under award 1K01AA028258. Joseph S.
Ross currently receives research support through Yale University
from Johnson and Johnson to develop methods of clinical trial
data sharing, from the Medical Device Innovation Consortium
as part of the National Evaluation System for Health Technology,
from the FDA for the Yale-Mayo Clinic Center for Excellence in
Regulatory Science and Innovation program (U01FD005938), from
the Agency for Healthcare Research and Quality (R01HS022882),
from the National Heart, Lung, and Blood Institute of the NIH
(R01HS025164, R01HL144644), and from the Laura and John
Arnold Foundation to establish the Good Pharma Scorecard at
Bioethics International; in addition, Joseph S. Ross is an expert
witness at the request of relator’s attorneys, the Greene Law Firm,
in a qui tam suit alleging violations of the False Claims Act and
Anti-Kickback Statute against Biogen Inc. No other disclosures were
reported.
Additional information Kyungwan Hong is a co–corresponding
author.