United States

**Objective** To systematically assess the characteristics and
current practice of developing methods guidance for health
researchers and explore opportunities for improvement.

**Design** A systematic survey of methods-guidance articles
published in general and methodology-focused, high-impact
medical journals indexed in MEDLINE in 2020 was
performed. Articles that explicitly stated the objective to
provide methods guidance for health research were eligible.
Characteristics related to findability, methods for guidance
development, and transparency were extracted.

**Results** A total of 105 guidance articles published in 12
different journals were included. Few articles had a
structured abstract (44 [42%]) or were indexed with Medical
Subject Headings (40 [38%]) or author keywords (18 [17%])
related to guidance. Of the 105 guidance articles, less than
half reported any methods for development (44 [42%]), most
frequently stakeholder involvement (30 [28%]), systematic
review of the methodological literature (21 [20%]), or
consensus process (21 [20%]). Use of explicit methods for
development differed between reporting guidelines (13 of 13
[100%] reported a development process) and other types of
methods guidance (ie, guidance for planning, conduct,
analysis, interpretation, or quality assessment: 31 of 92
articles [34%] reported a development process).
Transparency was limited, with few guidance articles
describing the authors’ expertise (23 [22%]). Conflicts of
interest, if reported (36 [34%]), were frequently unclear.

**Conclusions** Most methods-guidance articles published in
2020 were difficult to find in MEDLINE, were developed with
unclear methods, and lacked transparency regarding the
authors’ expertise and conflicts of interest. For health
researchers, those limitations implied important barriers to
the uptake of methods guidance. To improve findability, we
developed the new open-access Library of Guidance for
Health Scientists (LIGHTS; https://lights.science). More
research is required to inform methods for guidance
development and transparency considerations.

**Conflict of Interest Disclosures** None reported.

**Funding/Support** This work was supported by grant 190566 from
the Swiss National Science Foundation and grant 3MS1043 from the
University of Basel.

**Role of the Funder/Sponsor** The funders had no role in the
design and conduct of the study; collection, management, analysis,
and interpretation of the data; preparation, review, or approval of
the abstract; and decision to submit the abstract for presentation.

**Additional Information** Julian Hirt is a co–corresponding
author.

![](https://assets.underline.io/uploads/markdown_image/1/image/dbaccba0abde9f482b8175b3df1998be.png)

Peer Review Congress 2022

Characteristics and Opportunities for Improvement of Methods Guidance Published in General and Methodology-Focused Medical Journals

conflict of interest

reporting guidelines

research methods

The International Conference on Information and Communication Technologies for Ageing Well and e-Health aims to be a meeting point for those that study age and health-related quality of life and apply information and communication technologies for helping people stay healthier, more independent and active at work or in their community.

ICT4AWE 2020

poster

Our aim is to encourage research into the quality and credibility of peer review and scientific publication, to establish the evidence base on which scientists can improve the conduct, reporting, and dissemination of scientific research.
 
 **Welcome!**
 
 The JAMA Network, The BMJ, and METRICS welcome you to the Ninth International Congress on Peer Review and Scientific Publication. We have continued our efforts to broaden the scope of the Congress to all aspects of peer review and publication—from funding to postpublication—and to all sciences.
 
 We will have 3 days for presentations of new research into peer review and all aspects of scientific publication, bias, quality of reporting, and information access and dissemination. There are 50 plenary session research presentations and 5 plenary invited talks. Each plenary session research presentation will be followed by equal time for discussion and questions from the audience. In addition, there are 125 posters. All posters, including virtual posters, are available to view throughout the meeting. In-person posters will also be presented at designated times on September 9 and 10. If you are attending virtually, please visit the individual poster pages and ask your questions in the chat box. See the link to Posters on the left.
 
 We hope you will take an active part in the program, as we depend on your participation in the discussion sessions to make the Congress a success.

 [Peer Review Congress Organizers, Directors, and Advisory Board](https://peerreviewcongress.org/organizers-and-advisory-board/)
 
 [Health and Safety Policies](https://web.cvent.com/event/1b505794-6a0b-428c-a44c-6c46c4c50e99/websitePage:152962bb-4567-4561-9d81-21ebfa97f487)

![](https://assets.underline.io/uploads/markdown_image/1/image/7c9f0143dee383780cc4670269ade132.png)

A PDF of the schedule is available [here](https://drive.google.com/file/d/1-Jd191W-nxbWcN1prdX5o9nlSfeo9_-H/view?usp=sharing) and a flipbook version of the complete program and abstracts is included below. See also the link to the interactive schedule to the left.

<iframe width="700px" height="425px" src="https://online.fliphtml5.com/ebtyf/nlfn/#p=1" frameborder="0" allowfullscreen allowtransparency></iframe>

Registration is required to attend this event, please use the URL linked below to secure your ticket. Registration will close on Friday, September 2 at 5:00 PM CT. 

[**Register Here**]( https://web.cvent.com/event/1b505794-6a0b-428c-a44c-6c46c4c50e99/summary)

Our aim is to encourage research into the quality and credibility of peer review and scientific publication, to establish the evidence base on which scientists can improve the conduct, reporting, and dissemination of scientific research.



**Importance** Bias, spin, and lack of transparency are
responsible for an important avoidable waste in research.

**Observations** Evidence on research transparency, trial
result availability, completeness, and accuracy of reporting
will be highlighted. The role of the peer review process, the
research environment, and the research ecosystem will be
considered.

**Conclusions** As stated by Doug Altman and David Moher,
“The scientific community and the public at large deserve an
accurate and complete record of research; we need to make
changes to ensure that we will get one.”1

**Reference**
1. Altman DG, Moher D. Declaration of transparency for each
research article. BMJ. 2013;347:f4796. doi:10.1136/bmj.f4796

**Conflict of Interest Disclosures** Isabelle Boutron is an advisory
board member of the International Congress on Peer Review and
Scientific Publication but was not involved in the review or decision
for this abstract.

![](https://assets.underline.io/uploads/markdown_image/1/image/26287e00e2f2a5caf3c67f2a2efb9c0e.png)

Inaugural Drummond Rennie Lecture: Bias, Spin, and Problems With Transparency of Research, Isabelle Boutron

**Objective** It is generally believed that evidence from a
weaker body of evidence (eg, poorly designed and executed
and potentially biased studies, sparse studies and/or
heterogenous results) will generate inaccurate estimates
about treatment effects more often than evidence from a
stronger body of evidence. As a result, estimates of effects of
health interventions initially based on high certainty (quality)
of evidence (CoE) are expected to change less frequently than
the effects estimated by lower CoE, and the estimates of
magnitude of effect size are expected to differ between high
and low CoE. Empirical assessment of these foundational
principles of evidence-based medicine has been lacking.

**Design** The Cochrane Database of Systematic Reviews was
reviewed from January 2016 through May 2021 for pairs of
original and updated reviews for change in CoE assessments
based on the Grading of Recommendations Assessment,
Development and Evaluation (GRADE) method. The
difference in effect sizes between the original and updated
reviews were assessed as a function of change in CoE, which
was reported as a ratio of odds ratios (RORs). The RORs
generated in the studies that changed CoE from very low/low
(VL/L) to moderate/high (M/H) vs MH/H to VL/L were
compared. Heterogeneity and inconsistency were assessed
using the τ and I2
statistic. The change in precision of effect
size estimates was assessed by calculating the ratio of
standard errors (seR), and the absolute deviation in estimates
of treatment effects was assessed with adjusted RORs.

**Results** Overall, 419 pairs of reviews were included, of which
414 (207 × 2) informed the CoE appraisal and 384 (192 × 2)
the assessment of effect size. Certainty of evidence originally
appraised as VL/L had 2.1 (95% CI, 1.19-4.12; P = .01) times
higher odds to be changed in the future studies than those
with M/H CoE. However, the pooled effect size was not
different when the CoE changed from VL/L to M/H (ROR,
1.02; 95% CI, 0.74-1.39) compared with M/H CoE changing to VL/L (ROR,1.02; 95% CI, 0.44-2.37). Similarly, the overlap
in aROR between the VL/L CoE to M/H vs the M/H to VL/L
subgroups was observed (median [IQR], 1.12; 95% CI,
1.07-1.57 vs 1.21; 95% CI, 1.12-2.43). There was a large
inconsistency across ROR estimates (I2
= 99%). There was
larger imprecision in treatment effects when the CoE changed
from VL/L to M/H (seR = 1.46) than when it changed from
M/H to VL/L (seR = 0.72).

**Conclusions** This study found that low-quality evidence
changed more often than high CoE. However, the effect size
was not systematically different between studies with low vs
high CoE, indicating the need for improving contemporary
critical appraisal methods.

**Conflict of Interest Disclosures** None reported.

![](https://assets.underline.io/uploads/markdown_image/1/image/050ec883e962e972a46b1f81946c6345.png)


Comparison of Changes in High-Quality vs Low-Quality Evidence in Original and Updated Systematic Reviews

**Objective** Problems with reporting a scientific study can
affect research in different manners. For example, it is known
that study methods are frequently not described in adequate detail and that results are presented ambiguously,
incompletely, or selectively. A previous study1
confirmed that
most of the authors (approximately 67%) publishing
systematic reviews (SRs) in high-impact rehabilitation
journals did not mention the use of a reporting guideline.
Therefore, the objectives of this study were (1) to evaluate the
completeness of reporting of SRs published in rehabilitation
journals by evaluating their adherence to the Preferred
Reporting Items for Systematic Reviews and Meta-Analyses
(PRISMA) 2009 checklist, (2) to investigate the relationship
between completeness of reporting and risk of bias (ROB),
and (3) to study the association between completeness of
reporting and the characteristics of SRs and journals.

**Design** A random sample of 200 SRs published between
2011 and 2020 in the 68 rehabilitation journals was indexed
under the rehabilitation category in the InCites Journal
Citation Report. Two independent reviewers evaluated the
completeness of reporting as the adherence to the PRISMA
2009 checklist and assessed ROB using the ROBIS tool.
Overall adherence and adherence to each PRISMA 2009 item
and section were calculated as a percentage of the total
number of items described and reported of the total number
of applicable items. Regression analyses investigating the
association between completeness of reporting, ROB, and
other characteristics (impact factor, publication options, year
of publication, and study protocol registration) were
performed.

**Results** The mean overall PRISMA 2009 adherence was
61.4% (**Figure 34**). Studies with high ROB in ROBIS domain
1 had an overall adherence to the PRISMA 2009 checklist,
which was 5.70% (95% CI, −10.07% to −1.34%) lower than
those with low ROB. In domain 2, studies with high ROB had
5.41% (95% CI, −9.74% to −1.07%) lower adherence than
those with low ROB. Studies with high overall ROB had
7.06% (95% CI, −12.10% to −2.01%) lower adherence than
those with low overall ROB. Studies published in fourth-
quartile journals displayed an overall adherence of 7.24%
(95% CI, −13.19% to −1.29%) lower than those published in
the first quartile; there was an 11.95% (95% CI, 5.94%-
17.96%) increase in overall adherence if the SR protocol was
registered.

![](https://assets.underline.io/uploads/markdown_image/1/image/eadc9d58246daba8b9456d19c7356564.png)

**Conclusions** The completeness of reporting in SRs
published in rehabilitation journals is still suboptimal. High
ROB is associated with poorer completeness of reporting, indicating that if a study has better reporting, this could
positively affect the transparency of the ROB evaluation. The
registration of study protocols and journal ranking are also
associated with a more complete reporting. Authors of SRs
should improve adherence to the reporting standards, and
journal editors could implement strategies to optimize the
completeness of reporting.

**Reference**

1. Innocenti T, Salvioli S, Giagio S, Feller D, Cartabellotta N,
Chiarotto A. Declaration of use and appropriate use of
reporting guidelines in high-impact rehabilitation journals is
limited: a meta-research study. J Clin Epidemiol.
2021;131:43-50. doi:10.1016/j.jclinepi.2020.11.010

**Conflict of Interest Disclosures** None reported.

![](https://assets.underline.io/uploads/markdown_image/1/image/c486ab2b5159af9894c4233312feafc4.png)



Completeness of Reporting and Its Association With Risk of Bias in Systematic Reviews Published in Rehabilitation Journals: A Meta-research Study

**Objective** To assess the publication status and factors
associated with subsequent publication of all pediatric
intervention trials registered in the Clinical Trial Registry of
India (CTRI) over a period of 5 years.

**Design** This cross-sectional study was conducted from
December 2021 to February 2022 and included the first 100
pediatric intervention trials registered in the CTRI from 2008
to 2012. Registry records were identified from the CTRI
website using the keywords pediatric, paediatric, children,
adolescent, infant, newborn, neonate, kids, and school. Trial
characteristics (eg, blinding, type of intervention and
comparator, setting, funding, source of funding, single or
multicentric, and postgraduate thesis) were abstracted from
each registered trial. A list of all the randomized clinical trials
registered on the website was made and their subsequent
publication was systematically searched on PubMed and
Google Scholar using their registered CTRI number up to
December 2021. For trials that were not found, repeat
searches were performed, searching by the first author’s
name, second author’s name, and title of the registered trial.
The proportion of trials subsequently published and the time
to publication from the date of registration were analyzed.
Factors associated with publication were compared between
trials that were published and those not published using χ2
test and univariate analysis by calculating the odds ratio and
95% CI. Multivariable logistic regression was conducted for
factors with P < .50.

**Results** The first 100 pediatric intervention trials registered
from 2008 to 2012 were retrieved from the CTRI. The overall
proportion of trials published was 71%; 87% had
randomization with a comparator arm, 78% examined
intervention for treatment, 22% examined intervention for
process of care change, 62% had retrospective registration,
51% had funding, and 39% had government funding. The
proportion of trials that were performed at a single center was
75%, 92% of trials were hospital based, 84% were conducted
at teaching hospitals, and 43% were postgraduate thesis
based. The median (range) time to publication was 4 (1-9)
years. A non–statistically significant higher proportion of
postgraduate thesis–based trials vs non–thesis-based trials
(79% vs 65%; OR, 2.04; 95% CI, 0.81-5.09) and single-center
trials vs multicenter trials (76% vs 56%; OR, 2.48; 95% CI,
0.96-6.44) were published. On multivariable logistic
regression, none of the factors were associated with higher
odds of publication (**Table 85**).

![](https://assets.underline.io/uploads/markdown_image/1/image/b8780e831d5fc2d23c3c10d09f844e28.png)

**Conclusions** Seventy-one percent of pediatric intervention
trials registered in the CTRI were subsequently published
regardless of their blinding, funding status, type of
intervention, comparator, or setting. A non–statistically
significant higher proportion of postgraduate thesis–based
trials and single-center trials were published. Further studies
with a larger sample size are needed to demonstrate any
statistical significance. Journal editors, funding agencies, and
ethics committees could implement mandatory registration of
intervention trials, as registration seems to be associated with
better quality and a good chance of subsequent publication.
Following guidelines for preparing data sets for submission to
data repositories could help achieve more trial registrations.

**References**

1. Rao MVV, Maulik M, Juneja A, et al. Clinical Trials
Registry—India: a decadal perspective. Indian J Pharmacol.
2020;52(4):272-282. doi:10.4103/ijp.IJP_24_20
2. DePasse JM, Park S, Eltorai AEM, Daniels AH. Factors
predicting publication of spinal cord injury trials registered
on www.ClinicalTrials.gov. J Back Musculoskelet Rehabil.
2018;31(1):45-48. doi:10.3233/BMR-169628
3. National Heart, Lung, and Blood Institute. Guidelines for
preparing clinical study data sets for submission to the
NHLBI data repository. Accessed February 20, 2022. https://
www.nhlbi.nih.gov/grants-and-training/policies-and-
guidelines/guidelines-for-preparing-clinical-study-data-sets-
for-submission-to-the-nhlbi-data-repository

**Conflict of Interest Disclosures** None reported.

![](https://assets.underline.io/uploads/markdown_image/1/image/067a7989d5d81ff19bd6b13bc8fbf9c2.png)

Factors Affecting Publication of Pediatric Intervention Trials

**Objective** Over the past decade, there has been an
exponential increase in systematic reviews and meta-analyses
(SR/MAs) published in the medical literature, and some
concerns have arisen that many of these are duplicate studies
with redundant results or new studies of unclear quality.1-3
The aim of this study was to determine whether there was any
change in the methodological or reporting quality of SRs in
dermatology over the past decade.

**Design** This was a cross-sectional study of all SR/MAs
published in 2010 and 2019 in the 10 highest-ranked
dermatology journals by SCImago Journal. Methodological
quality was assessed through duplicate and independent
adjudication by 2 reviewers (A.M., L.K.L., J.K., A.L.) using the
Risk of Bias in Systematic reviews (ROBIS) tool and, for SR/
MA of interventions, the A Measurement Tool to Assess
Systematic Reviews (AMSTAR) 2 tool. Reporting quality was
assessed by 1 investigator (A.M.) using the Preferred
Reporting Items for Systematic Reviews and Meta-Analyses
(PRISMA) and PRISMA for Abstracts (PRISMA-A) tools, and study, author, and journal characteristics were abstracted.
Statistical analysis was performed with Jamovi, version
1.6.23. Methodological quality was analyzed with the χ2 test
for independent proportions, or the Fisher exact test if any of
the expected frequencies were less than 5, and difference in
proportions with 95% CIs. Reporting quality was analyzed
with the independent samples t test and difference in means
with 95% CIs.

**Results** In total, 21 SR/MAs from 2010 and 127 from 2019
were included There was little to no difference between 2010
and 2019 in the proportion of SR/MAs at high or unclear
overall risk of bias with ROBIS or with critically low
methodological quality using AMSTAR 2. The only
subdomain of ROBIS with a difference in proportion of SR/
MAs at high or unclear risk of bias between 2010 and 2019
was eligibility bias, with 27.3% more (95% CI, 5.42%-49.2%)
in 2010 (66.7%) than 2019 (39.4%) (P = .02). There was a
strong difference in proportion of PRISMA (t146 = 3.15;
P = .002) and PRISMA-A (t146 = 2.46; P = .02) checklist items
adequately reported between 2010 and 2019. The difference
in mean proportion of PRISMA checklist items adequately
reported was 3.8 items more (95% CI, 1.4-6.2 items more) in
2019 (mean [SD], 15.7 [5.1] items) than in 2010 (mean [SD],
11.9 [5.5]), and of PRISMA-A checklist items adequately
reported was 0.9 items more (95% CI, 0.2-1.6 items more) in
2019 (mean [SD], 5.9 [1.5] items) than in 2010 (mean [SD], 5
[1.9] items).

**Conclusions** While there was an improvement in the overall
reporting quality of SR/MAs between 2010 and 2019, there
was no improvement in the overall methodological quality as
assessed by the ROBIS and AMSTAR 2 tools. This suggests a
persistent potential for forming unreliable conclusions in SR/
MAs through a decade of dermatological research. These
results therefore highlight the urgency with which efforts to
improve SR/MA methodology should be undertaken.

**References**

1. Siontis KC, Ioannidis JPA. Replication, duplication, and
waste in a quarter million systematic reviews and meta-
analyses. Circ Cardiovasc Qual Outcomes.
2018;11(12):e005212.
2. Katsura M, Kuriyama A, Tada M, et al. High variability in
results and methodological quality among overlapping
systematic reviews on the same topics in surgery: a meta-
epidemiological study. Br J Surg. 2021;108(12):1521-1529.
3. Gómez-García F, Ruano J, Gay-Mimbrera J, et al. Most
systematic reviews of high methodological quality on
psoriasis interventions are classified as high risk of bias using
ROBIS tool. J Clin Epidemiol. 2017;92:79-88.

**Conflict of Interest Disclosures** None reported.

![](https://assets.underline.io/uploads/markdown_image/1/image/8b3c3da9a1f0f9cbc94a7a79a4532217.png)

Methodological and Reporting Quality of Systematic Reviews in Dermatology

**Objective** Existing literature reviews about retraction do not
analyze postretraction citation. This research synthesized a
subgroup of empirical studies about retraction and reports
what is known about postretraction citation.

**Design** This was a subanalysis of a previously reported
scoping review.1
A total of 386 items about retraction were
found by double screening a systematic search of PubMed,
Scopus, and Web of Science up to February 10, 2021; Scopus-
cited reference searching in January 2021; and hand
searching up to July 2021.1
Items comprised published and
unpublished (work in progress) research reports. Subsequent
to the previous report, a custom taxonomy2
of methods,
research goals, and research was iteratively created and made
searchable via an online bibliography.3
For the subanalysis,
the scope was narrowed to postretraction citation after
excluding items on citation-related implications for review
literature and authors’ careers, publicity, and altmetrics. A
codebook to guide data extraction was developed and piloted.
Data extraction and analysis is ongoing.

**Results** This subanalysis included 92 items up to July 2021
on postretraction citation of retracted papers. Items were
classified into 7 topics: database-focused analyses (n = 33)
(eg, PubMed and Web of Science); field-based case studies (n
= 20) (eg, genetics, radiology, and thoracic surgery); paper-
focused case studies of 1 to 125 selected papers (n = 15);
author-focused case studies of 1 or several authors with many
retracted publications (n = 15) (eg, Bruening, Darsee, and
Reuben); studies of retracted publications cited in review
literature (n = 8); geographic case studies (n = 4) (focusing on
Brazil, the European Union, Italy, and South Korea); and
studies selecting retracted publications by method (n = 2) (eg,
human subjects and randomized clinical trials). Five items
were classified as belonging to 2 topics each. Empirical
research about postretraction citation has been published in a
diffuse set of journals, primarily in journals of ethics,
information science, meta-science and scientometrics, and domain sciences, especially medical specialties. The earliest 2
studies identified were both published in 1990; 1 study was an
author-focused case study of citations to Stephen E.
Breuning’s publications and the other studied 82 Index
Medicus articles retracted, with retraction notices in Index
Medicus as of 1990 and citations from SCISEARCH. From
1990 to 2017, a total of 1 to 3 items were found per year
except in 2016 (5 items), and from January 2018 to July 2021,
11 to 18 items per year were found on postretraction citation.
Almost all of these items focused on health sciences (eg,
medicine, dentistry, nursing, psychology, and pharmacy),
with 1 item focused on arts and humanities and 2 items
focused on engineering.

**Conclusions** Postretraction citation has been studied
consistently since 1990, with increasing attention since 2018.
This analysis found an increasing number of items on
postretraction citation from January 2018 to July 2021.
However, relevant work published after July 2021 was not
included, and items published earlier may have been missed
from the scoping review. Inclusion of work in progress may
have increased publication counts for the most recent years.

**References**

1. Schneider J, Woods ND, Proescholdt R, Fu Y; RISRS Team.
Recommendations from the Reducing the Inadvertent Spread
of Retracted Science: shaping a research and implementation
agenda project. MetaArXiv Preprints. Preprint posted online
July 28, 2021. doi:10.31222/osf.io/ms579 2. Proescholdt R, Schneider J. Comparison set guide for
empirical retraction lit. University of Illinois IDEALS
Repository. Deposited September 10, 2021. https://hdl.handle.net/2142/110389
3. Proescholdt R, Schneider J; RISRS Team. Empirical
retraction lit bibliography. Version 2.20. Zenodo. September
9, 2021. doi:10.5281/zenodo.5498500

**Conflict of Interest Disclosures** Jodi Schneider has been an
invited speaker for scholarly publishing organizations Committee on
Publication Ethics, CrossRef, the European Association of Science
Editors, the International Society of Managing and Technical
Editors, the Institute of Electrical and Electronics Engineers,
the National Information Standards Organization, and STM;
has received data-in-kind from Retraction Watch and scite; and
has received usability testing compensation from the Institute of
Electrical and Electronics Engineers. No other disclosures were
reported.

**Funding/Support** This project was supported by the Alfred P.
Sloan Foundation (G-2020-12623).

**Role of the Funder/Sponsor** The study sponsor had no role
in the design and conduct of the study; collection, management,
analysis, and interpretation of the data; preparation, review, or
approval of the abstract; and decision to submit the abstract for
presentation.

![](https://assets.underline.io/uploads/markdown_image/1/image/750e15b2579ba642752748359dd97985.png)

Characterization of Publications on Post-Retraction Citation of Retracted Articles

**Objective** To investigate the utility of machine learning
algorithms in predicting the likelihood of publication of a
manuscript from peer reviewer scores. 
 **Design** Using a cross-sectional study design, 263
manuscripts that had undergone peer review between 2017
and 2021 were selected, and a final decision made on whether
to accept or reject a manuscript for publication. Excluded
were manuscripts with incomplete data on peer reviewer
scores. Data were collected on the manuscripts’ peer reviewer
scores and the final decision made by the journal. Peer
reviewers’ scores included ratings by 2 peer reviewers per
manuscript on originality, quality, interest, overall rating, and
priority for publishing. Two-thirds (174 manuscripts) of the
data were used for training and one-third (89 manuscripts)
for testing the algorithms. Microsoft Excel 2019 was used to
preprocess the data and Weka version 3.9.5 was used for
model assessment. Training and testing of the model were
conducted using various machine learning algorithms. The
model with the highest accuracy in predicting the likelihood
of a manuscript to be published would be further improved
and deployed for application. 
 **Results** One-hundred and thirty-four manuscripts were
accepted for publication and 129 rejected for the final
analysis. The performance of various machine learning
algorithms in predicting the likelihood of publication ranged
from 58.4% to 65.2% (**Table 27**). 
 ![](https://assets.underline.io/uploads/markdown_image/1/image/4d8feaa71c3089d11832644ff718f032.png) 
 **Conclusions** A machine learning model to reduce peer
review workload would ensure that scarce peer review
resources are utilized by optimizing desk rejections. Such a
model would promote efficiency in the publishing process and
improve overall journal output and satisfaction with authors.
To implement such a model, the in-house reviewers and
editorial team could score manuscripts and assess their
performance before advancing them for peer review. To
improve the model’s performance and reduce bias, there
would be a need to enhance the selection of data variables for
scoring the manuscripts with a greater focus towards
objective variables. Other limitations included small sample
size and possible interrater variability in scoring individual
manuscripts. 
 **References**
1. Checco A, Bracciale L, Loreti P, Pinfield S, Bianchi G.
AI-assisted peer review. Humanit Soc Sci Commun.
2021;8(1):25. doi.org/10.1057/s41599-020-00703-8 
 2. Heaven D. AI peer reviewers unleashed to ease publishing
grind. Nature. 2018;563:609-610. doi.org/10.1038/d41586-
018-07245-9 
 **Conflict of Interest Disclosures** Dr Kigera is a member of the
Peer Review Congress Advisory Board but was not involved in the
review or decision for this abstract. 
 ![](https://assets.underline.io/uploads/markdown_image/1/image/1a54813c25d43207c9340f6e22ccb977.png)

Utility of Machine Learning in Predicting Success of a Peer Review Paper From Peer Reviewer Scores

**Objective** To examine the levels of awareness, preferences,
experiences, and practices of researchers in the health and life
sciences regarding research and publication ethics. 
 **Design** In this cross-sectional study, a questionnaire was
deployed on Google Forms to a global audience. Reminder
emails and WhatsApp messages containing a brief description
and a link to the online survey were sent through the various
platforms until no new responses were received after 3
reminders. The form captured information regarding the
sociodemographic characteristics of respondents. The
questionnaire explored the level of awareness, training,
attitudes toward, preferences, and experiences with research
ethics committees. Regarding publication ethics, the
awareness of respondents regarding the International
Committee of Medical Journal Editors (ICMJE) authorship
criteria and their experiences with ghost authorship were
evaluated. All researchers working in the health and life
sciences were eligible to participate in the study. 
 **Results** A total of 500 researchers were contacted, and 264
responded (53.0%). Only 36.7% of respondents were aware of
the ICMJE authorship criteria. Less than a quarter (22.0%) of
the respondents were aware of the existence of an ethics code.
Respondents’ experience with their most recent ethics
approval application was poor (11.4%), good (36.0%), and
excellent (4.2%). The practice of research teams to include
authors with no or limited significant contribution to an
article was frequent (44.3%), common (29.5%), and systemic
(10.6%). More than 41.7% of the respondents had ever
conducted a study involving human participant research
without prior ethical approval. Respondents’ experience with
their most recent ethics approval application was poor
(11.4%), good (36.0%), and excellent (4.2%). Major challenges
in obtaining ethical approval were too much bureaucracy
(47.3%), ethical approval application cost (5.3%), and unduly
long review turnaround in receiving feedback and decisions
(3.8%). Most respondents (83.0%) worked in institutions that
hosted a research and ethics committee. Less than half
(42.4%) of the respondents had been formally trained in
publication ethics. 
 **Conclusions** Rates of ethical misconduct, such as having no
ethical approval prior to conducting a study or gift and ghost
authorships, were unacceptably high. Formal training in
research and publication ethics should be institutionalized in
the courses in universities and research institutions.
Academic journals and funders have the duty to support
researchers to uphold research ethics and research integrity
standards. Institutionalization and awareness raising
regarding these best practices are highly needed. 
 **Conflict of Interest Disclosures** None reported. 
 ![](https://assets.underline.io/uploads/markdown_image/1/image/67898825b9d16f150e5e3d08f5e15cb0.png)

Characteristics and Opportunities for Improvement of Methods Guidance Published in General and Methodology-Focused Medical Journals

Next from Peer Review Congress 2022

Inaugural Drummond Rennie Lecture: Bias, Spin, and Problems With Transparency of Research, Isabelle Boutron

Similar lecture

Digital Phenotyping and Machine Learning in the Next Generation of Digital Health Technologies: Utilising Event Logging, Ecological Momentary Assessment & Machine Learning

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES