United States

**Objective** This research aimed to perform a comparison of
basic bibliometrics of leading Chinese open access (OA)
journals from the Excellence Action Plan for Chinese Science,
Technology, and Medicine Journals. All journals used English
as the publication language.

**Design** This was a cross-sectional investigation reported
following the STROBE checklist for conference abstracts.
Journal data were extracted from the Web of Science (WoS)
and Directory of Open Access Journals databases. All
searches were performed in November 2021, and the Journal
Citation Reports 2020 impact factor release was considered
the latest available data set that allowed for retrieval of
relative data. The top 2 international OA journals in the first
quartile of the journal impact factor rank based on WoS
categories for each leading Chinese OA journal were chosen
as control journals. Bibliometric indicators, such as the
volume of published documents, citations, and Eigenfactor
score (based on items published between 2018 and 2020), were compared between leading Chinese OA journals and
control journals with nonparametric tests.

**Results** A total of 14 of 22 (63.64%) leading Chinese journals
were OA journals. For 2 OA journals, there was no
corresponding international OA journal; for another 2 OA
journals, there was only 1 corresponding OA journal.
Therefore, there were 22 control journals. The median (IQR)
number of published documents (articles and reviews) per
leading Chinese OA journal was less than that of the control
journals (302 [243-352] vs 715 [390-2010]; P = .004); the
median (IQR) citations per document (13 [9-26] vs 14 [9-25];
P = .99) and percentage of cited documents (94.81% [93.63%-
98.24%] vs 96.96% [92.64%-98.14%]; P = .76) were not
statistically different. The median (IQR) normalized citation
impact of the 2 groups was not significantly different (2.008
[1.221-2.933] vs 2.003 [1.353-3.093]; P = .97), but the median
(IQR) Eigenfactor score, which took into account both the
number of citations and the academic influence of citing
journals, of leading Chinese OA journals was lower than that
of control journals (0.005 [0.003-0.007] vs 0.013 [0.005-
0.052]; P = .02]. There were fewer top 1% documents in
leading Chinese OA journals (median [IQR], 10 [5-19] vs 34
[11-65]; P = .02); however, the percentage of top 1%
documents was not statistically different (median [IQR],
4.54% [1.18%-5.82%] vs 3.84% [1.86%-6.86%]; P = .94). The
average citations per top 1% document of leading Chinese OA
journals were more than that of control journals, but the
difference was not statistically significant (median [IQR], 103
[52-134] vs 72 [58-105]; P = .26).

**Conclusions** Compared with top international OA journals,
Chinese leading OA journals published fewer articles with
similar average citations, but they were less cited by highly
cited journals.

**Conflict of Interest Disclosures** None reported.

![](https://assets.underline.io/uploads/markdown_image/1/image/b465371259175e41a935b95b43682d2e.png)

Peer Review Congress 2022

Comparison of Bibliometrics of Leading Open Access Chinese Journals with Leading Non-Chinese Journals in Science, Technology, and Medicine

[![](https://assets.underline.io/uploads/markdown_image/1/image/81a3c0317d24f663b49b996875024d45.png)](https://aclanthology.org/volumes/2022.coling-1/)



### THE CONFERENCE WE KNOW AND WE WANT.
**COLING**, the International Conference on Computational Linguistics, is one of the premier conferences for the natural language processing and computational linguistics.

First established in 1965, the biennial COLING conference is held in diverse parts of the globe and attracts participants from both top-ranked research centers and emerging countries. Today, the most important developments in our field are taking place not only in universities and academic research institutes but also in industrial research departments including tech-startups. COLING provides opportunities for all these communities to showcase their exciting discovery.

In fall of 2022, COLING will be held in Gyeongju in a hybrid format. All participants can either present at the venue site or join virtually. As more people get vaccinated, we are happy to provide safer environments for our colleagues. We believe that COLING 2022 will be one of the conferences, free from the pandemic. The hybrid format gives presenters and sponsors a valuable opportunity to promote their companies in both an online and in-person venue. For the first time in a long time, customers can interact with their sponsor's products first-hand. The online venue, too, gives sponsors the chance to network with those unable to attend the in-person session.

The hybrid format ultimately lends itself to greater exposure for our sponsors: COLING2022 will let you reach more potential partners and customers than ever before!

To gain access to this event page you are required to register. Find more information and pay the registration fee on event the organizer’s website **[https://coling2022.org/reg](https://coling2022.org/reg)**

COLING 2022

COLING, the International Conference on Computational Linguistics, is one of the premier conferences for the natural language processing and computational linguistics.

poster

Our aim is to encourage research into the quality and credibility of peer review and scientific publication, to establish the evidence base on which scientists can improve the conduct, reporting, and dissemination of scientific research.
 
 **Welcome!**
 
 The JAMA Network, The BMJ, and METRICS welcome you to the Ninth International Congress on Peer Review and Scientific Publication. We have continued our efforts to broaden the scope of the Congress to all aspects of peer review and publication—from funding to postpublication—and to all sciences.
 
 We will have 3 days for presentations of new research into peer review and all aspects of scientific publication, bias, quality of reporting, and information access and dissemination. There are 50 plenary session research presentations and 5 plenary invited talks. Each plenary session research presentation will be followed by equal time for discussion and questions from the audience. In addition, there are 125 posters. All posters, including virtual posters, are available to view throughout the meeting. In-person posters will also be presented at designated times on September 9 and 10. If you are attending virtually, please visit the individual poster pages and ask your questions in the chat box. See the link to Posters on the left.
 
 We hope you will take an active part in the program, as we depend on your participation in the discussion sessions to make the Congress a success.

 [Peer Review Congress Organizers, Directors, and Advisory Board](https://peerreviewcongress.org/organizers-and-advisory-board/)
 
 [Health and Safety Policies](https://web.cvent.com/event/1b505794-6a0b-428c-a44c-6c46c4c50e99/websitePage:152962bb-4567-4561-9d81-21ebfa97f487)

![](https://assets.underline.io/uploads/markdown_image/1/image/7c9f0143dee383780cc4670269ade132.png)

A PDF of the schedule is available [here](https://drive.google.com/file/d/1-Jd191W-nxbWcN1prdX5o9nlSfeo9_-H/view?usp=sharing) and a flipbook version of the complete program and abstracts is included below. See also the link to the interactive schedule to the left.

<iframe width="700px" height="425px" src="https://online.fliphtml5.com/ebtyf/nlfn/#p=1" frameborder="0" allowfullscreen allowtransparency></iframe>

Registration is required to attend this event, please use the URL linked below to secure your ticket. Registration will close on Friday, September 2 at 5:00 PM CT. 

[**Register Here**]( https://web.cvent.com/event/1b505794-6a0b-428c-a44c-6c46c4c50e99/summary)

Our aim is to encourage research into the quality and credibility of peer review and scientific publication, to establish the evidence base on which scientists can improve the conduct, reporting, and dissemination of scientific research.



**Objective** This systematic review aimed to describe the
characteristics of clinical and biomedical research in
Malaysia.

**Design** A search was conducted on PubMed, Embase,
CINAHL, PsycINFO, and MyMedR (http://mymedr.afpm.
org.my/) for published clinical and biomedical research in primary care settings from 1962 to 2017 by Malaysian authors
in a Malaysian institution. Studies found were independently
screened by a team of reviewers and information was
extracted.1,2 In phase 1, the characteristics of the research and
profiles of the researchers and journals in which they were
published were reported descriptively. In phase 2, the quality
of studies included in phase 1 will be assessed using a newly
developed tool to ascertain risk of bias. Longitudinal trends of
the research characteristics, health conditions studied
(International Classification of Primary Care), and settings,
among other characteristics, were explored. No synthesis of
results was conducted as no effect estimates were available to
be pooled.

**Results** Of 4513 articles, 1078 were included in this
qualitative synthesis and 790 with complete data were
analyzed. Clinical studies (81.9%), primary research (81.1%),
and quantitative studies (74.2%), consisting mostly of
prevalence studies (67.7%) by cross-sectional sampling
(70.4%), were predominant. The number of studies increased
(**Figure 30**) and the number of characteristics also increased
after year 2000. Researchers from family medicine (39.3%)
and public health (15.2%) specialties were the main
contributors to the articles (**Figure 30**, A). Most of the
corresponding authors had a master of medicine degree
(46.5%) compared with a doctor of philosophy (PhD) (24.4%)
or doctor of medicine (MD) (5.2%) degree. Researchers with
PhD and MD degrees were more likely to conduct
interventional studies compared with those with master’s
degrees (8.0% vs 6.4% of studies; χ2 = 54.26; P = .03).
Publications were mainly original research (82.8%) in
international (48.6%) or local (35.0%) journals and were
evenly distributed between multidisciplinary (51.6%) and
discipline-specific (46.6%) journals. The number of authors
per article was most often fewer than 5 (73.6%), and the
number of collaborating institutions was predominantly
fewer than 3 (82.5%). Incidences of coauthorship and collaboration with overseas researchers were few but showed
a significant increasing trend in the last decade (Figure 30,
B). The top 5 conditions studied were general and unspecified
(37.9%); endocrine, metabolic, and nutritional (15.2%);
circulatory (7.8%); psychological (5.9%); and respiratory
(4.9%).

![](https://assets.underline.io/uploads/markdown_image/1/image/46f7fc54671d85ca7536705d5864ba71.png)

**Conclusions** The longitudinal and prospective trends of the
research characteristics assessed in this analysis provided
suggestions of improvement initiatives needed for primary
care research enterprise in Malaysia. This includes training
on the proper use of different study designs, on developing a
better supportive ecosystem for interventional clinical trials,
on skills for international research collaboration, and for
strategizing research topics that meet the issues of primary
medical care. Similar works in other disciplines could be
initiated and better conducted after this first experience. The
aim of phase 2 will be to validate a research-quality screening
tool based on domains of relevance, credibility of the
methods, and usefulness of the results.

**References**

1. Chew BH, Lim PY, Lee SWH, et al. A systematic review of
medical and clinical research landscapes and quality in
Malaysia and Indonesia [REALQUAMI]: the review protocol.
PROSPERO identifier: CRD42020152907. Updated April 28,
2020. Accessed July 14, 2022. https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=152907
2. Chew BH, Lim PY, Lee SWH, et al. A systematic review of
medical and clinical research landscapes and quality in
Malaysia and Indonesia [REALQUAMI]. OSF Registry for
Research on the Responsible Conduct of Research. May 1,
2019. Accessed July 14, 2022. https://osf.io/w85ce

**Conflict of Interest Disclosures** None reported.

![](https://assets.underline.io/uploads/markdown_image/1/image/37b0ecbcea3db3dda80fcaec6d325d90.png)

A Systematic Review of Medical and Clinical Research Landscapes in Primary Medical Care in Malaysia

**Objective** To compare financial conflict of interest (COI) as
declared by authors of systematic reviews (SRs) of
methylphenidate for attention-deficit disorder with publicly
available information and in relation to risk of bias (RoB).

**Design** SRs on the outcomes associated with
methylphenidate for attention-deficit disorder in all ages were
searched in Medline, Cochrane, Embase, and PsycInfo. The
Preferred Reporting Items for Systematic Reviews and
Meta-analyses (PRISMA) reporting guideline was followed
for the selection of relevant SRs. Two reviewers (A.S. and
S.C.) independently screened open websites and recent
publications for all authors of each SR, and data on financial
COI were extracted. All searches followed a preplanned and
similar routine. A time limit of 3 years from publication of the
index SR was applied. Findings were discussed between the
reviewers and repeated until consensus was reached. If no
data were found for any of the authors, the SR was
categorized as no COI. Two reviewers (A.S. and L.O.)
independently judged RoB of the SR using the Risk of Bias in
Systematic Reviews (ROBIS) tool. Any disagreement was
resolved in consensus. In addition, data were retrieved on
COI as declared by authors.

**Results** Of 651 unique publications, 44 relevant SRs
published between 2008 and 2021 were included. In all, 32
SRs (73%) were based on randomized clinical trials only, 18
(41%) reported positive effects only, 15 (34%) reported both
positive and adverse effects, and 11 (25%) reported adverse
effects only. Eleven SRs (25%) included only studies using
placebo for comparison. A meta-analysis was conducted in 26
of the SRs (59%). COI disclosure was missing for 2 of 44 SRs
(5%). For 15 SRs (34%), authors declared COI, and this
declaration was confirmed by open sources in all cases. For 27
SRs (61%), the authors declared no COI, but discordant
information was publicly available for 8 of 27 (30%). The
direction of COI for most SRs was not able to be assessed. The
RoB was high in 37 of 44 SRs (84%). Of the 7 SRs with low
RoB, 1 had no COI identified in open sources (**Table 78**).

![](https://assets.underline.io/uploads/markdown_image/1/image/df8e29168dd1f7fe42b80257f1d7c32c.png)

**Conclusions** The findings indicated an underreporting of
COI in SRs in studies on attention-deficit disorder, and most
SRs were compromised by high RoB. Owing to small
numbers, no firm conclusion on the association between COI
and RoB was possible.

**Conflict of Interest Disclosures** None reported.

![](https://assets.underline.io/uploads/markdown_image/1/image/43fce7efe9e39c5aa2156e3ba3ad6038.png)



Conflicts of Interest in Systematic Reviews on Methylphenidate for Attention-Deficit Disorder

**Objective** As gatekeepers for knowledge production, editors
use their backgrounds to determine what counts as high-
quality research, how research should be done, and whether it
offers novel insights. Individuals who come from racialized
backgrounds are often excluded from these conversations,
ensuring the publishing world continues to be led by the
perspectives of the dominant racial group.1
To address this
issue, the journal Teaching and Learning in Medicine (TLM)
created an internship program for early career scholars who
self-identified as Black, Latinx, or Indigenous to gain
experience in the publishing/editorial process.2
The purpose
of this study was to critically examine the outcomes of the
first 6 months and investigate its implications for other
journals interested in similar efforts.

**Design** A collaborative autoethnographical study was
designed to collectively analyze the program. Editorial staff
and interns reflected on their lived experiences participating
in the internship to understand the current culture of
academic publishing and explore how to improve.3
Data
sources included archival emails and program planning
documents, focus group data, and group exercises. After
transcribing the focus groups, data were analyzed using thematic analysis by both editorial staff and interns.
Institutional review board approval was not required for this
study.

**Results** Focus group data indicated that while TLM designed
the program with antiracist intent, mentors did not
foreground their race in the editorial/publishing process
outside of the intern’s minoritized status as a program
selection criterion. Early on, mentors viewed the internship
more as an opportunity to improve the journal’s editorial
work, for participants to gain experience working in a
historically guarded space, and to contribute to achieving
racial equity in medical education. Therefore, despite the
program’s specific antiracist focus, 6 months in, none of the
mentors (who identified as White individuals) had explicitly
discussed topics of race with their interns. Mentors’ reasons
for not discussing race varied, including uncertainty about
how to invite interns into such a discussion and not seeing
interns as racialized individuals. However, at the end of the
6 months, researchers realized the need to discuss this topic,
thus moving the program into explicit conversations about
race and the role it plays in publishing.

**Conclusions** Although the program met some antiracist
goals, stakeholders did not explicitly discuss the role that race
plays in the review process, therefore limiting the program’s
initial impact. Through this collaborative autoethnography,
TLM stakeholders critically reflected on the program in real
time and addressed this gap. In doing so, they engaged in the
ongoing critical action needed to support equity within the
editorial process. Editorial staff at TLM now have the
opportunity to address ongoing power dynamics between
interns and staff members that will advance the journal’s
efforts at antiracism. Plans are currently underway to ensure
interns’ experiences are incorporated.

**References**

1. Salazar JW, Claytor JD, Habib AR, Guduguntla V, Redberg
RF. Gender, race, ethnicity, and sexual orientation of editors
at leading medical and scientific journals: a cross-sectional
survey. JAMA Intern Med. 2021;181(9):1248-1251.
doi:10.1001/jamainternmed.2021.2363
2. Cianciolo AT. Letter from the editor—Teaching & Learning
in Medicine’s anti-racism strategy. Teach Learn Med.
2020;32(5):457-458. doi:10.1080/10401334.2020.1836461
3. Chang H, Ngunjiri FW, Hernandez KAC. Collaborative
Autoethnography. Left Coast Press; 2013.

**Conflict of Interest Disclosures** None reported.

**Disclaimer** This work was prepared by a military or civilian
employee of the US government as part of the individual’s official
duties and therefore is in the public domain. The opinions and
assertions expressed herein are those of the author(s) and do not
necessarily reflect the official policy or position of the Uniformed
Services University or the US Department of Defense.

**Additional Information** Anabelle Andon is a co–corresponding
author.

![](https://assets.underline.io/uploads/markdown_image/1/image/a8d384158802bc2cfad5c9d7e26fb85b.png)

Qualitative Assessment of an Antiracism Editorial Internship Program for Early Career Underrepresented Scholars at Teaching and Learning in Medicine

**Objective** Editorial decisions can depend on factors, like the
timing of submissions or the matching of editors and
reviewers, that are independent of the quality of the work.
This analysis investigates associations of these and other
external factors with editorial outcomes at eLife, a major
biomedical journal.

**Design** This study analyzed whether timing of submission
(weekend vs weekday) was associated with the decision to
send manuscripts for external review among senior editors,
who can desk-reject submissions; compared peer review
manuscript ratings by reviewer volume; and assessed whether
submission time of month or year, preceding decisions to
reject or review a manuscript, and matching of reviewer to
manuscript specialty influenced editorial decisions. Data were
analyzed by single variable regression.

**Results** Between January 2016 and December 2018 eLife
received 23,190 total submissions, 6498 of which were sent
for review. Among senior editors (n = 65), proportions of
manuscripts sent for external review ranged from 9.6% to
49.3% and were statistically significantly lower on weekends
(mean, 24% [SD, 1.3%]) than on weekdays (mean, 29% [SD,
0.6%]; P < .001), an association observed for most senior
editors. Average peer reviewer rating (range, 0-1) increased
with volume category: mean of 0.453 (SD, 0.003) for 1 to 5
submission reviews; 0.463 (SD, 0.008) for 6 to 10
submissions; and 0.472 (SD, 0.007) for 11 or more
submissions, and reviewers’ ratings increased with successive
reviews. In a nonquantitative inspection of submission and
decision trends, submission time of month or year, preceding
decisions to reject or review a manuscript, and matching of
reviewer to manuscript specialty did not appear to influence
editorial decisions.

**Conclusions** This study found a statistically significant
association between timing of submission during the week
and editorial decisions. Peer reviewer ratings increased with
review volume. Submission time of month or year, preceding
decisions to reject or review a manuscript, and matching of reviewer to manuscript specialty did not appear to influence
editorial decisions.

**Conflict of Interest Disclosures** None reported.

**Funding/Support** This research is supported by National Science
Foundation SCISIPBIO (A Science of Science Policy Approach to
Analyzing and Innovating the Biomedical Research Enterprise)
project 2022435.

**Additional Information** We thank Andy Collings and James
Gilbert from eLife for sharing the data and providing explanations
and eLife’s editorial leadership team for input and suggestions.

![](https://assets.underline.io/uploads/markdown_image/1/image/7aad5dd3d2641b2ae252900ca6469e10.png)

Analysis of Timing of Manuscript Submissions and Assignment of Editors and Reviewers on Editorial Decisions at eLife

**Objective** Many journals encounter difficulties obtaining
peer reviewers. However, it is unknown if this is associated
with publication outcomes.1,2 The purpose of this research
was to evaluate whether, for initial manuscript submissions
that were eventually peer reviewed, there was a significant
association between the number of external peer reviewers
sought, unsuccessful invites, and declined invites for external
peer review and the publication outcome.

**Design** This was a retrospective study of anonymized,
unique, original research submissions to the Emergency
Medicine Journal (EMJ) that received at least 1 external peer
review over a 5-year period (January 2016 to December
2020). A database of deidentified original manuscripts
submitted to EMJ during the study period was interrogated to determine if there were significant associations between the
number of unsuccessful external peer review invitations, the
number of total invitations needed, and the acceptance or
rejection of a research manuscript. Original submissions
without any external peer review were excluded. Statistical
review invitations were excluded from the data. Mann-
Whitney U test was used to assess differences between the
variables and their publication outcomes. Odds ratios (OR),
likelihood ratios (LR), and positive predictive value (PPV)
were used as measures of association for potential thresholds
for variables and publication outcomes.

**Results** There were 806 deidentified peer-reviewed original
submissions included with 85 manuscripts (10.6%) accepted
for publication during the study period (**Table 79**). The ORs
for a peer-reviewed original research submission eventually
being rejected, according to number of invitations, were 77.5
(95% CI, 24.2-248.2; P < .001; (LR, 21.0; 95% CI, 6.9-63.7;
PPV, 99.4%; 95% CI, 98.3-99.8) for submissions with 4 or
more invitations, 36.8 (95% CI, 9.0-150.9; P < .001; LR, 20.0;
95% CI, 5.1-78.8; PPV, 99.4%; 95% CI, 97.7-99.9) for
submissions with 3 or more unsuccessful external peer review
invitations, and 22.6 (95% CI, 3.1-163.8; LR, 18.0; 95% CI,
2.6-127.2; PPV, 99.4%; 95% CI, 95.6-99.9) for submissions
with 2 or more peer reviewers who declined review
invitations.

![](https://assets.underline.io/uploads/markdown_image/1/image/c65a6b7606fb2ba5ad006d5543dce3f1.png)

**Conclusions** The number of declined peer review invitations
and total review invitations prior to a decision were
associated with rejection of a manuscript. The wide 95% CIs
in these results could be due to the high variability of
underlying factors that could have influenced the difficulty in
getting peer reviews and their interplay with the decision to
publish. The findings of this study may also be potentially
different for different journals.3
Further research should be
done to provide further insights on specific factors that may
be associated with difficulties in getting peer reviewers.1-3

**References**

1. Tvina A, Spellecy R, Palatnik A. Bias in the peer review
process: can we do better? Obstet Gynecol. 2019;133(6):1081-
1083. doi:10.1097/AOG.0000000000003260
2. Tennant JP, Dugan JM, Graziotin D, et al. A multi-
disciplinary perspective on emergent and future innovations
in peer review. F1000Res. 2017;6:1151. doi:10.12688/
f1000research.12037.3
3. Stamm T, Meyer U, Wiesmann HP, Kleinheinz J, Cehreli
M, Cehreli ZC. A retrospective analysis of submissions,
acceptance rate, open peer review operations, and
prepublication bias of the multidisciplinary open access
journal. Head Face Med. 2007;3:27. doi:10.1186/1746-
160X-3-27

**Conflict of Interest Disclosures** None reported.

![](https://assets.underline.io/uploads/markdown_image/1/image/fe4b8ba267367c8f45db9c17a01bae41.png)

Association Between Number of External Peer Review Invites, Unsuccessful Invites, and Declined Reviews With Rejection of Manuscripts

**Objective** Over the past decade, several journals in Africa
have transitioned from email submission systems to web-
based platforms for manuscript submission and editorial
processes.1
Previous reports1,2 assessing experience with these
systems have revealed conflicting feedback from authors and
editorial teams, with varying reports of user friendliness.
Notably, a letter to the editor published in Nature3
stated that
such systems “make authors do all the work.” Over time,
online submission systems have sought to have in-built
programs to enable automation of the submission process to
ease the authors’ experience. The Annals of African Surgery
uses the submission prefill feature in its submission system
for the aforementioned purpose.

**Design** This study was cross-sectional. To establish efficacy
and gather authors’ experience of the online submission
system and the submission prefill (ie, the system that
automatically extracts and populates submission fields for
authors from an uploaded manuscript text file), authors
submitting articles to the Annals of African Surgery were
surveyed between September and December 2021. This
survey was delivered to authors using the manuscript
submission system, during the submission process, and at the
end of the editorial workflow after the final decision was
made.

**Results** A total of 44 authors completed the survey on
submission prefill, and 42 completed the survey on the
manuscript submission system. Because the surveys were embedded as part of the submission processes, the response
rate was 100%. Overall, the responses to the question of
whether the submission prefill was effective in reducing the
workload were divided evenly (50.0% agree vs 50.0%
neutral), but there was overwhelmingly positive feedback on
the effectiveness of manuscript submission system in terms of
its ease of use (88.1%) and in obtaining the status of the
manuscript (92.9%). The survey questions and their
respective responses are as outlined in **Table 80**.

![](https://assets.underline.io/uploads/markdown_image/1/image/e973f191539223b5f0e1324aee8ab981.png)

**Conclusions** There was overall positive feedback regarding
authors’ interaction with the web-based online submission
interface. Feedback from authors’ experience on whether
submission prefills reduce submission labor, however, still
remains divided. Measures taken to improve the prefill’s
capacity to extract manuscript details with better accuracy
would, hence, enhance authors’ experience.

**References**

1. Ware M. Online submission and peer-review systems.
Learned Publishing. 2005;18(4):245-250.
doi:10.1087/095315105774648771
2. Ware M. Authors say that they prefer online submission.
Nature. 2005;434(7033):559. doi:10.1038/434559d
3. Moore JP. Online submission makes authors do all the
work. Nature. 2005;433(7028):800. doi:10.1038/433800d

**Conflict of Interest Disclosures** James Kigera is a member of
the Peer Review Congress Advisory Board but was not involved in
the review or decision for this abstract. No other disclosures were
reported.

![](https://assets.underline.io/uploads/markdown_image/1/image/90eea1861b5ade470345f3fae96381f8.png)



Authors' General Experiences With Submitting Manuscripts and With Submission Prefill to a Manuscript Submission System for the Annals of African Surgery

**Objective** The objective of this research was to assess
whether an editorial workflow wherein submissions are
handled solely by dedicated protocol editors reduces the
average time from initial submission to a final decision
compared with standard peer review and whether this
reduction differs by protocol formatting (structured or
unstructured).

**Design** A cross-sectional study of the workflow timing for
protocol submissions to Trials was conducted. Type A
protocols demonstrate prior peer review as part of their
funding process by providing previous comments and
self-declaring this prior review in the submission process.
Type A protocols are handled by protocol editors as the sole
reviewers, whereas type B protocols are handled by other
editors and require full peer review (ie, 2 to 3 reviewers). The
workflow data stored in the Trials manuscript submission
and review system were extracted to compare the timing from
initial submission to first and final decisions for all protocols,
separated by type, before and after protocol editors were
implemented (January to December 2019 and January 2020
to November 2021, respectively). The timing was also
extracted for submissions that were formatted as
unstructured protocols with an accompanying SPIRIT
checklist or following the Structured Protocol Template
recommended by Trials. The workflow timing results are
descriptively summarized.

**Results** From January 2020 to November 2021, 1114 type A
(360 [32%]) and type B (754 [68%]) trial protocols were
submitted to Trials. Compared with type B protocols, 143 of
which (19%) used a structured template, 137 type A protocols
(38%) followed a structured format, possibly owing to self-selection or prior reviews leading to a more structured
approach in submission. A timeline of the workflow for each
protocol type by format as well as overall for each period is
shown in **Figure 31**. Overall, type A protocols had a mean of
118 days to final decision (from initial submission), whereas
type B protocols had a mean of 165 days to final decision.
Overall, across protocols in the period after protocol editors
were implemented, the time to final decision was reduced 30
days from the preceding year: 146 days (January 2020 to
November 2021) versus 176 days (January to December
2019). Among type A protocols, a structured template format
reduced the time to final decision compared with
unstructured—an association that was not seen in type B
protocols, possibly because multiple reviewers were required.

![](https://assets.underline.io/uploads/markdown_image/1/image/41a674ef8c75bec03869985bec7d5311.png)

**Conclusions** The use of dedicated editors to handle trial
protocols that had already undergone peer review was
associated with fewer days to final decision compared with
protocols that were handled by other editors and required full
peer review. Future work should examine the quality of peer
review in both workflows to determine whether use of
dedicated reviewers improves the quality of published works
as well as the time to publication.

**Conflict of Interest Disclosure**s Riaz Qureshi, Kirsty Loudon,
and Alexander Gough are protocol editors of Trials, BMC. Shaun
Treweek and Tianjing Li are editors in chief of Trials, BMC.

**Acknowledgments** We would like to thank Eleanor Cox, Bex
Chang, and Krishna Vairamani for their work as editorial staff and
for managing the workflow of peer review for Trials.

![](https://assets.underline.io/uploads/markdown_image/1/image/ce8b2b25aa2173d0019073c47efcd780.png)



Assessment of Use of Dedicated Editors for Handling and Reviewing Manuscripts With Previously Obtained Peer Reviews

**Objective** Peer review plays a fundamental role in scholarly
publishing, but its legitimacy has been increasingly
questioned. A growing literature discusses how reviewers’
demographic characteristics and biases might lead to
disparities in research dissemination.1,2 Because the extent to
which reviewers are able to determine the outcomes for
papers may vary, it is important to look at the relationship
between reviewers’ recommendations and editors’ decision-
making. However, reviewer recommendations are often
embedded in the text of the review. This work proposes a
method for automatic detection of recommendations based
on review text.

**Design** The automatic classification used a rule-based
algorithm that searched for the presence of 1 or more phrases
that signal the reviewer’s recommendation: accept, minor
revision, major revision, or reject categories, as defined on the
hand-coding process. The algorithm considered the different
combinations of signal phrases to define the outcome. The list
of signal phrases was iteratively built on 3 rounds of hand-
coding and fuzzy matching sentences, while the combinations
were defined to maximize the precision, on the hand-coded
cases. This study used Publons’ data set, which contained
3,310,791 reviews from 25,934 journals; while 600 cases were
hand-coded, a subset of 200 reviews was used to evaluate the
performance. The gender of reviewers was inferred by
matching first and last names to curated lists of country-
specific gendered names, including the US Census.3

**Results** The overall accuracy on the test was that 81% of
assigned recommendations were correct according to hand
coding (n = 149). Since the inclusion of additional phrases is
associated with lowered accuracy, this might indicate an
upper bound in our experiment, given the limits of the
current data and the idiosyncrasies of peer review language.
Nonetheless, the algorithm’s accuracy was comparable to the
rate of agreement between human hand coders (n = 60
[88%]). Over the full data, 14.3% of reviews were assigned a
recommendation by this method (n = 473,443). This was
comparable to the hand-coded identification of 18.3% of
reviews containing an explicit recommendation (n = 399).
From these results, we concluded that the inclusion of an
explicit recommendation remains relatively uncommon in
peer review, with the majority of peer reviewers leaving a final
decision on the manuscript as the responsibility of the editor,
but there was large variation between journals. Initial results
nonetheless showed gender differences in reviewing behavior,
with higher retrieval rates associated with reviewers who
identified as men.

**Conclusions** This work is among the first benchmarks for
automatic classification of review recommendations on a
large-scale, cross-domain database. Though preliminary, it
paves the way for future developments, including studies of potential biases and inequalities in scholarly publishing
through examination of the relationship between reviewer
characteristics and review outcomes.

**References**

1. Lee CJ, Sugimoto CR, Zhang G, Cronin B. Bias in peer
review. J Am Soc Inf Sci Tec. 2013;64(1):2-17. doi:10.1002/
asi.22784
2. Sun M, Barry Danfa J, Teplitskiy M. Does double-blind
peer review reduce bias? evidence from a top computer
science conference. J Am Soc Inf Sci Tec. 2022;73(6):811-819.
doi:10.1002/asi.24582
3. Larivière V, Ni C, Gingras Y, Cronin B, Sugimoto CR.
Bibliometrics: global gender disparities in science. Nature.
2013;504:211-213. doi:10.1038/504211a

**Conflict of Interest Disclosures** None reported.

**Funding/Support** This work was supported by the Doctoral
Training Unit on Data-Driven Computational Modelling and
Applications (DRIVEN), which is funded by the Luxembourg
National Research Fund under the PRIDE programme
(PRIDE17/12252781).

**Acknowledgment** We thank Brad Demarest and Chaoqun Ni for
contributions to an earlier phase of the project. We also thank the
team at Publons who implemented the gender inference algorithm
on their data before giving us access.

**AdditionalInformation** Diego Kozlowski is a co–corresponding
author.

![](https://assets.underline.io/uploads/markdown_image/1/image/3aed057350f3812bef4e49ec6d5137b7.png)

Automatic Classification of Peer Review Recommendation

**Objective** It is possible that most published research
findings are false.1
Effective peer review should detect the
study error and the mental model error. The study error can
be detected by traditional epidemiological/statistical analysis
and exists at the level of the study. The mental model error
exists at the level of underlying structural intellectual
knowledge and beliefs,2
in which accuracy provides the prior
probability and thus substantially affects the posterior
probability of the study.

**Design** The peer review process of The BMJ was assessed for
an inclusive sample of articles published with peer review
cycles (ie, editor and reviewer and author comments and
decision letters) in the research section in July 2021. Likert 5-point scales were used. The peer-review process for each
article was rated for the awareness within the process to
detect and manage study error and mental model error, with
5 representing high estimated awareness and efficacy. For
mental model error detection, higher Likert scores were
accorded in peer review cycles containing the following 4
elements: (1) active criticism of the underlying mental model
(possible errors in diagnostic categories, underlying beliefs of
therapy mechanisms, and the pathological appropriateness of
outcome measures) to estimate the final study posterior
probability; (2) awareness that studies produce contingent
fact claims and that intellectual validity requires analysis of
this contingency; (3) awareness that key patient outcomes
must exist outside the mental model to minimize outcome
contamination by the model and circular reasoning within the
model (eg, use of all-cause morbidity and mortality); and (4)
awareness that clinicians are most interested in the
probability of the mental model because they use the model
daily to inform numerous data decisions made in clinical
practice (eg, decisions regarding history, physical
examination, differential diagnosis, result, treatment
planning, and outcome assessments data). A single ranking
was determined for the totality of the review process for each
article provided by editors and reviewers and author
responses. Result populations of study error and mental
model error scores were compared to assess the separation of
the distributions using the Mann-Whitney test.

**Results** Forty-five peer reviewers produced 21 rounds of
review for 12 articles (**Table 81**). The study errormedian
score was 4.5, and the mental model error median score was 2
with the Mann-Whitney rejection of equal distributions at
P < .05.

![](https://assets.underline.io/uploads/markdown_image/1/image/b78047c705cc4da503421f1f0479a5de.png)

**Conclusions** This preliminary study found The BMJ peer
review process to be effective at detecting and managing
study errors but much less aware of mental model error
problems. Given that the underlying mental model probability may affect each study’s posterior probability,
excellent study internal validity may not secure a strong
warrant for clinical applicability. Peer review that detects and
manages both study error and mental model error may
improve clinical study applicability. Preliminary discussions
and later guideline formation may help correct this deficiency.

**References**

1. Ioannidis JP. Why most published research findings are
false. PLoS Med. 2005;2(8):e124.
2. Holland JH, Holyoak KJ, Nisbett RE, Thagard PR.
Induction: Processes of Inference, Learning, and Discovery.
Massachusetts Institute of Technology; 1986.

**Conflict of Interest Disclosures** None reported.

![](https://assets.underline.io/uploads/markdown_image/1/image/3b47aecf1afe0d03ad7af9ca6f7ba06b.png)

Assessment of Errors in Peer Reviews Published With Articles in The BMJ | VIDEO

**Objective** To study the manuscripts submitted to the
Journal of Clinical and Diagnostic Research (JCDR) that
were withdrawn or remained unattended by the authors after
receiving the external peer reviewer’s decision, and to further
evaluate the authors’ characteristics and reasons for
withdrawal stated in the correspondence.

**Design** JCDR is a broad-specialty, English-language,
peer-reviewed, monthly journal with no article processing
charges. This cross-sectional study was conducted on
manuscripts that were withdrawn or not followed up (ie,
abandoned) by authors between January 1, 2018, and
December 31, 2021, after posting the external peer review
report requesting revision. The manuscripts reported clinical
trials, observational studies, analytical studies, case reports,
literature reviews, systemic reviews, and meta-analyses. The
following data were recorded: dates of submission and the
email asking for a withdrawal, designation of the
corresponding author, whether a department head or
professor was among the authors, and the reason specified in
the emails. As a policy at JCDR, all the manuscripts that are
under process are considered withdrawn if there is no reply
from the author for more than 4 months since the last
correspondence.

**Results** There were 1080 manuscripts that were withdrawn
(n = 150) or abandoned (n = 930) during the 4-year study
period. Among these 1080 manuscripts, only 150 (13.8%) that
had an email from the corresponding author requesting a
withdrawal were further analyzed in the study. Of these 150
manuscripts, one-third (51 manuscripts [34.0%]) had a
professor as the corresponding author, whereas 44 (29.3%) had a department head or professor among the coauthors.
Thirteen manuscripts (8.6%) were withdrawn within a month
of the submission despite a quick response from the journal,
whereas 79 (52.6%) were withdrawn after more than 3
months with 1 or more revisions. Sixty-five emails (43.3%)
did not mention any specific reason for withdrawal, 35
(23.3%) stated that the authors were not willing to or unable
to revise the manuscripts as per the reviewers’ feedback, and
17 emails (11.3%) stated financial constraints as the reason for
withdrawal.

**Conclusions** It is a waste of journal resources and peer
reviewers’ labor when manuscripts are abandoned or
withdrawn. As a mark of good conduct, authors are expected
to have forthright communication with journals. If strong
reasons exist, authors may withdraw their manuscripts, but
clear communication is welcome. Notably, experienced
academicians (professors) also share the burden of such
misconduct. The effect of such withdrawals is harder for
journals that do not charge up-front processing fees. Along
with rising inflation, such withdrawals add to the economic
burden on a stand-alone journal.

**Conflict of Interest Disclosures** None reported

![](https://assets.underline.io/uploads/markdown_image/1/image/4b3514917c9ea492117196a7564af985.png)

Comparison of Bibliometrics of Leading Open Access Chinese Journals with Leading Non-Chinese Journals in Science, Technology, and Medicine

Next from Peer Review Congress 2022

A Systematic Review of Medical and Clinical Research Landscapes in Primary Medical Care in Malaysia

Similar lecture

The COVID That Wasn't: Counterfactual Journalism Using GPT

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES