United States

&lt;p&gt;Harleen K. Marwah,&lt;sup&gt;1&lt;/sup&gt; Abarna Pearl,&lt;sup&gt;1&lt;/sup&gt; Christos P. Kotanidis,&lt;sup&gt;1,2,3&lt;/sup&gt; Sarah Gorey,&lt;sup&gt;1,4&lt;/sup&gt; Darren Taichman,&lt;sup&gt;1,5&lt;/sup&gt; Mary Beth Hamel&lt;sup&gt;1,6&lt;/sup&gt;&lt;/p&gt;
&lt;h4&gt;Objective&lt;/h4&gt;
&lt;p&gt; Trials increasingly include participants from low- and middle-income countries (LMICs),&lt;sup&gt;1&lt;/sup&gt; but authorship representation may not be proportional. LMIC-affiliated authors on trials recruiting from LMICs offer important perspectives, which have implications for equity, expertise, and capacity building. Prior data have looked at specialty journals or non-Western country categories,&lt;sup&gt;2&lt;/sup&gt; but author representation specifically in trials with participants from LMICs is not known. This study evaluated trends over time in LMIC-affiliated authorship among trials recruiting participants from LMICs.&lt;/p&gt;
&lt;h4&gt;Design&lt;/h4&gt;
&lt;p&gt; Using PubMed, we extracted trials published during alternating months (6 mo/y) in the years 2008, 2013, 2018, and 2023 from 4 high-impact medical journals: the &lt;i&gt;New England Journal of Medicine&lt;/i&gt;, &lt;i&gt;The Lancet&lt;/i&gt;, &lt;i&gt;JAMA&lt;/i&gt;, and &lt;i&gt;BMJ&lt;/i&gt;. Four investigators manually identified trials that recruited participants from LMICs, defined by the Wellcome Trust&lt;i&gt; &lt;/i&gt;as countries with low-income or middle-income economies&lt;i&gt;.&lt;/i&gt;&lt;sup&gt;3&lt;/sup&gt; We designed and calculated 3 metrics for each trial: (1) an enrolling country ratio (the number of recruiting LMICs divided by the total number of recruiting countries), (2) an author ratio (the number of LMIC-affiliated authors divided by the total number of authors in a trial), and (3) a representation index (RI) (the author ratio divided by the enrolling country ratio). The RI controlled for the proportion of recruiting LMICs in each trial. An RI less than 1 indicated that the proportion of LMIC-affiliated authors was lower than the proportion of recruiting LMICs. An RI of 1 indicated that the 2 ratios were exactly proportionate.&lt;/p&gt;
&lt;h4&gt;Results&lt;/h4&gt;
&lt;p&gt; The PubMed search yielded 865 trials, 302 of which recruited participants from LMICs. Of these, 139 (46.0%) were published in the &lt;i&gt;New England Journal of Medicine&lt;/i&gt;, 117 (38.7%) in &lt;i&gt;The Lancet&lt;/i&gt;, 35 (11.6%) in &lt;i&gt;JAMA&lt;/i&gt;, and 11 (3.6%) in &lt;i&gt;BMJ&lt;/i&gt;. The number and proportion of trials recruiting participants from LMICs increased between 2008 (52/174, 29.9%) and 2023 (117/262, 44.7%). The mean enrolling country ratio decreased over time, from 0.62 in 2008 to 0.41 in 2023 (&lt;b&gt;Figure 25-1131&lt;/b&gt;, blue line). The mean author ratio decreased after 2008 and then remained similar; in 2008, it was 0.34 and in 2023, it was 0.23 (&lt;b&gt;Figure 25-1131&lt;/b&gt;, gray line). Between 2008 and 2023, the RI increased modestly from 0.49 to 0.53 (&lt;b&gt;Figure 25-1131&lt;/b&gt;, teal line).&lt;/p&gt;

![](https://assets.underline.io/markdown_image/1/image/8b793884c95e0b12aa0c86e9d286c00d.png)

&lt;h4&gt;Conclusions&lt;/h4&gt;
&lt;p&gt; This study highlights that LMIC-affiliated authorship in LMIC-recruiting trials has increased modestly over time but remains disproportionate to LMIC participant recruitment sites. Limitations of the study include the lack of information on the proportion of LMIC-recruited participants for each trial. This information could be a stronger indicator as to what LMIC-affiliated authorship representation should look like. Also, LMIC definitions may change over time and by defining organization. While there has been modest improvement in author representation, increased attention may help promote representative and ethical research.&lt;/p&gt;
&lt;h4&gt;References&lt;/h4&gt;
&lt;p&gt;1. Number of clinical trials by year, country, WHO region and income group (1999-2024). World Health Organization. December 2024. Accessed April 15, 2025. &lt;a href=&quot;https://www.who.int/observatories/global-observatory-on-health-research-and-development/monitoring/number-of-clinical-trials-by-year-country-who-region-and-income-group&quot;&gt;&lt;span class=&quot;Hyperlink CharOverride-6&quot;&gt;https://www.who.int/observatories/global-observatory-on-health-research-and-development/monitoring/number-of-clinical-trials-by-year-country-who-region-and-income-group&lt;/span&gt;&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;2. De Oliveira-Gomes D, Guilliod C, Gebran K, et al. Equity and representation in cardiology research: a comprehensive analysis of authorship from low and low-middle income countries in cardiology journals. &lt;i&gt;Curr Probl Cardiol&lt;/i&gt;. 2024;49(8):102667. doi:10.1016/j.cpcardiol.2024.102667&lt;/p&gt;
&lt;p&gt;3. Low- and middle-income countries. Wellcome Trust. Accessed January 8, 2025. &lt;a href=&quot;https://wellcome.org/grant-funding/guidance/prepare-to-apply/low-and-middle-income-countries&quot;&gt;&lt;span class=&quot;Hyperlink CharOverride-6&quot;&gt;https://wellcome.org/grant-funding/guidance/prepare-to-apply/low-and-middle-income-countries&lt;/span&gt;&lt;/a&gt;&lt;/p&gt;
&lt;sup&gt;1&lt;/sup&gt;&lt;i&gt;New England Journal of Medicine&lt;/i&gt;, Boston, MA, US, hmarwah@nejm.org; &lt;sup&gt;2&lt;/sup&gt;Heart and Vascular Center, Division of Cardiovascular Medicine, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, US; &lt;sup&gt;3&lt;/sup&gt;Division of Cardiovascular Medicine, Radcliffe Department of Medicine, University of Oxford, UK; &lt;sup&gt;4&lt;/sup&gt;Stroke Clinical Trials Network Ireland, University College Dublin Clinical Research Centre, Mater Misericordiae University Hospital, Dublin, Ireland; &lt;sup&gt;5&lt;/sup&gt;Department of Medicine, Division of Pulmonary, Allergy and Critical Care Medicine, University of Pennsylvania School of Medicine, Philadelphia, PA, US; &lt;sup&gt;6&lt;/sup&gt;Division of General Medicine and Primary Care, Department of Medicine, Harvard Medical School, Beth Israel Deaconess Medical Center, Boston, MA, US.&lt;/p&gt;
&lt;h4&gt;Conflict of Interest Disclosures&lt;/h4&gt;
&lt;p&gt; Harleen K. Marwah, Abarna Pearl, Christos P. Kotanidis, and Sarah Gorey are editorial fellows at the &lt;i&gt;New England Journal of Medicine&lt;/i&gt; (2024-2025). Darren Taichman is a deputy editor and Mary Beth Hamel is executive editor of the &lt;i&gt;New England Journal of Medicine&lt;/i&gt;. No other disclosures were reported.&lt;/p&gt;


The 10th Peer Review Congress

Representation of Authors From Low- and Middle-Income Countries (LMIC) in Trials With Participants From LMICs

low-resource setting challenges

authorship and contributorship

diversity and inclusion

The Tenth Peer Review Congress featured 3 days for presentations of new research into peer review and all aspects of scientific publication, bias, quality of reporting, and information access and dissemination.

This year’s meeting was hybrid, and all plenary sessions were livestreamed with opportunities to view presentations and engage with participants in person and virtually during and after the meeting. All recordings are now available on Underline for viewing.

<div style="position:relative;padding-top:max(60%,324px);width:100%;height:0;"><iframe style="position:absolute;border:none;width:100%;height:100%;left:0;top:0;" src="https://online.fliphtml5.com/ebtyf/mget/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe></div>


#### Why? 
Our aim is to encourage research into the quality and credibility of peer review and scientific publication, to establish the evidence base on which scientists can improve the conduct, reporting, and dissemination of scientific research.

#### Who? 
Participants include editors and publishers of scientific peer-reviewed journals, researchers, funders, bibliometric and informatics experts, information innovators, librarians, journalists, policymakers, ethicists, scientific information disseminators, and anyone interested in the progress of the scientific information enterprise and the quality of scientific evidence.

The Congress embraces a wide range of disciplines, including biomedicine, health and life sciences, applied sciences, basic sciences, physical and chemical sciences, mathematics, computer sciences, engineering, economics, and social sciences as well as new and emerging disciplines.

#### What? 
The Congress provides a forum for the presentation and discussion of new research into the quality and credibility of editorial and funding peer review, scientific and scholarly publication, research methods and reporting practices, identification and management of biases, transparency, reproducibility, access, publication and dissemination metrics, identification and management of threats to the integrity of science, and advances in information exchange.

**Follow us**

[![](https://assets.underline.io/markdown_image/1/image/662a22766b9788ed3766cb23cd156e32.png)](https://x.com/peerrevcongress)  [![](https://assets.underline.io/markdown_image/1/image/bb105afb1773a01fa169aa97ca1af49a.png)](https://www.youtube.com/channel/UCsG_0nKi3_Zfyv0yRPdxjWQ)  [![](https://assets.underline.io/markdown_image/1/image/034e1e70a194a9a4178fdf470ef796db.png)](https://bsky.app/profile/peerreviewcongress.bsky.social)



You need to log in with the email address you registered with. 



Please log in to explore this event.

Registration is required to attend this event, please [here](https://peerreviewcongress.org/registration-information/) to secure your ticket. Please check your spam/promotional inbox too.



Please register!

The Tenth Peer Review Congress features 3 days for presentations of new research into peer review and all aspects of scientific publication, bias, quality of reporting, and information access and dissemination.

<p>Harleen K. Marwah,<sup>1</sup> Abarna Pearl,<sup>1</sup> Christos P. Kotanidis,<sup>1,2,3</sup> Sarah Gorey,<sup>1,4</sup> Darren Taichman,<sup>1,5</sup> Mary Beth Hamel<sup>1,6</sup></p>
<h4>Objective</h4>
<p> Trials increasingly include participants from low- and middle-income countries (LMICs),<sup>1</sup> but authorship representation may not be proportional. LMIC-affiliated authors on trials recruiting from LMICs offer important perspectives, which have implications for equity, expertise, and capacity building. Prior data have looked at specialty journals or non-Western country categories,<sup>2</sup> but author representation specifically in trials with participants from LMICs is not known. This study evaluated trends over time in LMIC-affiliated authorship among trials recruiting participants from LMICs.</p>
<h4>Design</h4>
<p> Using PubMed, we extracted trials published during alternating months (6 mo/y) in the years 2008, 2013, 2018, and 2023 from 4 high-impact medical journals: the <i>New England Journal of Medicine</i>, <i>The Lancet</i>, <i>JAMA</i>, and <i>BMJ</i>. Four investigators manually identified trials that recruited participants from LMICs, defined by the Wellcome Trust<i> </i>as countries with low-income or middle-income economies<i>.</i><sup>3</sup> We designed and calculated 3 metrics for each trial: (1) an enrolling country ratio (the number of recruiting LMICs divided by the total number of recruiting countries), (2) an author ratio (the number of LMIC-affiliated authors divided by the total number of authors in a trial), and (3) a representation index (RI) (the author ratio divided by the enrolling country ratio). The RI controlled for the proportion of recruiting LMICs in each trial. An RI less than 1 indicated that the proportion of LMIC-affiliated authors was lower than the proportion of recruiting LMICs. An RI of 1 indicated that the 2 ratios were exactly proportionate.</p>
<h4>Results</h4>
<p> The PubMed search yielded 865 trials, 302 of which recruited participants from LMICs. Of these, 139 (46.0%) were published in the <i>New England Journal of Medicine</i>, 117 (38.7%) in <i>The Lancet</i>, 35 (11.6%) in <i>JAMA</i>, and 11 (3.6%) in <i>BMJ</i>. The number and proportion of trials recruiting participants from LMICs increased between 2008 (52/174, 29.9%) and 2023 (117/262, 44.7%). The mean enrolling country ratio decreased over time, from 0.62 in 2008 to 0.41 in 2023 (<b>Figure 25-1131</b>, blue line). The mean author ratio decreased after 2008 and then remained similar; in 2008, it was 0.34 and in 2023, it was 0.23 (<b>Figure 25-1131</b>, gray line). Between 2008 and 2023, the RI increased modestly from 0.49 to 0.53 (<b>Figure 25-1131</b>, teal line).</p>

![](https://assets.underline.io/markdown_image/1/image/8b793884c95e0b12aa0c86e9d286c00d.png)

<h4>Conclusions</h4>
<p> This study highlights that LMIC-affiliated authorship in LMIC-recruiting trials has increased modestly over time but remains disproportionate to LMIC participant recruitment sites. Limitations of the study include the lack of information on the proportion of LMIC-recruited participants for each trial. This information could be a stronger indicator as to what LMIC-affiliated authorship representation should look like. Also, LMIC definitions may change over time and by defining organization. While there has been modest improvement in author representation, increased attention may help promote representative and ethical research.</p>
<h4>References</h4>
<p>1. Number of clinical trials by year, country, WHO region and income group (1999-2024). World Health Organization. December 2024. Accessed April 15, 2025. <a href="https://www.who.int/observatories/global-observatory-on-health-research-and-development/monitoring/number-of-clinical-trials-by-year-country-who-region-and-income-group"><span class="Hyperlink CharOverride-6">https://www.who.int/observatories/global-observatory-on-health-research-and-development/monitoring/number-of-clinical-trials-by-year-country-who-region-and-income-group</span></a></p>
<p>2. De Oliveira-Gomes D, Guilliod C, Gebran K, et al. Equity and representation in cardiology research: a comprehensive analysis of authorship from low and low-middle income countries in cardiology journals. <i>Curr Probl Cardiol</i>. 2024;49(8):102667. doi:10.1016/j.cpcardiol.2024.102667</p>
<p>3. Low- and middle-income countries. Wellcome Trust. Accessed January 8, 2025. <a href="https://wellcome.org/grant-funding/guidance/prepare-to-apply/low-and-middle-income-countries"><span class="Hyperlink CharOverride-6">https://wellcome.org/grant-funding/guidance/prepare-to-apply/low-and-middle-income-countries</span></a></p>
<sup>1</sup><i>New England Journal of Medicine</i>, Boston, MA, US, hmarwah@nejm.org; <sup>2</sup>Heart and Vascular Center, Division of Cardiovascular Medicine, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, US; <sup>3</sup>Division of Cardiovascular Medicine, Radcliffe Department of Medicine, University of Oxford, UK; <sup>4</sup>Stroke Clinical Trials Network Ireland, University College Dublin Clinical Research Centre, Mater Misericordiae University Hospital, Dublin, Ireland; <sup>5</sup>Department of Medicine, Division of Pulmonary, Allergy and Critical Care Medicine, University of Pennsylvania School of Medicine, Philadelphia, PA, US; <sup>6</sup>Division of General Medicine and Primary Care, Department of Medicine, Harvard Medical School, Beth Israel Deaconess Medical Center, Boston, MA, US.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> Harleen K. Marwah, Abarna Pearl, Christos P. Kotanidis, and Sarah Gorey are editorial fellows at the <i>New England Journal of Medicine</i> (2024-2025). Darren Taichman is a deputy editor and Mary Beth Hamel is executive editor of the <i>New England Journal of Medicine</i>. No other disclosures were reported.</p>


poster

<p><span lang="fr-FR">Vincent Kipkorir,</span><sup>1</sup><span lang="fr-FR"> Godfrey Sama,</span><sup>1</sup><span lang="fr-FR"> Mumba Chalwe,</span><sup>1</sup><span lang="fr-FR"> Tihitena Negussie,</span><sup>1</sup><span lang="fr-FR"> Michael Mwachiro,</span><sup>1</sup><span lang="fr-FR"> Robert Parker,</span><sup>1</sup><span lang="fr-FR"> Seke Kazuma,</span><sup>1</sup><span lang="fr-FR"> Stella Itungu,</span><sup>1</sup><span lang="fr-FR"> Abebe Bekele</span><sup>1</sup></p>
<h4>Objective</h4>
<p> Underrepresentation along geographical and gender lines has historically been observed in academic surgery.<sup>1</sup> However, efforts have been made in recent years to ensure equity and inclusion in the surgical learning environment to increase female representation and ensure geographically diversified research engagements.<sup>2</sup> We set out to explore the evolution of authorship diversity in African surgical research using publications featured in an African surgical journal.</p>
<h4>Design</h4>
<p> This was a retrospective cross-sectional study. Data were collected from all articles published in the <i>East and Central African Journal of Surgery</i> between 2003 and 2023. Article metadata and author biodata were extracted from each article for subsequent analysis. Author gender was established by 2 independent reviewers through an online search, confirmed by a minimum of 2 web pages including LinkedIn, editorial profiles, institutional websites, and ResearchGate. For original articles, case series, and case reports, geographical location was determined by the primary study site. For the other articles, we used the author’s institutional affiliation.</p>
<h4>Results </h4>
<p>A total of 889 articles were included featuring 3001 authors; 2609 (86.9%) were male and 392 (13.1%) were female. In total, 92 females (3.1%) were designated as first authors, 93 (3.1%) as corresponding authors, and 90 (3.0%) as last authors. There was a predominance of males in the positions of first, last, and corresponding author. Despite this, the number of females represented in these categories had a steady increase over the years. The gender disparity ratio increased in the first 15 years, then dropped in the last 5 years. There were 2523 African authors (84.1%), with Ethiopia having the largest number of authors (155 [5.2%]). The distribution according to country and the trends in female authorship are presented in <b>Figure 25-1149</b>.</p>

![](https://assets.underline.io/markdown_image/1/image/21e0e5a9b54c9d6917f17d43e8d66dbb.png)

<h4>Conclusions </h4>
<p>Our findings show disparities in the ratio of male to female authors, with a substantial male predominance. While it is encouraging to note the steady increase in the number of female authors over the years, a wide gap remains. The steady rise may be attributed to the overall rise in female surgeons following the recent establishment of women in surgery associations and better advocacy and funding toward empowerment of female surgeons in Africa. With a female surgical workforce density of about 11% in College of Surgeons of East, Central and Southern Africa member countries, further efforts are needed to ensure that these potential female authors are well represented, especially as first, last, and corresponding authors.<sup>3</sup> Strategies currently in place to foster surgical research in Africa need to be sustained to maintain the current trajectory of African representation in the global surgical research landscape. This study is limited by the subjective nature of ascertaining gender and location; however, the level of bias was reduced by having 2 independent assessors.</p>
<h4>References</h4>
<p>1. Rathna RB, Biswas J, D’Souza C, Joseph JM, Kipkorir V, Dhali A. Authorship diversity in general surgery-related Cochrane systematic reviews: a bibliometric study. <i>BJS</i>. 2023;110(8):989-990. doi:10.1093/bjs/znad117</p>
<p>2. Burks CA, Russell TI, Goss D, et al. Strategies to increase racial and ethnic diversity in the surgical workforce: a state of the art review. <i>Otolaryngol Head Neck Surg</i>. 2022;166(6):1182-1191.</p>
<p>3. Osei-Kuffour D, Banda CH, Campion A, et al. The evolution of the specialist surgeon workforce in East, Central and Southern Africa. World J Surg. 2025;49(4):946-954. doi:10.1002/wjs.12545</p>
<sup>1</sup><i>East and Central African Journal of Surgery</i>, College of Surgeons of East, Central and Southern Africa; managing_editor@cosecsa.org.</p>
<h4>Conflict of Interest Disclosures </h4>
<p>None reported.</p>


Evolution of Authorship Diversity in African Surgical Research Over 2 Decades

<p>Tais Freire Galvão,<sup>1,2</sup> Everton Nunes da Silva,<sup>2</sup> Jorge Otávio Maia Barreto,<sup>3</sup> Marcus Tolentino Silva<sup>4</sup></p>
<h4>Objective</h4>
<p> Detailed descriptions of journals’ workflows exist,<sup>1</sup> but empirical studies assessing their effects are scarce. In 2024, a new workflow was implemented at <i>Epidemiologia e Serviços de Saúde: revista do SUS</i> (<i>RESS</i>) aimed at getting timely and quality reviews. This study assessed the effect of a new workflow on the journal’s time to decision-making.</p>
<h4>Design</h4>
<p> This before and after study using historical controls assessed submissions to <i>RESS</i> from June to December 2024 (intervention) and June to December 2023 (control), followed up until January 2025. The intervention started in June 2024, based on inclusion of a peer review administrator in charge of reviewers’ assignments and adding new and trained associate editors<sup>2</sup>; previous evaluation by the editor in chief (scope) and scientific editor (methodological quality); and checklists to standardize reviews.<sup>3</sup> The comparator was the previous workflow, centralized on the associate editor’s assignment of reviewers, and final decisions made by the editor in chief and a member of the editorial committee. Outcomes were processing time from submission to decision, number of reviewers invited, and reviews received. We summarized means and SDs or medians and IQRs. Normality was tested using the Shapiro-Wilk test and square root or logarithmic transformations preceded t-tests, with the Levene test used to assess variances’ homogeneity. In unsuitable t-tests, the Mann-Whitney test was applied. Residual normality and heteroscedasticity were examined by Breusch-Pagan test. We adjusted by the length of the paper (full or short) to minimize potential confounding.</p>
<h4>Results</h4>
<p> Out of 928 submissions in 2024, 218 proceeded to peer review (140 ongoing until cohort closure) and 78 had a decision (36 accepted, 42 rejected). In 2023, from 970 submissions, 77 were peer reviewed and resulted in 28 acceptances and 49 rejections (<i>P</i> = .22). Mean (SD) time in days to decision was significantly shorter in the intervention (62.8 [40.5]) than in the controls (130.1 [64.5]; <i>P</i> &lt; .001), which was also faster in the intervention according to acceptance (92.8 [39.8] vs 133.5 [61.6]; <i>P</i> = .002) or rejection (37.1 [16.0] vs 128.1 [66.7]; <i>P</i> &lt; .001). The mean (SD) number of invited reviewers per manuscript was higher in the intervention (10.1 [6.3]) than in the controls (7.8 [5.5]; <i>P</i> = .02), without significant difference according to decision when compared with the historical controls (approved, 10.0 [7.0] vs 6.3 [2.8]; <i>P</i> = .07; rejected, 10.3 [5.8] vs 8.6 [6.5]; adjusted <i>P</i> = .11). The median (IQR) number of reviews received was also higher (4 [4-4] vs 2 [2-3]; <i>P</i> &lt; .001), and was similarly higher when comparing decisions across groups (approved, 4 [4-4] vs 2 [2-3]; <i>P</i> &lt; .001; rejected, 4 [4-4] vs 2 [2-3]; <i>P</i> &lt; .001) (<b>Table 25-0909</b>).</p>

![](https://assets.underline.io/markdown_image/1/image/c5b3c1bf721bf5ccfd8c212b70f07e17.png)

<h4>Conclusions</h4>
<p> The new workflow reduced the duration of editorial processing and allowed a higher number of peer reviews to inform decision-making. The outcomes were not defined before the intervention was planned; the length of follow-up, new editors, and submission system were confounders that may have influenced the findings.</p>
<h4>References</h4>
<p>1. Mark H, Ragon T, Funning G, et al. Editorial workflow of a community-led, all-volunteer scientific journal: lessons from the launch of Seismica. <i>Seismica</i>. 2023;2(2):1091. doi:10.26443/seismica.v2i2.1091</p>
<p><span lang="pt-PT">2. Galvão TF, Silva EN, Araújo WN, Barreto JOM. Editorial improvement in </span><i>Epidemiologia e Serviços de Saúde</i><span lang="pt-PT"> in 2024. </span><i>Epidemiol Serv Saude</i><span lang="pt-PT">. 2024;33:e20241002. doi:</span><a href="https://doi.org/10.1590/S2237-96222024v33e20241002.en"><span class="Hyperlink CharOverride-6" lang="pt-PT">10.1590/S2237-96222024v33e20241002.en</span></a></p>
<p><span lang="pt-PT">3. Silva MT, Galv</span><span lang="ar-SA">ã</span><span lang="pt-PT">o TF. Systematization of peer review in </span><i>Epidemiologia e Serviços de Saúde</i><span lang="pt-PT">. </span><i>Epidemiol Serv Saude</i><span lang="pt-PT">. 2024;33:e20241001. doi:</span><a href="https://doi.org/10.1590/S2237-96222024v33e20241001.en"><span class="Hyperlink CharOverride-6" lang="pt-PT">10.1590/S2237-96222024v33e20241001.en</span></a></p>
<p class="abstract_conflict" lang="fr-FR"><sup>1</sup>Faculdade de Ciências Farmacêuticas<span lang="pt-PT">, Universidade Estadual de Campinas, Campinas, Brazil, taisgalvao@gmail.com; </span><sup>2</sup><span lang="pt-PT">Secretaria de Vigilância em Saúde e Ambiente, Ministério da Saúde</span>, Brasília, Brazil; <sup>3</sup><span lang="pt-PT">Faculdade d</span>e Ciências e Tecnologias em Saúde, Universidade de Brasília, Brasília, Brazil; <sup>4</sup><span lang="pt-PT">Gerência Regional de Brasília, Fundação Oswaldo Cruz, </span>Brasília, Brazil; <sup>5</sup><span lang="pt-PT">Faculdade d</span>e Ciências da Saúde, Universidade de Brasília, Brasília, Brazil.</p>
<h4>Conflict of Interest Disclosures </h4>
<p>Tais Freire Galv<span lang="ar-SA">ã</span>o, Everton Nunes da Silva, and Jorge Otávio Maia Barreto are editors of <i>Epidemiologia e Serviços de Saúde: revista do SUS</i>. Marcus Tolentino Silva has no conflicts of interest to declare. Tais Freire Galv<span lang="ar-SA">ã</span>o is a member of the Peer Review Congress Advisory Board but was not involved in the review or decision for this abstract.</p>


Effects of Peer Review and Editorial Workflows in Decision-Making at a Diamond Open Access Journal

<p>Hillary Wnfried Ramirez,<sup>1,2,3</sup> Malena Chiaborelli,<sup>1,2,3</sup> Christof M. Schönenberger,<sup>1</sup> Katie Mellor,<sup>4,5</sup> Alexandra N. Griessbach,<sup>1</sup> Paula Dhiman,<sup>4,6</sup> Pooja Gandhi,<sup>7</sup> Szimonetta Lohner,<sup>8,9</sup> Arnav Agarwal,<sup>10,11</sup> Ayodele Odutayo,<sup>5,12</sup> Michael M. Schlussel,<sup>4,6</sup> Philippe Ravaud,<sup>13,14</sup> David Moher,<sup>15,16 </sup>Matthias Briel,<sup>1,10</sup> Isabelle Boutron,<sup>13,14</sup> Sally Hopewell,<sup>4</sup> Sara Schroter,<sup>17,18 </sup>Benjamin Speich<sup>1,4</sup></p>
<h4>Objective </h4>
<p>Two randomized controlled trials (RCTs) conducted at journal level have shown that reminding peer reviewers about the 10 most important and underreported reporting items did not improve the reporting quality in published articles.<sup>1</sup> With this pooled in-depth analysis of peer reviewer reports, we aimed to assess at what stage the intervention failed.</p>
<h4>Design</h4>
<p> A subsample of peer reviewer reports from the control group (receiving no reminder) and the intervention group (receiving a reminder of the 10 most important reporting items) were analyzed. In brief, 2 blinded authors independently extracted from peer reviewer reports how many of the 10 key reporting items were flagged by peer reviewers for clarification. The main outcome of this analysis was the mean proportion of the 10 selected reporting items for which at least 1 peer reviewer requested clarification, assessed at the manuscript level. Furthermore, we assessed how many requested changes were later adequately reported in published articles.</p>
<h4>Results </h4>
<p>Across the RCTs, we had access to peer reviewer reports for 533 manuscripts (265 in the intervention group, assessing comments from 740 peer reviewers; and 268 in the control group, assessing comments from 719 peer reviewers). Our results indicate that reviewers in the intervention group requested clarification on more reporting items than those in the control group. Overall, reviewers in the intervention group flagged 21.1% of the 10 reporting items for clarification compared with 13.1% in the control group (mean difference, 8.0 percentage points (pp); 95% CI, 4.9-11.1 pp). However, the overall mean difference between groups was diluted from 8.0 to 4.2 pp when only assessing accepted and published articles and decreased even further to 2.6 pp when only considering changes that were then implemented by authors of manuscripts (<b>Table 25-0956</b>). Approximately 55% of reporting items that were criticized by peer reviewers were later adequately reported in the published article (intervention group, 173 of 308 [56.2%]; control group, 105 of 197 [53.3%]).</p>

![](https://assets.underline.io/markdown_image/1/image/80fe50628cb162743e45faadd6604b8b.png)

<h4>Conclusions</h4>
<p> Reminding peer reviewers to check reporting items increased their focus on reporting guidelines, leading to more reporting-related requests in their review reports. However, the effect was diluted during the peer review process (particularly due to rejected articles and requests not being implemented by authors). Journals should therefore make sure that requested clarifications are adequately addressed in revised manuscripts.</p>
<h4>Reference</h4>
<p>1. Speich B, Mann E, Schönenberger CM, et al. Reminding peer reviewers of reporting guideline items to improve completeness in published articles: primary results of 2 randomized trials. <i>JAMA Netw Open</i>. 2023;6(6):e2317651.</p>
<sup>1</sup>CLEAR Methods Center, Division of Clinical Epidemiology, Department Clinical Research, University Hospital Basel, University of Basel, Basel, Switzerland, benjamin.speich@usb.ch; <sup>2</sup>Swiss Tropical and Public Health Institute, Basel, Switzerland; <sup>3</sup>University of Basel, Basel, Switzerland; <sup>4</sup>Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK; <sup>5</sup>Clinical Outcomes Assessment, Clarivate, London, UK; <sup>6</sup>The EQUATOR Network, Oxford, UK; <sup>7</sup>Department of Communication Sciences and Disorders, Faculty of Rehabilitation Medicine, University of Alberta, Edmonton, AL, Canada; <sup>8</sup>Cochrane Hungary, Medical School, University of Pécs, Pécs, Hungary; <sup>9</sup>MTA–PTE Lendület “Momentum” Evidence in Medicine Research Group, Department of Public Health Medicine, Medical School, University of Pécs, Pécs, Hungary; <sup>10</sup>Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada; <sup>11</sup>Division of General Internal Medicine, Department of Medicine, McMaster University, Hamilton, ON, Canada; <sup>12</sup>Division of Nephrology, Toronto General Hospital, University Health Network, Toronto, ON, Canada; <sup>13</sup>Centre d’Épidémiologie Clinique, Hôpital Hôtel-Dieu, Assistance Publique Hôpitaux de Paris, Paris, France;<sup>14</sup>Université de Paris, CRESS, Inserm, INRA, Paris, France; <sup>15</sup>Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada; <sup>16</sup>Faculty of Medicine, School of Epidemiology and Public Health, University of Ottawa, Ottawa, Ontario, Canada; <sup>17</sup><i>The BMJ</i>, London, UK; <sup>18</sup>Faculty of Public Health &amp; Policy, London School of Hygiene &amp; Tropical Medicine, London, UK.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> Benjamin Speich and Matthias Briel reported receiving unrestricted grants from Moderna for studies unrelated to the presented work. Sara Schroter is employed by BMJ Publishing Group. Katie Mellor is employed by Clarivate. David Moher, Sally Hopewell, and Isabelle Boutron are members of the Consolidated Standards for Reporting Trials (CONSORT) executive board and authors of the CONSORT 2010 statement. David Moher is an author of the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) 2013 statement. David Moher, Michael M. Schlussel, Paula Dhiman, and Philippe Ravaud are members of the Enhancing the Quality and Transparency of Research (EQUATOR) network. Isabelle Boutron and David Moher are members of the Peer Review Congress Advisory Board but were not involved in the review or decision for this abstract. No other disclosures were reported.</p>
<h4>Funding/Support </h4>
<p>Benjamin Speich was supported by a Return Postdoc.Mobility (P4P4PM_194496) grant from the Swiss National Science Foundation. Christof M. Schönenberger was funded by the Janggen Pöhn Foundation and the Swiss National Science Foundation (MD-PhD grant No. 323530_221860). Szimonetta Lohner was supported by the Hungarian Academy of Sciences (MTA) within the framework of the Lendület Programme.</p>
<h4>Role of the Funder/Sponsor</h4>
<p> The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.</p>
<h4>Additional Information</h4>
<p> Both trials (CONSORT-PR and SPIRIT-PR) were prospectively registered on Open Science Framework (<a href="https://osf.io/c4hn8"><span class="Hyperlink CharOverride-6">https://osf.io/c4hn8</span></a> and <a href="https://osf.io/z2hm9"><span class="Hyperlink CharOverride-6">https://osf.io/z2hm9</span></a>).</p>


Reminding Peer Reviewers to Comment on Reporting Items as Instructed by the Journal? An Analysis of 2 Randomized Trials

<p>Mario Malički,<sup>1,2,3</sup> Ana Jerončić,<sup>4</sup> Gerben ter Riet,<sup>5,6</sup> Lex Bouter,<sup>7,8</sup> John P. A. Ioannidis,<sup>2,3,9,10</sup> IJsbrand Jan Aalbersberg,<sup>11 </sup>Steven N. Goodman<sup>1,2,3,9,12</sup> </p>
<h4>Objective </h4>
<p>Previous research has indicated a knowledge gap on changes brought on by peer review and journal publishing.<sup>1</sup> We are conducting a living evidence synthesis of studies that analyzed differences between manuscript versions (eg, preprinted or submitted versions vs peer reviewed journal versions).</p>
<h4>Design </h4>
<p>This is a living synthesis of studies with meta-analyses of proportions (following PRISMA guidelines for reporting).<sup>2</sup> Studies were identified based on authors’ knowledge of the field, from 18 previous systematic reviews on peer review,<sup>3</sup> and by checking all research presented at the peer review conferences. All references and citations of identified studies (using Google Scholar up to January 2025) were checked. For all included studies that reported on changes, we extracted descriptive variables (including year of publication, sampling method, change comparison method, and data, wherever available, on how many analyzed version-pairs had changes for 11 outcomes). We assessed risk of bias for each outcome based on 3 domains: sample selection, measurement reliability, and conflicts of interest. To pool the data, we used random effects meta-analysis with Freeman-Tukey transformed proportions and reported the <i>I</i><sup>2</sup> index of heterogeneity.</p>
<h4>Results </h4>
<p>We identified 67 studies published from 1978 through the end of 2024, of which 31 (46%) analyzed changes between preprint and journal versions, 23 (34%) between submitted and accepted or published versions, 8 (12%) between rejected versions and those later published in other journals, 4 (6%) between multiple version sets, and 1 (1.5%) between different preprinted versions. Most studies, 45 (67%), analyzed changes manually, 13 (19%) used computational methods, and 9 (12%) combined manual and computational methods. The median number of analyzed version-pairs was 113 (IQR, 51-429). By discipline, 43 (64%) studies looked only at health research, 6 (9%) at life sciences, 6 (9%) at social sciences, 4 (6%) at physical sciences, and 8 (12%) at multiple disciplines. Meta-analyses showed the highest frequency of changes in title, authorship, conflicts of interest, and numerical results, with the lowest frequency in study conclusions (<b>Table 25-1077</b>).</p>

![](https://assets.underline.io/markdown_image/1/image/e3c3d82a0d010bd96adfaadbf9984c49.png)

<h4>Conclusions </h4>
<p>Current evidence indicates that while different manuscript sections experience varying degrees of changes from their submitted, preprinted, or rejected versions to their peer-reviewed journal version, the primary conclusions rarely change. Our results in this stage are limited by not conducting a systematic search of bibliographic databases and lack of (identified) data points from many subdisciplines, especially arts and humanities, as well as studies that explored possible factors associated with the extent of changes (eg, researchers’ or editors’ career stage or prestige, quality of initial manuscript version).</p>
<h4>References</h4>
<p>1. Tennant JP, Ross-Hellauer T. The limitations to our understanding of peer review. <i>Res Integr Peer Rev</i>. 2020;5:6. doi:10.1186/s41073-020-00092-1</p>
<p>2. Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. <i>BMJ</i>. 2021;372:n71. doi:10.1136/bmj.n71</p>
<p>3. European Association of Science Editors. What is peer review? December 28, 2024. Accessed July 15, 2025. <a href="https://ease.org.uk/communities/peer-review-committee/peer-review-toolkit/What-is-peer-review/"><span class="Hyperlink CharOverride-6">https://ease.org.uk/communities/peer-review-committee/peer-review-toolkit/What-is-peer-review/</span></a></p>
<sup>1</sup>Stanford Program on Research Rigor and Reproducibility (SPORR), Stanford University, Stanford, California, US, mmalicki@stanford.edu; <sup>2</sup>Department of Epidemiology and Population Health, Stanford University School of Medicine, Stanford, California, US; <sup>3</sup>Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, US; <sup>4</sup>Department of Research in Biomedicine and Health, University of Split School of Medicine, Split, Croatia; <a id="_idTextAnchor069"></a><sup>5</sup>Urban Vitality Centre of Expertise, Amsterdam University of Applied Sciences, Amsterdam, The Netherlands; <sup>6</sup>Amsterdam University Medical Centers, Department of Cardiology, Amsterdam, The Netherlands; <sup>7</sup>Department of Philosophy, Faculty of Humanities, Vrije Universiteit, Amsterdam, The Netherlands; <sup>8</sup>Amsterdam University Medical Centers, Department of Epidemiology and Data Science, Amsterdam, The Netherlands; <a id="_idTextAnchor070"></a><sup>9</sup>Department of Medicine, Stanford University School of Medicine, Stanford, California, US<a id="_idTextAnchor071"></a>; <sup>10</sup>Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, California, US; <sup>11</sup>Independent scholar; <sup>12</sup>Department of Health Policy, Stanford University School of Medicine, Stanford, California, US.</p>
<p class="abstract_conflict" lang="fr-FR"><span class="text-bold">Conflict of Interest Disclosures </span>Mario Malički was a co–editor in chief of Research Integrity and Peer Review, in which 1 of the included studies was published (he was not involved in the processing or decision making for that study, and Research Integrity and Peer Review full peer review histories are available to the public on the journal website). Mario Malički was also an external reviewer for 1 study. Gerben ter Riet is a co-author of 1 of the included studies and Steven N. Goodman is a co-author of 1 of the included studies. In both of these cases, data extractions and primary data analyses were done by Mario Malički and Ana Jerončić. IJsbrand Jan Aalbersberg was a Senior Vice President of Research Integrity at Elsevier. John P. A. Ioannidis, Lex Bouter, and Steven N. Goodman are members of the Peer Review Congress Advisory Board but were not involved in the review or decision for this abstract.</p>
<h4>Funding/Support</h4>
<p> The work on this project started in 2020, and in 2020 Elsevier funding was awarded to Stanford University for a 1-year METRICS postdoctoral position that supported Dr Malički’s work. From 2021 to 2025, Dr Malički’s work has been supported by the Stanford School of Medicine Research Office, which also supports the work of Dr Goodman. At the start of the project, IJsbrand Jan Aalbersberg was an employee of Elsevier.</p>
<h4>Acknowledgments </h4>
<p>In 2022’s Peer Review Congress, we presented preliminary findings (<a href="https://peerreviewcongress.org/abstract/a-synthesis-of-studies-on-changes-manuscripts-underwent-between-submission-or-preprint-posting-and-peer-reviewed-journal-publication/"><span class="Hyperlink CharOverride-6">https://peerreviewcongress.org/abstract/a-synthesis-of-studies-on-changes-manuscripts-underwent-between-submission-or-preprint-posting-and-peer-reviewed-journal-publication/</span></a>) that covered 25 studies published between 1990 and 2021. The current results include studies published before 1990 and up to January 2025.</p>
<h4>Additional Information</h4>
<p> A website with computationally reproducible results from this living review will be available soon.</p>


Differences Between Manuscript Versions: A Living Review and Series of Meta-Analyses

<p>Melecia Miller,<sup>1</sup> <span lang="it-IT">Vera Nezgovorova,</span><sup>2</sup> Ilana Kersh,<sup>3</sup> Mohamed I. Elsaid,<sup>4</sup> Marina Broitman<sup>2</sup></p>
<h4>Objective</h4>
<p> This study’s goal was to develop a way to assess patient review quality based on author and editor reactions to reviews. We used patient reviews of draft final research reports (DFRRs) of Patient Centered Outcomes Research Institute (PCORI)–funded research that were peer reviewed by clinical, statistical, and patient reviewers.<sup>1</sup> Review quality ratings were based on prior research on PCORI peer review and other research on patient reviewers.<sup>2,3</sup> We hypothesized that specific, substantive changes to the report and patient-focused comments would be associated with a favorable mention of the patient review by the editor as well as a substantive response from the author.</p>
<h4>Design</h4>
<p> Pilot testing on 13 patient reviews for 13 clinical research DFRRs submitted from 2021 to 2022 was completed in 2025. We conducted a content analysis of patient reviewer comments and developed a coding scheme based on prior research. Coding was performed at the sentence level and included 2 categories: (1) specificity and constructiveness of the comment and (2) patient focus of the comment (<b>Table 25-1110</b>). Each review was coded independently by 2 raters each for category 1 and category 2, with a final Cohen κ of 0.83 and 0.66, respectively, indicating moderate to substantial interrater agreement. Coding discrepancies between reviewers received the most conservative (ie, lowest value) code. Raters assessed whether the editor mentioned the patient review at least once in their review summary and either encouraged (code 1) or discouraged (code 2) an author response vs no mention (code 0). Author responses to reviewer comments were coded by paragraph indicating whether authors responded directly to patient-reviewer comments (code 1) or made changes to the report because of the comments (code 2) vs neither (code 0). We used hierarchical modeling to calculate the odds of editors encouraging authors to respond (code 1 vs codes 0 or 2) and mixed-effects logistic regression to calculate the odds of authors responding substantively (codes 1 or 2 vs code 0) to patient reviews with more specific, constructive comments.</p>
<h4>Results</h4>
<p> <b>Table 25-1110</b> shows counts for associations between category 1 and category 2. More specific and constructive comments and patient-focused comments were not associated with higher odds of the editor mentioning the patient review (N = 13; odds ratio [OR], 0.36; 95% CI, 0.28-0.45 and OR, 0.65; 95% CI 0.54-0.78, respectively). Patient review paragraphs with more than 50% specific and constructive comments were associated with higher odds of authors responding substantively, compared with other comment types (N = 147; OR, 4.00; 95% CI, 1.67-9.59; <i>P</i> = .002).</p>
<a href="/wp-content/uploads/2025_conference/tables/prc25-1110t1.png"> <img loading="lazy" src="/wp-content/uploads/2025_conference/tables/prc25-1110t1.png"> </a>
<h4>Conclusions</h4>
<p> This pilot study shows the feasibility of establishing codes rating the quality of patient reviews and determining whether higher quality patient reviews are associated with mentions by the editor and author responses. The results are limited because of the sample size and the uniqueness of DFRRs.</p>
<h4>References</h4>
<p>1. Broitman M, Sox HC, Slutsky J. A model for public access to trustworthy and comprehensive reporting of research. <i>JAMA</i>. 2019;321(15):1453-1454. doi:10.1001/jama.2019.2807</p>
<p>2. Ghazaryan E, Broitman M, Sox H. Differences in the style and quantity of reviewer comments in structured vs unstructured peer review forms. Poster presented at: Ninth International Congress on Peer Review and Scientific Publication; September 10, 2022; Chicago, IL.</p>
<p>3. Huml AM, Albert JM, Beltran JM, et al. Community members as reviewers of medical journal manuscripts: a randomized controlled trial. <i>J Gen Intern Med</i>. 2022;38(6):1393-401. doi:10.1007/s11606-022-07802-z</p>
<sup>1</sup>KnowledgeWorks Global, Ltd., 410 MN-25, Brainerd, MN, US; <span lang="it-IT">mmiller@originreview.org; </span><sup>2</sup>Patient-Centered Outcomes Research Institute, Washington, DC, US; <sup>3</sup>University of Pittsburgh, Pittsburgh, PA, US; <sup>4</sup>The Ohio State University College of Medicine, Columbus, Ohio, US.</p>
<h4>Conflict of Interest Disclosures </h4>
<p>None reported.</p>


Quality of Patient Reviewer Comments and Association With Author and Editor Responses

<p>Matthew B. Stanbrook,<sup>1</sup> Shannon Charlebois,<sup>1</sup> George A. Tomlinson,<sup>1</sup> Meredith Weinhold<sup>1</sup></p>
<h4>Objective</h4>
<p> The opportunity to appeal a journal’s decision to reject a manuscript enhances procedural fairness toward authors and allows editors to correct errors in judgment but wastes time for both authors and editors when unsuccessful. We sought to identify factors associated with outcomes of appeals of research manuscripts at the <i>Canadian Medical Association Journal</i> (<i>CMAJ</i>), a general medical journal that explicitly highlights the option to appeal in rejection letters.</p>
<h4>Design</h4>
<p> We conducted a retrospective cohort study of all appeal requests for manuscripts initially rejected after submission to the Research section of <i>CMAJ</i> between 2007 and 2024. We obtained data on manuscript characteristics and editorial decisions from <i>CMAJ</i>’s manuscript submission database (ScholarOne). Three editors independently reviewed the text of the initial manuscript decision letters to identify reasons for rejection, with disagreements resolved by consensus. We included manuscripts rejected at any stage of the editorial process. The primary outcome was whether appeal requests were granted or denied. A secondary outcome was whether appealed manuscripts were ultimately published. We estimated associations of multiple prespecified characteristics with appeal outcomes using bayesian logistic regression.</p>
<h4>Results</h4>
<p> Among 14,351 submitted manuscripts, 13,419 were rejected, and 616 of these rejections (4.6%) were appealed. Among appeals, 214 (34.7%) were granted, of which 58 (27.1% of granted appeals and 9.4% of all appeals) were ultimately published. Compared with manuscripts rejected without external peer review, granted appeals were more likely for manuscripts rejected after peer review without editorial discussion (adjusted odds ratio [AOR], 2.71; 95% credible interval [CrI], 1.40-5.27) but similarly likely for manuscripts rejected after peer review and editorial discussion (AOR, 1.19; 95% CrI, 0.74-1.91) (<b>Table 25-1161</b>). Granted appeals were more likely if rejection letters included positive comments (AOR, 1.82; 95% CrI, 1.21-2.76) or cited lack of clarity (AOR, 1.84; 95% CrI, 1.12-3.04) and less likely if corresponding authors were based outside Canada (AOR, 0.38; 95% CrI, 0.24-0.59). The odds of an appeal being granted also varied significantly based on the individual editor making the appeal decision (median AOR, 1.90; 95% CrI, 1.24-4.37). Among factors associated with appeals being granted, only positive comments (AOR, 1.98; 95% CrI, 1.10-3.59) were also significantly associated with successful publication after appeal. However, appealed manuscripts rejected after peer review and editorial discussion were more likely to be published than were those rejected without peer review (AOR, 4.93; 95% CrI, 2.47-10.25).</p>

![](https://assets.underline.io/markdown_image/1/image/ca4a766cb379b500fce70628c15c12cb.png)

<h4>Conclusions</h4>
<p> Characteristics of authors, editors, editorial process, and decision communication were associated with successful appeals of initially rejected research manuscripts. Lack of awareness of the small fraction of appeals that ultimately achieve publication may raise false expectations among authors.</p>
<sup>1</sup><i>Canadian Medical Association Journal </i>(<i>CMAJ</i>), Toronto, Ontario, Canada, matthew.stanbrook@cmaj.ca.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> All authors are employees or paid consultants of CMAJ. Matthew B. Stanbrook, Shannon Charlebois, and George A. Tomlinson participated in editorial decision making for some of the manuscripts included in the study. No other disclosures were reported.</p>


Factors Associated With Outcomes of Appeals of Manuscripts Initially Rejected by a General Medical Journal

<p>Daniel S. Evanko,<sup>1</sup> Deondre Jordan<sup>1</sup></p>
<h4>Objective </h4>
<p>Journals ask authors to comply with many reporting requirements to improve study transparency, reproducibility, and reuse. Manual monitoring and enforcement of requirements is difficult, and reporting checklists or vendor-based solutions can be untargeted, burdensome, expensive, or of limited effectiveness. For example, since June 2020 the American Association for Cancer Research has used a methods review tool (SciScore) to help improve author compliance with reporting requirements at its 10 journals. This tool greatly increased author assignment of research resource identifiers (RRIDs),<sup>1</sup> but compliance plateaued at approximately 30% of original research articles published in the association journals. This cross-sectional study describes a simple, inexpensive, automated approach to further overcome limitations of common approaches and quantitatively evaluates its performance using 3 example interventions.</p>
<h4>Design </h4>
<p>Custom Python scripts coupled with structured language queries were used for targeted extraction of manuscript data from the EJournal Press journal submission system. For RRID reporting compliance, once every day the submission system was queried for manuscripts that completed initial quality control checks and did not contain the text RRID. A templated e-mail alert was sent to the corresponding author that explained what RRIDs are, why they are important, and how to add them. The system was also adapted with a custom supervised machine-learning method to identify studies that generated new sequencing data or that were computationally intensive to encourage authors to deposit sequence data and provide computer code used in their study and used to send different alerts for these manuscripts at different designated points in the editorial process.</p>
<h4>Results </h4>
<p>The new automated RRID process was activated August 6, 2024, at all the association journals, resulting in missing-RRID alerts sent for 84% (4949 of 5899) of manuscripts at completion of initial quality control checks through the end of 2024. Performance was assessed by searching for “RRID” in published articles as done previously.<sup>1</sup> An immediate sharp increase in articles containing RRIDs was observed that quickly increased the percentage of published research articles containing RRIDs from 33% (176 of 526) in April through July to 65% (330 of 505) in November through February <b>(Figure 25-1181</b>). Targeted alerts to authors of manuscripts asking them to deposit sequencing data (sent after peer review started) or supply code in a computational science software (Code Ocean) capsule (sent after a revise decision was sent to the author) were turned on on August 26, 2024, and resulted in sequence deposition alerts sent for 43% (1416 of 3261) of manuscripts and Code Ocean alerts sent for 39% (614 of 1594) of manuscripts through April 30, 2025. The sequencing alert triggered more than 100 author e-mail communications about deposition, demonstrating author engagement with this request and allowing resolution early in the editorial process. The percentage of accepted manuscripts with linked Code Ocean capsules increased from 1.4% (21 of 1555) in 2024 to 4.6% (25 of 547) in the January to May 2025 period, a more than 3-fold increase.</p>

![](https://assets.underline.io/markdown_image/1/image/87ac32761435115531ebf1a5f6ddec09.png)

<h4>Conclusions </h4>
<p>This fully automatic and highly targeted process engages with authors in a context-dependent way at predefined times in the editorial process that are appropriate for the specific communication, thus achieving substantial author engagement and improved compliance with journal requirements.</p>
<h4>Reference</h4>
<p>1. Roelandse M, Ozykurt IB, Evanko D, Bandrowski A. Assessing the effectiveness of SciScore in supporting the reproducibility of scientific research. <i>Sci Ed</i>. 2023;46:46-52.</p>
<sup>1</sup>American Association for Cancer Research, Philadelphia, PA, US, daniel.evanko@aacr.org.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> Daniel S. Evanko is a member of the Peer Review Congress Advisory Board but was not involved in the review or decision for this abstract.</p>


Automated targeted emails for improving author compliance with study reporting requirements and other editorial processes

<p>Franki Y. H. Kung,<sup>1</sup> Mariam Aly,<sup>2</sup> Shahana Ansari,<sup>3</sup> Eliana Colunga,<sup>4</sup> M. J. Crockett,<sup>5,6</sup> Amanda B. Diekman,<sup>3</sup> Pablo Gomez,<sup>7</sup> Paul C. McKee,<sup>8</sup> Miriam Pérez,<sup>9</sup> Sarah M. Stilwell,<sup>10</sup> Matthew Goldrick<sup>11,12</sup></p>

<h4>Objective</h4> Accumulating data suggest peer reviews can exhibit bias<i> </i>(ie, systematically devaluing certain topics, methods, or the study of nondominant populations<sup>1</sup>) and can be unfair with respect to process (eg, decisions are made inconsistently) or people (eg, authors are disrespected).<sup>2</sup> Recent studies have shown that biases and unfair behaviors in peer review tend to undermine the self-reported experiences and career progression of researchers, including those from underserved populations, such as women, nonbinary individuals, and people from racial and ethnic groups other than White.<sup>1-3</sup> This study examined whether peer review exercises increased psychology and neuroscience trainees’ perceived readiness to respond to biased and unfair reviewers.</p>

<h4>Design </h4>Single-session synchronous online trainings (conducted during 2023-2024) allowed for trainer-trainee and trainee-trainee interactions. Trainees were recruited for this study through social media, targeted academic organizations, and graduate programs. Trainers (professors and a postdoctoral fellow with multiple years of peer review experience) reviewed processes occurring between journal submission and receipt of a decision letter. Trainees were then given a simplified, anonymized decision letter and reviews. Exercises and discussions illustrated how trainees could plan manuscript revisions and compose responses to critiques. Finally, trainees were given examples of biased and unfair reviews, along with strategies for responding to them while acknowledging the power dynamics inherent to review. Discussion included how to maintain motivation when encountering such reviews. Participants were asked to complete surveys before and after online training in which they rated their ability to perform the following targeted skills on a scale from 1 (very little skill) to 5 (great deal of skill): define and identify bias in peer reviews; define and identify unfair language in peer reviews; respond to reviews using fair language; respond to biased or unfair reviews; and promote an inclusive and equitable peer-review culture.</p>

<h4>Results </h4>A total of 64 participants (42 PhD students [66%], 10 predoctoral students [16%], 8 postdoctoral trainees [13%], and 4 other trainees [5%]) rated their level of skill for 5 targeted skills. Ordinal regressions showed participants had significantly higher ratings for all skills at posttest. For example, the mean (SD) self-ratings for the skill to respond to biased on unfair reviews increased from 2.0 (1.01) on the pretest to 4.09 (0.89) on the posttest (<b>Table 25-0893</b>).</p>

![](https://assets.underline.io/markdown_image/1/image/f801de28833ed0c4470dce4f2815a32b.png)

<h4>Conclusions </h4>
<p>After these peer review exercises, psychology and neuroscience trainees had an increase in their perceived readiness to confront biased and unfair reviews. Work is ongoing to scale up this training using on-demand materials and further assess training impact, including measures beyond trainee self-reports.</p>
<h4>References</h4>
<p>1. Rogers LO, Moffitt U, McLean KC, Syed M. Research as resistance: naming and dismantling the master narrative of “good” science. <i>Am Psychol</i>. 2024;79:484-496. doi:10.1037/amp0001246</p>
<p>2. Silbiger NJ, Stubler AD. Unprofessional peer reviews disproportionately harm underrepresented groups in STEM. <i>PeerJ</i>. 2019;7:e8247. doi:10.7717/peerj.8247</p>
<p>3. Aly M, Colunga E, Crockett MJ, et al. Changing the culture of peer review for a more inclusive and equitable psychological science. <i>J Exp Psychol Gen</i>. 2023;152:3546-3565. doi:10.1037/xge0001461</p>
<sup>1</sup>Department of Psychological Sciences, Purdue University, West Lafayette, IN, US; <sup>2</sup>Department of Psychology, University of California Berkeley, Berkeley, CA, US; <sup>3</sup>Department of Psychological and Brain Sciences, Indiana University Bloomington, Bloomington, IN, US; <sup>4</sup>Department of Psychology and Neuroscience, University of Colorado Boulder, City, CO, US; <sup>5</sup>Department of Psychology, Princeton University, Princeton, NJ, US; <sup>6</sup>Department of Psychology, University Center for Human Values, Princeton University, Princeton, NJ, US; <sup>7</sup>Department of Psychology, Skidmore College, Saratoga Springs, NY, US; <sup>8</sup>Department of Psychology and Neuroscience, Duke University, Durham, NC, US; <sup>9</sup>Department of Psychology, North Park University, Chicago, IL, US; <sup>10</sup>Department of Health Behavior and Health Equity, University of Michigan, Ann Arbor, MI, US; <sup>11</sup>Department of Linguistics, Northwestern University, Evanston, IL, US, matt-goldrick@northwestern.edu; <sup>12</sup>Cognitive Science Program, Northwestern University, Evanston, IL, US.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> None reported.</p>
<h4>Funding/Support</h4>
<p> This work was supported by grants from the National Science Foundation (DGE-2224777, DGE-2224779, DGE-2436430). </p>
<h4>Role of the Funder/Sponsor</h4>
<p> The funder of the study had no role in design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the abstract; and decision to submit the abstract for presentation.</p>
<h4>Additional Information</h4>
<p> Training materials are freely available (<a href="https://osf.io/658ev"><span class="Hyperlink CharOverride-6">https://osf.io/658ev</span></a>).</p>


Peer Review Exercises to Enhance Trainees' Readiness to Confront Unfair or Biased Reviews

<p>Christopher Baethge,<sup>1,2</sup>Hannah Jergas<sup>3</sup></p>
<h4>Objective</h4>
<p> Although quotations are central to scientific publications, they often misrepresent the cited reference.<sup>1</sup> Inaccurate quotations may mislead readers, undermine the line of argument of a paper, and threaten the integrity of the scientific record.<sup>2,3</sup> We updated an earlier meta-analysis<sup>1</sup> to assess improvement in recent years.</p>
<h4>Design</h4>
<p> This systematic review and meta-analysis, preregistered on Open Science Framework (OSF), was conducted following the Cochrane Handbook and PRISMA statement. It focuses on quotation inaccuracy—errors of content—as opposed to bibliographical errors, often referred to as citation errors. We searched MEDLINE, PubMed Central via PubMed, and Web of Science from January 1, 2014, through October 1, 2023, for studies on quotation errors in the medical literature, with no date, language, or source restrictions, and we hand-searched all reference lists of included studies. Literature screening, data extraction, and risk of bias estimation were carried out by 2 authors independently. We adopted the original studies’ definitions<sup>1</sup> of major inaccuracies (ie, not at all in accordance with the claim of the original authors), minor inaccuracies (ie, inconsistencies and factual errors not severe enough to contradict a statement by citing authors), secondary quotations (ie, quotations of sources referencing an original study), and total quotation errors. Summary proportions of quotation errors were estimated in random-effects meta-analyses (log-transformed proportions and DL τ<sup>2</sup> estimates), with heterogeneity measures, 95% confidence intervals, and 95% prediction intervals. Time trends were investigated in meta-regressions.</p>
<h4>Results</h4>
<p> Based on 46 studies included, with 32,074 quotations/references checked, 16.9% were incorrect (95% CI, 14.1%-20.0%). Of these, 8.0% (95% CI, 6.4%-10.0%) were major inaccuracies and 7.8% (95% CI, 5.7%-10.5%) were minor inaccuracies. Heterogeneity throughout all meta-analyses was high (<b>Table 24-0811</b>). Rates of total errors have not improved during the last decade (slope: −0.002; 95% CI, −0.03 to 0.02; <i>P</i> = .85) and neither have rates of major errors (slope: 0.002; 95% CI, −0.02 to 0.03; <i>P</i> = .89). There were no associations with risk of bias, publication bias, number of references, or medical specialty (surgical, nonsurgical, other), but the total error rate was statistically significantly (<i>P</i> = .04; binomial, n = 25) and negatively correlated with Journal Impact Factor (−0.25; Spearman, n = 23). Interrater reliability was sufficient (Cohen κ: 0.73), and summary estimates were supported by sensitivity analyses employing arcsine square root data transformation and Sidik and Jonkman τ<sup>2</sup> estimates.</p>

![](https://assets.underline.io/markdown_image/1/image/6088d5f3f296b8d165f7d47e3340260a.png)

<h4>Conclusions</h4>
<p> Quotation inaccuracy remains prevalent in scientific medical texts. Despite seemingly broad awareness among journal editors and probably among many authors, there is no evidence of improvement in recent years. It is uncertain how the utilization of artificial intelligence in writing articles and in double-checking quotation inaccuracy will impact the problem, but human effort will likely remain important in creating and vetting quotes—on the part of authors, editors, reviewers, and readers.</p>
<h4>References</h4>
<p>1. Jergas H, Baethge C. Quotation accuracy in medical journal articles—a systematic review and meta-analysis. <i>PeerJ</i>. Published online October 27, 2015. doi:10.7717/peerj.1364</p>
<p>2. Baethge C. Importance, errors, and patterns of quotations to psychiatric original articles. <i>Pharmacopsychiatry</i>. 2020;53(6):247-255. doi:10.1055/a-1167-35673</p>
<p>3. Peoples N, Østbye T, Yan LL. Burden of proof: combating inaccurate citation in biomedical literature. <i>BMJ</i>. Published online November 6, 2023. doi:10.1136/bmj-2023-076441</p>
<sup>1</sup><i>Deutsches Ärzteblatt </i>and<i> Deutsches Ärzteblatt International</i>, Editorial Offices, Cologne, Germany, <span lang="de-DE">baethge@aerzteblatt.de; </span><sup>2</sup>Department of Psychiatry and Psychotherapy, Faculty of Medicine, University of Cologne, Cologne, Germany; <sup>3</sup>Department of Neurology, Faculty of Medicine, University of Cologne, Cologne, Germany.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> Both authors have published on this topic before. Christopher Baethge is employed by a general medical journal (<i>Deutsches Arzteblatt, Deutsches Arzteblatt International</i>).</p>
<h4>Acknowledgment</h4>
<p> The authors gratefully acknowledge Joan Albert Hammerstein’s help in screening the literature.</p>
<h4>Additional Information</h4>
<p> This study is registered on Open Science Framework (OSF) at <a href="https://osf.io/95rej/?view_only=8c5d2b51a8814278bff1fec1e40d0bb9"><span class="Hyperlink CharOverride-6">https://osf.io/95rej/?view_only=8c5d2b51a8814278bff1fec1e40d0bb9</span></a></p>


Quotation inaccuracy in medicine - A systematic review and meta-analysis

<p>Seba Qussini,<sup>1,2</sup> Farizah Mezer Anami,<sup>2</sup> Kris Dierickx<sup>1</sup></p>
<h4>Objective </h4>
<p>Many peer review attributes are widely criticized and remain poorly investigated, particularly in the context of proposals’ peer review.<sup>1</sup> This study aims to explore the diverse perspectives of stakeholders regarding the role of unmasking in funding proposals’ peer review and the implications of open peer review as part of the recent open science movement, specifically in biomedical research proposals’ peer review.</p>
<h4>Design </h4>
<p>To describe participants’ perspectives as constructed through their recent experiences, we have conducted a generic descriptive qualitative study<sup>2</sup> within a constructivist paradigm, using semistructured interviews to gather insights from reviewers, applicants, and peer review scholars. A total of 23 participants were selected through purposive and snowball sampling from funding agencies in Belgium and Qatar with whom no prior relationship had been established. Interviews were conducted between June 2024 and February 2025 by the first author (S.Q.). Transcribed interviews were analyzed according to the 6-step thematic framework analysis described by Braun and Clarke.<sup>3</sup> Initially, autogenerated transcripts were read and checked for in-depth familiarization with the data, which was followed by a line-by-line inductive coding, conducted iteratively after each set of 2 interviews. We followed the Consolidated Criteria for Reporting Qualitative Research (COREQ) guidelines.</p>
<h4>Results </h4>
<p>Codes with shared characteristics were grouped into categories, and ultimately 3 overarching themes were generated: (1) the importance of increased transparency in fund allocation procedures while maintaining anonymized reviewer identities; (2) open peer review as a feasible approach for enhancing transparency and accountability in funding proposals’ peer review; and (3) a growing critical stance toward traditional peer review systems, calling for alternative models like baseline or lottery funding procedures. Collectively, the results shed light on the perceived advantages and limitations of different peer review models and provide an understanding of how unmasking identities influences the fairness and objectivity of fund allocation decisions.</p>
<h4>Conclusions </h4>
<p>There is persistent preference for double-anonymous review among the scientific community; however, researchers are increasingly aware of the shortcoming of anonymized review, especially in light of current challenges within the funding landscape. Simultaneously, they recognize the importance of greater openness in peer review and increased transparency in fund allocation procedures.</p>
<h4>References</h4>
<p>1. Qussini S, MacDonald RS, Shahbal S, Dierickx K. Blinding models for scientific peer-review of biomedical research proposals: a systematic review. <i>J Empir Res Hum Res Ethics</i>. 2023;18(4):250-262. doi:10.1177/15562646231191424</p>
<p>2. Qussini S, Dierickx K. Multi-stakeholder perspectives on current attitudes toward (un)blinding reviewers’ identity in biomedical research proposals peer review: a qualitative study. OSF Registries. <a href="https://osf.io/dp65f"><span class="Hyperlink CharOverride-6">https://osf.io/dp65f</span></a></p>
<p>3. Clarke V, Braun V. Thematic analysis. <i>J Pos Psychol</i>. 2017;12(3):297-298. doi:10.1080/17439760.2016.1262613</p>
<sup>1</sup>Centre for Biomedical Ethics and Law, Faculty of Medicine, KU Leuven, Leuven, Belgium, seba.qussini@student.kuleuven.be; <sup>2</sup>Hamad Medical Corporation, The Medical Research Center, Doha, Qatar.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> None reported.</p>
<h4>Funding/Support</h4>
<p> The publication and conference fees will be funded by the Medical Research Center at Hamad Medical Corporation through grant MRC-01-24-305.</p>
<h4>Role of the Funder/Sponsor</h4>
<p> The Medical Research Center had no role in the design, analysis, or interpretation of this study.</p>


Multi-Stakeholder Perspectives on Current Attitudes Toward Unmasking Reviewers' Identity in Biomedical Research Proposals Peer Review: A Qualitative Study

Next from The 10th Peer Review Congress

Evolution of Authorship Diversity in African Surgical Research Over 2 Decades

Similar lecture

Changes to Research Article Abstracts Between Submission and Publication

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES