United States

&lt;p&gt;Annie Whamond,&lt;sup&gt;1&lt;/sup&gt; Adrian G. Barnett,&lt;sup&gt;2&lt;/sup&gt; Jennifer A. Byrne&lt;sup&gt;1,3&lt;/sup&gt;&lt;/p&gt;
&lt;h4&gt;Objective&lt;/h4&gt;
&lt;p&gt; Paper mills are unethical organizations that provide low-value or fraudulent content to client authors.&lt;sup&gt;1&lt;/sup&gt; Although image manipulation and incorrect reagent detection tools are available,&lt;sup&gt;1&lt;/sup&gt; these approaches can require reviewers and readers to learn new skills. Scaled manuscript production through templates may also result in paper mill articles showing other common features. We aimed to help readers identify possible indicators of paper mill support by simply scanning article text and references. We focused on discussion sections for this study.&lt;/p&gt;
&lt;h4&gt;Design&lt;/h4&gt;
&lt;p&gt; We used an exploratory, cross-sectional design&lt;sup&gt;2&lt;/sup&gt; to develop a descriptive analysis of discussion text and references in high Impact Factor (IF) cancer journals, defined as IF of 7 or higher for journal categories oncology, biochemistry and molecular biology, or cell biology. We downloaded the Retraction Watch database on October 16, 2024, and filtered for retraction reason &lt;i&gt;paper mill&lt;/i&gt;, subject &lt;i&gt;cancer&lt;/i&gt;, and journals on our high IF list. To identify comparison article cohorts, we randomly sampled original cancer articles in (1) the same 8 journals as retracted articles or (2) 8 independent journals with no paper mill retractions and high IF maintained for 20 years or longer. For each article, we recorded the percentages of references first cited in the introduction, methods, results, or discussion section. Focusing on discussion sections, we classified each sentence as providing background, summary, comparison, interpretation or implication, limitation, or future direction&lt;sup&gt;3&lt;/sup&gt; and recorded whether individual discussion sentences cited new references. &lt;/p&gt;
&lt;h4&gt;Results &lt;/h4&gt;
&lt;p&gt;We found 22 retracted paper mill articles from 8 journals published between June 21, 2016, and June 16, 2022. Both comparison groups (50 articles each) were restricted to articles published between January 1, 2016, and October 31, 2024. Articles in all 3 groups included similar numbers of total references per article (median [IQR]: 42 [36-46] retracted articles; 47 [36-55] articles in same journals; 51 [45-60] articles in independent journals) and percentages of discussion sentences per article (median [IQR]: 34 [30-38] retracted articles; 40 [30-48] same journals; 34 [26-43] independent journals). Analyses of discussion sections indicated that retracted paper mill articles included higher percentages of references that were first cited in the discussion and higher percentages of discussion sentences that described background information and cited new references (&lt;b&gt;Figure 25-1022&lt;/b&gt;).&lt;/p&gt;

![](https://assets.underline.io/markdown_image/1/image/d20ae00ed6b6328fe12ed01f027e066e.png)

&lt;h4&gt;Conclusions&lt;/h4&gt;
&lt;p&gt; Our analyses suggest that some discussion sections in retracted paper mill cancer research articles reiterate background information that is supported by new and possibly superfluous references. While recognizing that genuine studies also cite new references in discussion sections, superficial and redundant second introductions in discussion sections could help readers identify potentially problematic articles in high IF cancer journals, particularly when combined with other features of paper mill support.&lt;sup&gt;1&lt;/sup&gt;&lt;/p&gt;
&lt;h4&gt;References&lt;/h4&gt;
&lt;p&gt;1. Byrne JA, Abalkina A, Akinduro-Aje O, et al. A call for research to address the threat of paper mills. &lt;i&gt;PLoS Biol&lt;/i&gt;. 2024;22(11):e3002931. doi:10.1371/journal.pbio.3002931&lt;/p&gt;
&lt;p&gt;2. Kesmodel US. Cross-sectional studies—what are they good for? &lt;i&gt;Acta Obstet Gynecol Scand&lt;/i&gt;. 2018;97(4):388-393. doi:10.1111/aogs.13331&lt;/p&gt;
&lt;p&gt;3. Toronto CE, Remington R. Discussion and conclusion. In: &lt;i&gt;A Step-by-Step Guide to Conducting an Integrative Review&lt;/i&gt;. Toronto CE, Remington R, eds. Springer International; 2020:71-84.&lt;/p&gt;
&lt;sup&gt;1&lt;/sup&gt;School of Medical Sciences, Faculty of Medicine and Health, University of Sydney, Sydney, New South Wales, Australia, jennifer.byrne@health.nsw.gov.au; &lt;sup&gt;2&lt;/sup&gt;School of Public Health and Social Work, Queensland University of Technology, Kelvin Grove, Queensland, Australia; &lt;sup&gt;3&lt;/sup&gt;NSW Health Statewide Biobank, NSW Health Pathology, Camperdown, New South Wales, Australia.&lt;/p&gt;
&lt;h4&gt;Conflict of Interest Disclosures&lt;/h4&gt;
&lt;p&gt; None reported.&lt;/p&gt;
&lt;h4&gt;Funding/Support&lt;/h4&gt;
&lt;p&gt; Adrian G. Barnett and Jennifer A. Byrne acknowledge grant funding from the National Health and Medical Research Council of Australia (ideas grant APP2029249). This grant supports Annie Whamond’s PhD candidature.&lt;/p&gt;
&lt;h4&gt;Role of Funder/Sponsor&lt;/h4&gt;
&lt;p&gt; The funding body played no role in the study design, data collection, management, analysis, or interpretation, and will play no role in the writing of any report, or the decision to submit the report for publication.&lt;/p&gt;


The 10th Peer Review Congress

Analysis of Cancer Research Discussion Text and References in High-Impact Factor Journals for Possible Indicators of Paper Mills

paper mills

quality of the literature

research integrity

The Tenth Peer Review Congress featured 3 days for presentations of new research into peer review and all aspects of scientific publication, bias, quality of reporting, and information access and dissemination.

This year’s meeting was hybrid, and all plenary sessions were livestreamed with opportunities to view presentations and engage with participants in person and virtually during and after the meeting. All recordings are now available on Underline for viewing.

<div style="position:relative;padding-top:max(60%,324px);width:100%;height:0;"><iframe style="position:absolute;border:none;width:100%;height:100%;left:0;top:0;" src="https://online.fliphtml5.com/ebtyf/mget/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe></div>


#### Why? 
Our aim is to encourage research into the quality and credibility of peer review and scientific publication, to establish the evidence base on which scientists can improve the conduct, reporting, and dissemination of scientific research.

#### Who? 
Participants include editors and publishers of scientific peer-reviewed journals, researchers, funders, bibliometric and informatics experts, information innovators, librarians, journalists, policymakers, ethicists, scientific information disseminators, and anyone interested in the progress of the scientific information enterprise and the quality of scientific evidence.

The Congress embraces a wide range of disciplines, including biomedicine, health and life sciences, applied sciences, basic sciences, physical and chemical sciences, mathematics, computer sciences, engineering, economics, and social sciences as well as new and emerging disciplines.

#### What? 
The Congress provides a forum for the presentation and discussion of new research into the quality and credibility of editorial and funding peer review, scientific and scholarly publication, research methods and reporting practices, identification and management of biases, transparency, reproducibility, access, publication and dissemination metrics, identification and management of threats to the integrity of science, and advances in information exchange.

**Follow us**

[![](https://assets.underline.io/markdown_image/1/image/662a22766b9788ed3766cb23cd156e32.png)](https://x.com/peerrevcongress)  [![](https://assets.underline.io/markdown_image/1/image/bb105afb1773a01fa169aa97ca1af49a.png)](https://www.youtube.com/channel/UCsG_0nKi3_Zfyv0yRPdxjWQ)  [![](https://assets.underline.io/markdown_image/1/image/034e1e70a194a9a4178fdf470ef796db.png)](https://bsky.app/profile/peerreviewcongress.bsky.social)



You need to log in with the email address you registered with. 



Please log in to explore this event.

Registration is required to attend this event, please [here](https://peerreviewcongress.org/registration-information/) to secure your ticket. Please check your spam/promotional inbox too.



Please register!

The Tenth Peer Review Congress features 3 days for presentations of new research into peer review and all aspects of scientific publication, bias, quality of reporting, and information access and dissemination.

<p>Annie Whamond,<sup>1</sup> Adrian G. Barnett,<sup>2</sup> Jennifer A. Byrne<sup>1,3</sup></p>
<h4>Objective</h4>
<p> Paper mills are unethical organizations that provide low-value or fraudulent content to client authors.<sup>1</sup> Although image manipulation and incorrect reagent detection tools are available,<sup>1</sup> these approaches can require reviewers and readers to learn new skills. Scaled manuscript production through templates may also result in paper mill articles showing other common features. We aimed to help readers identify possible indicators of paper mill support by simply scanning article text and references. We focused on discussion sections for this study.</p>
<h4>Design</h4>
<p> We used an exploratory, cross-sectional design<sup>2</sup> to develop a descriptive analysis of discussion text and references in high Impact Factor (IF) cancer journals, defined as IF of 7 or higher for journal categories oncology, biochemistry and molecular biology, or cell biology. We downloaded the Retraction Watch database on October 16, 2024, and filtered for retraction reason <i>paper mill</i>, subject <i>cancer</i>, and journals on our high IF list. To identify comparison article cohorts, we randomly sampled original cancer articles in (1) the same 8 journals as retracted articles or (2) 8 independent journals with no paper mill retractions and high IF maintained for 20 years or longer. For each article, we recorded the percentages of references first cited in the introduction, methods, results, or discussion section. Focusing on discussion sections, we classified each sentence as providing background, summary, comparison, interpretation or implication, limitation, or future direction<sup>3</sup> and recorded whether individual discussion sentences cited new references. </p>
<h4>Results </h4>
<p>We found 22 retracted paper mill articles from 8 journals published between June 21, 2016, and June 16, 2022. Both comparison groups (50 articles each) were restricted to articles published between January 1, 2016, and October 31, 2024. Articles in all 3 groups included similar numbers of total references per article (median [IQR]: 42 [36-46] retracted articles; 47 [36-55] articles in same journals; 51 [45-60] articles in independent journals) and percentages of discussion sentences per article (median [IQR]: 34 [30-38] retracted articles; 40 [30-48] same journals; 34 [26-43] independent journals). Analyses of discussion sections indicated that retracted paper mill articles included higher percentages of references that were first cited in the discussion and higher percentages of discussion sentences that described background information and cited new references (<b>Figure 25-1022</b>).</p>

![](https://assets.underline.io/markdown_image/1/image/d20ae00ed6b6328fe12ed01f027e066e.png)

<h4>Conclusions</h4>
<p> Our analyses suggest that some discussion sections in retracted paper mill cancer research articles reiterate background information that is supported by new and possibly superfluous references. While recognizing that genuine studies also cite new references in discussion sections, superficial and redundant second introductions in discussion sections could help readers identify potentially problematic articles in high IF cancer journals, particularly when combined with other features of paper mill support.<sup>1</sup></p>
<h4>References</h4>
<p>1. Byrne JA, Abalkina A, Akinduro-Aje O, et al. A call for research to address the threat of paper mills. <i>PLoS Biol</i>. 2024;22(11):e3002931. doi:10.1371/journal.pbio.3002931</p>
<p>2. Kesmodel US. Cross-sectional studies—what are they good for? <i>Acta Obstet Gynecol Scand</i>. 2018;97(4):388-393. doi:10.1111/aogs.13331</p>
<p>3. Toronto CE, Remington R. Discussion and conclusion. In: <i>A Step-by-Step Guide to Conducting an Integrative Review</i>. Toronto CE, Remington R, eds. Springer International; 2020:71-84.</p>
<sup>1</sup>School of Medical Sciences, Faculty of Medicine and Health, University of Sydney, Sydney, New South Wales, Australia, jennifer.byrne@health.nsw.gov.au; <sup>2</sup>School of Public Health and Social Work, Queensland University of Technology, Kelvin Grove, Queensland, Australia; <sup>3</sup>NSW Health Statewide Biobank, NSW Health Pathology, Camperdown, New South Wales, Australia.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> None reported.</p>
<h4>Funding/Support</h4>
<p> Adrian G. Barnett and Jennifer A. Byrne acknowledge grant funding from the National Health and Medical Research Council of Australia (ideas grant APP2029249). This grant supports Annie Whamond’s PhD candidature.</p>
<h4>Role of Funder/Sponsor</h4>
<p> The funding body played no role in the study design, data collection, management, analysis, or interpretation, and will play no role in the writing of any report, or the decision to submit the report for publication.</p>


poster

<p>Alexandre Clausse,<sup>1</sup> Guillaume Cabanac,<sup>1,2</sup> Pascal Cuxac,<sup>3</sup> Cyril Labbé<sup>4</sup></p>
<h4>Objective</h4>
<p> The Problematic Paper Screener (PPS) features 10 detectors to identify problematic articles in the scientific record.<sup>1</sup> The detectors search through metadata, references, or textual contents. The tortured phrases<sup>2</sup> detector uses a list of more than 7000 identified suspect expressions (<b>Table 25-1159</b>) to query a database (Dimensions; Digital Science) with approximately 130 million scientific articles.<sup>3</sup> The PPS displays the screening results on a public website. In this qualitative study, we introduce an alternative screening algorithm based on regular expressions (regex) and benchmark its effectiveness. Publishers are welcome to use this stand-alone algorithm that does not require a subscription to the database that we used. </p>

![](https://assets.underline.io/markdown_image/1/image/8b16462ed041c6b4f82712aec9be879f.png)

<h4>Design</h4>
<p> As of May 2025, we designed an algorithm that uses regex based on the PPS fingerprints list in the database to capture different forms of tortured phrases, such as “man-made consciousness,” “profound learning AND deep learning,” and “128 pieces”~5’ [128 bits] (ie, a tortured phrase with components in a 5-word sliding window). This approach is independent from search engines, as it matches fingerprints against the textual content of each article, ignoring their metadata, figure and table contents (including captions), and references. We benchmarked this new approach on the Hindawi extensible markup language (XML) corpus, which contains several hundred articles with tortured phrases, focusing on articles published from 2020 to 2022—the period with the most PPS-flagged articles (n <i>= </i>3400).</p>
<h4>Results</h4>
<p> The regex-based algorithm flagged 2455 problematic articles, with 1948 also flagged by PPS, yielding a 58% overlap. After removing duplicates, 401 articles flagged by the PPS were missing from the Hindawi XML corpus, amounting to 3371 problematic articles in 139 different journals. The evaluation is ongoing; we analyzed the top 200 results and found 48 false-positive results due to database indexing issues related to special characters and figures and PPS querying issues. We also found 52 false-negative results due to fingerprints list issues and because we excluded figures, tables, and references. Overall, we obtained 100 true-positive results, as both detectors (ie, PPS and regex-based algorithm) extracted expressions on the fingerprints list from the same articles.</p>
<h4>Conclusions</h4>
<p> The regex-based algorithm yielded results comparable to the current PPS screening process. Other configurations should be tested, including figures, tables, and references in the screening, as tortured phrases can also appear there. We invite publishers to use this regex-based approach to screen incoming manuscripts. We noticed several querying issues with the database that we used, and some fingerprints need to be redesigned. We will investigate further the validation assessment, as there are still 3351 articles to be reassessed. This research could contribute to improving the regex and updating the PPS fingerprints list, while offering detailed feedback and bug reports to the database developer.</p>
<h4>References</h4>
<p>1. Cabanac G, Labbé C, Magazinov A. The ‘Problematic Paper Screener’ automatically selects suspect publications for post-publication (re)assessment. <i>arXiv</i>. Preprint posted online October 7, 2022. doi:10.48550/arXiv.2210.04895</p>
<p>2. Cabanac G, Labbé C, Magazinov A. Tortured phrases: A dubious writing style emerging in science—evidence of critical issues affecting established journals. <i>arXiv</i>. Preprint posted online July 12, 2021. doi:10.48550/arXiv.2107.06751</p>
<p>3. Herzog C, Hook D, Konkiel S. Bringing down barriers between scientometricians and data. <i>Quantitative Science Studies</i>. 2020;1:387-395. doi:10.1162/qss_a_00020</p>
<sup>1</sup>Université de Toulouse, IRIT (UMR 5505), Toulouse, France, alexandre.clausse@irit.fr; <sup>2</sup>Institut Universitaire de France, Paris, France; <sup>3</sup>INIST CNRS, UAR 67, Vandoeuvre-lès-Nancy, France; <sup>4</sup>Université Grenoble Alpes, CNRS, Grenoble INP, LIG UMR 5217, Grenoble, France.</p>
<h4>Conflicts of Interest Disclosures </h4>
<p>Guillaume Cabanac is the administrator of the Problematic Paper Screener, a public platform that uses metadata from Digital Science and PubPeer via no-cost agreements. Guillaume Cabanac and Cyril Labbé have been in touch with most of the major publishers and their integrity officers, offering pro bono consulting regarding detection tools to various actors in the field, including Clear Skies, Morressier, River Valley, Signals, and STM.</p>
<h4>Funding/Support </h4>
<p>Alexandre Clausse, Guillaume Cabanac, and Cyril Labbé received funding from the European Research Council. Guillaume Cabanac received funding from the Institut Universitaire de France.</p>


Screening articles for tortured phrases with a regex-based detector

<p>Melissa L. Rethlefsen,<sup>1</sup> Carrie Price,<sup>2</sup> Sara Schroter<sup>3</sup></p>
<h4>Objective</h4>
<p> To explore the perspectives of librarians and information specialists (LISs) who participated in a randomized controlled trial of the effect of LIS involvement on reporting quality of systematic reviews (SRs).<sup>1</sup></p>
<h4>Design</h4>
<p> We surveyed LISs who completed a peer review of an SR in a trial conducted in <i>BMJ</i>, <i>BMJ Open</i>, and <i>BMJ Medicine</i> from January 3, 2023, to January 2, 2024. LISs were not told they were peer reviewing manuscripts as part of a trial but were sent invitations to review as part of the usual process. The questionnaire sought to understand their experience, what aspects of manuscripts they focused on, perceived impact on editorial decision-making and authors’ revisions, and willingness to peer review again. To better understand factors that might impact decisions to review again, we contacted 27 respondents to participate in a focus group concentrating on facilitators and barriers to peer reviewing SRs. </p>
<h4>Results</h4>
<p> Of the 88 LISs invited to participate in the survey, 70 (79.5%) responded. Most respondents had 6 or more years of experience as an LIS (67/70; 95.7%) and advising researchers on doing SRs (55/70; 78.6%) and had peer reviewed for a journal prior to the study (57/70; 81.4%). Most focused on the search and SR methods when reviewing but also commented on aspects such as research question formulation, plagiarism, and study results and conclusions. Two-thirds (44/66; 66.7%) believed they impacted editors’ decision-making and 59.1% (39/66) believed they impacted the authors’ revisions. Only 3 factors were considered extremely or very likely to impact their decision to review again: their schedule and/or lack of time, review turnaround time, and their sense of professional duty (<b>Table 24-0830</b>). Seventeen of 27 invited LISs (63.0%) participated in a focus group. Time was the primary barrier identified in the focus groups, followed by a sense of intimidation. LISs reported being motivated by feeling valued by editors, the enjoyment of peer reviewing, the desire to improve SR quality, and peer review as a learning experience. Several expressed surprise and delight at being asked to peer review for the journals.</p>

![](https://assets.underline.io/markdown_image/1/image/c26ed0c4df287a1d953b680e49fbcda6.png)

<h4>Conclusions</h4>
<p> A select sample of highly engaged LIS respondents believed they made a difference through their peer reviews and said they were very likely to agree to peer review in the future. LISs may be an underused peer reviewing resource with methodological experience that can help editors make decisions and improve the quality of SRs.</p>
<h4>Reference</h4>
<p>1. Rethlefsen ML, Schroter S, Bouter LM, et al. Improving peer review of systematic reviews and related review types by involving librarians and information specialists as methodological peer reviewers: a randomised controlled trial. <i>BMJ Evid Based Med</i>. Published online March 11, 2025. doi:10.1136/bmjebm-2024-113527</p>
<sup>1</sup>Health Sciences Library &amp; Informatics Center, University of New Mexico, Albuquerque, NM, US, mlrethlefsen@gmail.com; <sup>2</sup>ToxStrategies, A Blue Ridge Life Sciences Company, Baltimore, MD, US; <sup>3</sup>BMJ Publishing Group, London, UK.</p>
<h4>Conflict of Interest Disclosures </h4>
<p>This study follows up on a study that was conducted as part of Melissa L. Rethlefsen’s self-funded PhD project registered at Maastricht University, the Netherlands, in collaboration with BMJ Publishing Group. Carrie Price was employed at the National Institutes of Health Library at the time of this research. Sara Schroter is a full-time employee of BMJ Publishing Group but is not involved in decision-making on individual research submissions.</p>


Librarian and Information Specialist Perceptions of Peer Reviewing Systematic Reviews

<p><span lang="fr-FR">Abdelghani Maddi,¹ Ahmad Yaman Abdin,²</span><sup>,</sup><span lang="ar-SA">³ Francesco De Pretis⁴</span><sup>,</sup><span lang="fr-FR">⁵</span></p>

<h4>Objective</h4>
<p> To evaluate, using an empirically calibrated, agent-based model (ABM), whether a structured reviewer-training intervention is associated with improved peer review quality and reviewer engagement and reduced positive outcome bias across single- and multijournal settings. Existing randomized and quasi-experimental trials of peer review interventions are limited in scope and effect size,¹ and evolutionary models suggest competition among journals shapes reviewer effort.²</p>
<h4>Design </h4>
<p>We built an ABM populated by authors, reviewers, and editors. Calibration used the PeerRead International Conference on Learning Representations (ICLR)&#160;2017-2019 corpus (8151 reviews, 4905 submissions).³ Simulations followed the STRESS-ABM recommendations. Single-journal scenarios assumed 2500 submissions per year, while multijournal portfolios comprised 5 titles handling 10,000 combined submissions. The intervention increased the share of reviewers completing a certified online training module from 0% to 40%. Main outcomes were (1) review quality score (1-10 ordinal), (2) reviewer engagement (report length, number of words), and (3) positive outcome bias (difference in acceptance probability between high- and low-significance manuscripts). Each scenario used 10,000 Monte-Carlo replications; 95% confidence intervals were obtained with percentile bootstrap.</p>
<h4>Results </h4>
<p>In the single-journal baseline, mean review quality score was 5.52 (95%&#160;CI,&#160;5.49-5.55), and mean reviewer engagement was 604&#160;words (95% CI, 598-610); positive outcome bias was 0.33 (95% CI, 0.32-0.35) (<b>Table 25-0961</b>). With training, mean scores increased to 5.80 (95% CI, 5.77-5.83; Δ&#160;+0.28), engagement to 682&#160;words (95% CI, 675-689; Δ&#160;+78 words), and bias decreased to 0.26 (95% CI, 0.25-0.28; Δ&#160;-0.07). Multijournal results were similar (score, 5.54&#160;vs&#160;5.82; engagement, 607&#160;vs&#160;685&#160;words; bias, 0.34&#160;vs&#160;0.27). Across 10,000 replications per arm, each difference was significant at P &lt; .001.</p>

![](https://assets.underline.io/markdown_image/1/image/bbc8009dbf735846f879b676a4babc28.png)

<h4>Conclusions</h4>
<p> A pragmatic reviewer training package was associated with modest but consistent improvements in review quality, reviewer engagement, and reduction of positive outcome bias in both single- and multijournal simulations. These data support further real-world trials of low-cost training interventions.</p>
<h4>References</h4>
<p>1. Heim A, Ravaud P, Baron G, Boutron I. Designs of trials assessing interventions to improve the peer review process: a vignette-based survey. <i>BMC Med</i>. 2018;16(1):191. doi:10.1186/s12916-018-1167-7</p>
<p>2. Radzvilas M, De Pretis F, Peden W, Tortoli D, Osimani B. Incentives for research effort: an evolutionary model of publication markets with double-blind and open review. <i>Comput Econ</i>. 2023;61(4):1433-1476. doi:10.1007/s10614-022-10250-w</p>
<p>3. Kang D, Ammar W, Dalvi B, et al. A dataset of peer reviews (PeerRead): collection, insights, and NLP applications. In: <i>Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics (HLT)</i>. Association for Computational Linguistics; 2018;1. doi:10.18653/v1/N18-1149</p>
<sup>1</sup>Sorbonne Université, CNRS, Groupe d’Étude des Méthodes de l’Analyse Sociologique de la Sorbonne, GEMASS, Paris, France; ²Division of Pharmasophy, School of Pharmacy, Saarland University, Saarbrücken, Germany; ³Division of Bioorganic Chemistry, School of Pharmacy, Saarland University, Saarbrücken, Germany; ⁴Department of Environmental and Occupational Health, School of Public Health, Indiana University Bloomington, Bloomington, IN, US, francesco.depretis@unimore.it; ⁵Department of Communication and Economics, University of Modena and Reggio Emilia, Reggio Emilia, Italy.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> None reported.</p>
<h4>Funding/Support</h4>
<p> This research was partially supported by the French Research Agency ANR via the OPENIT project (projet AAPG 2024, Agence Nationale de la Recherche).</p>
<h4>Role of the Funder/Sponsor</h4>
<p> The funding organizations had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the abstract; and decision to submit the abstract for presentation.</p>
<h4>Additional Information</h4>
<p> GPT-o3 (OpenAI) was used for code and writing and editing on January 31, 2025. All authors take responsibility for the integrity of the content.</p>


An Agent-Based Modeling Approach for Evaluating Interventions to Optimize Peer Review

<p>Cyril Labbé,<sup>1</sup> <span lang="fr-FR">Gilles Hubert,</span><sup>2</sup><span lang="fr-FR"> </span>Wendeline Swart,<sup>2</sup> <span lang="fr-FR">Guillaume Cabanac</span><sup>2,3</sup></p>
<h4>Objective </h4>
<p>Paper mills are well documented, and the existence of review mills has been suggested.<sup>1</sup> Our aim was to test tools and methods that would report evidence of such mills. We studied 4 datasets of peer review reports and found evidence for template-based peer review report practices.</p>
<h4>Design </h4>
<p>Attempting to maximize diversity, we collected 4 datasets gathering 148,159 peer review reports. We used webscraped peer review reports from MDPI journals (11 journals from 2018-2025: 47,593 articles and 122,831 reports), <i>BMJ</i> (97 articles from 2021-2024: 308 reports), <i>PeerJ</i> journals listed in the Multidisciplinary Open Peer Review Dataset (MOPRD)<sup>2</sup> (7 journals from 2015-2022: 6292 articles and 12,959 reports), as well as the available dataset NLPeer<sup>3</sup> (12,061 reports from 5 conferences in NLP: years 2016, 2017, 2020, and 2022, and F1000Research platform: year 2022). We investigated only plain text reports (excluding attached files) considering “round 1” only. We computed statistics on report length, common sequences of terms (CST), and similarity measures between reports.</p>
<h4>Results</h4>
<p> Depending on the dataset, the mean report length ranged from 251 to 530 words, and the median from 197 to 433 words. Reports with fewer than 20 words (from 1 to 2825, depending on the dataset) were mostly coming from resubmissions inaccurately reported as a round 1 submission. Regarding CST, excluding stop words (respectively including stop words), 8% to 12% (respectively 11.16% to 24.72%) of reports shared common sequences of at least 10 terms. This represented from 15% to 28% (CST excluding stop words) of articles having at least 1 such report. Regarding similarity, approximately 0.5% to 1.3% of reports were highly similar to another report (ie, cosine similarity excluding stop words greater than 0.75). This represented 0.8% to 3% of articles having at least 1 such report. After having automatically highlighted common CST in reports sharing more than 100 words, we analyzed them qualitatively in order to identify potential templates used by reviewers to write their reports. We were able to identify very generic chunks that are reused from report to report, sometimes by different identifiable reviewers.</p>
<h4>Conclusions</h4>
<p> Our results showed evidence of template-based peer review reports practices. We observed that the extent of these practices varies from one dataset to another. This might be due to reviewers’ practices, discrepancies in dataset sizes, differences in scientific fields, or to journals’, editors’, or publishers’ habits. More openly available review report datasets would help further characterize and understand this phenomenon.</p>
<h4>References</h4>
<p>1. Oviedo-García MÁ. The review mills, not just (self-)plagiarism in review reports, but a step further. <i>Scientometrics</i>. 2024;129:5805-5813. doi:10.1007/s11192-024-05125-w</p>
<p>2. Lin J, Song J, Zhou Z, Chen Y, Shi X. MOPRD: a multidisciplinary open peer review dataset. Neural Comput Applications. 2023;35:24191-24206. https://doi.org/10.1007/s00521-023-08891-5</p>
<p>3. Dycke N, Kuznetsov I, Gurevych I. NLPeer: a unified resource for the computational study of peer review. <i>arXiv</i>. Preprint posted online November 12, 2022.doi:10.18653/v1/2023.acl-long.277</p>
<sup>1</sup>Univ. Grenoble Alpes, CNRS, Grenoble INP, LIG (Laboratoire d’Informatique de Grenoble LIG - équipe SIGMA), Domaine Universitaire de Saint-Martin-d’Hères, Grenoble, France, cyril.labbe@imag.fr; <sup>2</sup>IRIT, Université de Toulouse, CNRS, Toulouse INP, UT, Laboratoire IRIT - équipe IRIS, Toulouse, France; <sup>3</sup>Institut Universitaire de France (IUF), Paris, France.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> None reported.</p>
<h4>Funding/Support </h4>
<p>Cyril Labbé and Guillaume Cabanac acknowledge the NanoBubbles project (<a href="https://nanobubbles.hypotheses.org/"><span class="Hyperlink CharOverride-6">https://nanobubbles.hypotheses.org/</span></a>) that has received Synergy grant funding from the European Research Council (ERC) within the European Union’s Horizon 2020 program, grant agreement number 951393. Guillaume Cabanac received funding from the Institut Universitaire de France (IUF).</p>
<h4>Additional Information</h4>
<p> Cyril Labbé and Gilles Hubert (gilles.hubert@irit.fr) are co–corresponding authors.</p>


Evidence of Use of Template-Based Peer-Review Reports and Concern About Review Mills

<p>Markus K Heinemann,<sup>1</sup> Andreas Boening,<sup>2</sup> Kazunori Okabe,<sup>3</sup> Jessica Bogensberger,<sup>4</sup> Zulfugar Timur Taghiyev<sup>2</sup></p>
<h4>Objective</h4>
<p> Double-anonymized peer review is thought to enhance objectivity but causes additional work for editorial staff.<sup>1-3</sup> The aim of this study was to evaluate its efficiency in a midsized cardiothoracic surgical journal.</p>
<h4>Design</h4>
<p> In July 2017, the journal introduced anonymization, performed by a secretary during the first editorial office check, and introduced custom questions to the review form asking reviewers if they had an idea about the origin of a manuscript despite its anonymization and, if so, why. Descriptive statistics were used for analysis, and a 2-proportion <i>z</i> test was utilized as applicable.</p>
<h4>Results</h4>
<p> Between July 2017 and November 2024, a total of 1735 manuscripts were amenable for anonymization and sent for peer review to a mean of 2 reviewers. For 525 manuscripts (30.3%), at least 1 reviewer had an idea about its origin for at least 1 of 4 potential reasons (multiple entries possible; n = 597): faulty anonymization (232), references (172), familiar with work (146), and characteristic language (47). In 82 manuscripts (15.6%), more than 1 reason was given by at least 1 reviewer. Forty guesses (7.6%) were incorrect for the following reasons: misled by impressions of being familiar with the work (20), references (9), faulty anonymization (6), and characteristic language (5). Multiple reviewers guessed for the same reason in 8 cases and for different reasons in 32 cases. One of the reviewers was incorrect in 5 manuscripts; all reviewers guessed correctly in 35. Among 649 accepted manuscripts (37.4% of the total manuscripts), 223 (34.3%) were deanonymized by reviewers, whereas 426 stayed anonymized (65.6%). The 2-proportion <i>z</i> test demonstrated higher acceptance rates for anonymized compared with deanonymized manuscripts (426 vs 223; <i>z</i> = −11.27).</p>
<h4>Conclusions</h4>
<p> Perfect anonymization remains challenging, if not impossible, because hints identifying origins can be hidden throughout a manuscript. Roughly 30% of manuscripts were correctly deanonymized by the reviewers of an admittedly limited specialty surgical community. Even though the results seem to support different editorial dispositions of anonymized vs deanonymized manuscripts, editors must be aware of the confined reliability of anonymization and weigh it against the additional administrative workload.</p>
<h4>References</h4>
<p>1. Parmanne P, Laajava J, Järvinen N, Harju T, Marttunen M, Saloheimo P. Peer reviewers’ willingness to review, their recommendations and quality of reviews after the Finnish Medical Journal switched from single-blind to double-blind peer review. Res Integr Peer Rev. 2023;8:14. doi:10.1186/s41073-023-00140-6</p>
<p>2. Ucci MA, D’Antonio F, Berghella V. Double- vs single-blind peer review effect on acceptance rates: a systematic review and meta-analysis of randomized trials. <i>Am J Obstet Gynecol MFM</i>. 2022;4(4):100645. doi:10.1016/j.ajogmf.2022.100645</p>
<p>3. Kmietowicz, Z. Double blind peer reviews are fairer and more objective, say academics. <i>BMJ</i>. 2008;336(7638):241. doi:10.1136/bmj.39476.357280.DB</p>
<sup>1</sup>German Society for Thoracic and Cardiovascular Surgery (DGTHG), Germany, heinemann@uni-mainz.de; <sup>2</sup>Universitaetsklinik Giessen, Germany; <sup>3</sup>Bell Land General Hospital, Osaka, Japan; <sup>4</sup>Thieme Publishers, Stuttgart, Germany.</p>
<h4>Conflict of Interest Disclosures </h4>
<p>Jessica Bogensberger is an employee of Thieme Publishers, the publishing house of the journal investigated.</p>


Efficiency of Author Anonymization in Peer Review

<p>Yurong Fei-Bloom,<sup>1</sup> Matthew Welch<sup>2</sup></p>
<h4>Objective </h4>
<p><i>Molecular Biology of the Cell</i> (<i>MBoC</i>) serves as our community cell science journal for members of the American Society for Cell Biology (ASCB) and cell scientists worldwide. ASCB members contribute high-quality research manuscripts to MBoC, enhancing its reputation as a platform for enduring work. Nearly half of the articles published in <i>MBoC</i> come from ASCB members, and most submissions from members are accepted. In our annual surveys, ASCB members expressed great value in having a community-led society journal where they can publish their research, which ensures that funds used to publish remain within our community to support meetings, awards, mentorship, and professional training. The objective of this study was to examine the implications of guaranteed peer review for submissions from ASCB members. </p>
<h4>Design</h4>
<p> Starting in January 2024, <i>MBoC </i>guaranteed that all newly submitted research manuscripts with current ASCB members as corresponding authors would undergo the peer review process.<sup>1</sup> It is important to note that guaranteed peer review did not imply guaranteed acceptance; editorial standards<sup>2</sup> remained the same for all submissions, whether from ASCB members or non-members. Editors were aware of which manuscripts were submitted by members, but reviewers were not aware of the member status. The Editor in Chief carefully assessed each nonmember submission to determine whether it should be desk rejected or moved forward to the peer-review stage. The peer-review process was overseen by dedicated monitoring editors with subject matter expertise. This structure enabled these editors to manage the peer review effectively, fostering a fair and unbiased system that upholds the integrity of the review for all submissions, including those from members.</p>
<h4>Results</h4>
<p> <i>MBoC</i> received 120 submissions (new or revisions) from ASCB members in 2023 and 204 in 2024, which is a 70% increase from the previous year. The total submissions rose by 13%, from 373 in 2023 to 421 in 2024. In 2023, the journal accepted 180 manuscripts, including 102 (57%) from members. In 2024, it accepted 165 manuscripts, with 140 (85%) from members. In 2023, the journal rejected 180 manuscripts, including 12 from members. In 2024, it rejected 233 manuscripts, including 12 from members. The rejection rate of the member manuscripts was 9% in 2023 and 6% in 2024, suggesting that our members consistently submit high-quality work.<i> </i>The average time from submission to the first decision for peer-reviewed articles was 34 days, 2 days fewer than in 2023 (36 days). </p>
<h4>Conclusions</h4>
<p><i> </i>Findings of<i> </i>this study suggest that guaranteed peer review increased submissions from ASCB members while maintaining the quality and timing of the peer review process. The data also suggest that the community values straightforward and guaranteed access to constructive peer feedback.<sup>3</sup> </p>
<h4>References</h4>
<p>1. Fei-Bloom Y, Welch M. New ASCB Member Benefit—Guaranteed Peer Review in MBoC. ASCB Member News, Publishing, Society News. December 22, 2023. Accessed July 15, 2025. <a href="https://www.ascb.org/society-news/new-member-benefit-guaranteed-peer-review-in-mboc/">https://www.ascb.org/society-news/new-member-benefit-guaranteed-peer-review-in-mboc/</a></p>
<p>2. Baum B. Reviewing papers as you would like your papers to be reviewed. <i>Mol Biol Cell</i>. 2019;30:3013-3014. doi.org/10.1091/mbc.E19-05-0273</p>
<p>3. Drubin DG. Any jackass can trash a manuscript, but it takes good scholarship to create one (how <i>MBoC</i> promotes civil and constructive peer review). <i>Mol Biol Cell</i>. 2011;22:525-527. doi.org/10.1091/mbc.e11-01-0002</p>
<sup>1</sup>American Society for Cell Biology, Rockville, MD, US, yfeibloom@ascb.org; <sup>2</sup>Department of Molecular and Cell Biology, University of California, Berkeley, Berkeley, CA, US.</p>
<h4>Conflict of Interest Disclosures </h4>
<p>None reported.</p>


Manuscript Submissions Following Implementation of Guaranteed Peer Review

<p>Douglas K. Novins,<sup>1</sup> Mary K. Billingsley,<sup>2</sup> Robert R. Althoff<sup>3</sup> </p>
<h4>Objective</h4>
<p> For many journals in science and medicine, a large number of submitted manuscripts may be rejected by editors before peer review. Given the high rates of such “desk rejections,” it is important that editorial reviews minimize bias in this decision-making process. One potential source of bias is the editor having access to information regarding the authors of manuscripts, which may result in editorial decision being influenced by authors’ academic reputation as well as other author characteristics.<sup>1,2</sup> Unfortunately, a recent review was unable to identify any empirical studies of such “triple-blind” peer review.<sup>3</sup> In this pilot study, we masked author information in the initial stage of editorial review to explore whether this changes patterns in the articles that our journal sends out for peer review.</p>
<h4>Design</h4>
<p> The <i>Journal of the American Academy of Child and Adolescent Psychiatry</i> utilizes a 2-round editor in chief (EIC) review process during which the EIC is aware of author identity in both rounds of review. In the standard round 1 review, the EIC completes a high-level reading of the manuscript and the cover letter, checks for test recycling, and reviews study preregistration. In the standard round 2 review, the EIC completes a detailed read of the manuscript. In this preregistered pretest/posttest study (<a href="https://doi.org/10.17605/OSF.IO/TNYFX">https://doi.org/ 10.17605/OSF.IO/TNYFX</a>), we followed the above standard process in the pretest. In the posttest, the identity of the authors was masked in the round 1 review and tasks that unmask author identity, such as reading the cover leader, were moved to the round 2 review. </p>
<h4>Results</h4>
<p> A total of 95 manuscripts were included in the pretest and 102 were included in the posttest. In the masked process, significantly more manuscript decisions were made in round 2 (49/102) than in the unmasked process (26/95) (χ² = 8.91; 1 degree of freedom; <i>P</i> &lt; .01) (<b>Figure 25-1112</b>). Overall, EIC decisions (reject, transfer, assignment to an action editor) did not differ between the pretest and posttest (χ² = 2.48; 2 degrees of freedom; <i>P</i> = .29). </p>

![](https://assets.underline.io/markdown_image/1/image/aafe3983a7ba8c694206726665c8c41c.png)

<h4>Conclusions </h4>
<p>The shift of decisions from round 1 to round 2 in the posttest review process is consistent with the shift of key EIC tasks to the round 2 review, resulting in more manuscripts also receiving a detailed read of their manuscript than in the pretest. </p>
<h4>References</h4>
<p>1. Fox CW, Meyer J, Aimé E. Double-blind peer review affects reviewer ratings and editor decisions at an ecology journal. <i>Funct Ecol</i>. 2023;37(5):1144-1157. doi:10.1111/1365-2435.14259</p>
<p>2. Srivastava DS, Bernardino J, Marques AT, et al. Editors are biased too: an extension of Fox et al. (2023)’s analysis makes the case for triple-blind review. Funct Ecol. 2024;38(2):278-283. doi:10.1111/1365-2435.14483</p>
<p>3. Polnaszek BE, Mei J, Cheng C, et al. Triple-blind peer review in scientific publishing: a systematic review. <i>Am J Obstet Gynecol MFM</i>. 2024;6(4):101320. doi:10.1016/j.ajogmf.2024.101320</p>
<sup>1</sup><i>Journal of the American Academy of Child and Adolescent Psychiatry</i> (University of Colorado Anschutz Medical Campus); <sup>2</sup><i>Journal of the American Academy of Child and Adolescent Psychiatry</i> (American Academy of Child and Adolescent Psychiatry); <sup>3</sup><i>Journal of the American Academy of Child and Adolescent Psychiatry</i> (University of Vermont).</p>
<h4>Conflict of Interest Disclosures</h4>
<p> Douglas K. Novins reported receiving honorarium for serving as editor in chief for the <i>Journal of the American Academy of Child and Adolescent Psychiatry</i>, American Academy of Child and Adolescent Psychiatry and grant support from the National Institutes of Health. Robert R. Althoff reported receiving honorarium for serving as associate editor for the <i>Journal of the American Academy of Child and Adolescent Psychiatry</i> from the American Academy of Child and Adolescent Psychiatry; grant support from National Institutes of Health and Klingenstein Third Generation Foundation; and stock or Equity in WISER Systems, LLC.</p>
<h4>Acknowledgment</h4>
<p> We acknowledge the contributions of our journal’s senior editorial team in designing this study.</p>


Editor Initial Manuscript Review: A Masked Pilot Study

<p>Andrea Corvillon,<sup>1</sup> John Carpenter,<sup>1</sup> Nihar B. Shah<sup>2</sup></p>
<h4>Objective </h4>
<p>As the Atacama Large Millimeter/Submillimeter (ALMA) telescope, the largest radio telescope in the world, transitioned from panel-based to distributed peer review to manage increasing proposal volumes, new challenges emerged in aligning reviewer expertise with proposal content. Building on prior work highlighting the importance of reviewer-proposal match quality, this study evaluates a machine learning and optimization framework to improve assignment fairness and accuracy.</p>
<h4>Design</h4>
<p> The new assignment process has 2 steps: (1) using machine learning to measure similarity between a proposal’s topic and a reviewer’s expertise and (2) optimizing assignments based on a fairness metric. Proposal topics were inferred using latent Dirichlet allocation (LDA), trained on ALMA proposals submitted between 2012 and 2023; the topics were known based on a set of keywords selected by the principal investigators. Reviewer expertise was estimated using the same model trained on each reviewer’s past proposals. Both were represented as topic vectors, and cosine similarity was used to assess alignment. Each proposal was assigned to 10 reviewers, and each reviewer received 10 proposals. Assignments were optimized using the PeerReview4All algorithm,<sup>1</sup> which prioritizes proposals with the lowest similarity scores and least reviewer availability, ensuring fairer matches by improving assignments for the most disadvantaged proposals. This method was compared with ALMA’s approach from 2021 to 2022, which relied on direct keyword overlap. In 2021 and 2022, 1016 and 1087 reviewers assessed 1497 and 1729 proposals, resulting in 14,970 and 17,290 assignments, respectively.</p>
<h4>Results</h4>
<p> The new method was implemented in 2023, involving 1098 reviewers and 1635 proposals. Similarity scores were validated using survey data from cycles 2021 and 2022. On average, 89% of the reviewers rated their expertise. The median similarity of assignments in which reviewers identified themselves as experts was 0.35 compared with 0.04 for nonexpert assignments, confirming the metric’s reliability. Retrospective application of the similarity metric to cycles 2021 and 2022 using the same LDA model showed that the median similarity increased from 0.20 in 2022, the final cycle of the old algorithm, to 0.71 in 2023, the first cycle of the new algorithm, demonstrating a better alignment between reviewers and proposal assignments. Every cycle, on average, 87% of reviewers rated their expertise level on their assigned proposals. With the new algorithm, the percentage of reviewers identifying themselves as experts increased from 45% to 65%, while self-identified nonexperts decreased from 10% to 5%. This tendency continued in cycle 2024, when the new algorithm was also used (<b>Figure 25-1127</b>). Additionally, the new algorithm eliminated manual reassignments, reducing manual effort time from 3 to 5 days to nearly zero. These findings will be updated with analysis of new data from 2024 and 2025.</p>

![](https://assets.underline.io/markdown_image/1/image/1bbe2eac526f172fa408e3aeb5da040a.png)

<h4>Conclusions </h4>
<p>ALMA’s machine learning–based assignment framework improved reviewer-proposal alignment, increased expertise match rates, and eliminated manual reassignments.</p>
<h4>Reference</h4>
<p>1. Stelmakh I, Shah N, Singh A. PeerReview4All: fair and accurate reviewer assignment in peer review. <i>J Machine Learning Res</i>. 2021;22(163):1-66. Accessed July 16, 2025. <a href="https://jmlr.csail.mit.edu/papers/volume22/20-190/20-190.pdf"><span class="Hyperlink CharOverride-6">https://jmlr.csail.mit.edu/papers/volume22/20-190/20-190.pdf</span></a></p>
<sup>1</sup>Joint ALMA Observatory, Santiago, Chile, andrea.corvillon@alma.cl; <sup>2</sup>Carnegie Mellon University, Pittsburgh, PA, US.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> Andrea Corvillon and John Carpenter are employed by the Joint ALMA Observatory (JAO), which is jointly managed by Associated Universities, Inc/National Radio Astronomy Observatory, the European Organization for Astronomy Research in the Southern Hemisphere (ESO), and the National Astronomical Observatory of Japan (NAOJ) on behalf of the ALMA partnership. Nihar B. Shah is employed by the Carnegie Mellon University and is a member of the Peer Review Congress Advisory Board but was not involved in the editorial review or decision for this abstract.</p>
<h4>Funding</h4>
<p> The study was funded by grant 1942124 from the National Science Foundation (NSF) to the JAO.</p>
<h4>Role of Funder/Sponsor</h4>
<p> This work is carried out as part of the duties of the authors at the Joint ALMA Observatory, which receives funding from the NSF through NRAO. The NSF did not contribute to the analysis presented in this contribution.</p>
<h4>Acknowledgments</h4>
<p> ALMA is a partnership of ESO (representing its member states), NSF (US), and the National Institute of Natural Sciences (Japan), together with the National Research Council of Canada (Canada), the National Science and Technology Council and the Institute of Astronomy &amp; Astrophysics, Academia Sinica (Taiwan), and the Korea Astronomy and Space Science Institute (Republic of Korea), in cooperation with the Republic of Chile.</p>


Optimizing Proposal Assignments in the Distributed Peer Review System of the World's Largest Radio Telescope Observatory

<p>Pravin Bolshete,<sup>1</sup> Madhulika Bolshete,<sup>1</sup> Priyanka Mate<sup>1</sup></p>
<h4>Objective</h4>
<p> To assess the current status of journals and publishers formerly listed in Beall’s List, including their operational status, recent publication activity, and indexing in recognized databases.</p>
<h4>Design</h4>
<p> A systematic evaluation of journals and publishers listed in Beall’s List was conducted. The list was accessed from <a href="https://beallslist.net/"><span class="Hyperlink CharOverride-6">https://beallslist.net/</span></a>, considering Beall’s original list has been defunct since January 2017. The current version of the list is hosted anonymously and was used solely as a tool to access the archived entities listed by Beall, excluding updates made after 2017. Each website link was checked for functionality, and the operational status of the respective journal or publisher was recorded from November to December 2024. Among journals that remained active, we further assessed whether they had published articles in 2024 and their indexing status, listed on their website, in databases such as PubMed/Medline, Scopus, Embase, Directory of Open Access Journals, and Web of Science.</p>
<h4>Results</h4>
<p> Of the total number of journals (N = 1310) and publishers (N = 1163) assessed, 617 (47.1%) and 624 (53.7%), respectively, were found to be defunct, with their websites no longer accessible. Among the journals that remained active<span class="CharOverride-4"> </span>(n = 693 [52.9%]), 461 (66.5%) had published articles in 2024. Indexing analysis revealed that 154 (22.2%) of these journals mentioned that they are listed in at least 1 recognized academic database.</p>
<h4>Conclusions</h4>
<p> A significant proportion of predatory journals and publishers listed in Beall’s List have ceased operations, yet a substantial number remain active and continue to publish research. Some of these journals have managed to be indexed in reputed databases, raising concerns about the persistence and evolving strategies of predatory publishing. The operational status of a journal and its indexing do not confirm or refute its current quality. Some journals may have reformed and implemented peer review and ethical publishing practices. However, we cannot confirm whether active journals have improved, remained predatory, or now operate under misleadingly enhanced credibility. Additionally, although the use of Beall’s archived list offers a consistent reference point, the anonymous nature of its current hosting presents limitations. Overall, these findings underscore the need for ongoing scrutiny and awareness required to safeguard academic integrity and prevent researchers from inadvertently submitting their work to predatory journals.</p>
<sup>1</sup>Sqarona Medical Communications, Pune, India, pravinbolshete@gmail.com.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> All authors are affiliated with Sqarona Medical Communications, which provides medical writing services.</p>


Persistence and Indexing of Predatory Journals and Publishers: A Follow-Up Evaluation of Beall's List

<p>Marija Purgar,<sup>1,2</sup> Edward R. Ivimey-Cook,<sup>3</sup> Antica Culina,<sup>1,4</sup> Joshua D. Wallach<sup>2,5</sup></p>
<h4>Objective</h4>
<p> Preprints—preliminary research reports that have not yet undergone peer review—are becoming increasingly common across scientific fields.<sup>1</sup> However, little is known about preprint policies in journals that publish ecology and evolutionary biology research.<sup>2</sup></p>
<h4>Design</h4>
<p> In this cross-sectional analysis, we identified all journals included under the Web of Science 2023 Journal Citation Reports categories of Ecology and Evolutionary Biology. We reviewed journal and publisher websites to explore specific preprint policies, including guidelines regarding the possible locations for posting preprints (eg, on a preprint server), disclosure of preprints at time of manuscript submission and the required location (eg, in the cover letter), timing of preprint posting relative to manuscript submission (eg, before submission to a journal), and any additional actions on publication of the final article (eg, linking the preprint to the published version). The journal-level and publisher-level policies were classified as preprints allowed without restrictions, preprints considered on a case-by-case determination (manuscripts with preprints evaluated by the journal on an individual basis), or preprints prohibited. Journals with no identified policy were categorized as having no preprint policy. We used a Wilcoxon rank sum test to compare 2023 Journal Impact Factors between journals with a preprint policy at either journal- or publisher-level and those without any preprint policy.</p>
<h4>Results</h4>
<p> We identified 230 eligible ecology and evolutionary biology journals published by 69 different publishers, of which 119 (51.7%) included preprint policies in their author guidelines—either through journal-specific policies (n = 109) or by directly referencing their publisher’s preprint policies (n = 10). There were 73 additional journals (31.7%) without their own preprint policies, which did not directly reference their publisher’s preprint policies but were associated with publishers that had favorable policies. Overall, there were 38 journals (16.5%) without any journal-level or publisher-level preprint policies. Of the 192 journals with either journal-level or publisher-level preprint policies, 191 (99.5%) explicitly allowed preprints and 1 (0.5%) considered preprints on a case-by-case basis. The median (IQR) Journal Impact Factor was higher among journals with a journal-level or publisher-level preprint policy (2.4 [1.7-3.7]) compared with those without any preprint policy (0.6 [0.5-0.9]) (<i>P</i> &lt; .001). Among the 109 journals with journal-level preprint policies (<b>Table 25-1002</b>), 62 (56.9%) provided information about specific platforms (eg, bioRxiv, SSRN, arXiv), 24 (22.0%) required disclosure of preprints during manuscript submission, and 35 (32.1%) provided specific guidance on the timing of preprint posting relative to manuscript submission. </p>

![](https://assets.underline.io/markdown_image/1/image/9a918b235beaea8ae656fcd347310f00.png)

<h4>Conclusions</h4>
<p> In this study, only one-half of identified ecology and evolutionary biology journals provided preprint policies in their journal-level author guidelines or offered clear instructions regarding the use of preprints. This lack of clarity may lead to confusion among authors about whether posting a preprint before submitting to a peer-reviewed journal conflicts with the journal’s policies on prior publication. </p>
<h4>References</h4>
<p>1. Abdill RJ, Blekhman R. Meta-research: tracking the popularity and outcomes of all bioRxiv preprints. <i>eLife</i>. 2019;8:e45133. doi:10.7554/eLife.45133</p>
<p>2. Noble DW, Xirocostas ZA, Wu NC, et al. The promise of community-driven preprints in ecology and evolution. <i>EcoEvoRxiv</i>. Preprint posted online June 13, 2024. doi:10.32942/X2SS46</p>
<sup>1</sup>Division for Marine and Environmental Research, Ruđer Bošković Institute, Zagreb, Croatia; <sup>2</sup>Department of Epidemiology, Rollins School of Public Health, Emory University, Atlanta, GA, US, joshua.wallach@emory.edu; <sup>3</sup>School of Biological Sciences, One Health &amp; Veterinary Medicine, University of Glasgow, Scotland, UK; <sup>4</sup>Netherlands Institute of Ecology, Royal Netherlands Academy of Arts and Sciences, Wageningen, the Netherlands; <sup>5</sup>Collaboration for Regulatory Rigor, Integrity, and Transparency, Yale School of Medicine, New Haven, CT, US.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> Marija Purgar, Edward R. Ivimey-Cook, and Antica Culina are members of the <i>Society for Open, Reliable, and Transparent Ecology and Evolutionary Biology</i>; Marija Purgar and Edward R. Ivimey-Cook are on the current Board of Directors. Joshua D. Wallach reports funding from Arnold Ventures to the Yale Collaboration for Regulatory Rigor, Integrity, and Transparency, the National Institute on Alcohol Abuse and Alcoholism of the National Institutes of Health under award 1K01AA028258, Johnson &amp; Johnson (through the Yale Open Data Access Project), and the US Food and Drug Administration. No other disclosures were reported. </p>
<h4>Funding/Support </h4>
<p>Marija Purgar was funded by the Croatian Science Foundation project number DOK-2021-02-6688. Marija Purgar gratefully acknowledges the financial support for this research by the Fulbright US Student Program, which is sponsored by the US Department of State and the Croatian-American Fulbright Commission. Antica Culina was funded by the Croatian Science Foundation project number IP-2022-10-2872.</p>
<h4>Additional Information</h4>
<p> Marija Purgar is a co–corresponding author (mpurgar@irb.hr).</p>


Downloads

Next from The 10th Peer Review Congress

Screening articles for tortured phrases with a regex-based detector

Similar lecture

An audit and feedback intervention to reduce inappropriate citation of retracted literature in the pain and anaesthesiology fields

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES