United States

&lt;p&gt;Ivan D. Florez,&lt;sup&gt;1,2&lt;/sup&gt; Alberto Henriquez,&lt;sup&gt;3,4&lt;/sup&gt; Andrés F. Estupinan-Bohorquez&lt;sup&gt;3,5&lt;/sup&gt;&lt;/p&gt;
&lt;h4&gt;Objective &lt;/h4&gt;
&lt;p&gt;We described the influence of retracted systematic reviews and meta-analyses (SRMAs) on clinical practice guidelines (CPGs) and the characteristics of retractions in CPGs.&lt;/p&gt;
&lt;h4&gt;Design &lt;/h4&gt;
&lt;p&gt;This cross-sectional study was conducted in 2 stages based on searches focused on the Retraction Watch (RW) database and MEDLINE from inception to November 30, 2024. In the first stage, we included SRMAs. We described the reasons for retractions, and we recategorized them based on our assessment. We identified the CPGs that cited the SRMA in the Google Scholar database. In the second stage, we included the retracted CPGs, described the reasons for retractions, and recategorized them into ethical or nonethical reasons based on our assessment. Nonethical reasons were categorized as editorial or administrative or outdated guidelines, while ethical reasons were reported according to RW categories. We used descriptive statistics to summarize the findings.&lt;/p&gt;
&lt;h4&gt;Results &lt;/h4&gt;
&lt;p&gt;In the first stage, we included 377 SRMAs, of which 211 (56.0%) were retracted due to peer review or publication manipulation (eg, detected “fake” reviewers); 30 (8.0%), due to duplicate or redundant publication; and 136 (36.1%), due to intellectual or authorship disputes, plagiarism, outdated publication, retraction of included studies, methodologic or data errors, and conflicts of interests. For 49 (13.0%), specific reasons were not provided. Of the retracted SRMAs, 41 (10.9%) were cited in CPGs; 19 (46.3%) of these SRMAs were retracted due to research integrity issues and 12 (29.3%), due to data errors or being outdated. For 10 (24.4%), specific reasons were not provided. Most retractions were due to manipulation of the publication or peer review process. The median time between publication and retraction of the SRMA used in CPGs was 12.0 (IQR, 3.5-25.0) months, and the median number of SRMA citations was 40 (IQR, 22-191). In the second stage, we included 36 CPGs of the 138 potential CPGs identified. Nine CPGs (25.0%) were retracted because of ethical reasons and 22 (61.1%) for nonethical reasons; the rest had no available information. The most common ethical reasons were plagiarism, authorship or intellectual property disputes, lack of disclosure of conflicts of interest, and discrepancies between the content and the cited evidence. Among the 22 CPGs retracted for nonethical reasons, 11 were due to dual publication or incorrect citations and 9 were due to outdated recommendations. The median publication to retraction time was 10 (range, 3-96) months. All of these CPGs were cited after their retraction date, and in all cases, the citations were used to support the background of the research studies.&lt;/p&gt;
&lt;h4&gt;Conclusions &lt;/h4&gt;
&lt;p&gt;Retracted SRMAs have been informing CPGs, which provide recommendations in practice and policy. The retraction of CPGs has been neglected. The RW database should be revised according to the specificities of CPGs. The most concerning reasons are ethical. Retracted CPGs continue to be cited after their retraction, mainly to inform the background sections of articles.&lt;/p&gt;
&lt;sup&gt;1&lt;/sup&gt;Department of Pediatrics, University of Antioquia, Medellín, Colombia, ivan.florez@udea.edu.co; &lt;sup&gt;2&lt;/sup&gt;School of Rehabilitation Science, McMaster University, Hamilton, Ontario, Canada; &lt;sup&gt;3&lt;/sup&gt;Universidad del Norte, Barranquilla, Colombia; &lt;sup&gt;4&lt;/sup&gt;Universidad Metropolitana, Barranquilla, Colombia; &lt;sup&gt;5&lt;/sup&gt;EPICLINICA SAS, Barranquilla, Colombia.&lt;/p&gt;
&lt;h4&gt;Conflict of Interest Disclosures&lt;/h4&gt;
&lt;p&gt; None reported.&lt;/p&gt;


The 10th Peer Review Congress

Retraction of Systematic Reviews and Clinical Practice Guidelines

errors and corrections

retractions

research integrity

<p>Ivan D. Florez,<sup>1,2</sup> Alberto Henriquez,<sup>3,4</sup> Andrés F. Estupinan-Bohorquez<sup>3,5</sup></p>
<h4>Objective </h4>
<p>We described the influence of retracted systematic reviews and meta-analyses (SRMAs) on clinical practice guidelines (CPGs) and the characteristics of retractions in CPGs.</p>
<h4>Design </h4>
<p>This cross-sectional study was conducted in 2 stages based on searches focused on the Retraction Watch (RW) database and MEDLINE from inception to November 30, 2024. In the first stage, we included SRMAs. We described the reasons for retractions, and we recategorized them based on our assessment. We identified the CPGs that cited the SRMA in the Google Scholar database. In the second stage, we included the retracted CPGs, described the reasons for retractions, and recategorized them into ethical or nonethical reasons based on our assessment. Nonethical reasons were categorized as editorial or administrative or outdated guidelines, while ethical reasons were reported according to RW categories. We used descriptive statistics to summarize the findings.</p>
<h4>Results </h4>
<p>In the first stage, we included 377 SRMAs, of which 211 (56.0%) were retracted due to peer review or publication manipulation (eg, detected “fake” reviewers); 30 (8.0%), due to duplicate or redundant publication; and 136 (36.1%), due to intellectual or authorship disputes, plagiarism, outdated publication, retraction of included studies, methodologic or data errors, and conflicts of interests. For 49 (13.0%), specific reasons were not provided. Of the retracted SRMAs, 41 (10.9%) were cited in CPGs; 19 (46.3%) of these SRMAs were retracted due to research integrity issues and 12 (29.3%), due to data errors or being outdated. For 10 (24.4%), specific reasons were not provided. Most retractions were due to manipulation of the publication or peer review process. The median time between publication and retraction of the SRMA used in CPGs was 12.0 (IQR, 3.5-25.0) months, and the median number of SRMA citations was 40 (IQR, 22-191). In the second stage, we included 36 CPGs of the 138 potential CPGs identified. Nine CPGs (25.0%) were retracted because of ethical reasons and 22 (61.1%) for nonethical reasons; the rest had no available information. The most common ethical reasons were plagiarism, authorship or intellectual property disputes, lack of disclosure of conflicts of interest, and discrepancies between the content and the cited evidence. Among the 22 CPGs retracted for nonethical reasons, 11 were due to dual publication or incorrect citations and 9 were due to outdated recommendations. The median publication to retraction time was 10 (range, 3-96) months. All of these CPGs were cited after their retraction date, and in all cases, the citations were used to support the background of the research studies.</p>
<h4>Conclusions </h4>
<p>Retracted SRMAs have been informing CPGs, which provide recommendations in practice and policy. The retraction of CPGs has been neglected. The RW database should be revised according to the specificities of CPGs. The most concerning reasons are ethical. Retracted CPGs continue to be cited after their retraction, mainly to inform the background sections of articles.</p>
<sup>1</sup>Department of Pediatrics, University of Antioquia, Medellín, Colombia, ivan.florez@udea.edu.co; <sup>2</sup>School of Rehabilitation Science, McMaster University, Hamilton, Ontario, Canada; <sup>3</sup>Universidad del Norte, Barranquilla, Colombia; <sup>4</sup>Universidad Metropolitana, Barranquilla, Colombia; <sup>5</sup>EPICLINICA SAS, Barranquilla, Colombia.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> None reported.</p>


poster

The Tenth Peer Review Congress featured 3 days for presentations of new research into peer review and all aspects of scientific publication, bias, quality of reporting, and information access and dissemination.

This year’s meeting was hybrid, and all plenary sessions were livestreamed with opportunities to view presentations and engage with participants in person and virtually during and after the meeting. All recordings are now available on Underline for viewing.

<div style="position:relative;padding-top:max(60%,324px);width:100%;height:0;"><iframe style="position:absolute;border:none;width:100%;height:100%;left:0;top:0;" src="https://online.fliphtml5.com/ebtyf/mget/"  seamless="seamless" scrolling="no" frameborder="0" allowtransparency="true" allowfullscreen="true" ></iframe></div>


#### Why? 
Our aim is to encourage research into the quality and credibility of peer review and scientific publication, to establish the evidence base on which scientists can improve the conduct, reporting, and dissemination of scientific research.

#### Who? 
Participants include editors and publishers of scientific peer-reviewed journals, researchers, funders, bibliometric and informatics experts, information innovators, librarians, journalists, policymakers, ethicists, scientific information disseminators, and anyone interested in the progress of the scientific information enterprise and the quality of scientific evidence.

The Congress embraces a wide range of disciplines, including biomedicine, health and life sciences, applied sciences, basic sciences, physical and chemical sciences, mathematics, computer sciences, engineering, economics, and social sciences as well as new and emerging disciplines.

#### What? 
The Congress provides a forum for the presentation and discussion of new research into the quality and credibility of editorial and funding peer review, scientific and scholarly publication, research methods and reporting practices, identification and management of biases, transparency, reproducibility, access, publication and dissemination metrics, identification and management of threats to the integrity of science, and advances in information exchange.

**Follow us**

[![](https://assets.underline.io/markdown_image/1/image/662a22766b9788ed3766cb23cd156e32.png)](https://x.com/peerrevcongress)  [![](https://assets.underline.io/markdown_image/1/image/bb105afb1773a01fa169aa97ca1af49a.png)](https://www.youtube.com/channel/UCsG_0nKi3_Zfyv0yRPdxjWQ)  [![](https://assets.underline.io/markdown_image/1/image/034e1e70a194a9a4178fdf470ef796db.png)](https://bsky.app/profile/peerreviewcongress.bsky.social)



You need to log in with the email address you registered with. 



Please log in to explore this event.

Registration is required to attend this event, please [here](https://peerreviewcongress.org/registration-information/) to secure your ticket. Please check your spam/promotional inbox too.



Please register!

The Tenth Peer Review Congress features 3 days for presentations of new research into peer review and all aspects of scientific publication, bias, quality of reporting, and information access and dissemination.

<p>Caden Seraphine,<sup>1</sup> Abigail Chambers,<sup>1</sup> Susan Galandiuk<sup>2</sup></p>
<h4>Objective</h4>
<p> In recent years, <i>Diseases of the Colon &amp; Rectum</i> (<i>DCR</i>), a single society-owned specialty journal, sought to grow its online presence. The journal launched a YouTube channel in 2018, featuring videos of monthly English-language journal clubs (JCs) discussing topical publications beginning in 2020, followed by article discussions in Spanish (2022-2023), Japanese (2024), and Korean (2024). The journal maintained its Facebook and X (formerly Twitter) presence and started LinkedIn and Instagram accounts in 2023. Articles were given a nonpaywalled status for 1 month, and upcoming discussions with article hyperlinks were announced 4 times per month via social media. Articles were accessible by hyperlinks, and article citations were displayed in JC discussant slides. We sought to explore how these activities impacted measures of journal interaction and global reach.<sup>1,2</sup></p>
<h4>Design</h4>
<p> We performed a 7-year (2018-2024) retrospective cohort study of JC video-based educational content provided by <i>DCR</i> with respect to journal website and social media access following STROBE guidelines.<sup>3</sup> We examined efforts toward global engagement, including addition of journal-specific non–English-language (2 Spanish, 1 Korean, and 1 Japanese) discussions. Analyses of 4 popular social media platforms’ native analytics reporting and website engagement data examined viewer demographics (age, language, viewing country, and traffic source) and journal web-based interaction (number of views, number of unique visitors, and country of access) by month and year, trends in individual JC videos, and overall page engagement. We compared the median number of citations of JC articles (cumulative Web of Science citations) and their Altmetric scores (indicating article attention received) with mean citations of <i>DCR</i> articles per year.</p>
<h4>Results</h4>
<p> Only 1 JC featured videos of surgical procedures; all others discussed original articles. The growth in YouTube engagement since the journal’s YouTube channel launched in 2018 grew exponentially with the addition of monthly JCs in 2020. LinkedIn and Instagram were implemented in 2023, and cumulative traffic to the journal’s website from social media platforms tripled within 1 year, from 15,414 clicks in 2023 to 56,609 in 2024 (Facebook, X, Instagram, LinkedIn, and YouTube). Throughout the period, individuals aged 25 to 34 years led YouTube viewership. International engagement increased with <i>DCR</i>’s implementation of non-English JC discussions in 2022. Publications discussed during JCs consistently had higher Web of Science citations compared with the mean citations in the journal that year (<span class="CharOverride-4"><b>Table 25-0959</b></span>).</p>

![](https://assets.underline.io/markdown_image/1/image/dd454d0ce367b165c77b22150974e865.png)

<h4>Conclusions</h4>
<p> Younger audiences engage beyond traditional platforms of scholarly publishing in ways that hold inherent educational value. Journal Impact Factors exclude this reach and overlook the impact of web-based peer-reviewed educational content. Inclusion of social media and video-based learning tools, as well as efforts toward global reach, contribute to journal impact.</p>
<h4>References</h4>
<p>1. Narayan RR, Fleming AM, Gunder M, et al. Reflections from the Annals of Surgical Oncology social media committee: the impact of promoting surgical science online. Ann Surg Oncol. 2025;32(2):656-664. doi:10.1245/s10434-024-16420-4</p>
<p>2. Özkent Y. Social media usage to share information in communication journals: an analysis of social media activity and article citations. PLoS One. 2022;17(2):e0263725. doi:10.1371/journal.pone.0263725</p>
<p>3. von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP; STROBE Initiative. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. J Clin Epidemiol. 2008;61(4):344-349. doi:10.1016/j.jclinepi.2007.11.008</p>
<sup>1</sup>Price Institute of Surgical Research, University of Louisville, Louisville, KY, US; <sup>2</sup>Division of Colorectal Surgery, Department of Surgery, University of Louisville, Louisville, KY, US, s0gala01@louisville.edu.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> Susan Galandiuk receives a stipend for services as editor for the journal Diseases of the Colon &amp; Rectum from the American Society of Colon and Rectal Surgeons. No other disclosures were reported.</p>
<h4>Acknowledgments</h4>
<p> Margaret Abby (Managing Editor, Diseases of the Colon &amp; Rectum; Research Grants Coordinator, Department of Surgery, University of Louisville, Louisville, KY), helped with obtaining data on articles discussed during journal clubs and their Web of Science citations.</p>


Video and Social Media Performance at a Surgical Journal with Video Journal Clubs

<p>Jeremy Y. Ng,<sup>1,2,3</sup> Daivat Bhavsar,<sup>4</sup> Laura Duffy,<sup>4</sup> Hamin Jo,<sup>4</sup> Cynthia Lokker,<sup>4</sup> R. Brian Haynes,<sup>4,5</sup> Alfonso Iorio,<sup>4,5</sup> Ana Marušić<sup>6</sup></p>
<h4>Objective </h4>
<p>This study examined the policies implemented by academic publishers regarding authors’ use of generative artificial intelligence (GenAI) tools, focusing on their regulation, disclosure requirements, and role in ensuring the integrity of scientific publications. By analyzing the prevalence and content of these policies, this study aimed to provide insight into the current landscape and inform future policy development in the rapidly evolving field of artificial intelligence (AI)–assisted research and publication.</p>
<h4>Design </h4>
<p>A cross-sectional audit was conducted on the publicly available policies of 163 academic publishers listed as members of the International Association of Scientific, Technical, and Medical Publishers. Policies were collected and analyzed between September 1 and December 31, 2023. Publishers without publicly accessible policies specific to GenAI use by authors were excluded. Data extraction and analysis were conducted independently in duplicate, with a third reviewer resolving discrepancies. The key policy components analyzed included authorship accreditation, disclosure requirements, and permissions for tasks such as research methods, content generation, image creation, and proofreading. Descriptive statistics were used to summarize the findings. Our protocol was registered.<sup>1</sup></p>
<h4>Results </h4>
<p>Of 163 academic publishers, 56 (34.4%) had publicly available policies guiding GenAI use by authors. None permitted authorship accreditation for AI tools, citing accountability concerns and alignment with ethical guidelines. Nearly all publishers with policies (49 of 56 [87.5%]) mandated disclosure of GenAI use, primarily in the Methods or Acknowledgments section. However, disclosure practices varied, with some publishers providing standardized templates while others left requirements vague. Four publishers completely prohibited GenAI use in manuscript preparation, while others allowed their use for specific tasks. Most (33 of 56 [58.9%]) publishers permitted GenAI for drafting nonmethodological sections (eg, Introductions), while 18 (32.1%) permitted their use in research methods, such as data analysis and organization. Few publishers addressed GenAI use in image generation (14 of 163 [8.6%]) or proofreading (15 of 163 [9.2%]). Only 1 publisher (0.6%) allowed citation of AI as primary sources, while 19 (11.6%) explicitly prohibited such citations. Our study has been published.<sup>2</sup></p>
<h4>Conclusions </h4>
<p>This audit highlights the inconsistent development of GenAI policies among academic publishers, with large variability in scope and clarity. While the prohibition of AI authorship and the emphasis on mandatory disclosure are consistent themes, inconsistencies in regulating specific tasks suggest a need for standardized and comprehensive policies. As AI technology and its applications in research evolve, publishers must adapt to safeguard scientific integrity. Given that the AI landscape is fast moving, future research includes updating this audit and comparing and contrasting current policies with those found in this study. Future research should also assess how policies are implemented and enforced by examining samples of published articles, as well as explore how these policies affect editors and reviewers, taking into account potential risks, such as privacy breaches and bias.</p>
<h4>References</h4>
<p>1. Bhavsar D, Lokker C, Haynes RB, Iorio A, Marusic A, Ng JY. Academic publisher artificial intelligence chatbot policies for authors: a cross-sectional audit. OSF Registries. Accessed July 11, 2025. doi:10.17605/OSF.IO/937ES</p>
<p><span lang="fr-FR">2. Bhavsar D, Duffy L, Jo H, et al. Policies on artificial intelligence chatbots among academic publishers: a cross-sectional audit. </span><i>Res Integr Peer Rev</i><span lang="fr-FR">. 2025;10(1):1. </span>doi:10.1186/s41073-025-00158-y</p>
<sup>1</sup>Institute of General Practice and Interprofessional Care, University Hospital Tübingen, Tübingen, Germany, jeremyyng.phd@gmail.com; <sup>2</sup>Robert Bosch Center for Integrative Medicine and Health, Bosch Health Campus, Stuttgart, Germany; <sup>3</sup>Centre for Journalology, Ottawa Hospital Research Institute, Ottawa, Canada; <sup>4</sup>Department of Health Research Methods, Evidence, and Impact, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada; <sup>5</sup>Department of Medicine, McMaster University, Hamilton, Ontario, Canada; <sup>6</sup>Department of Research in Biomedicine and Health and Center for Evidence-Based Medicine, University of Split School of Medicine, Split, Croatia.</p>
<h4>Conflicts of Interest Disclosures </h4>
<p>The authors declare no conflicts of interest. Ana Marušić is a member of the Peer Review Congress Advisory Board but was not involved in the review or decision for this abstract.</p>


[V] Policies on Artificial Intelligence Among Academic Publishers

<p>Eva Sarachaga,<sup>1</sup> Ines Steffens<sup>1</sup></p>
<h4>Objective </h4>
<p>In September 2023, <i>Eurosurveillance</i> adopted an artificial intelligence (AI) policy encouraging authors and reviewers to use AI tools responsibly. The policy states that authors should disclose tools such as large language models (LLMs), chatbots, and image-generating algorithms used in the production and writing of the manuscript. It further outlines that reviewers should declare whether they have used a chatbot or LLM tool in the generation of their review or in their correspondence to authors or editors.<sup>1</sup> Fifteen months after launching the policy, we investigated how many authors and reviewers had declared using AI tools and how. Additionally, we examined whether possible AI use was flagged by the plagiarism detection system, iThenticate, which checks whether incoming submissions overlap with published material.</p>
<h4>Design </h4>
<p>Data were extracted from the article submission system, Editorial Manager. In a cross-sectional design, we included all articles and reviews in the system since the policy was published in September 2023 until November 2024. Results were analyzed by domain, AI tool, and tool version. To investigate whether results from iThenticate differed between articles where AI use was declared versus where it was not declared, we compared scores of a convenience sample of 43 articles in each group and checked manually thereafter.</p>
<h4>Results </h4>
<p>Only 7% of authors (70 of 961) and 2% of reviewers (11 of 707) declared using AI tools. Language was the most frequent domain in both groups: 61 of 70 authors and 10 of 11 reviewers. Authors declared other domains, such as coding, data analysis, prediction identification, correspondence, and figure creation. ChatGPT was the most used AI tool in both groups: 54 of 70 authors and 6 of 11 reviewers. The second most used AI tools were DeepL for authors (2 of 70) and Paperpal for reviewers (2 of 11). Our results were in line with results from other studies.<sup>2,3</sup> AI use was not declared according to the policy: only 20% of authors (13 of 70) and 20% of reviewers (2 of 11) declared the tool version as required. The plagiarism score was lower in articles where AI use was declared (35 of 43) versus where it was not declared (39 of 43).</p>
<h4>Conclusions </h4>
<p>In line with findings by others, only a limited proportion of authors and reviewers declared using AI tools following implementation of an AI policy. The proportion was similar for authors and reviewers, with language being the most frequent domain and ChatGPT the most used tool in both groups. Authors and reviewers need to be reminded to comply better with the AI policy and to declare tool version.</p>
<h4>References</h4>
<p>1. E<span class="text-italic">urosurveillance. </span>Editorial policy: Responsible use of artificial intelligence (AI) tools. Accessed July 10, 2025. <a id="_idTextAnchor012"></a><a href="https://www.eurosurveillance.org/editorial-policy"><span class="Hyperlink CharOverride-6">https://www.eurosurveillance.org/editorial-policy#AI%20policy</span></a></p>
<p>2. Salvagno M, De Cassai A, Zorzi S, et al. The state of artificial intelligence in medical research: a survey of corresponding authors from top medical journals. <i>PLOS One</i>. 2024;19(8):e0309208. doi:10.1371/journal.pone.0309208</p>
<p>3. Else H. Should researchers use AI to write papers? Group aims for community-driven standards. <em>Science</em>. 2024;384(6693). doi:10.1126/science.z9gp5zo</p>
<sup>1</sup>European Centre for Disease Prevention and Control (ECDC), Stockholm, Sweden, evasarachaga01@gmail.com.</p>
<h4>Conflict of Interest Disclosures </h4>
<p>None reported.</p>
<h4>Acknowledgment</h4>
<p> We would like to thank the Eurosurveillance editorial team (Alina Buzdugan, Anirban Dey, Kathrin Hagmaier, Megan Osler, Elina Tast-Lahti, and Karen Wilson), with special thanks to Kathrin Hagmaier, at the ECDC who provided their feedback on this work.</p>


[V] Use of Generative Artificial Intelligence Tools by Authors and Reviewers of of the Journal Eurosurveillance

<p>Pawin Taechoyotin,<sup>1</sup> Daniel E. Acuna<sup>1</sup></p>
<h4>Objective </h4>
<p>Nowadays, there is an excessive load on human reviewers to the point where the quality of peer reviews is often compromised. To alleviate this excessive load, we explored the possibility of a multiagent AI peer review panel that we have developed that considers text, figures, and citations to produce peer reviews. This study is based on the advancement of large language models<sup>1,2</sup> and the study of AI agents for peer review,<sup>3</sup> which so far only utilize the textual content within the manuscript.</p>
<h4>Design</h4>
<p> The AI panel consisted of a leader agent, an experiment agent, an impact agent, and a clarity agent. Each agent utilized zero-shot prompting to the LLM model Claude Sonnet 3.5. Prompt engineering was used to craft the system and task prompt. The system prompts guided the agents to assume their specific role, and the task prompts provided clear instructions on the tasks to perform. Novelty was assessed by comparing the manuscript with similar articles from an external database of published literature. The manuscript, novelty assessment, and figures were sent to each agent for analysis, and the leader agent compiled the information from all agents, requested clarifications if needed, and produced the final review. Our AI multiagent panel was evaluated and compared with human reviews as well as simple prompt-based review<sup>2</sup> and findings of 2 previous studies.<sup>1,3</sup> Evaluation was done by graduate students via an arena where pairs of systems were selected based on their Elo score. The Elo score is a measure of the relative capability of each system based on their win/loss in the arena against other systems and was calculated using the Bradley-Terry model. The system names were hidden from the participants to reduce bias. The participants evaluated the reviews based on technical quality, constructiveness, clarity, and overall assessment before making their choice.</p>
<h4>Results</h4>
<p> Eleven graduate students in the computer science department participated in this study, which resulted in 140 matches between pairs of systems. The systems with similar Elo scores were most likely to be paired together. Mean Elo scores of the reviews produced from our system were higher than those of human review scores (<b>Table 25-0971</b>). Further analysis showed that reviews produced by our system included more detailed assessment of the limitations and possible improvements of the manuscript than reviews produced by humans. However, our system tended to overly praise work, which humans rarely do.</p>

![](https://assets.underline.io/markdown_image/1/image/e9b3e219d84889a96133d3b20f6fe73d.png)

<h4>Conclusions</h4>
<p> Our AI peer review system demonstrated potential in aiding the peer review process of publications as an addition to human reviews. It might act as a quick response to authors while human reviewers are reviewing the manuscript. Although there might be bias within the review, the author can choose to verify and utilize comments from this quick feedback as needed. Harm or the potential for negative effects were not studied. We acknowledge that the number of participants in this study was small. Thus, a larger study might have different results from our findings.</p>
<h4>References</h4>
<p>1. Liang W, Zhang Y, Cao H, et al. Can large language models provide useful feedback on research papers—a large-scale empirical analysis. <i>NEJM AI</i>. 2024;1(8):AIoa2400196. doi:10.1056/AIoa2400196</p>
<p>2. Wang H, Fu T, Du Y, et al. Scientific discovery in the age of artificial intelligence. <i>Nature</i>. 2023;620(7972):47-60. doi:10.1038/s41586-023-06221-2</p>
<p>3. D’Arcy M, Hope T, Birnbaum L, Downey D. Marg: multiagent review generation for scientific papers. <i>arXiv</i>. Preprint posted online January 8, 2024. doi:10.48550/arXiv.2401.04259</p>
<sup>1</sup>Science of Science and Computational Discovery Lab, Department of Computer Science, University of Colorado Boulder, Boulder, CO, US, daniel.acuna@colorado.edu.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> Daniel E. Acuna reported being the founder of ReviewerZero AI LLC (<a href="https://www.reviewerzero.ai"><span class="Hyperlink CharOverride-6">https://www.reviewerzero.ai</span></a>), a company focused on research integrity solutions, which may be relevant to the topic of this publication.</p>


[V] Use of an AI Peer Review Panel to Assess Clarity, Novelty, and Impact

<p>Yidan Sun,<sup>1</sup> Mayank Kejriwal<sup>2</sup></p>
<h4>Objective </h4>
<p>Platforms like OpenReview<sup>1</sup> have become a popular standard for open peer review in conference-oriented fields such as computing.<sup>2</sup> Using OpenReview data from a machine learning conference, we examined (1) whether higher reviewer confidence is associated with more polarized scoring and (2) whether greater score variability is associated with rejection among papers with neutral mean scores. We also explored the assistive potential of a language model to assign sentiment labels to reviews of neutrally rated papers based on text alone.</p>
<h4>Design </h4>
<p>We used the OpenReview application planning interface to collect data on all International Conference on Learning Representations 2019 Conference Track submissions submitted by the September 27, 2018, deadline. The dataset included 4332 reviews across 1419 papers. Each review contained an integer score (1-10), a confidence rating (1-5), and a final accept or reject decision from a meta-reviewer. Analyses were conducted in January 2025. We used all 4332 reviews to assess whether reviewer confidence was associated with score extremity, defined as the absolute difference between a review score and the rubric midpoint (5), using Pearson correlation. We focused on 775 neutral<i> </i>papers—those with all review scores between 4 and 7 (inclusive)—and measured score variability as the SD of scores within each paper. We compared variability between accepted and rejected papers using 2-sided <i>t</i> tests. We then trained a Bidirectional Encoder Representations From Transformers (BERT)–based sentiment classifier<sup>3</sup> using polarized reviews scored less than 4 (negative) and greater than 7 (positive). The model achieved 90.9% accuracy on held-out review-level validation data. We then applied the model to all 2356 reviews from the 775 neutral papers and took the mean of predicted sentiment scores across each paper to produce a paper-level sentiment score. We labeled the top 246 papers—matching the number of actually accepted neutral papers—as “Accept” based on model sentiment. As a baseline, we ranked papers by mean score and labeled the top 246 as “Accept” by score.</p>

<h4>Results</h4> Reviewer confidence was positively correlated with score extremity (<i>r</i> = 0.15 [95% CI, 0.10-0.20]; <i>P</i> &lt; .001). Among the 775 neutral papers, 246 (31.7%) were accepted and 529 (68.3%) were rejected (<b>Table 25-1167</b>). Rejected neutral papers showed greater score variability than accepted papers (mean SD of 0.80 vs 0.59; difference, 0.21 [95% CI, 0.15-0.28]; <i>P</i> &lt; .001), supporting both hypotheses. Of the 246 papers labeled “Accept” by BERT, 137 were accepted. Of the 246 labeled “Accept” by the baseline, 203 were accepted. Among rejected papers, both methods labeled 20 as “Accept.”</p>

![](https://assets.underline.io/markdown_image/1/image/e390106030daa9fb11f54986bee7520b.png)

<h4>Conclusions</h4> Higher reviewer confidence was associated with more decisive scoring. Rejected neutral<i> </i>papers showed greater disagreement among reviewers. Despite moderate scores, review text often conveyed clear positive or negative sentiment, as inferred by the BERT model.</p>
<h4>References</h4>
<p class="abstract_abstract-text ParaOverride-4">1. OpenReview: an open platform for peer review. Accessed July 15, 2025. <a href="https://openreview.net"><span class="Hyperlink CharOverride-6">https://openreview.net</span></a></p>
<p class="abstract_abstract-text ParaOverride-4">2. Ford E. Defining and characterizing open peer review: a review of the literature<i>. J Scholarly Pub</i>. 2013;44(4):311-326. doi:10.3138/jsp.44-4-00</p>
<p class="abstract_abstract-text ParaOverride-3">3. Wolf T, Debut L, Sanh V, et al. Transformers: state-of-the-art natural language processing. In <i>Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations</i>. Association for Computational Linguistics; 2020:38-45.</p>
<sup>1</sup>Department of Industrial and Systems Engineering, University of Southern California, Los Angeles, CA, US, yidans@usc.edu; <sup>2</sup>Research Associate Professor, Principal Scientist, Information Sciences Institute, University of Southern California, Los Angeles, CA, US.
<h4>Conflict of Interest Disclosures</h4>
<p> None reported.</p>


[V] Reviewer Rating Variability and Confidence and Language Model Sentiment Prediction of Machine Learning Conference Papers

<p>Maud Bernisson<sup>1,2</sup></p>
<h4>Objective</h4>
<p> Researchers have raised concerns about scientific articles written by medical education and communication companies (MECCs) after litigations revealed that MECCs have been using unethical practices like ghostwriting.<sup>1</sup> Little is known about the literature produced by medical writing. While previous methods to assess this literature include surveying authors about ghostwriting,<sup>2,3</sup> this study offers a new method to collect this literature and makes it available to further explore medical writing.</p>
<h4>Design</h4>
<p> The method of collection of this literature used 3 different sources: (1) Web of Science, from which I collected 26,858 DOIs of articles, abstracts, and documents mentioning “medical writ*” in the acknowledgments; (2) the websites of 15 MECCs, found during the mapping of MECCs for another study, that publicize the literature they produce, and from which I collected 4555 additional DOIs; and (3) the Industry Documents Library of University of California, San Francisco (<a href="https://www.industrydocuments.ucsf.edu"><span class="Hyperlink CharOverride-6">https://www.industrydocuments.ucsf.edu</span></a>), in which I found the publication trackers of 5 companies and collected 229 DOIs of publications. The final database includes the metadata of 31,642 documents published between 1972 and 2025, collected from Web of Science, cleaned in R, and analyzed with the library dplyr (version 1.1.4).</p>
<h4>Results</h4>
<p> Transparency practices of disclosure of medical writing differed according to data sources. Only 252 of 4784 publications (5.3%) acknowledged medical writing in the Industry Documents Library and the MECCs websites. In contrast, a timeline of the literature collected from Web of Science identified an increase in transparency practices, with 138 publications acknowledging medical writing in 2008, 1475 in 2016, and 4265 in 2024. Topics also differed according to data sources. Four of the top 10 publishing journals of the MECC literature focused on health economics and outcomes research (<i>Value in Health</i>, <i>Journal of Medical Economics</i>, <i>Current Medical Research and Opinion</i>, <i>Pharmacoeconomics</i>). These 4 journals published 348 documents written by MECCs, which represents 7.6% of the data collected from MECCs websites. In this study, journals that published the MECC-written literature covered a range of topics. Among the journals assessed, a rheumatology journal published 605 MECC-written publications from 2008 to 2024; of these, 524 (86.6%) were meeting abstracts, which represented 43.3% of the 1211 meeting abstracts in the database. Limitations of this study include the difficulty in accessing sources for collecting this literature and that medical writing is not always acknowledged in published articles.</p>
<h4>Conclusions </h4>
<p>This study highlights the need to further explore use and disclosure of medical writing in the health economics and outcomes research literature, as it targets decision-makers such as payers and policymakers. The database used in this study offers different routes to study medical writing in more detail.</p>
<h4>References</h4>
<p>1. Sismondo S. <i>Ghost-Managed Medicine: Big Pharma’s Invisible Hands</i>. Mattering Press; 2018.</p>
<p>2. Flanagin A, Carey LA, Fontanarosa PB, et al. Prevalence of articles with honorary authors and ghost authors in peer-reviewed medical journals. <i>JAMA</i>. 1998;280(3):222-224. doi:10.1001/jama.280.3.222</p>
<p>3. Moffatt B, Elliott C. Ghost marketing: pharmaceutical companies and ghostwritten journal articles. <i>Perspect Biology Med</i>. 2007;50(1):18-31. doi:10.1353/pbm.2007.0009</p>
<sup>1</sup>LISIS, CNRS, INRAE, Université Gustave Eiffel, France, maud.bernisson@cnrs.fr; <sup>2</sup>Institute for Science in Society, Radboud University, the Netherlands.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> None reported.</p>
<h4>Funding/Support</h4>
<p> The author was part of the NanoBubbles project, which has received Synergy grant funding from the European Research Council, within the European Union’s H2020 programme (grant agreement 951393).</p>
<h4>Additional Information</h4>
<p> The database for this study, including the Web of Science identifiers, DOIs (when they exist), and sources (MECCs websites, Industry Documents Library, and Web of Science) of 31,642 documents involving medical writing, collected on January 9-13, 2025, is available at <a href="https://zenodo.org/records/15537678"><span class="Hyperlink CharOverride-6">https://zenodo.org/records/15537678</span></a>.</p>


[V] Compiling the Publications Produced by Medical Writing

<p>Hannah Grace Varkey,<sup>1</sup> Florian P. Thomas,<sup>1,2</sup> Elli Gourna Paleoudis<sup>3,4</sup></p>
<h4>Objective</h4>
<p> Current scientific practices and culture are largely focused on producing high numbers of publications and positive or newsworthy data. Recently, some journals have dedicated space to the publication of negative results; nevertheless, negative results often remain unpublished, leading to duplication of efforts, lack of transparency, and publication bias. To investigate the evolution of negative data publications over a decade, we reviewed published studies that did not reach statistical significance or reported inconclusive findings or trends.</p>
<h4>Design</h4>
<p> We searched for negative results in 300 publications per year from 2014 to 2024 using Web of Science’s Medline. To limit selection bias, we selected the first 300 articles of each year based on the Web of Science publication date in chronological order, without regard to the scientific or treatment area. Additional requirements included clinical trial or other experimental design, presence of an abstract, and English language; case reports, commentaries, editorials, and case series were excluded. Records from each year were reviewed independently by 2 researchers to ensure reliability and accuracy. Any disagreements were resolved by a tie-breaking third researcher.</p>
<h4>Results</h4>
<p> Over 10 years, we found 2 to 6 articles annually (<span class="CharOverride-4"><b>Figure 25-0879</b></span>) with solely negative results, with a downward trend (from 6 in 2014 to 3 in 2024). This assessment was applied to the primary outcome measure if and as identified. For the purposes of this project, negative results were defined as results that were inconclusive (ie, they did not prove a hypothesis or yield unexplained outcomes) or statistically nonsignificant based on the predefined measure of statistical significance (usually a <i>P</i> value). Identified articles with negative outcomes were most commonly from the fields of cellular biology, biomechanics, and genetics.</p>

![](https://assets.underline.io/markdown_image/1/image/ec21b6408d4e7695210a278eec82df46.png)
	
<h4>Conclusions</h4></span> Recently, there has been increased attention to and support for publishing negative findings in traditional and new scientific journals and search engines (eg, BioNØT<sup>1</sup>) focused specifically on these results. The notion that all research, regardless of outcome, can contribute to scientific advancement is gaining traction among journal editors, funders, and the scientific community overall.<sup>2</sup> This is particularly true in the artificial intelligence and machine learning era, where models are created based on existing (ie, published) data and research is driven by large datasets.<sup>3</sup> Access to negative results would not only limit duplication of effort but also improve planning and interpretation of data. Thus, it may be surprising that publications with negative data remain few in number, and such results are often rejected by journals, forgotten as unpublished data, or buried in supplemental sections.</p>
<h4>References</h4>
<p>1. Agarwal S, Yu H, Kohane I. BioNØT: a searchable database of biomedical negated sentences. <i>BMC Bioinformatics</i>. 2011;12(1):420. doi:10.1186/1471-2105-12-420</p>
<p class="abstract_reference-text ParaOverride-4">2. Sansom O, Bogani D, Reichenbach L, Wells S. Negative equity—the value of reporting negative results. <i>Dis Model Mech</i>. 2024;17(8):dmm050937. doi:10.1242/dmm.050937</p>
<p class="abstract_reference-text ParaOverride-4">3. Taniike T, Takahashi K. The value of negative results in data-driven catalysis research. <i>Nat Catal</i>. 2023;6:108-111. doi:10.1038/s41929-023-00920-9</p>
<p class="abstract_conflict ParaOverride-6" lang="fr-FR"><sup>1</sup>Department of Neurology, Hackensack Meridian School of Medicine, Nutley, NJ, US; <sup>2</sup>Hackensack University Medical Center, Nutley, NJ, US; <sup>3</sup>Investigator Initiated Research Program and Support Services, Office of Research Administration, Hackensack Meridian Health Research Institute, Nutley, NJ, US; <sup>4</sup>Department of Medical Sciences, Hackensack Meridian School of Medicine, Hackensack Meridian Health, Nutley, NJ, US; elli.gournapaleoudis@hmhn.org.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> None reported.</p>


[V] Biomedical Studies Published With Negative Results Over the Past Decade

<p>Julia Soto Rizzato,<sup>1</sup> Marcus Tolentino Silva,<sup>2</sup> David Moher,<sup>3</sup> Tais Freire Galvão<sup>1</sup></p>
<h4>Objective </h4>
<p>To assess publication trends in epidemics estimated to end by 2030 according to the Sustainable Development Goals (SDGs) in pharmaceutical journals in the past 25 years and the association of the adoption of these SDGs in 2015 with this trend. Assessments of the transition from Millennium Development Goals to SDGs are available,<sup>1</sup> but investigations are lacking on the impact of the adoption of these SDGs on publications of a specific target.</p>
<h4>Design</h4>
<p> This cross-sectional study assessed all journals listed in the Pharmacology and Pharmacy category of the Journal Citation Reports. The primary outcome was the proportion of articles published addressing AIDS, tuberculosis, and malaria—epidemics targeted to end according to SDG 3.3. We consulted the Clarivate Web of Science Core Collection on December 19, 2024, to identify the total number of articles and the number of publications in each journal from 2000 to 2024 related to these epidemics. Data on country were collected from Web of Science metadata, which are based on authors’ affiliations and then classified into low-, middle-, or high-income countries based on World Bank country classifications by income level. A paper was considered from a low- or middle-income country if at least 1 author was from 1 of the countries in the metadata. An interrupted time series analysis was conducted to calculate the regression coefficient (β) and 95% CI of the proportion of papers (per 1000 publications) on SDG 3.3 before and after its adoption in 2015. Stata, version 14.2 was used for statistical analyses.</p>
<h4>Results </h4>
<p>Three hundred fifty-five journals were included, which published 1,313,671 articles, of which 49,687 (3.8%) addressed SDG 3.3 from 2000 to 2024 and 37,521 (2.9%) were published by authors from high-income countries and 12 166 (0.9%) were published by authors from low- and middle-income countries. Between 2000 and 2015, there was an overall growing trend in published papers related to SDG 3.3 (β, 0.47; 95% CI, 0.18-0.76; <i>P</i> = <i>.</i>003), which was more pronounced in developed countries (β, 0.69; 95% CI, 0.38-1.00; <i>P</i> &lt; .001),  while the trend from developing countries was not significant (β, 0.21; 95% CI, −0.19 to 0.06; P = .30) (<b>Figure 25-0901</b></span>). After the adoption of SDGs in 2015, SDG 3.3 publications started to decrease until 2024 (β, −1.00; 95% CI, −1.54 to −0.47; P = .001) in a similar pattern in highincome countries (β, −1.40; 95% CI, −2.00 to −0.81; P < .001), whereas a nonsignificant increase was observed in lower-income countries (β, 0.05; 95% CI, −0.05 to −0.73; P = .89) in the same period.</p>

![](https://assets.underline.io/markdown_image/1/image/3a82bbfa1f2eb9d809c2927e383599b4.png)
	
<h4>Conclusions </h4>
<p>The adoption of SDGs in 2015 did not seem to affect the publication priorities of pharmaceutical journals until 2024, which indicates that these measures may not have stimulated research efforts and publications. Even when considering that the results indicating research agenda priorities would take a few years to be published, the trend decreased until the end of the series, almost a decade after the adoption of SDGs.</p>
<h4>Reference</h4>
<p>1. Díaz-López C, Martín-Blanco C, De la Torre Bayo JJ, Rubio-Rivera B, Zamorano M. Analyzing the scientific evolution of the sustainable development goals. <i>Appl Sci</i>. 2021;11(18):8286. doi:0.3390/app11188286</p>
<p class="abstract_conflict" lang="fr-FR"><span class="CharOverride-3" lang="pt-PT"><sup>1</sup></span><span lang="pt-PT">Faculdade de Ciências Farmacêuticas, Universidade Estadual de Campinas, Campinas, Brazil, </span>taisgalvao@gmail.com<span lang="pt-PT">;</span> <span class="CharOverride-3" lang="pt-PT"><sup>2</sup></span><span lang="pt-PT">Faculdade de Ciências de Saúde, Universidade de Brasília, Brasília, Brazil; </span><sup>3</sup>Centre for Journalology, Methods Centre of the Ottawa Hospital Research Institute, Ottawa, Canada.</p>
<h4>Conflict of Interest Disclosures </h4>
<p>David Moher and <a id="_idTextAnchor039"></a>Tais <span lang="pt-PT">Freire Galvão</span> report being advisory board members of the International Congress on Peer Review and Scientific Publication but were not involved in the review or decision for this abstract. The other authors declare no conflict of interest.</p>
<h4>Funding/Support<a id="_idTextAnchor040"></a> </h4>
<p>This study was funded in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior–Brasil (Finance Code 001, granted to <span lang="pt-PT">Julia Soto Rizzato)</span>. <span lang="pt-PT">Tais Freire Galvão</span> receives a productivity scholarship from the National Council for Scientific and Technological Development (grant 313431/2023-00).</p>
<h4>Role of the Funder/Sponsor </h4>
<p>The funders had no role in this research.</p>


[V] Publication Trends on Priority Epidemics According to the Sustainable Development Goals in Pharmaceutical Journals, 2000-2024

<p>Qinyue Liu,<sup>1</sup> Yagmur <span lang="ar-SA">Öztürk</span>,<sup>1</sup> Cyril Labbé<sup>1</sup></p>
<h4>Objective </h4>
<p>Abstracts of scientific publications are widely used in scientometrics and text mining because they can be extracted in large quantities from bibliometric databases, and they convey important information about the publications. However, we found misregistered abstracts in Crossref published by <i>Science</i> and the <i>Proceedings of the National Academy of Sciences</i> (<i>PNAS</i>). We aimed at estimating the proportion of misregistered abstracts in our dataset and providing the textual similarity between the correct abstracts and the misregistered texts. A previous study also found misregistered metadata in Crossref.<sup>1</sup></p>
<h4>Design </h4>
<p>Having recently worked on citation analysis, we developed a script to automatically collect a dataset of citations extracted from articles published between 2016 and 2024 in <i>The Lancet, Cell</i>, and <i>Joule</i>. Each citation context is paired with the cited abstract mainly extracted using Crossref API. This dataset includes 19,822 citation context-abstract pairs. We noticed that certain abstracts on Crossref are misregistered. For <i>Science</i>, a section that introduces and cites the current article, and for <i>PNAS</i>, a section called Significance, are misregistered as abstracts on Crossref. We developed a script to automatically find these cases from our dataset by searching for distinctive features (string “et al” for <i>Science</i> and “Significance” for <i>PNAS</i>). For the cases in <i>Science</i>, we also manually collected the correct abstracts, and used the Sentence-BERT (SBERT)<sup>2</sup> model to calculate the textual similarity between the misregistered abstracts and the correct abstracts.</p>
<h4>Results </h4>
<p>Of the 821 abstracts of <i>Science</i> publications, 402 (48%) contained the distinctive features.<i> </i>We manually verified the results, and 400 of 402 abstracts were misregistered. The 2 correctly registered abstracts were for report types of publication where there was no other text to be misregistered. Of the 669 <i>PNAS</i> abstracts, 243 (36%) contained Significance at the beginning. Moreover, most of the misregistered abstracts in our dataset, as estimated, had high textual similarity scores (cosine similarity) with the correct abstracts. The scores range from 0.58 to 0.99, with an average of 0.83. This phenomenon occurs because the misregistered abstracts summarize the article and mention key points, resembling the original abstracts.</p>
<h4>Conclusions </h4>
<p>Metadata misregistration in large repositories like Crossref can compromise data quality. Although our method detected misregistered abstracts in a small portion of the dataset using distinctive features, its scalability requires validation with larger datasets.</p>
<h4>References</h4>
<p>1. Cioffi A, Coppini S, Massari A, et al. Identifying and correcting invalid citations due to DOI errors in Crossref data. <i>Scientometrics</i>. 2022;127:3593-3612. doi:10.1007/s11192-022-04367-w</p>
<p>2. Reimers N, Gurevych I. Sentence-BERT: Sentence embeddings using Siamese BERT-Networks. In: <i>Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)</i>. Association for Computational Linguistics; 2019:3982-3992. doi:10.18653/v1/D19-1410</p>
<sup>1</sup>Université Grenoble Alpes, French National Centre for Scientific Research, Grenoble INP, Laboratoire d’Informatique de Grenoble, Grenoble, France, qinyue.liu@univ-grenoble-alpes.fr.</p>
<h4>Conflict of Interest Disclosures</h4>
<p> None reported.</p>
<h4><a id="_idTextAnchor041"></a>Funding/Support</h4>
<p> <a id="_idTextAnchor042"></a>The NanoBubbles project has received Synergy grant funding from the European Research Council, within the European Union’s Horizon 2020 program, grant 951393.</p>
<h4>Role of Funder/Sponsor </h4>
<p>The European Research Council had no role in design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the abstract; and decision to submit the abstract for presentation.</p>


[V] Science Journal Abstracts Misregistered in the Crossref Database

<p>Younwoo Ki,<sup>1,2</sup> Chungsoo Kim,<sup>3,4</sup> Yuan Lu,<sup>3</sup> Joshua D. Wallach,<sup>5</sup> Behnood Bikdeli,<sup>4,6,7</sup> Milton Packer,<sup>8,9</sup> Harlan M. Krumholz,<sup>4</sup> Seng Chan You<sup>1,2</sup></p>
<h4>Objective </h4>
<p>Recent trends in cardiovascular journals suggest a growing emphasis on publishing guidelines, reviews, and statements.<sup>1,2</sup> Citations, often used to assess the scholarly impact of journals, also influence researcher evaluation and library purchasing decisions. To understand these changes and their implications, we examined the evolving contribution of original research to citation metrics in leading cardiovascular journals.</p>
<h4>Design </h4>
<p>Our cross-sectional bibliometric study used Scopus to identify all articles published in the <i>Journal of the American College of Cardiology</i> (JACC), <i>Circulation</i> (CIRC), and <i>European Heart Journal</i> (EHJ) from 2008 to 2018, with associated citations. Using section headings on respective journal websites and article metadata (title, abstract, and pages), we manually classified articles into (1) original research articles (ORAs, including nonsystematic meta-analyses); (2) reviews as nonoriginal research articles (NORA-reviews, narrative or systematic); (3) guidelines, statements, and statistics as NORAs (NORA-GSS), and (4) uncitable items (brief reports, case reports, commentaries, and editorials). Uncitable items were excluded from analysis. Outcomes were (1) ORAs’ and NORAs’ mean 2-year citation counts, (2) proportion of 2-year citations by each category among total 2-year citations to all citable items, and (3) publication count and proportions of each category among total citable items.</p>
<h4>Results</h4>
<p> Among 22,740 articles, 9117 (39.9%) were ORAs, 2286 (10.0%) were NORA-reviews, 909 (4.0%) were NORA-GSS, and 10,566 (46.2%) were uncitable items. From 2008 to 2018, Journal Impact Factors (JIFs) in each journal increased, but their 2-year mean (SD) citations of ORA plateaued (JACC, 24.9 [24.3] to 26.4 [28.9]; CIRC, 24.00 [26.3] to 23.4 [23.6]; and EHJ, 15.8 [16.9] to 25.2 [20.8]) (<b>Figure 25-0977</b>, A). The annual publication of ORAs among total citable items declined across all 3 journals, although to varying extents (counts with proportions: JACC, 333 [80.5] to 245 [65.8]; CIRC, 415 [79.2] to 267 [78.1]; and EHJ, 251 [87.9] to 165 [69.6]), reflecting the decreased proportion of ORA 2-year citations among total citations (proportion of 2-year citations: JACC, 71.9% [9249/12,858] to 55.1% [7561/13,728]; CIRC, 68.1% [10,657/15,646] to 55.8% [6708/12,027]; and EHJ, 70.1% [4023/5739] to 36.8% [5073/13,771]) (<b>Figure 25-0977</b>, B). Although the annual publications of NORA-GSS were small in portion (counts with proportions: JACC, 33 [7.5%] to 22 [5.7%]; CIRC, 40 [7.0%] to 47 [13.0%]; and EHJ, 9 [3.1%] to 29 [11.9%]), the proportion of their 2-year citations among the total substantially increased (proportion of 2-year citations: JACC, 10.6% [1362/12,858] to 20.0% [2749/13,728]; CIRC, 19.3% [3019/15,646] to 37.7% [4535/12,027]; and EHJ, 16.6% [955/5739] to 52.4% [7211/13,771]) (<b>Figure 25-0977</b>, B).</p>

![](https://assets.underline.io/markdown_image/1/image/047d3162fe848a6fbfe0c11af9284812.png)

<h4>Conclusions </h4>
<p>Rising JIFs in major cardiology journals were associated primarily with an expanding share of nonoriginal content. Overreliance on JIF in academic evaluation may unconsciously steer journals toward prioritizing highly cited nonoriginal content, systematically undermining the value of primary research.</p>
<h4>References</h4>
<p>1. Genuis SJ. The proliferation of clinical practice guidelines: professional development or medicine-by-numbers? <i>J Am Board Fam Pract</i>. 2005;18(5):419-425. doi:10.3122/jabfm.18.5.419 </p>
<p>2. Shuaib W, Khan MS, Shahid H, Valdes EA, Alweis R. Bibliometric analysis of the top 100 cited cardiovascular articles. <i>Am J Cardiol</i>. 2015;115(7):972-981. doi:10.1016/j.amjcard.2015.01.029</p>
<sup>1</sup>Institute for Innovation in Digital Healthcare, Yonsei University, Seoul, Korea; <sup>2</sup>Department of Biomedical Systems Informatics, Yonsei University College of Medicine, Seoul, Korea, chandryou@yuhs.ac; <sup>3</sup>Section of Cardiovascular Medicine, Department of Internal Medicine, Yale University School of Medicine, New Haven, CT, US; <sup>4</sup>Center for Outcomes Research and Evaluation, Yale New Haven Hospital, New Haven, CT, US; <sup>5</sup>Department of Epidemiology, Rollins School of Public Health, Emory University, Atlanta, GA, US; <sup>6</sup>Division of Cardiovascular Medicine, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, US; <sup>7</sup>Thrombosis Research Group, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, US; <sup>8</sup>Baylor Heart and Vascular Institute, Baylor University Medical Center, Dallas, TX, US; <sup>9</sup>Imperial College London, London, UK.</p>
<h4>Conflict of Interest Disclosures </h4>
<p>Several authors hold editorial positions at the Journal of the American College of Cardiology (JACC): Yuan Lu and Behnood Bikdeli as executive associate editors, Joshua D. Wallach and Seng Chan You as associate editors, Milton Packer as senior consulting editor, and Harlan M. Krumholz as editor in chief. Joshua D. Wallach reported receiving grants from Johnson &amp; Johnson (through the Yale Open Data Access Project), Arnold Ventures, the National Institutes of Health, and the US Food and Drug Administration, as well as consulting fees from Hagen Berman Sobol Shapiro LLP and Dugan Law Firm APLC outside the submitted work. Seng Chan You reported being a chief technology officer of PHI Digital Healthcare and receiving grants from Daiichi Sankyo outside the submitted work.</p>
<h4>Funding/Support</h4>
<p> This study was supported by a Severance Hospital Research Fund for Clinical Excellence (SHRC) (C-2024-0011).</p>


Next from The 10th Peer Review Congress

Video and Social Media Performance at a Surgical Journal with Video Journal Clubs

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES