United States

**Importance** Bias, spin, and lack of transparency are
responsible for an important avoidable waste in research.

**Observations** Evidence on research transparency, trial
result availability, completeness, and accuracy of reporting
will be highlighted. The role of the peer review process, the
research environment, and the research ecosystem will be
considered.

**Conclusions** As stated by Doug Altman and David Moher,
“The scientific community and the public at large deserve an
accurate and complete record of research; we need to make
changes to ensure that we will get one.”1

**Reference**
1. Altman DG, Moher D. Declaration of transparency for each
research article. BMJ. 2013;347:f4796. doi:10.1136/bmj.f4796

**Conflict of Interest Disclosures** Isabelle Boutron is an advisory
board member of the International Congress on Peer Review and
Scientific Publication but was not involved in the review or decision
for this abstract.

![](https://assets.underline.io/uploads/markdown_image/1/image/26287e00e2f2a5caf3c67f2a2efb9c0e.png)

Peer Review Congress 2022

Inaugural Drummond Rennie Lecture: Bias, Spin, and Problems With Transparency of Research, Isabelle Boutron

editorial and peer review process

dissemination of information

bias

[![](https://assets.underline.io/uploads/markdown_image/1/image/81a3c0317d24f663b49b996875024d45.png)](https://aclanthology.org/volumes/2022.coling-1/)



### THE CONFERENCE WE KNOW AND WE WANT.
**COLING**, the International Conference on Computational Linguistics, is one of the premier conferences for the natural language processing and computational linguistics.

First established in 1965, the biennial COLING conference is held in diverse parts of the globe and attracts participants from both top-ranked research centers and emerging countries. Today, the most important developments in our field are taking place not only in universities and academic research institutes but also in industrial research departments including tech-startups. COLING provides opportunities for all these communities to showcase their exciting discovery.

In fall of 2022, COLING will be held in Gyeongju in a hybrid format. All participants can either present at the venue site or join virtually. As more people get vaccinated, we are happy to provide safer environments for our colleagues. We believe that COLING 2022 will be one of the conferences, free from the pandemic. The hybrid format gives presenters and sponsors a valuable opportunity to promote their companies in both an online and in-person venue. For the first time in a long time, customers can interact with their sponsor's products first-hand. The online venue, too, gives sponsors the chance to network with those unable to attend the in-person session.

The hybrid format ultimately lends itself to greater exposure for our sponsors: COLING2022 will let you reach more potential partners and customers than ever before!

To gain access to this event page you are required to register. Find more information and pay the registration fee on event the organizer’s website **[https://coling2022.org/reg](https://coling2022.org/reg)**

COLING 2022

COLING, the International Conference on Computational Linguistics, is one of the premier conferences for the natural language processing and computational linguistics.

lecture

Our aim is to encourage research into the quality and credibility of peer review and scientific publication, to establish the evidence base on which scientists can improve the conduct, reporting, and dissemination of scientific research.
 
 **Welcome!**
 
 The JAMA Network, The BMJ, and METRICS welcome you to the Ninth International Congress on Peer Review and Scientific Publication. We have continued our efforts to broaden the scope of the Congress to all aspects of peer review and publication—from funding to postpublication—and to all sciences.
 
 We will have 3 days for presentations of new research into peer review and all aspects of scientific publication, bias, quality of reporting, and information access and dissemination. There are 50 plenary session research presentations and 5 plenary invited talks. Each plenary session research presentation will be followed by equal time for discussion and questions from the audience. In addition, there are 125 posters. All posters, including virtual posters, are available to view throughout the meeting. In-person posters will also be presented at designated times on September 9 and 10. If you are attending virtually, please visit the individual poster pages and ask your questions in the chat box. See the link to Posters on the left.
 
 We hope you will take an active part in the program, as we depend on your participation in the discussion sessions to make the Congress a success.

 [Peer Review Congress Organizers, Directors, and Advisory Board](https://peerreviewcongress.org/organizers-and-advisory-board/)
 
 [Health and Safety Policies](https://web.cvent.com/event/1b505794-6a0b-428c-a44c-6c46c4c50e99/websitePage:152962bb-4567-4561-9d81-21ebfa97f487)

![](https://assets.underline.io/uploads/markdown_image/1/image/7c9f0143dee383780cc4670269ade132.png)

A PDF of the schedule is available [here](https://drive.google.com/file/d/1-Jd191W-nxbWcN1prdX5o9nlSfeo9_-H/view?usp=sharing) and a flipbook version of the complete program and abstracts is included below. See also the link to the interactive schedule to the left.

<iframe width="700px" height="425px" src="https://online.fliphtml5.com/ebtyf/nlfn/#p=1" frameborder="0" allowfullscreen allowtransparency></iframe>

Registration is required to attend this event, please use the URL linked below to secure your ticket. Registration will close on Friday, September 2 at 5:00 PM CT. 

[**Register Here**]( https://web.cvent.com/event/1b505794-6a0b-428c-a44c-6c46c4c50e99/summary)

Our aim is to encourage research into the quality and credibility of peer review and scientific publication, to establish the evidence base on which scientists can improve the conduct, reporting, and dissemination of scientific research.



**Objective** The International Committee of Medical Journal Editors (ICMJE) provided a set of minimum criteria for authorship.1 These recommendations have been adapted for all sciences in an article by McNutt and colleagues.2 The main difference between these 2 sets of recommendations is that the science-wide recommendations do not require authors to draft or revise the work.1,2 This study aimed to identify the proportion of authors who, based on their self-compiled Contributor Role Taxonomy (CRediT) statements, did not meet the minimum criteria for authorship (ie, were honorary authors) according to these 2 sets of recommendations. Furthermore, the study aimed to identify the proportion of authors who supplied only resources and/or funding to a study. Such authors were identified as “supply authors,” and this practice is considered to be a subtype of honorary authorship. 
 **Design** Cross-sectional study of CRediT statements published in scholarly articles. The Public Library of Science (PLOS) provided CRediT statements and associated data for authors of articles published in PLOS journals between July 2017 and October 2021. Two investigators independently evaluated the authorship recommendations1,2 and developed logical operations of CRediT items for each recommendation. A third investigator acted as an arbiter in case of disagreement. The criteria related to approval and accountability for both recommendations could not be verified because of current CRediT items. For the second objective of the study, authors who contributed only to roles funding acquisition and/or resources were identified. 
 **Results** A total of 629,046 CRediT statements (1 per author) originating from 82,683 journal articles were included. Of the CRediT statements, 34.8% (n = 218,563; 95% CI, 34.7%34.9%) indicated that the contributions provided by the author were not sufficient to qualify for authorship according to the ICMJE recommendations. Based on science-wide recommendations, 3.6% (n = 22,575; 95% CI, 3.5%-3.6%) of the authors did not qualify for authorship. Sensitivity analyses accounting for potentially ambiguous CRediT items provided similar results. The odds of fulfilling only 1 of the recommendations steadily decreased from 2017 to 2021 (Table 1), and authors of articles published in nonmedical journals had 1.11 times the odds to fulfill only 1 of the recommendations compared with authors of articles published in medical journals. Overall, 8394 authors (1.33%; 95% CI, 1.31%-1.36%) were “supply authors.” Their prevalence decreased in the years from 2017 to 2019 (difference in proportion, 0.6%; 95% CI, 0.57%-0.63%) but remained unchanged from 2019 to 2021 (0%; 95% CI, −0.02% to 0.02%). 
 **Conclusions** Based on self-compiled CRediT statements, honorary authorship is still prevalent in science, although it seems to have steadily decreased in recent years. A seemingly minor edit applied to the ICMJE recommendations resulted in substantially different authorship requirements. Efforts should be directed toward developing consensus on core tasks to qualify for authorship that are widely applicable in science. 
Additional strategies should be implemented to address 
“supply authorship.” 
 **Table 1. Prevalence of Honorary Authorship Based on CRediT Statements Depending on 2 Authorship Recommendations and 
the Association of Year and Journal Area With Disagreements 
Between Recommendations** 
 ![](https://assets.underline.io/uploads/markdown_image/1/image/e020149a07644589bc36ea5518d9e003.png)
![](https://assets.underline.io/uploads/markdown_image/1/image/b2f5d13c1f1a80d2be85c7493c4f9bd0.png)
**References**
1.	International Committee of Medical Journal Editors (ICMJE). Recommendations for the conduct, reporting, editing, and publication of scholarly work in medical journals. Updated December 2021. Accessed February 27, 2022. http://www.icmje.org/icmje-recommendations.pdf 
2.	McNutt MK, Bradford M, Drazen JM, et al. Transparency in authors’ contributions and responsibilities to promote integrity in scientific publication. Proc Natl Acad Sci U S A. 2018;115(11):2557-2560. doi:10.1073/pnas.1715374115
1College of Veterinary Medicine, Cornell University, Ithaca, 
NY, USA, nicoladiggi@gmail.com; 2Journal of Small Animal 
Practice, British Small Animal Veterinary Association, 
Gloucestershire, UK; 3Department of Oral and Maxillofacial 
Surgery, Amsterdam University Medical Center, University of Amsterdam, Amsterdam, the Netherlands; 4Private practice of orthodontics, Milan, Italy; 5École de bibliothéconomie et des sciences de l’information, Université de Montréal, Montréal, QC, Canada; 6School of Electrical and Computer 
Engineering, Oklahoma State University, Stillwater, OK, USA 
 **Conflict of Interest Disclosures** Nicola Di Girolamo is an editor in chief of 2 peer-reviewed journals, 1 published by Elsevier and 1 by Wiley. No other disclosures were reported. 
 **Additional Information** The protocol of this research study was registered in the Open Science Framework (https://osf.io/ezpxs/).

![](https://assets.underline.io/uploads/markdown_image/1/image/3b7cc05bc88eb604a84a4c5bbd38244a.png)

Prevalence of Honorary Authorship According to Different Authorship Recommendations and Contributor Role Taxonomy (CRediT) Statements | VIDEO

**Objective** Image reuse is a common problem with the integrity of image data reported in scientific articles. Considerable resources are expended by the community in trying to detect, communicate, and respond to potential image reuse. These efforts are manually intensive and often take place after publication, when resolution is difficult and time-consuming. The study objective was to evaluate the use of an artificial intelligence–based tool to identify potential image reuse prior to acceptance and address problems then. 
 **Design** In this cross-sectional study designed to systematically identify and act on instances of image reuse in manuscripts submitted to 9 journals published by the American Association for Cancer Research, a machineassisted process was implemented for detecting potential image duplication between images in a manuscript. Prior to issuing a provisional acceptance decision, an internal editor selected original research manuscripts containing images susceptible to duplication and detection and uploaded them to a commercial tool (Proofig). The tool’s assessment was followed by editor evaluation and refinement of the results and communication of potentially problematic image duplications to the authors using a standard report generated by the tool. Author responses and the outcomes of these queries were recorded in a tracking sheet using a standardized classification system. 
 **Table 2. Author Responses to the Report of Image Duplication and Outcome of the Author Query**
![](https://assets.underline.io/uploads/markdown_image/1/image/6f66f47263c443faa948f69e34a80286.png)
![](https://assets.underline.io/uploads/markdown_image/1/image/ec436191191089fad9b874222d12d5ef.png)
**Results** From January 2021 through May 2022, a total of 207 image duplication queries were sent to authors for their response. This represented 9.3% of 2220 original research manuscripts that reached this editorial stage and 15% of the 1367 manuscripts selected for analysis. The distribution of duplicate quantity per manuscript was 104 (50%) with 1 duplicate, 46 (22%) with 2 duplicates, 19 (9%) with 3 duplicates, 28 (14%) with 4 to 10 duplicates, and 11 (5%) with 11 or more duplicates. Responses from authors indicated that 63% (n = 131) of duplications were unintentional, for example, from general image mishandling, and 28% (n = 58) were intentional duplications for presentation purposes (see **Table 2** for details). In 2% (n = 5) of cases the author said they were investigating or provided no explanation. These manuscripts were withdrawn or rejected. In all other cases, changes were made to address the duplication(s). Only in 12 cases did the potential duplication turn out not to be a duplication. To compare the time required to identify seemingly real duplications for communication to authors, 5 editors first analyzed 27 manuscripts purely manually and then using the tool followed by manual evaluation and refinement. The mean time per manuscript for tool-assisted analysis was 4.4 minutes (11 duplications identified in 8 manuscripts) vs 8 minutes (5 duplications identified in 3 manuscripts) for manual analysis. 
 **Conclusions** The use of this artificial intelligence–based tool effectively identified real duplications between and within figures in 14% (195/1367) of manuscripts intended for acceptance that contained images susceptible to duplication and detection. This allowed these problems to be addressed prior to publication with minimal manual effort. 
 **Conflict of Interest Disclosures** Daniel S. Evanko is a member of the Peer Review Congress Advisory Board but was not involved in the review or decision for this abstract.

![](https://assets.underline.io/uploads/markdown_image/1/image/638857f5929475017e03295d84ac618c.png)

Use of an Artificial Intelligence-Based Tool for Detecting Image Duplication Prior to Manuscript Acceptance

**Objective** Paper mills represent an offer for on-demand writing of fraudulent academic manuscripts for a fee. There is evidence of increasing infiltration of the academic literature with fraudulent papers originating from paper mills. The majority of known paper mills originate from China and less is known about their operations in other countries, namely Iran and Russia, where paper mills also operate.1 This study attempted to shed light on the activity of paper mills using the experience of a Russian mill to detect papers originating from the Russian-based paper mill International Publisher LLC and to identify a set of factors associated with fraudulent papers. 
 **Design** Offers published during 2019-2021 were collected from the 123mi.ru website. The details of the offer as the title/ topic of the paper, number of coauthors, country/region of a journal, date of publication, and in some cases country of the author and abstract were analyzed to identify auctioned papers. Many fraudulent papers were detected because they were published with identical or closely worded titles. The correctness of detection was confirmed by matches of other offer details. 
 **Results** A total of 1009 offers were published during the study period, and the study identified at least 436 papers (43%) published in 154 journals potentially linked to the 
paper mill as of mid-March 2022; 22 of the papers appeared in 3 hijacked journals. More than 800 scholars from at least 39 countries purchased coauthorships from this paper mill.
Fraudulent papers were published in predatory journals, journals of reputable publishers, and hijacked journals that represent cybercriminal publishers who clone titles and other metadata of legitimate journals (**Figure 1**). The list of identified papers can be accessed via a shared spreadsheet.2 This study found collaboration anomalies in questionable papers in which authors did not have common research interests, specialized in different disciplines, or were affiliated with different universities that did not focus research on the paper’s subject or might not specialize in the topic of the paper. There was also evidence of data fabrication in these papers. The study detected dishonest collaboration with journals (purchase of an entire issue) or editors who were listed as coauthors of fraudulent papers. This paper mill also applied a “one paper–one journal” principle, ie, submission of a problematic paper to an individual legitimate journal only once. 
 **Figure 1. Number of Articles per Individual Journal**
![](https://assets.underline.io/uploads/markdown_image/1/image/36ccf052cd29559d1116f599c5519033.png)
![](https://assets.underline.io/uploads/markdown_image/1/image/b3aaf14cc6092b68003954157b9a862e.png)
 **Conclusions** The production of this paper mill was difficult to detect due to individually tailored papers being submitted to 154 journals. Journals themselves had no opportunity to notice irregularities from 1 paper. Detection of irregularities can require regular upgrades of the system of detection of fraudulent papers by publishers and COPE (Committee on Publication Ethics), which is based on tracing similarity patterns among manuscripts. Because the study analyzed a single paper mill, it is likely that the number of papers with forged authorship is much higher. 
 **References**
1.	Else H, Van Noorden R. The fight against fake-paper factories that churn out sham science. Nature. 2021;591:516519. doi:10.1038/d41586-021-00733-5
2.	Abalkina A. Paper mill “International Publisher.” https://docs.google.com/spreadsheets/d/1vzjtRPX7kd2KczdtKONEpRZb2F-4lj5Sd9jL6DfbiBk/edit?usp=sharing 
 **Conflict of Interest Disclosures** None reported. 
 ![](https://assets.underline.io/uploads/markdown_image/1/image/2b4a2d896142a5537f9159199f35eb9b.png)

Publication and Collaboration Anomalies in Academic Papers Originating From a Russian-Based Paper Mill

**Objective** Retracted clinical trials may be
influential in systematic reviews and
guidelines. Fanelli et al 1 examined the impact of
retractions on 50 meta-analyses and found
little influence but suggested that the impact
was likely variable and context specific. Using
retracted clinical trial reports from one
research group, 2 responses of editors and
authors to notifications that systematic reviews
and guidelines they had published had cited
the retracted trials were evaluated. 
 **Design** Between November 2019 and January
2020, for 27 retracted trials (published 1997 to
2012, retracted 2016 to 2019) in osteoporosis
and neurology, searches were conducted in
Web of Science, Scopus, and researchers’ files
for systematic reviews and clinical guidelines
that cited the trials as evidence. Citing
publications that acknowledged that they had
cited retracted work were excluded. For each
citing publication, 2 researchers independently
coded the likely impact of removing the
retracted trial reports, including findings likely
to change (yes, no, or uncertain) and size of change
(substantial, moderate, or minor). In a factorial design with 4
groups, authors of citing publications were randomized to
receive up to 3 emails (if no reply) to the contact author and
journal editor vs contact author only and, for citing
publications with 2 or more authors, to an email to the
contact author only vs up to 3 authors (selected from among
contact, first, second, and final authors). Emails giving details
of the retracted trials were sent monthly September to
October 2020. Follow-up was 1 year for outcomes, assessed
by replies to emails and notices in the public domain,
including any reply (yes or no), time for a reply from first
author, time for any reply, and action taken. Comparisons
were undertaken using χ² tests. Email replies were analyzed
for content by 3 researchers. 
 **Results** A total of 88 citing publications (published 2003 to 2020) were identified; 2 were corrected before emails were sent. Authors/editors were emailed about 86 citing publications. A total of 45 of 88 citing publications (51%) had findings coded likely to change if the retracted trials were removed, and 39 of these (87%) were likely substantial impacts. Replies were received for 44 of 86 citing publications (51%). Emailing 3 authors was more likely to elicit a reply than emailing the contact author alone (26 of 42 [62%] vs 16 of 40 [40%]; P = .03), but including the editor did not increase replies (23 of 44 [52%] with editor vs 21 of 44 [48%] without editor; P = .66). Including more authors and/or the editor, whether findings were coded likely to change and the size of the likely change, had no effect on published corrective action. One year after emails were sent, only 9 publications had been corrected. 
 **Conclusions** Retracted trials impact systematic reviews and guidelines. Emailing more authors to notify them of the retractions yielded more replies but did not increase corrections. Email alerts to authors and editors are inadequate to correct the impact of retracted publications in citing systematic reviews and guidelines. Publications with retracted citations should be marked until authors resolve concerns. 
 **References**
1.	Fanelli D, Wong J, Moher D. What difference might retractions make? an estimate of the potential epistemic cost of retractions on meta-analyses. Account Res. Published online July 14, 2021. doi:10.1080/08989621.2021.1947810 
 2.	Bolland MJ, Avenell A, Gamble GD, Grey A. Systematic review and statistical analysis of the integrity of 33 randomized controlled trials. Neurology. Published online November 9, 2016. doi:10.1212/WNL.0000000000003387 
 **Conflict of Interest Disclosures** None reported. 
 **Funding/Support** Mark J. Bolland is the recipient of an HRC Clinical Practitioner Fellowship. The Health Services Research Unit is funded by the Chief Scientist Office of the Scottish Government Health and Social Care Directorates. 
 ![](https://assets.underline.io/uploads/markdown_image/1/image/c24532dbbbf609fe14bad507ca0cd465.png)

Effect of Alerting Authors of Systematic Reviews and Guidelines That Research They Cited Had Been Retracted: A Randomized Controlled Trial

**Objectives** To investigate women’s representation among peer reviewers of medical journals overall and according to the gender of the editor in chief and women’s representation in the editorial board. 
 **Design** This cross-sectional study included journals of the BMJ Publishing Group that reported the names of their peer reviewers in 2020. For each journal, the gender of the editor in chief, deputy editors, and associate editors was determined based on photographs and pronouns available on the journal website or professional affiliations. The package genderizeR in R was used to predict the gender of the peer reviewers based on given names, which were extracted from full names and assigned as woman or man. 
 **Results** Overall, this study included 47 of the 74 journals in the BMJ Publishing Group because data were not publicly available for the remaining journals (Table 3). Women accounted for 30.2% of the 42,539 peer reviewers, with marked variation ranging from 8% in the Journal of ISAKOS to 50% in Medical Humanities. Women represented 33.4% of the 555 editors, including 19.2% of the 52 editors in chief. There were 5 journals with more than 1 editor in chief, all of which had 2 men as editors in chief. There were 5 journals with no woman among the editors and 12 journals in which women’s representation as editors was 50% or greater. Among those 12 journals, 7 had a woman as editor in chief. There was a moderate positive correlation between the percentage of women as editors and as reviewers (Spearman correlation coefficient, 0.590; P < .001). The percentage of women as editors excluding editors in chief was higher when the editor in chief was a woman than a man (53.3% vs 29.2%; P < .001). The percentage of women as peer reviewers was also higher in journals with a woman as editor in chief compared with a man (32.0% vs 26.4%; P < .001). There was no significant correlation between women’s representation and the journal CiteScore (Spearman correlation coefficient, −0.288; P = .07) or impact factor (Spearman correlation coefficient, −0.343; P = .09). This study has some limitations, such as using binary prediction of gender based on given names, relying on data for accepted rather than invited reviewers, and not accounting for the number of reviews performed by each individual. 
 ![](https://assets.underline.io/uploads/markdown_image/1/image/2dfaac8ba7785221c799d813135c0ddd.png)![](https://assets.underline.io/uploads/markdown_image/1/image/2e2d56fb188047c3e0ee0b87f9d73c3c.png)
![](https://assets.underline.io/uploads/markdown_image/1/image/8125764c399777c8413734c0ee19b4f0.png)**Conclusions** Women account for less than 1 in 3 peer reviewers of medical journals. Women’s representation as peer reviewers is higher in journals with a higher percentage of women as editors or with a woman as editor in chief, suggesting that increasing women’s representation as editors may be one of many necessary steps toward gender equity. The gender gap among peer reviewers of medical journals mimics that previously found among authors and editors,1-3 thus emphasizing the need to address the persisting gender gap at all levels of the publishing system. 
 **References**
1.	Lundine J, Bourgeault IL, Clark J, et al. The gendered system of academic publishing. Lancet. 
2018;391(10132):1754-1756. doi:10.1016/S01406736(18)30950-4
2.	Hart KL, Perlis RH. Trends in proportion of women as authors of medical journal articles, 2008-2018. JAMA Intern Med. 2019;179(9):1285-1287. doi:10.1001/ jamainternmed.2019.0907
3.	Pinho-Gomes AC, Vassallo A, Thompson K, et al. Representation of women among editors in chief of leading medical journals. JAMA Netw Open. 2021;4(9):e2123026. doi:10.1001/jamanetworkopen.2021.23026 
 **Conflict of Interest Disclosures** Ana-Catarina PinhoGomes reported receiving funding through an Academic Clinical 
Fellowship by the National Institute for Health Research. Mark Woodward is a consultant for Amgen, Kyowa Kirin, and Freeline and reported receiving grants from the National Health and Medical Research Council of Australia. Sanne A. E. Peters reported receiving support through a UK Medical Research Council Skills Development Fellowship. No other disclosures were reported.

![](https://assets.underline.io/uploads/markdown_image/1/image/1b9a73ff68953942366e96aec9a60f58.png)

Women's Representation Among Peer Reviewers of Medical Journals

**Objective** To examine the association of having editorial staff members affiliated with low- and middle-income countries with publications from low- and middle-income countries in leading biomedical journals. It was hypothesized that greater representation from low- and middle-income countries among editorial staff would be associated with more published articles from low- and middle-income countries. 
 **Design** A cross-sectional study was conducted of biomedical journals in 2020 in fields whose content covers the largest disease burden globally. To obtain editorial staff country affiliations, webpages of the 5 leading journals in each of the following fields were reviewed: general medicine, pediatrics, surgery, obstetrics and gynecology, cancer, cardiovascular diseases, infectious diseases, psychiatry, and nutrition. Original research articles in each journal were reviewed through MEDLINE. The study country of each original research article (ie, where the study was conducted) was determined by searching the article title, abstract, keyword, and medical subject heading using EndNote, and by 2 authors reviewing the full text of each article to assign a study country. Editorial staff country affiliations and study country location(s) were classified according to the World Bank income brackets and regions. Descriptive statistics were used to describe the proportion of editorial staff affiliated with each income bracket and region. Spearman ρ was used to assess the relation between the proportion of editorial staff affiliated with low- and middle-income countries and published articles reporting work conducted in these countries. 
 **Results** There were 3819 editorial staff in the 45 included journals: 3637 (95.2%) were affiliated with high-income countries, 140 (3.7%) with upper−middle income countries, 37 (1.0%) with lower−middle income countries, and 5 (0.1%) with low-income countries. Every editor in chief (n = 48; 100.0%) was affiliated with a high-income country. Of the 459 associate editors, 445 (96.9%) were affiliated with highincome countries, 10 (2.2%) with upper−middle income countries, 4 (0.9%) with lower−middle income countries, and 0 with low-income countries. Editorial staff were most commonly affiliated with North American (2120 [55.5%]) and European and Central Asian countries (1256 [32.9%]). Of the 10,096 original research articles included, 7857 (77.8%) reported research conducted in high-income countries, 1562 (15.5%) in upper−middle income countries, 507 (5.0%) in lower−middle income countries, and 170 (1.7%) in lowincome countries. Greater editorial staff representation correlated moderately with more published articles reporting research conducted in low- and middle-income countries among all articles (Spearman ρ, 0.51; P < .001) and among articles reporting multicountry studies (Spearman ρ, 0.42; P = .005) (Figure 2). 
 ![](https://assets.underline.io/uploads/markdown_image/1/image/14edf99e92065c3e95ce0751d301a2f3.png) 
 **Conclusions** The inclusion of editorial staff affiliated with low- and middle-income countries may be an approach to promoting the publication of research conducted in low- and middle-income countries.
1Precision Vaccines Program, Boston Children’s Hospital, Boston, MA, USA, chris.rees@emory.edu; 2Division of Pediatric Emergency Medicine, Emory University School of Medicine, Atlanta, GA, USA; 3Children’s Healthcare of Atlanta, Atlanta, GA, USA; 4School of Public Health and Social Sciences, Muhimbili University of Health and Allied Sciences, Dar es Salaam, Tanzania; 5Division of Infectious Diseases, Department of Paediatrics, The Hospital for Sick Children, Toronto, ON, Canada; 6Department of Physiology and Nutrition, St. John’s Medical College, Bengaluru, India; 7Center for Nutrition, Division of Gastroenterology, Hepatology and Nutrition, Boston Children’s Hospital, Boston, MA, USA; 8Department of Nutrition, Harvard T. H. Chan School of Public Health, Boston, MA, USA 
 **Conflict of Interest Disclosures** None reported. 
 **Funding/Support** Christopher P. Duggan was supported in part by the National Institutes of Health (K24DK104676 and 2P30 DK040561). 
 **Role of the Funder/Sponsor** The funder had no role in the study design or in the collection, analysis, or interpretation of the data. The funder did not write the report and had no role in the decision to submit the abstract for presentation. 
 **Acknowledgments** Gandolina Melhem and Chris A. Rees are co−first authors. We thank Chloe Rotman, at Boston Children’s Hospital, for her assistance in developing the PubMed query as well as acquiring the full-text articles reviewed in this study.

![](https://assets.underline.io/uploads/markdown_image/1/image/bf5305ad43d9e2c5cd859aec4fec7415.png)

Association Between International Editorial Staff and International Publications in Leading Biomedical Journals

**Objective** This study measured diversity in scholarly
journals’ editors and editorial boards and evaluated
associations between editor and board diversity and journal
policies toward open access and open science. This study
extends the literature on international diversity in editorial
leadership positions 1,2 to a large multidisciplinary corpus of
journals. 
 **Design** Multiple data sources were integrated to assemble a
novel database describing the composition of editors and
editorial boards, building on Pacher et al, 3 which used web
mining to harvest editor-level information. These data were
cleaned, coded, and supplemented with journal-level
classifications of journal discipline and journal policies
derived from the 2018 Australian national research
evaluation, Directory of Open Access Journals (DOAJ), and
Center for Open Science TOPS initiative (TOPS). This
dataset, collected in February 2021, comprised 17 publishers,
6090 journals, and 478,563 named editor roles. Diversity
measures were computed for 14,228 journal editorial boards.
Editor gender was imputed from editor names by comparison
with Social Security Administration, Census, and social media
corpora. Editor country was determined by applying
geoentity extraction and gazetteer lookup of organizational
affiliation. Because chief editorship is often limited to a single
post, diversity was measured across the pool of chief editors
within that category. Boards were stratified by type (review,
editor, chief) based on each journal’s labeling of the board
member’s specific role. Journal policies were classified by
access policy (ie, open or closed) based on whether the
journal met DOAJ criteria (this corresponds approximately to
gold and diamond open access categories) and by the
presence of an open-science policy (applying the TOPS
criteria). Means were calculated with bootstrapped 95% CIs. 
 **Results** The grand mean across board types was 28.7% for
women representation and 69.7% for international diversity,
as measured with the Herfindahl-Hirschman Index. There
was substantial variation across disciplines: women
composition of editorial boards ranged from 26% in
engineering journals to 53% in education journals. Figure 3
shows mean diversity by board type. Women composition of
open access journal editorial boards was lower than that of
closed-access journal editorial boards (−7%); in journals with
open science policy vs those without (−4%), and in journals
that were both open access and had an open science policy vs
those without either (−6%). In contrast, open-access journal
editorial boards were associated with higher international
diversity (&gt;11%) compared with closed-access journal boards.
Furthermore, editorial boards were, on average,
disproportionately male and USA-centric or UK-centric,
although the specific level of diversity varied substantially by
discipline. 
 ![](https://assets.underline.io/uploads/markdown_image/1/image/8d52c5ce66b5a531bb11b9d994bd21dd.png) 
 **Conclusions** In this cross-sectional study of gender balance
and international diversity in editorial boards, open-access
journals were associated with lower women representation
and more international diversity than their closed-access
counterparts .
 **References**
1, Bhaumik S, Jagnoor J. Diversity in the editorial boards of
global health journals. BMJ Global Health. 2019;4:e001909.
doi:10.1136/bmjgh-2019-001909 
 2. Wu D, Lu X, Li J, Li J. Does the institutional
diversity of editorial boards increase journal quality? the case
economics field. Scientometrics. 2020;124:1579-1597.
doi:10.1007/ s11192-020-03505-6 
 3. Pacher A, Heck T, Schoch K. Open editors: a dataset
of scholarly journals’ editorial board positions. SocArXiv.
Preprint published online March 10, 2021. Updated October
2, 2021. doi:10.31235/osf.io/jvzq7 1 MIT Libraries, Massachusetts Institute of Technology,
Cambridge, MA, USA, escience@mit.edu; 2 Department of
Sociology, University of Maryland, College Park, MD, USA 
 **Conflict of Interest Disclosures** None reported. 
 **Funding/Support** Supported in part by the Institute of Museum
and Library Services, grant LG-250130-OLS-21.

![](https://assets.underline.io/uploads/markdown_image/1/image/76dcc80c878d332b5162c37479d9a3df.png)

Patterns of Gender and International Diversity of Editors and Editorial Boards Among Journals With Open Access Licenses and Open Science Policies

**Objective** Geographical disparities have been observed in
the acquisition of research grants and in the submission and
publication of research articles. Peer reviewers are selected
primarily based on their expertise through publication
records. Response to peer invitation has been shown to be
related to relevance of topic to own work, reviewer
availability, journal attributes (impact factor and type of
reviewer blinding) and compensation. 1,2 However, diversity in
terms of geographical distribution in response to peer
invitations remains almost unexplored. 3 
 **Design** Retrospective cohort study of all research
manuscripts submitted to 21 BMJ Publishing Group
biomedical journals between January 1, 2018, and May 31,
2021, and subsequently sent for review. Data were collected
on geographical affiliation, income level of the country of
affiliation (according to World Bank Data 2020), journal
impact factor, and peer-review process (open vs
anonymized). The primary outcome was response (agreed vs
not agreed) to review invitation. A multivariable mixed-
effects logistic regression model with random factors on the
intercept at journal and manuscript levels was performed. 
 **Results** A total of 257,025 reviewers were invited to review
and 90,467 (35.2%) agreed. The distribution of geographical
affiliations of the invited reviewers were as follows: 10.0% in
Africa, 8.8% in Asia, 47.6% in Europe, 26.0% in North
America, 6.3% in Oceania, and 1.2% in South America.
Among invited reviewers, 217,682 (84.7%) were affiliated
with a high-income country. **Figure 4** summarizes results
from multivariable analysis. Agreement was higher among
reviewers from Asia (2.13 [95% CI, 2.05-2.21]), Oceania (1.22
[95% CI, 1.17-1.27]), or South America (2.24 [95% CI,
2.06-2.45]) and lower among reviewers from Africa (0.43
[95% CI, 0.42-0.45] compared with Europe (P &lt; .001).
Agreement was significantly lower when the last author was
from Asia (0.85 [95% CI, 0.83-0.87]) or Oceania (0.91 [95%
CI, 0.87-0.95]) compared with Europe (P &lt; .001). Reviewers
agreed significantly more often when the associated editor
had a North American institutional affiliation compared with
a European affiliation (1.07 [95% CI, 1.02-1.12]). Compared
with high-income countries, agreement was higher among
reviewers from lower middle–income countries (3.26 [95%
CI, 3.06-3.48]) and low-income countries (2.99 [95% CI,
2.57-3.48]) (P &lt; .001). Agreement was also lower when the
last author was from an upper middle–income country (0.94
[95% CI, 0.91-0.98]) or low-income country (0.88 [95% CI,
0.81-0.96]) compared with a high-income country (P &lt; .001).
Agreement was associated with impact factor (higher for
impact factors between 5 and 10, or &gt;10, compared with &lt;5:
1.73 [95% CI, 1.29-2.32] and lower when peer-review process
was open compared with anonymized: 0.43 [95% CI, 0.29-
0.64]). 
 ![](https://assets.underline.io/uploads/markdown_image/1/image/791669f6cb25e359afbf7ed2548f41d5.png)
**Conclusions** The geographical affiliation of reviewers was
an independent factor associated with agreement to review.
To avoid bias and increase diversity, journal editors need to
invite more reviewers from upper middle–income or
lowincome countries. 
 **References**
1. Ellwanger JH, Chies JAB. We need to talk about
peerreview: experienced reviewers are not endangered
species, but they need motivation. J Clin Epidemiol.
2020;125:201-205. doi:10.1016/j.jclinepi.2020.02.001
2. Tite L, Schroter S. Why do peer reviewers decline to
review? a survey. J Epidemiol Community Health.
2007;61(1):9-12. doi:10.1136/jech.2006.049817
3. Masukume G, Grech V. The Lancet peer reviewers:
global pattern and distribution. Lancet.
2018;391(10140):2603-2604. doi:10.1016/S0140-
6736(18)31136-X
1 Department of Health and Community Medicine, University of
Geneva, Geneva, Switzerland, Khaoula.BenMessaoud@
unige.chkhaoula.benmessaoud@unige.ch; 2 Division of Clinical
Epidemiology, University Hospitals of Geneva, Geneva, Switzerland;
3 The BMJ, London, UK; 4 Article Transfer Service, BMJ Publishing
Group, London, UK 
 **Conflict of Interest Disclosures** All authors have completed the
ICMJE uniform disclosure form at https://www.icmje.org/disclosure-of-interest/ 
and declare support from the University
Hospitals of Geneva, Geneva Medical School, and the Swiss National
Science Foundation for the submitted work. Sara Schroter and Mark
Richards are full-time employees of BMJ Publishing Group but were
not involved in decision making on congress submissions. There were
no other relationships or activities that could appear to have
influenced the submitted work. 
 **Funding/Support** This study was supported by grant 192374 from
the Swiss National Science Foundation. 
 **Role of the Funder/Sponsor** The funder had no role in the design
and conduct of the study; collection, management, analysis, and
interpretation of the data; or preparation, review, or approval of the
abstract.

![](https://assets.underline.io/uploads/markdown_image/1/image/6563cf53c81909fac38fb8e6c8100ebc.png)

Factors Associated With Geographical Diversity of Reviewers Invited and Agreeing to Review for 21 Biomedical Journals

**Objective** Previous research found limited progress in
reporting race and socioeconomic status in research
published in medical journals from 2015 to 2019. 1 This study
compared race and ethnicity reporting in 3 JAMA Network
medical journals before and after implementation of Updated
Guidance on Reporting Race and Ethnicity in Medical and
Science Journals in August 2021. 2 
 **Design** All major research articles published in JAMA,
JAMA Internal Medicine, and JAMA Pediatrics in the first 3
months of 2019 and 2022 and the 3 months immediately
before guidance implementation (May-July 2021) were
included. Articles were reviewed independently by 2
reviewers for the following: study included human
participants; race and ethnicity, age, sex and gender, and
measures of socioeconomic status reported; where race and
ethnicity was reported (abstract, methods, results, tables);
number and order of racial and ethnic categories reported; if
the category “other” was included and if that was defined;
and if the article indicated how race and ethnicity were
determined. Comparisons between years were calculated with
χ² for 2-sided P values, odds ratios (ORs), and Wald test 95%
CIs using R version 4.2.0. 
 **Results** Of 258 research articles published during the study
periods, 249 (96.5%) included human participants and were
included in this analysis. In 2019, 49 of 86 articles (57.0%)
reported race and ethnicity of study participants compared
with 42 of 77 (54.5%) in 2021 and 58 of 86 (67.4%) in 2022
(**Table 4**). Compared with articles reporting race and
ethnicity, higher proportions of articles reported participants’
age and sex or gender and a lower proportion reported
socioeconomic status measures in all years. There were no
significant differences in article location of reporting race and
ethnicity or the proportion of articles that reported racial and
ethnic categories. Of articles that included “other” as a
collective racial and ethnic category, the proportion that
defined specific categories included in “other” was 26.7% in
2019, 70.4% in 2021, and 84.8% in 2022, with a significant
difference observed before guidance implementation in 2021
vs 2019 (OR, 6.53; 95% CI, 2.05-20.76; P &lt; .001). A
significant difference after reporting guidance
implementation was observed for articles listing categories in
alphabetical order (92.6% in 2022 vs 16.7% in 2021 [OR
73.75; 95% CI, 20.15-269.99; P &lt; .001]) and articles
indicating how race and ethnicity were determined (74.1% in
2022 vs 50.0% in 2021 [OR, 2.87; 95% CI, 1.23-6.66; P =
.01]). 
 ![](https://assets.underline.io/uploads/markdown_image/1/image/a3043a5da67d9fb8c3ca555aca5ece37.png) 
 **Conclusions** In this analysis, higher proportions of articles
reported how race and ethnicity were determined and listed
categories in alphabetical order in 2022 following the
implementation of reporting guidance. Some improvement
was noted before 2021 and may have been associated with
internal guidance shared in October 2020 and publication of
an early draft of the guidance in February 2021. 3 Overall, race
and ethnicity were still underreported compared with age,
sex, and gender. 
 **References**
1. Alegria M, Sud S, Steinberg BE, Gai N, Siddiqui A.
Reporting of participant race, sex, and socioeconomic
status in randomized clinical trials in general medical
journals, 2015 vs 2019. JAMA Netw Open.
2021;4(5):e2111516. doi:10.1001/
jamanetworkopen.2021.11516 
 2. Flanagin A, Frey T, Christiansen SL; AMA Manual of Style
Committee. Updated guidance on the reporting of race and
ethnicity in medical and
science journals. JAMA. 2021;326(7):621-627.
doi:10.1001/jama.2021.13304 
 3. Flanagin A, Frey T, Christiansen SL, Bauchner H. The
reporting of race and ethnicity in medical and science
journals: comments invited. JAMA. 2021;325(11):1049-
1052.
doi:10.1001/jama.2021.2104 
 1 JAMA Network, Chicago, IL, USA, annette.flanagin@jamanetwork.
org; 2 Berry Consultants LLC, Austin, TX, USA; 3 Harbor-UCLA
Medical Center, Torrance, CA, USA; 4 David Geffen School of
Medicine at UCLA, Los Angeles, CA, USA 
 **Conflict of Interest Disclosures** Stacy L. Christiansen is
chair and Annette Flanagin and Tracy Frey are members of
the AMA Manual of Style committee. Annette Flanagin is
executive director of the Peer Review Congress but was not
involved in the review or decision for this abstract.

![](https://assets.underline.io/uploads/markdown_image/1/image/b256ab4aaa1299e825ad845ac4afcf71.png)

Comparison of Reporting Race and Ethnicity in Medical Journals Before and After Implementation of Reporting Guidance, 2019-2022 | VIDEO

**Objective** In 2019, Neurology appointed 2 equity, diversity,
and inclusion (EDI) editors to review manuscripts dealing
with sex and gender, race and ethnicity, and other categories
of marginalization. 1,2 Manuscripts requiring EDI review are
identified by the EDI editor by title review or selected by the
handling editor during the review process. The EDI review is
obtained only for manuscripts that are being considered for
publication if suitably revised. Final editorial decisions are
made considering the EDI reviews and the responses from
the authors. The objectives of this retrospective analysis were
to describe the outcomes of the process and highlight the
issues identified by the EDI editors. 
 **Design** For this mixed methods study, the research articles
that were reviewed by the EDI editors between May 1, 2019,
and December 31, 2021, were identified. For this analysis, 75
of these manuscripts were selected at random. Two raters
with expertise in EDI issues read the reviews and classified
the issues raised by the EDI reviewers as related to
terminology or conceptual issues in reference to sex and
gender, race or ethnicity, or other marginalization categories.
The analysis was limited to initial reviews. The data were
summarized with descriptive statistics. 
 **Results** In the study time frame, 7841 research articles were
submitted to the journal. The EDI editors reviewed 101
manuscripts (1.3%). After evaluating the 75 reviews selected
for analysis, 2 were excluded because the EDI editor reviewed
them as a topic expert and not for EDI concerns. Among the
73 papers included in the analysis, the EDI reviewers
identified at least 1 terminology or conceptual issue in 64,
and there were at least 2 issues in 41 manuscripts. In relation
to sex and gender, the EDI editors identified terminology and
conceptual issues in 23 and 19 manuscripts, respectively; the
matching numbers in relation to race or ethnicity were 23
and 56; and for other categories, 6 manuscripts had
conceptual issues. The main themes identified during the EDI
review process are presented in **Table 5**. As of May 31, 2022,
45 of the 73 manuscripts in the analysis had been accepted, 27 had
been rejected, and 1 was undergoing major revision. The
proportion of rejected manuscripts was lower for those in
which an issue was identified by the EDI editor (22 of 64 vs 5
of 9; difference, 22%; 95% CI, −9.4% to 49.5%). 
 ![](https://assets.underline.io/uploads/markdown_image/1/image/f90a4f65913d4d195f2a1eb790752e1d.png)**Conclusions** The EDI editors identified several conceptual
and nomenclature issues spanning various themes. The fact
that the proportion of rejected manuscripts was lower when
EDI concerns were identified suggests that the authors were
able to address the concerns raised. The implementation of
EDI review led to identification and substantive corrections
related to categories of marginalization in manuscripts
submitted to Neurology. 
 **References**
1. Baskin PK, Pieper KM, Gross RA. Efforts by the
journal to promote perspectives of equity, diversity, and
inclusion and address disparities. Neurology.
2019;93(12):521-522. doi:10.1212/WNL.0000000000008021 
 2. Hamilton RH, Hinson HE. Introducing the associate
editors for equity, diversity, and inclusion: aligning editorial
leadership with core values in Neurology®. Neurology.
2019;93(15):651-652. doi:10.1212/WNL.0000000000008235 
 1 Neurology, Minneapolis, MN, USA, jose.g.merino@gunet.
georgetown.edu; 2 Department of Neurology, University of
Pennsylvania, Philadelphia, PA, USA; 3 Department of Neurology,
Oregon Health and Science University, Portland, OR, USA;
4 Department of Neurology, Dana-Farber Cancer Institute,
Massachusetts General Hospital, Harvard Medical School, Boston,
MA, USA; 5 Department of Neurology, University of California, San
Francisco, CA, USA; 6 Department of Neurology, University of
Rochester, Rochester, NY, USA; 7 Department of Pharmacology
and Physiology, University of Rochester, Rochester, NY, USA;
8 Department of Neurology, Georgetown University Medical Center,
Washington, DC, USA 
 **Conflict of Interest Disclosures** Roy H. Hamilton
receives salary support from the American Academy of
Neurology for his role as associate editor for Equity,
Diversity, and Inclusion. H. E. Hinson receives salary support
from the American Academy of Neurology for her role as
associate editor for Equity, Diversity, and Inclusion. She
consults for Biogen and conducts research supported by
National Institute of Neurological Disorders and Stroke grant
1K23NS110828. Joshua A. Budhu receives salary support
from the American
Academy of Neurology for his role as section editor for
Inclusion, Diversity, Equity, Anti-racism, and Social Justice.
The Commonwealth Fund provides salary and research
support. Nicole Rosendale receives research support from the
American Academy of Neurology. Patricia K. Baskin’s salary
is paid by the American Academy of Neurology. José G.
Merino receives salary support from the American Academy
of Neurology for his role as editor in chief of Neurology. No
other disclosures were reported. 
 **Additional Information** Roy H. Hamilton and H. E.
Hinson are co–first authors.

![](https://assets.underline.io/uploads/markdown_image/1/image/f851fdbc824d1a0dbeb8c5c898ff05ea.png)

Inaugural Drummond Rennie Lecture: Bias, Spin, and Problems With Transparency of Research, Isabelle Boutron

Downloads

Next from Peer Review Congress 2022

Prevalence of Honorary Authorship According to Different Authorship Recommendations and Contributor Role Taxonomy (CRediT) Statements | VIDEO

Similar lecture

SOS: Systematic Offensive Stereotyping Bias in Word Embeddings

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Inaugural Drummond Rennie Lecture: Bias, Spin, and Problems With Transparency of Research, Isabelle Boutron

.css-70qvj9{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}Downloads

Next from Peer Review Congress 2022

Prevalence of Honorary Authorship According to Different Authorship Recommendations and Contributor Role Taxonomy (CRediT) Statements | VIDEO

Similar lecture

SOS: Systematic Offensive Stereotyping Bias in Word Embeddings

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES

Downloads