**Objective** Peer-reviewed journals ask reviewers to rate the perceived impact or priority of a manuscript. Previous research has suggested an association between reviewer priority scores and citations.1 Altmetrics (alternative metrics) provide an alternative view on social impact (ie, uptake on factor, >5). This journal asks peer reviewers to rate the priority (defined as potential impact) of a manuscript on an ordinal rating scale with possible scores of 1, 2, 5, and 10 (highest priority). Manuscripts are typically reviewed by 2 reviewers. The mean priority score of all reviewers for a manuscript in the first review round constitutes the Manuscript Average Priority Score (MAPS). For this analysis, manuscripts were categorized into 4 quartiles (Qs), with the groups labeled as Q4 (MAPS score, ≤3) to Q1 (MAPS score, >5). The dependent variables, citations, and altmetric scores were obtained from the Dimensions database in February 2022; manuscripts and published articles were similarly stratified into quartiles, with the citation (or altmetrics) quartile Q1 containing the group of articles with the highest citation count (or altmetric score). The association between independent variables (MAPS scores) and citation or altmetric scores was measured using χ² tests for 4 × 4 contingency tables for the quartiles and using Spearman rank correlation between MAPS score ranks and citation or altmetric rank, respectively.

**Results** The MAPS scores for 451 published articles ranged from 1.5 to 10; citations, from 0 to 253; and altmetric scores, from 1 to 849. Although both mean and median citations as well as altmetric scores were higher in the higher MAPS quartiles (**Table 46**), the results of χ² tests were not statistically significant for citations (P = .46) but were statistically significant for altmetric scores (P = .03). The Spearman rank correlation between citation ranks and MAPS score ranks was statistically significant but weak (ρ = .0955; r2 = .009; P = .03). In contrast, altmetric score ranks had a stronger correlation with MAPS score ranks (ρ = .1313; r2 = .017; P = .002).

![](https://assets.underline.io/uploads/markdown_image/1/image/e9506e3de590eff1a8fbd4b75fe758f1.png)

**Conclusions** This longitudinal bibliometric cohort study found that in the Journal of Medical Internet Research, a journal whose subject matter lends itself to the type of attention measured by altmetrics, altmetric scores seemed to be better correlated than citations with a manuscript’s potential impact as assessed by reviewers. Peer reviewers may interpret priority and impact in terms of social impact, rather than citations, raising further questions about the appropriateness of citation-based metrics to measure impact as understood by reviewers.

**References**

1. Opthof T, Coronel R, Janse MJ. The significance of the peer review process against the background of bias: priority ratings of reviewers and editors and the prediction of citation, the role of geographical bias. Cardiovasc Res. 
2002;56(3):339-346. doi:10.1016/S0008-6363(02)00712-5

2. Eysenbach G. Can tweets predict citations? metrics of social impact based on twitter and correlation with traditional metrics of scientific impact. J Med Internet Res. 2011;13(4):e123. doi:10.2196/jmir.2012

3. Araujo AC, Vanin AA, Nascimento DP, et al. What are the variables associated with altmetric scores? Syst Rev. 2021;10:193. doi:10.1186/s13643-021-01735-0

**Conflict of Interest Disclosures Gunther** Eysenbach reported equity in JMIR Publications.

![](https://assets.underline.io/uploads/markdown_image/1/image/78cc867faeefd28c67dc78d1ef488c9f.png)


Association Between Peer Reviewers' Priority Ratings of Impact of Research Manuscripts With Citations and Altmetric Scores of Subsequently Published Articles in the Journal of Medical Internet Research

**Objective** In 2019, JMIR Publications announced the
creation of a new series of journals, which are called
superjournals, with JMIRx-Med the initial journal launched
in that new journal series. 1
Superjournals are a type of overlay journal that sit on top of
preprint servers, offering rapid peer review and formal
publication of revised preprints. 1 
 **Design** An editorial prospecting platform (XDash) that
invites preprint authors from medRxiv or bioRxiv was
developed **(Figure 5)**. All authors received an online survey
that asked about motivations on why they posted the
preprint, plans to submit to a journal, and whether they
would be interested in a peer review of their preprint.
Authors of new preprints were contacted as soon as their
preprints were posted; no reminders were sent. Only
preprints that were not already under peer review at a journal
were eligible for consideration for JMIRx. After the peer
review process, all peer review reports, author responses, and
revised and accepted preprints are published in a JMIRx
journal and deposited in PubMed Central/PubMed and
institutional repositories of member institutions or, via the
Manuscript eXchange Marketplace, offered to other journals
for publication. 
 ![](https://assets.underline.io/uploads/markdown_image/1/image/4669f533c5d8376696983a90ba71d483.png)**Results** Between December 19, 2019, and March 9, 2022,
11,143 responses were received. In the same period, 113,724
preprints were posted (response rate of 9.7%, although it is
not known how many emails were actually extracted correctly
and delivered). Forty-five percent of respondents (5011)
submitted their manuscripts to a journal immediately after
deposit as a preprint; for 1288 of respondents (11.6%),
journals deposited a submitted manuscript on the preprint
server on behalf of the authors. Together, these 2 groups
represented 56.5% of respondents (n = 6299), whose preprint
was already under peer review. The remaining 4844
respondents (43.5%) had not submitted their preprint to a
journal yet. A total of 869 (7.8%) had no plans to submit to a
journal, 3409 preprint authors (30.6%) planned to submit
the preprint to a journal, and 1976 (17.7%) expressed interest
in a peer review. A total of 7676 authors (68.9%) also
indicated they absolutely need to publish the preprint in a
journal with an impact factor, and 6560 authors (58.9%)
indicated they wanted their publication indexed in PubMed. 
 **Conclusions** The JMIRx concept is welcomed by some
authors looking for a rapid publication venue. 
 **References**
1. Eysenbach G. Celebrating 20 years of open access
and innovation at JMIR Publications. J Med Internet Res.
2019;21(12):e17578. doi:10.2196/17578 
 2. Stern BM, O’Shea EK. A proposal for the future of
scientific publishing in the life sciences. PLoS Biol.
2019;17(2):e3000116. doi:10.1371/journal.pbio.3000116 
 **Conflict of Interest Disclosures** Gunther Eysenbach has an
equity stake in and receives a salary from the publisher.

![](https://assets.underline.io/uploads/markdown_image/1/image/5fee415064d5603a1f9c0174be1827f6.png)

Early Experiences of the Preprint Overlay Journal JMIRx

This is an inperson poster session. If you are attending virtually please visit the individual poster pages and ask your questions in the chat box.

PS1 - Peer Review

poster

Our aim is to encourage research into the quality and credibility of peer review and scientific publication, to establish the evidence base on which scientists can improve the conduct, reporting, and dissemination of scientific research.
 
 **Welcome!**
 
 The JAMA Network, The BMJ, and METRICS welcome you to the Ninth International Congress on Peer Review and Scientific Publication. We have continued our efforts to broaden the scope of the Congress to all aspects of peer review and publication—from funding to postpublication—and to all sciences.
 
 We will have 3 days for presentations of new research into peer review and all aspects of scientific publication, bias, quality of reporting, and information access and dissemination. There are 50 plenary session research presentations and 5 plenary invited talks. Each plenary session research presentation will be followed by equal time for discussion and questions from the audience. In addition, there are 125 posters. All posters, including virtual posters, are available to view throughout the meeting. In-person posters will also be presented at designated times on September 9 and 10. If you are attending virtually, please visit the individual poster pages and ask your questions in the chat box. See the link to Posters on the left.
 
 We hope you will take an active part in the program, as we depend on your participation in the discussion sessions to make the Congress a success.

 [Peer Review Congress Organizers, Directors, and Advisory Board](https://peerreviewcongress.org/organizers-and-advisory-board/)
 
 [Health and Safety Policies](https://web.cvent.com/event/1b505794-6a0b-428c-a44c-6c46c4c50e99/websitePage:152962bb-4567-4561-9d81-21ebfa97f487)

![](https://assets.underline.io/uploads/markdown_image/1/image/7c9f0143dee383780cc4670269ade132.png)

A PDF of the schedule is available [here](https://drive.google.com/file/d/1-Jd191W-nxbWcN1prdX5o9nlSfeo9_-H/view?usp=sharing) and a flipbook version of the complete program and abstracts is included below. See also the link to the interactive schedule to the left.

<iframe width="700px" height="425px" src="https://online.fliphtml5.com/ebtyf/nlfn/#p=1" frameborder="0" allowfullscreen allowtransparency></iframe>

Registration is required to attend this event, please use the URL linked below to secure your ticket. Registration will close on Friday, September 2 at 5:00 PM CT. 

[**Register Here**]( https://web.cvent.com/event/1b505794-6a0b-428c-a44c-6c46c4c50e99/summary)

Peer Review Congress 2022

Our aim is to encourage research into the quality and credibility of peer review and scientific publication, to establish the evidence base on which scientists can improve the conduct, reporting, and dissemination of scientific research.



Gunther Eysenbach

2

5

Presentations

Association Between Peer Reviewers' Priority Ratings of Impact of Research Manuscripts With Citations and Altmetric Scores of Subsequently Published Articles in the Journal of Medical Internet Research

Early Experiences of the Preprint Overlay Journal JMIRx

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES