Objective Scientific journals may counter the misuse, misreporting, and misinterpretation of statistical methods by offering statistical guidance to authors. 1,2,3 This study assessed the nature and prevalence of statistical guidance in topranked journals across 22 scientific disciplines.

Design Statistical guidance from journal websites of 15 journals (top-ranked by impact factor) in each of 22 scientific disciplines (330 journals) was extracted and classified (in duplicate). Disagreements were resolved through discussion. Information was recorded on whether journals had dedicated statistical guidance sections and/or referred to guidance in external sources. For journals that provided their own statistical guidance, advice on each of 20 prespecified topics was recorded. For 6 topics that were considered in advance (based on author intuition) to be hotly debated in the statistical literature (P values, statistical significance, confidence intervals, effect sizes, sample size justification, and bayesian statistics), 1 investigator classified whether the journal indicated opposition or endorsement and whether this was implicit or explicit.

Results Of 330 journals, 160 (48%) provided statistical guidance and 93 (28%) had a dedicated statistical guidance section in their author instructions (Figure 9, A). Statistical guidance was most common in health and life sciences journals. Notably, all 15 clinical medicine journals offered some statistical guidance. In 2 disciplines (computer science and mathematics), no journals offered any statistical guidance. Some journals shared the same publisher-level guidance, including 31 Nature Research journals (9%), 12 Cell Press journals (4%), and 2 Frontiers Media journals (0.6%). A total of 137 journals (42%) referred authors to statistical guidance in 80 individual external sources, 49 of which were reporting guidelines (the remainder were primarily journal articles). Among 20 prespecified statistical topics (Figure 9, B), only 2 were mentioned in more than a quarter of the journals: confidence intervals (90 27%) and P values (88 27%). Guidance on these topics was inconsistent across journals. For 6 hotly debated topics, only 3 journals explicitly opposed the use of statistical significance; more commonly, journals implicitly endorsed the use of P values (77 23%), statistical significance (35 11%), and bayesian statistics (39 12%) and explicitly endorsed reporting of effect sizes (62 19%), confidence intervals (85 26%), and sample size justifications (67 20%).

Figure 9. Statistical Guidance

Conclusions The results of this study suggest that there are large gaps and inconsistent coverage in the statistical guidance provided by top-ranked journals across scientific disciplines. Future studies should investigate whether journal statistical guidance to authors is associated with improved selection, use, reporting, or interpretation of statistical analyses.

References 1. Strasak AM, Zaman Q, Marinell G, Pfeiffer K, Ulmer H. The use of statistics in medical research: a comparison of the New England Journal of Medicine and Nature Medicine. Am Statistician. 2007;61:47-55. doi:10.1198/000313007X170242

2. Altman DG, Gore SM, Gardner MJ, Pocock SJ. Statistical guidelines for contributors to medical journals. Br Med J (Clin Res Ed). 1983;286:1489-1493.

3. Schriger DL, Arora S, Altman DG. The content of medical journal instructions for authors. Ann Emerg Med. 2006;48:743-749. doi:10.1016/j.annemergmed.2006.03.028

Conflict of Interest Disclosures Tom E. Hardwicke receives funding from the European Union’s Horizon 2020 research and innovation program under Marie Skłodowska-Curie grant 841188. Maia Salholz-Hillel is employed as a researcher under research grants from the German Bundesministerium für Bildung und Forschung. Mario Malički is a co–editor in chief of Research Integrity and Peer Review. John P. A. Ioannidis is a codirector of the Peer Review Congress but was not involved in the review or decision of this abstract. The Meta-Research Innovation Center at Stanford is supported by a grant from the Laura and John Arnold Foundation, and the Meta-Research Innovation Center Berlin is supported by a grant from the Einstein Foundation and Stiftung Charité. No other disclosures were reported.