Objective Over the past decade, there has been an exponential increase in systematic reviews and meta-analyses (SR/MAs) published in the medical literature, and some concerns have arisen that many of these are duplicate studies with redundant results or new studies of unclear quality.1-3 The aim of this study was to determine whether there was any change in the methodological or reporting quality of SRs in dermatology over the past decade.

Design This was a cross-sectional study of all SR/MAs published in 2010 and 2019 in the 10 highest-ranked dermatology journals by SCImago Journal. Methodological quality was assessed through duplicate and independent adjudication by 2 reviewers (A.M., L.K.L., J.K., A.L.) using the Risk of Bias in Systematic reviews (ROBIS) tool and, for SR/ MA of interventions, the A Measurement Tool to Assess Systematic Reviews (AMSTAR) 2 tool. Reporting quality was assessed by 1 investigator (A.M.) using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and PRISMA for Abstracts (PRISMA-A) tools, and study, author, and journal characteristics were abstracted. Statistical analysis was performed with Jamovi, version 1.6.23. Methodological quality was analyzed with the χ2 test for independent proportions, or the Fisher exact test if any of the expected frequencies were less than 5, and difference in proportions with 95% CIs. Reporting quality was analyzed with the independent samples t test and difference in means with 95% CIs.

Results In total, 21 SR/MAs from 2010 and 127 from 2019 were included There was little to no difference between 2010 and 2019 in the proportion of SR/MAs at high or unclear overall risk of bias with ROBIS or with critically low methodological quality using AMSTAR 2. The only subdomain of ROBIS with a difference in proportion of SR/ MAs at high or unclear risk of bias between 2010 and 2019 was eligibility bias, with 27.3% more (95% CI, 5.42%-49.2%) in 2010 (66.7%) than 2019 (39.4%) (P = .02). There was a strong difference in proportion of PRISMA (t146 = 3.15; P = .002) and PRISMA-A (t146 = 2.46; P = .02) checklist items adequately reported between 2010 and 2019. The difference in mean proportion of PRISMA checklist items adequately reported was 3.8 items more (95% CI, 1.4-6.2 items more) in 2019 (mean SD, 15.7 5.1 items) than in 2010 (mean SD, 11.9 5.5), and of PRISMA-A checklist items adequately reported was 0.9 items more (95% CI, 0.2-1.6 items more) in 2019 (mean SD, 5.9 1.5 items) than in 2010 (mean SD, 5 1.9 items).

Conclusions While there was an improvement in the overall reporting quality of SR/MAs between 2010 and 2019, there was no improvement in the overall methodological quality as assessed by the ROBIS and AMSTAR 2 tools. This suggests a persistent potential for forming unreliable conclusions in SR/ MAs through a decade of dermatological research. These results therefore highlight the urgency with which efforts to improve SR/MA methodology should be undertaken.

References

1. Siontis KC, Ioannidis JPA. Replication, duplication, and waste in a quarter million systematic reviews and meta- analyses. Circ Cardiovasc Qual Outcomes. 2018;11(12):e005212.
2. Katsura M, Kuriyama A, Tada M, et al. High variability in results and methodological quality among overlapping systematic reviews on the same topics in surgery: a meta- epidemiological study. Br J Surg. 2021;108(12):1521-1529.
3. Gómez-García F, Ruano J, Gay-Mimbrera J, et al. Most systematic reviews of high methodological quality on psoriasis interventions are classified as high risk of bias using ROBIS tool. J Clin Epidemiol. 2017;92:79-88.

Conflict of Interest Disclosures None reported.