Reproducibility is an important feature of science; experiments are retested, and analyses are repeated. Trust in the findings increases when consistent results are achieved. Despite the importance of reproducibility, significant work is often involved in these efforts, and some published findings may not be reproducible due to oversights or errors. In this paper, we examine a myriad of features in scholarly articles published in computer science conferences and journals and test how they correlate with reproducibility. We collected data from three different sources that labeled publications as either reproducible or irreproducible and employed statistical significance tests to identify features of those publications that hold clues about reproducibility. We found the readability of the scholarly article and accessibility of the software artifacts through hyperlinks to be strong signals noticeable amongst reproducible scholarly articles.

Reproducibility Signals in Science: A preliminary analysis

[![](https://assets.underline.io/uploads/markdown_image/1/image/fa066bd3696d61289bf9217fc0e3f64e.png)](https://zoom.us/rec/play/8onUtAb7Bn7_pODVmCNcXuafSKjmVvye33w9oZ3c3vCm7Ln-0tcB8oVaoWsI93o1fnV1g-7RwLFomWY.ehdiv5McYi-nrKW1?autoplay=true&startTime=1668958212000) 

[![](https://assets.underline.io/uploads/markdown_image/1/image/f34062db3f5ffa4d48aa0635e5f22a87.png)](https://zoom.us/rec/play/ZEFoEo89MnuE9QLT96_tu_owR3a_aldaXq9OCyYm_bkM81wsfAfugWaNVyKzyLeChTakfBKFT1O_cOMD.04LEbdBmYe2OU325?autoplay=true&startTime=1668967245000)
 
 **Workshop Organizers:**
 Tirthankar Ghosal, Sergi Blanco-Cuaresma, Alberto Accomazzi, and Robert Patton

First Workshop on Information Extraction from Scientific Publications (WIESP)

workshop paper

The AACL, Asia-Pacific Chapter of the Association for Computational Linguistics, and the AFNLP, Asian Federation of Natural Language Processing, invite you to participate in the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (AACL) and the 10th International Joint Conference on Natural Language Processing (IJCNLP).

A registration is required to access this event. Access to the event site will be granted to all registered attendees on **November 16th**. If you have not already done so, please register using the link below:
 
 https://conference.iis.sinica.edu.tw/servlet/Register?ConferenceID=411

Registration Is Required

IJCNLP-AACL 2022

The AACL, Asia-Pacific Chapter of the Association for Computational Linguistics, and the AFNLP, Asian Federation of Natural Language Processing, invite you to participate in the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (AACL) and the 10th International Joint Conference on Natural Language Processing (IJCNLP). 

Akhil Pandey Akella

1

1

SHORT BIO

Presentations

Reproducibility Signals in Science: A preliminary analysis

Stay up to date with the latest Underline news!

PRESENTATIONS

CONFERENCES

COMPANY

RESOURCES