Spain

science studies

biomedical research materials

journal guidelines

Following the success of the 1st PEERE International Conference on Peer Review, the PEERE COST Action follow-up community organised the second edition of the PEERE International Conference on Peer Review  and was celebrated virtually on 29 September – 1 October 2020.

The conference welcomed any contribution (e.g., original research, position papers, literature reviews) that used any method (e.g., quantitative, experimental or qualitative) to investigate peer review in a variety of scientific domains (e.g., scholarly journals, funding agencies, research assessment). Interdisciplinary research is especially welcome.

The program is published here: http://tiny.cc/peere2020 


PEERE

Problem
One of the mechanisms proposed to improve reporting of research results is the installation of reporting guidelines in scholarly journals. These guidelines have been issued in a variety of research contexts and with a diverse set of aims, including improving identification and validation of biomedical research materials. Considering these journal initiatives from a science policy perspective, they constitute rule-based or ‘regulatory’ policy, attempting to change researchers’ behaviour through regulation. Rules require policing, which is often expensive, and rules tend to fail if they lack community support. 
Approach
To analyse the effectiveness of journal reporting guidelines we assessed such guidelines in two contexts within biomedical science. We first analysed reporting guidelines in the context of misidentified cell lines, which constitutes a long-known and large-scale issue in biomedical research. Secondly, we assessed guidelines in the context of antibody validation.
Using bibliometric techniques, we set out to study how journal guidelines affect the number of journal articles reporting on misidentified cell lines and the provision of sufficient antibody validation information. For both cases, we compared journals with and without relevant reporting guidelines, as well as reporting before and after the introduction of guidelines.
Results
In the case of cell lines, we observed a decrease in the percentage of articles that use misidentified cell lines in journals with guidelines. However, this decrease already starts before the implementation of the guidelines and does accelerate when the guidelines are implemented. Similarly, we also find a decrease in the number of articles on misidentified cells in journals not introducing guidelines. In addition, the percentage of articles that use misidentified cell lines is higher in journals with guidelines than in those without.
Conclusions
Our results suggest that journal reporting guidelines are an only moderately effective mechanism to steer individual behaviour. In addition, the context and precise format in which they are implemented highly influences the likelihood of guidelines actually leading to improved reporting. In our studies, the implementation of guidelines seems to have resulted in symbolic or minimal compliance. The observed trend in reporting on misidentified cells, suggests that journal guidelines only formalise reporting behaviour that has already been adopted by the community.

Biomedical journal reporting guidelines: what they can and cannot do.

technical paper

Platform for Responsible Editorial Policies

Local versus global inter-rater reliability for evaluating the internal validity of grant peer review: Considerations of measurement

Does social influence matter in peer review?

Introducing a Data Accessibility Policy for journals at IOP Publishing: Measuring the impact on authors and editorial teams

Changing the shoulders I am standing on: Describing the changes that occurred in publications' reference lists after peer review

In Review on Research Square: a journal-integrated preprint system for Springer Nature

Peer Reviews' Prediction in Proposals' Funding Success: A Sentiment Analysis of Grant Reviews

Large-scale language analysis of peer review reports

Software scaffolds for quality feedback in peer review

Journal Competition and the Sustainability of Peer review: An agent based model

Biomedical journal reporting guidelines: what they can and cannot do.

Downloads

Next from PEERE

Platform for Responsible Editorial Policies

Similar lectures

Introducing a Data Accessibility Policy for journals at IOP Publishing: Measuring the impact on authors and editorial teams