Bret Beheim
I use the tools of evolutionary ecology to help understand cumulative cultural dynamics in human societies. Specifically, I look at innovation and strategic retention of ideas and technology in noisy environments, and the demographic aspects of cultural change.
Bret's lectures
Cultural Evolution Society 2021
Estimating the reproducibility of social learning research, 1955-2018
Low rates of data availability and results reproducibility have been observed across the biological and behavioural sciences. Here, we attempt to quantify the rate of reproducibility in the study of social learning research. We surveyed 560 empirical publications on this topic, published between 1955 and 2018 in animal behaviour, behavioural ecology, cultural evolution, and evolutionary psychology.
Data was recoverable online or through direct data requests for 30% of this sample. Data recovery declines exponentially with time since publication, halving every 6 years, and up to every 9 years for human experimental data. Essentially no data is available for any publication before the year 2000. However, when data for a publication is available, we estimate by hierarchical modeling a high probability of subsequent data usability (87%), analytical clarity (97%), and agreement of published results with reproduced findings (96%), giving an overall reproducibility rate of any given result in the literature of 23% 0.89 HPDI: 15-28%.
Our results suggest that the primary threat to the reproducibility of published results in cultural evolution is the lack of long-term data archiving, rather than mistakes in analyses or results. Although recent innovations in research transparency, such as the FAIR data guidelines and use of version-controlled analysis code, address these failure points, individual adoption of these novel research practices, as well as incentives promoting data reuse and openness at the funding and publication level, must become more widespread to prevent similar decay of social learning results in the future.