Before outlining his theory of change for academic science, Marcus discussed the problems in science today creating the need for change. These include an academic culture that is rooted in the 19th century, leading to distorted scientific processes due to misaligned incentives, pressure to publish, and pressure to win grants. These problems are exacerbated by cognitive biases that scientists bring to their work, leading to overinterpretation of data and drawing conclusions that may not be fully supported – without malicious intent. Munafo emphasizes the importance of adhering to good practices in scientific research in order to ensure scientific rigor, and notes that by focusing on evaluating papers as the end product of the scientific process without sufficient focus on quality control throughout the entire process can lead to less reliable outputs, slow the rate of generating knowledge, and create blind alleys in research.
Munafo then introduced the theory of change from the Center for Open Science which focuses on five leverage points within the scientific ecosystem. By targeted changes to infrastructure, user experience, culture and communities, incentives, and policy, the Center for Open Science aims to make it possible, easy, normative, rewarding, and required for researchers to engage in open research practices.
Marcus noted that many of the changes that are being proposed at these levels fall under the domain of stakeholders such as journals, publishers, and funders, and that coordinating the efforts of these different stakeholders is crucial in order to bring about change. He also mentioned that there are concerns at a high level in many countries, such as the UK where there's an inquiry launched into reproducibility and research integrity by the Science and Technology committee and that if the sector doesn't address these concerns, there's a risk of being regulated by politicians in a way that may not be ideal for the academic freedom and intellectual freedom in the academic world.
To address these issues, the UK Reproducibility Network (UKRN) was established in 2019 to bring researchers on the ground together with institutions, funders, publishers, and other organizations in the research ecosystem to improve the integrity and reproducibility of scientific research. The network is a peer-led consortium made up of three main parts: local networks, institutional members, and external stakeholders. UKRN aims to connect, coordinate, and collaborate across these different elements and focuses on open research.
The UKRN is working on various initiatives to improve the integrity and reproducibility of scientific research. They introduced the registered reports funding partnership model, where researchers submit their study protocol to a journal before any data collection happens and the journal evaluates the study protocol based on the importance of the research question and the robustness of the methodology. This model is being adopted by major funders and used in a number of journals across publishers. The UKRN also provides training and resources for researchers to understand and engage in open research practices, such as open research primers and train-the-trainer courses.
Marcus concludes that beyond a coherent theory of change illustrating the desired change we need the right structures in place to bring together the different elements of the research ecosystem for coordination. This way the solutions being implemented can be informed by the voice of the grassroots research community itself while making sure that other stakeholders like the institutions, funders, publishers, and learned societies are also shape the change. This coordination should ideally be happening not only at a national level but also at a supranational level.
