Do multiple experimenters improve the reproducibility of animal studies?

von Kortzfleisch, Vanessa Tabea; Ambrée, Oliver; Karp, Natasha A; Meyer, Neele; Novak, Janja; Palme, Rupert; Rosso, Marianna; Touma, Chadi; Würbel, Hanno; Kaiser, Sylvia; Sachser, Norbert; Richter, S Helene (2022). Do multiple experimenters improve the reproducibility of animal studies? PLoS biology, 20(5), e3001564. Public Library of Science 10.1371/journal.pbio.3001564

[img]
Preview
Text
journal.pbio.3001564.pdf - Published Version
Available under License Creative Commons: Attribution (CC-BY).

Download (1MB) | Preview

The credibility of scientific research has been seriously questioned by the widely claimed "reproducibility crisis". In light of this crisis, there is a growing awareness that the rigorous standardisation of experimental conditions may contribute to poor reproducibility of animal studies. Instead, systematic heterogenisation has been proposed as a tool to enhance reproducibility, but a real-life test across multiple independent laboratories is still pending. The aim of this study was therefore to test whether heterogenisation of experimental conditions by using multiple experimenters improves the reproducibility of research findings compared to standardised conditions with only one experimenter. To this end, we replicated the same animal experiment in 3 independent laboratories, each employing both a heterogenised and a standardised design. Whereas in the standardised design, all animals were tested by a single experimenter; in the heterogenised design, 3 different experimenters were involved in testing the animals. In contrast to our expectation, the inclusion of multiple experimenters in the heterogenised design did not improve the reproducibility of the results across the 3 laboratories. Interestingly, however, a variance component analysis indicated that the variation introduced by the different experimenters was not as high as the variation introduced by the laboratories, probably explaining why this heterogenisation strategy did not bring the anticipated success. Even more interestingly, for the majority of outcome measures, the remaining residual variation was identified as an important source of variance accounting for 41% (CI95 [34%, 49%]) to 72% (CI95 [58%, 88%]) of the observed total variance. Despite some uncertainty surrounding the estimated numbers, these findings argue for systematically including biological variation rather than eliminating it in animal studies and call for future research on effective improvement strategies.

Item Type:

Journal Article (Original Article)

Division/Institute:

05 Veterinary Medicine > Department of Clinical Research and Veterinary Public Health (DCR-VPH) > Veterinary Public Health Institute > Animal Welfare Division
05 Veterinary Medicine > Department of Clinical Research and Veterinary Public Health (DCR-VPH) > Veterinary Public Health Institute

UniBE Contributor:

Novak, Janja

Subjects:

600 Technology > 630 Agriculture

ISSN:

1544-9173

Publisher:

Public Library of Science

Language:

English

Submitter:

Pubmed Import

Date Deposited:

06 May 2022 09:20

Last Modified:

08 May 2022 02:05

Publisher DOI:

10.1371/journal.pbio.3001564

PubMed ID:

35511779

BORIS DOI:

10.48350/169765

URI:

https://boris.unibe.ch/id/eprint/169765

Actions (login required)

Edit item Edit item
Provide Feedback