Research in the biomedical sciences: Transparent and reproducible

Abstract

Biomedical research findings should be validated for reproducibility by independent studies and if necessary may require scientific self-correction. Otherwise, these cannot contribute to the advancement of science. A survey published in 2016 in ‘Nature’ found that 70 per cent responders failed to reproduce an experiment. At present, ‘reproducibility’ is a crisis and a chronic problem which needs to be addressed. The present book, authored by three eminent scientists with a cumulative experience of a century or more focuses on this problem and discusses possible solutions for the same. The book consists of six main chapters with several sub-headings. The first chapter, ‘Reproducibility in Biomedical Research’ lists and discusses four important factors namely honest errors, sloppy science, introduction of bias and noise which are responsible for failure of reproducibility. A new lexicon to better describe issues in reproducibility of methods, results and inference is suggested. It is stated that besides misuse of statistics other environmental factors can also contribute to non-reproducibility of research findings, such as lack of a credible hypothesis, poor experimental design, execution, inadequate power, lack of appropriate controls, lack of blinding, wrong interpretation and other experimental issues. These are explained with appropriate examples such as the genetics of mouse behaviour study, and Caenorhabditis Intervention Testing Program (CITP). Topics such as reproducibility issues in a project involving ‘big science’ of large-scale databases ranging from genomics, proteomics, to others, research fraud, misconduct, retractions, etc are also covered. A table listing five serious frauds which resulted in serious societal consequences (eg. measles, mumps and rubella vaccine publications resulting in measles outbreak) is also provided. The last section of this chapter discusses reproducibility in translational medical research. Limitations in using animal models for human diseases such as amyotrophic lateral sclerosis (ALS) and stroke are also discussed. The second chapter focuses on some established issues and the respective resolutions in planning and execution of experiments. It discusses the hypothesis generation which includes the concept of falsification and null hypothesis. The American Statistical Association statement of 2016 on statistical significance and P value is given in tabular form. Under the experimental planning section, the differences between preliminary, exploratory (hypothesis generating) and confirmatory (hypothesis confirmation) studies are explained with suitable examples. One of the tables provides details of Assay Capability Tools developed by Pfizer. It addresses three major issues namely: (i) aligning assay capability with research objectives, (ii) enabling assay capability by managing variability, and (iii) objectivity in assay conduct. In order to ensure reproducibility, many aspects of an experiment need to be validated, such as compound authentication and characterization, validation of dose-response curves, antibodies, animal models, equipment and cell line authentications. This chapter ends with a list of ‘Dos’ and ‘Don’ts’ which will increase the probability of generating meaningful results. Appropriate use of statistics in research is essential to generate replicable data. The third chapter elaborates on this issue. It discusses the use and misuse of statistics, basics of sampling bias, descriptive statistics and measures of dispersion. Guidance for analysis of exploratory studies, visualization of exploratory data by bar charts, scatter plots and frequency histograms is described with simple figures. A section is devoted to sample size, power analysis and its misuse. Three flow diagrams to select the appropriate statistical hypothesis tests for parametric and non-parametric data are a useful addition. P-hacking encompasses several types of manipulation of data analysis which may reduce the chance of reproducibility. This has been dealt with briefly. Chapter 4 is about ‘Reporting Results’. This chapter elaborates on preparing manuscripts, IMRAD (Introduction, Methods, Results and Discussion) format, importance of reading guidelines for authors and better writing of various parts of the manuscript. Despite available books on this topic, the uniqueness of this book is that it highlights the inappropriate image manipulation, supplementary materials submission and practical aspects of peer review process and authors’ response to reviewer feedback which will be useful for potential authors. Chapter 5 discusses various issues concerning ‘Reproducibility’. Some of these such as ineffective peer review and bias in peer review pertain to the editorial office functioning. Post-publication peer review process, post-publication commentary, open data peer review systems and financing of Open Access journals are also detailed. The pros and cons of journal impact factor and its future are also discussed. Compilation of important biomedical research guidelines for animal studies in a table may be useful for readers. The last chapter addresses the multiple challenges in resolving reproducibility issues in the 21st century. It dwells on the impact of technological advances, alternative approaches in drug discovery and development. The authors state that researcher training, motivation and incentivisation contribute to reproducibility. The chapter ends with discussing two series of frameworks for improving reproducibility. Overall, this book adequately addresses the important issue of reproducibility with remedial measures. It will be useful for researchers for improving the quality and reportability of their data and conclusions. It is recommended for research scholars and senior scientists across specializations.