Reproducibility: Principles, Problems, Practices, and Prospects

A comprehensive, insightful treatment of the reproducibility challenges facing science today and of ways in which the scientific community can address them.

–– Kathleen Hall Jamieson, Elizabeth Ware Packard Professor of Communication, University of Pennsylvania

How can we make sure that reproducible research remains a key imperative of scientific communication under increasing commercialization, media attention, and publication pressure? This handbook offers the first interdisciplinary and fundamental treatment of this important question.

                                    –– Torsten Hothorn, Professor of Biostatistics, University of Zurich

Featuring peer–reviewed contributions from noted experts in their fields of research, Reproducibility: Principles, Problems, Practices, and Prospects presents state–of–the–art approaches to reproducibility, the gold standard sound science, from multi– and interdisciplinary perspectives. Including comprehensive coverage for implementing and reflecting the norm of reproducibility in various pertinent fields of research, the book focuses on how the reproducibility of results is applied, how it may be limited, and how such limitations can be understood or even controlled in the natural sciences, computational sciences, life sciences, social sciences, and studies of science and technology.

The book presents many chapters devoted to a variety of methods and techniques, as well as their epistemic and ontological underpinnings, which have been developed to safeguard reproducible research and curtail deficits and failures. The book also investigates the political, historical, and social practices that underlie reproducible research in contemporary science studies, including the difficulties of good scientific practice and the ethos of reproducibility in modern innovation societies. 

Reproducibility: Principles, Problems, Practices, and Prospects is a guide for researchers who are interested in the general and overarching questions behind the concept of reproducibility, for active scientists who are confronted with practical reproducibility problems in their everyday work, and for economic stakeholders and political decision makers who need to better understand the challenges of reproducibility. In addition, the book is a useful in–depth primer for upper–undergraduate and graduate–level courses in scientific methodology and basic issues in the philosophy and sociology of science from a modern perspective.

Harald Atmanspacher, PhD, is Associate Fellow and staff member at Collegium Helveticum, ETH and University Zurich and is also President of the Society for Mind–Matter Research.  He has pioneered advances in complex dynamical systems research and in a number of topics concerned with the relation between the mental and physical.

Sabine Maasen, PhD, is Professor for Sociology of Science and Director of the Munich Center for Technology in Society (TU Munich) and Associate Fellow at Collegium Helveticum (ETH and University Zurich).  Her research focuses on the interface of science, technology, and society, notably with respect to neuroscience and its applications. 

Introduction 1
Harald Atmanspacher and Sabine Maasen

PART I: CONTEXTUAL BACKGROUNDS

Introductory Remarks 9
Harald Atmanspacher

Reproducibility, Objectivity, Invariance 13
Holm Tetens

Reproducibility between Production and Prognosis 21
Walther Ch. Zimmerli

Stability and Replication of Experimental Results: A Historical Perspective 39
Friedrich Steinle

Reproducibility of Experiments: Experimenters′ Regress, Statistical Uncertainty Principle, and the Replication Imperative 65
Harry Collins

PART II: STATISTICAL ISSUES

Introductory Remarks 83
Harald Atmanspacher

Statistical Issues in Reproducibility 87
Werner A. Stahel

Model Selection, Data Distributions and Reproducibility 115
Richard Shiffrin and Suyog Chandramouli

Reproducibility from the Perspective of Meta–Analysis 141
Werner Ehm

Why Are There so Many Clustering Algorithms, and How Valid Are Their Results? 169
Vladimir Estivill–Castro

PART III: PHYSICAL SCIENCES

Introductory Remarks 201
Harald Atmanspacher

Facilitating Reproducibility in Scientic Computing: Principles and Practice 205
David H. Bailey, Jonathan M. Borwein, and Victoria Stodden

Methodological Issues in the Study of Complex Systems 233
Harald Atmanspacher and Gerda Demmel

Rare and Extreme Events 251
Holger Kantz

Science under Social Scrutiny: Reproducibility in Climate Science 269
Georg Feulner

PART IV: LIFE SCIENCES

Introductory Remarks 287
Harald Atmanspacher

From Mice to Men: Translation from Bench to Bedside 291
Marianne Martic–Kehl and P. August Schubiger

A Continuum of Reproducible Research in Biomedical Science? 315
Gerd Folkers and Sabine Baier

Randomness as a Building Block for Reproducibility in Local Cortical Networks 325
Johannes Lengler and Angelika Steger

Neural Reuse and the in–Principle Limitations on Reproducibility in Cognitive Neuroscience 341
Michael Anderson

On the Difference between Persons and Things: Reproducibility in Social Contexts 361
Kai Vogeley

PART V: SOCIAL SCIENCES

Introductory Remarks 383
Sabine Maasen and Harald Atmanspacher

Order Effects in Sequential Judgments and Decisions 389
Zheng Wang and Jerome Busemeyer

Reproducibility in the Social Sciences 405
Martin Reinhart

Accurate But Not Reproducible? The Possible Worlds of Public Opinion Research 423
Felix Keller

Depending on Numbers 445
Theodore M. Porter

Science between Trust and Control: Non–Reproducibility in Scholarly Publishing 465
Martina Franzen

PART VI: WIDER PERSPECTIVES

Introductory Remarks 485
Sabine Maasen and Harald Atmanspacher

Repetition with a Difference: Reproducibility in Literature Studies 489
Ladina Bezzola Lambert

Repetition Impossible: Co–Affection by (Self–)Mimesis and the Transfiguration of Reality 509
Hinderk Emrich

Relevance Criteria for Reproducibility: The Contextual Emergence of Granularity 525
Harald Atmanspacher

The Quest for Reproducibility Viewed in the Context of Innovation Societies 539
Sabine Maasen

Index 561