Qualitative Inquiry in Evaluation: From Theory to Practice

Qualitative inquiry methods and philosophy, with practical real–world applications Qualitative Inquiry in Evaluation brings together evaluation theory and practice to provide an accessible, practical introduction to qualitative evaluation for students, and a useful reference for practicing evaluators. Contributions from leading practitioners describe the design, implementation, and reporting of qualitative evaluations in real–world settings, and case–based examples illustrate the benefits and challenges faced in those settings. Readers learn how qualitative inquiry fits into major evaluation approaches, and gain the skills and knowledge required to produce a high–quality evaluation. Compiled by active, recognized experts in the field, this book provides a unique insight into the evaluator’s methodological toolbox, as well as utilization techniques that can ease the everyday challenges presented by complicated or nuanced settings. Understand the origins and significance of qualitative inquiry in evaluation Conceptualize and implement qualitative inquiry within commonly used evaluation approaches Discover the key competencies that make an effective qualitative evaluator Learn how to optimize stakeholder relationships with dialogue and mutual learning Examine strategies for balancing roles and navigating ethical conundrums Effective evaluation requires a particular skillset and a deep understanding of prominent theory and methodology. Qualitative Inquiry in Evaluation draws from the wisdom and experience of industry leaders to provide a comprehensive, practical reference.

Figures and Tables xi Introduction xiii About the Editors xxi About the Contributors xxv Acknowledgments xxxi PART ONE: The Intersection of Qualitative Inquiry and Evaluation Approaches 1 CHAPTER ONE: THE ORIGINS, MEANING, AND SIGNIFICANCE OF QUALITATIVE INQUIRY IN EVALUATION 3 Thomas A. Schwandt and Timothy J. Cash The Genesis of Qualitative Evaluation 4 What Is Qualitative Evaluation? 7 Enduring Contributions 11 Conclusion 18 Key Concepts 19 Discussion Questions 19 References 20 CHAPTER TWO: QUALITATIVE INQUIRY IN UTILIZATION–FOCUSED EVALUATION 25 Michael Quinn Patton The Focus on Evaluation Use as the Primary Evaluation Outcome 27 Basic Definitions 28 Involving Intended Users in Making Evaluation Decisions 28 Qualitative Inquiry into Evaluation Use: Emergence of the Personal Factor 30 Considerations for Incorporating Qualitative Inquiry into Utilization–Focused Evaluations 35 Particularly Appropriate Uses of Qualitative Evaluations 37 Value of Incorporating Qualitative Approaches 46 Challenges Associated with Qualitative Inquiry in Utilization–Focused Evaluations 49 Conclusion 52 Key Concepts 53 Discussion Questions 53 References 54 CHAPTER THREE: QUALITATIVE INQUIRY FOR POLICY MAKERS 55 George F. Grob Who Are the Policy Makers and How Do They Operate? 57 Why Qualitative Analysis Is Important to Policy Makers 59 Principles for Influencing Policy Makers 60 Conclusion 73 Key Concepts 74 Discussion Questions 74 References 75 CHAPTER FOUR: QUALITATIVE INQUIRY WITHIN THEORY–DRIVEN EVALUATION: PERSPECTIVES AND FUTURE DIRECTIONS 77 Katrina L. Bledsoe Theory–driven Evaluation: A Brief History 81 Key Elements in Theory–driven Evaluation 83 Qualitative Inquiry: An Imperative to Theory–driven Evaluation 89 Conclusion 92 Key Concepts 94 Discussion Questions 95 References 95 CHAPTER FIVE: CONCEPTUAL AND PRACTICAL INTERSECTIONS BETWEEN PARTICIPATORY EVALUATION AND QUALITATIVE INQUIRY 99 Jill Anne Chouinard and J. Bradley Cousins Participatory Evaluation: Concepts and Principles of Practice 103 Qualitative Inquiry: A Brief Overview of Key Characteristics 107 Locating Participatory Evaluation and Qualitative Inquiry: Conceptual Intersections 109 Epistemological Intersections 110 Methodological Linkages 117 Methods 118 Practical Intersections 119 Key Benefits 128 Potential Challenges 129 Conclusion 130 Key Concepts 131 Discussion Questions 131 References 132 PART TWO: Tales from the Field of Qualitative Evaluation 139 CHAPTER SIX: WHAT DOES IT TAKE TO BE AN EFFECTIVE QUALITATIVE EVALUATOR? ESSENTIAL COMPETENCIES 141 Laurie Stevahn and Jean A. King Competencies for Professional Practice 143 Toward Competencies for Evaluators 144 Evaluator Competency Taxonomies 145 Competencies for Qualitative Evaluators 151 Core Competency Domains 151 Commitment to the Qualitative Paradigm 154 Technical Expertise 156 The Qualitative Sixth Sense 160 Conclusion 162 Key Concepts 162 Discussion Questions 163 References 164 CHAPTER SEVEN: DEALING WITH ASYMMETRIC RELATIONS BETWEEN STAKEHOLDERS: FACILITATING DIALOGUE AND MUTUAL LEARNING THROUGH QUALITATIVE INQUIRY 167 Vivianne E. Baur and Tineke A. Abma Evaluators’ Outlook and Identity 168 The Project 171 Evaluators’ Methodological Choices and Actions 175 Conclusion 183 Key Concepts 184 Discussion Questions 185 References 185 CHAPTER EIGHT: BALANCING INSIDER–OUTSIDER ROLES AS A NEW EXTERNAL EVALUATOR 189 Norma Mart´ýnez–Rubin and Stuart Hanson Responding to the Sponsoring Organization’s Request for Proposals 191 The Use of Qualitative Inquiry 197 Personal Influences on Evaluation Design 198 Trends in Health Education 199 Delving into the Formality of Evaluation Design 200 Data Collection, Analysis, and Reporting 201 Reflections on Evaluator Background 206 Reflections on Our Evaluation Process 207 Reflections on Our Use of Qualitative Inquiry 209 Conclusion 210 Key Concepts 210 Discussion Questions 211 References 211 CHAPTER NINE: WHOSE STORY IS THIS, ANYWAY? NAVIGATING AN ETHICAL CONUNDRUM 213 Sharon F. Rallis The Context: Who’s Doing What? 215 The Moral Lens on Qualitative Inquiry: Principles and Ethical Theories 217 Challenges: Ethical and Logistic 221 Conclusion 242 Acknowledgments 246 Key Concepts 247 Discussion Questions 247 Notes 248 References 248 CHAPTER TEN: ELEMENTS OF QUALITY IN QUALITATIVE EVALUATION 251 Leslie Goodyear, Jennifer Jewiss, Janet Usinger, and Eric Barela Understanding and Acknowledging Personal Identity 255 Building Trusting Relationships 258 Reflecting on and Managing Relationships 260 Employing Sound and Explicit Methodology, and Being Transparent about It 265 Staying True to the Data 268 Fostering Learning 272 Key Concepts 275 Notes 276 References 276 Index 277

LESLIE GOODYEAR, P H D is a principal research scientist at EDC, with nearly twenty years in the evaluation profession. She is a long–term, active member of the American Evaluation Association, associate editor of the American Journal of Evaluation , and co–chair of the AEA Qualitative Methods Topical Interest Group. JENNIFER JEWISS, EdD is a research assistant professor in the Department of Leadership and Developmental Sciences at the University of Vermont and co–chair of the AEA Qualitative Methods Topical Interest Group. Her work has appeared in the American Journal of Evaluation and Practical Assessment Research & Evaluation. JANET USINGER, P H D is associate professor in the Department of Educational Leadership at the University of Nevada, Reno, and program co–chair of the AEA Qualitative Methods Topical Interest Group. ERIC BARELA, P H D is chief knowledge and impact officer at Partners in School Innovation, and program co–chair of the AEA Qualitative Methods Topical Interest Group.