Mixed Methods and Credibility of Evidence in Evaluation: New Directions for Evaluation, Number 138

Mixed methods in evaluation have the potential to enhance the credibility of evaluation and the outcomes of evaluation. Advances in understanding mixed methods are explored in philosophical, theoretical, and methodological terms with specific illustrations of the application of these concepts in evaluation practice. Leading thinkers in the mixed methods evaluation community provide frameworks and strategies that are associated with improving the probability of reaching the goals of enhanced credibility for evaluations, the evidence they produce, and the actions taken as a result of the evaluation findings.

EDITORS NOTES 1
Donna M. Mertens, Sharlene Hesse–Biber

1. Mixed Methods and Credibility of Evidence in Evaluation 5
Donna M. Mertens, Sharlene Hesse–Biber

Mixed methods can potentially contribute to an increase in the credibility of evidence in evaluation because it can provide a fuller understanding than a single method alone. However, evaluators need to be cognizant of the assumptions that they are making when choosing to use mixed methods.

2. Pragmatism, Evidence, and Mixed Methods Evaluation 15
Jori N. Hall

Pragmatism according to Dewey emphasizes intelligent action. The author uses this form of pragmatism to explore intelligent action in mixed methods research.

3. What Does a Transformative Lens Bring to Credible Evidence in Mixed Methods Evaluations? 27
Donna M. Mertens

Mixed methods have a role to play in conducting and using evaluations that are designed to address issues of social justice and human rights; a transformative lens facilitates the identification of appropriate strategies.

4. Considering the Evidence–and–Credibility Discussion in Evaluation Through the Lens of Dialectical Pluralism 37
R. Burke Johnson, Tres Stefurak

Dialectical pluralism is a philosophy that allows for engaging differences through dialogue. The application of dialectical pluralism to methodological approaches in evaluation allows for dialogue about the differences in the use of quantitative and qualitative approaches in a mixed methods study.

5. Thinking Outside the Randomized Controlled Trials Experimental Box: Strategies for Enhancing Credibility and Social Justice 49
Sharlene Hesse–Biber

Weaving and shifting methodological perspectives and methods into randomized controlled trials (RCT)–based evaluations prior to, during, or after the RCT design holds promise of enhancing the credibility and social–justice RCT praxis.

6. The Use of Mixed Methods in Randomized Control Trials 61
Howard White

Impact evaluation is about determining if an intervention had the desired impact; randomized control trials are the best method for determining this type of causation. Mixed methods can be used to address questions related to the quality of the implementation and reaching the targeted population.

7. The Contribution of Pluralistic Qualitative Approaches to Mixed Methods Evaluations 75
Nollaig Frost, Sevasti–Melissa Nolas

Use of multiple qualitative approaches in the same study provides evidence that can be used to enhance efforts to scale up small interventions. The authors draw lessons from this use of multiple qualitative approaches for evaluators using mixed methods.

8. Establishing Interpretive Consistency When Mixing Approaches: Role of Sampling Designs in Evaluations 85
Kathleen M. T. Collins, Anthony J. Onwuegbuzie

Mixed methods studies involve the use of samples for both the quantitative and qualitative portions of the study. Quality criteria are presented for the sampling processes used in mixed methods evaluations.

9. Incorporating Qualitative Evidence in Systematic Reviews: Strategies and Challenges 97
Valerie J. Caracelli, Leslie J. Cooksy

Criteria for quality in quantitative and qualitative approaches have implications for synthesizing studies that use mixed methods. The authors discuss the challenges and put forth ideas for conducting syntheses of mixed methods studies in evaluation.

10. Reflections and Ruminations 109
Jennifer C. Greene

Mixed methods in evaluation encompasses philosophical, theoretical, and practice issues. Credibility of evidence is dependent on clarity in these respects, as well as on the relationships that are formed in an evaluation.

INDEX 121