Improvement Science in Evaluation: Methods and Uses New Directions for Evaluation, Number 153

While improvement science has experienced a surge of interest over the past 30 years, applications of it are rare in the evaluation literature.

This issue promotes the cross-fertilization of ideas, techniques, and tools between evaluation and improvement science. There are at least four areas where this cross-fertilization is particularly relevant: learning from error, examining variation, appreciating context, and focusing on systems change. This volume considers:

• the conceptual similarities and distinctions between improvement science and evaluation;
• the intellectual foundations, methods, and tools that collectively comprise improvement science; and
• case chapters that offer an inspiring review of state-of-the-art improvement science applications.

Cutting across all of these applications is a shared grounding in systems thinking, a determination to capture and better understand variation and contextual complexity, as well as a sustained commitment to generative learning about projects and programs—all issues of great concern to evaluators. The issue offers producers and users of evaluations the potential benefits of a closer engagement with improvement science.

This is the 153rd issue in the New Directions for Evaluation series from Jossey-Bass. It is an official publication ofthe American Evaluation Association.

Moira Inkelas is an associate professor in the Department of Health Policy and Management in the UCLA Fielding School of Public Health and assistant director of the Center for Healthier Children, Families and Communities.

Sebastian Lemire is a doctoral candidate in the Social Research Methodology Division in the Graduate School of Education and Information Studies, University of California, Los Angeles.

Editors’ Notes 7
Christina A. Christie, Moira Inkelas, Sebastian Lemire

1. Understanding the Similarities and Distinctions Between Improvement Science and Evaluation 11
Christina A. Christie, Sebastian Lemire, Moira Inkelas

This chapter considers the similarities and distinctions between improvement science and evaluation according to use, valuing, and methods.

2. The Methods and Tools of Improvement Science 23
Sebastian Lemire, Christina A. Christie, Moira Inkelas

This chapter introduces the intellectual foundation, core principles, and selected tools of improvement science.

3. Timely and Appropriate Healthcare Access for Newborns: A Neighborhood-Based, Improvement Science Approach 35
Courtney M. Brown, Robert S. Kahn, Neera K. Goyal

This chapter illustrates the application of quality improvement science techniques to iteratively address family- and system-level barriers to primary care.

4. Improvement for a Community Population: The Magnolia Community Initiative 51
Moira Inkelas, Patricia Bowie, Lila Guirguis

This chapter illustrates how a network of diverse organizations can use iterative learning cycles to come up with promising ideas, test and prototype these ideas, and spread and sustain what is found to work for a community population.

5. Breaking the “Adopt, Attack, Abandon” Cycle: A Case for Improvement Science in K–12 Education 65
Kristen Rohanna

This chapter describes the implementation of rapid cycles of evaluations (Plan–Do–Study–Act cycles) to adapt interventions to local school contexts.

6. Online Learning as a Wind Tunnel for Improving Teaching 79
James W. Stigler, Karen B. Givvin

This chapter considers the potential value of combining improvement science and online learning.

7. Value and Opportunity for Improvement Science in Evaluation 93
Moira Inkelas, Christina A. Christie, Sebastian Lemire

This concluding chapter reflects on the case examples comprising this volume and considers the major benefits and implications of integrating improvement science more firmly in evaluation.

Index 103

The New copy of this book will include any supplemental materials advertised. Please check the title of the book to determine if it should include any access cards, study guides, lab manuals, CDs, etc.

The Used, Rental and eBook copies of this book are not guaranteed to include any supplemental materials. Typically, only the book itself is included. This is true even if the title states it includes any access cards, study guides, lab manuals, CDs, etc.