Making Every Study Count: Learning From Replication Failure to Improve Intervention Research

December 16, 2019

Kim, J. S. (2019). Making every study count: Learning from replication failure to improve intervention research. Educational Researcher48(9), 599-607.

Overview

Making Every Study Count: Learning From Replication Failure to Improve Intervention Research by James S. Kim has been published in Educational Researcher. Kim leads the K-3 Intervention area of focus in Reach Every Reader.

Abstract

Why, when so many educational interventions demonstrate positive impact in tightly controlled efficacy trials, are null results common in follow-up effectiveness trials? Using case studies from literacy, this article suggests that replication failure can surface hidden moderators—contextual differences between an efficacy and an effectiveness trial—and generate new hypotheses and questions to guide future research. First, replication failure can reveal systemic barriers to program implementation. Second, it can highlight for whom and in what contexts a program theory of change works best. Third, it suggests that a fidelity first and adaptation second model of program implementation can enhance the effectiveness of evidence-based interventions and improve student outcomes. Ultimately, researchers can make every study count by learning from both replication success and failure to improve the rigor, relevance, and reproducibility of intervention research.

Read the full article