Association for Public Policy Analysis & Management

The Ed Tech Rapid Cycle Evaluation Coach is a free, web-based resource to guide a user, step by step, through an evaluation of educational technology in a school or district. The Coach streamlines the evaluation process, reducing the time required for district staff to conduct evaluations, and automates the technical and methodological aspects of evaluation, making rigorous evaluation more accessible to local education agencies. The Coach frames the research process around the specific decision to be made, providing scaffolding for the process of turning evidence into action. On the back-end, the RCE Coach uses innovative Bayesian methods to conduct the analyses, providing actionable findings with smaller sample sizes, and making the findings, presented as probabilities, more intuitive to program administrators and policymakers.

The Coach has been used by dozens of districts to evaluate technologies and other interventions used in schools. The Coach facilitates knowledge building by highlighting district experiences through case studies published on the site and by allowing users to share their findings with others.

This poster will introduce the Coach and highlight how the user's inputs for a meaningful effect and the level of confidence required to make a particular decision are used to provide actionable results. Funded by the Office of Educational Technology at the U.S. Department of Education, the Coach focuses assisting schools in evaluating educational technology in school settings. However, the Coach’s tools and concepts apply to a variety of interventions and contexts, and districts and schools have used the Coach to evaluate many types of initiatives.