Association for Public Policy Analysis & Management

A distinctive feature of a clustered observational study is a multilevel or nested data structure arising from the assignment of treatment, in a non-random manner, to groups or clusters of individuals.  Examples are ubiquitous in the health and social sciences including patients in hospitals, employees in firms, and students in schools. In this paper we explore best practice for using optimal matching methods with multilevel data.  In particular, we highlight use of a matching method that attempts to replicate a paired clustered randomized study by finding the largest sample of matched pairs of treated and control clusters. We explore these issues by trying to recover an experimental benchmark from the Achieve3000 randomized trial conducted by the Wake County School District.  Achieve3000 is an early literacy program that seeks to improve reading outcomes for elementary school students that was randomly assigned to some schools in Wake County. We will use the schools included in the Access3000 intervention and compare them to schools that were not part of the initial randomization. With these data, we explore how well matching methods perform in educational settings. We also use the data to demonstrate important design choices when conducting observational studies with multilevel data.