Panel Paper: On the Computation of Minimum Detectable Effects for Analyses of Symmetrically-Predicted Endogenous Subgroups

Saturday, November 9, 2013 : 4:30 PM
3017 Monroe (Washington Marriott)

*Names in bold indicate Presenter

David Judkins, Shawn Moulton and Laura Peck, Abt Associates, Inc.
The Analysis of Symmetrically-Predicted Endogenous Subgroups method developed in Peck (2002, 2003, 2013) provides a framework for researchers to create experimentally valid subgroups defined by some post random assignment event or path.  In brief, this analytic approach uses observed baseline characteristics which are exogenous to the treatment indicator to create experimentally valid subgroups that are associated with some post-random assignment (endogenous) event or experience.  While this method is part of many applied analyses, including recent analyses of the Moving to Opportunity (Moulton, Peck, and Dillman, 2013) and Head Start Impact (Peck and Bell, 2012) studies, prior work has not considered how to compute Minimum Detectable Effects (MDEs) associated with Analyses of Symmetrically-Predicted Endogenous Subgroups.  The MDE is the smallest true effects of an intervention that researchers can expect to detect as statistically significant when analyzing samples of a given size. Although designed to increase the utility of existing experimental data, the approach is increasingly being incorporated prospectively in evaluation design plans.  At this early stage in a research project, it is common to consider whether the likely MDEs are in sync with sample availability and other design options.  This paper develops a method for computing MDEs that can be expected when anticipating use of the Analysis of Symmetrically-Predicted Endogenous Subgroups method.  An application of the MDEs from a social policy evaluation will be included.