How Can Design-Based Methods be Extended to Multi-Armed RCTs?
*Names in bold indicate Presenter
This presentation will discuss results from a paper that addresses several key topics for estimating average treatment effects (ATEs) for multi-armed designs using design-based methods. In particular, the presentation will focus on the key question: How do design-based ATE estimators for the two-group design need to be modified for the multi-armed design when comparing pairs of research groups to each other?
As formalized in the paper, key components of the design-based theory for the two-group design apply also to the multi-armed context. However, two simple modifications are required:
- Under the finite-population (FP) model where impact finings are to be generlatized to the study sample only, ATE estimators for each pairwise contrast pertain to the entire randomized sample, not just to the two groups being compared. Thus, variance estimators for the FP model for the two-group design need to be adjusted slightly to reflect the broader inference population.
- For similar reasons, for blocked FP designs, weights for aggregating the block-specific impact estimates need to be modified to reflect the size of the full randomized sample in each block.
The presentation will also briefly address several other topics related to multi-armed trials:
What multiple comparison adjustments should be used when conducting hypothesis tests across pairwise contrasts to identify the most effective interventions? The presentation will provide a brief summary of adjustment methods that align with the non-parametric underpinnings of the design-based approach with some simulation evidence.
What assumptions are required to identify and estimate the complier average causal effect (CACE) parameter for multi-armed RCTs? The presentation will briefly show using an instrumental variable (IV) framework, that identification becomes much more complex in the multi-armed context, and may not be possible in some cases.