*Names in bold indicate Presenter
This work builds off of previous research on the appropriate interpretation and analysis of data in individually randomized trials. The talk will cover:
- A review of the implications of service provider selection and effects when defining “the causal” effects in individually randomized experiments. The review will describe how individual randomization can make it difficult or impossible to disentangle the effect due to the intervention from the effect due to the service providers. While not the central theme of this work, clarity on this issue is fundamental.
- The three potential sources of lack of independence of observations in individually randomized experiments: (1) non-random sorting into groups, (2) provider effects, and (3) peer effects. Importantly, these sources cannot be disentangled without modeling assumptions, but may need to be accounted for in analyses.
- The fixed and random effects approaches to accounting for lack of independence of observations, which have been described in detail in past literature. This includes discussion of the intended inferences of each approach (this is not intended to be new, but rather it is necessary background information).
- Presentation of analytical results and a related set of simulations that demonstrate and elucidate when fixed effects are appropriate, when random effects are appropriate, and when neither are appropriate. In addition, the simulations provide control over the process that sorts individuals into groups – permitting a demonstration of how this sorting can bias the estimates of the standard error of the impact estimates under both the fixed and random effects paradigms.
- Suggestions on how to proceed in real world experiments.