Association for Public Policy Analysis & Management

In the era of open data, researchers have found an abundance of opportunities to understand the consequences of public policy and inform future decisions. However, these researchers must be extremely careful when using open data to make causal claims. We discuss these challenges and include an example investigating the effect of reducing New York City’s speed limit using an open dataset of motor vehicle collisions from the Police Department. We identify several important preprocessing/postprocessing steps necessary to achieve reasonable causal estimates.

New York City Mayor de Blasio proposed a comprehensive traffic safety reform in 2014 called Vision Zero. Working with State and City legislators, de Blasio signed Vision Zero’s flagship legislation, reducing the default speed limit in New York City streets from 30 to 25 miles per hour. His administration reasoned in the 2014 Vision Zero Action Plan that the change would reduce deaths because "people hit by a car going 25 MPH are half as likely to die as those hit by a car going 30 MPH".

As an illustrative example, we investigate whether the Vision Zero speed limit reduction actually achieved a reduction in fatalities using the potential outcomes approach. We compute the estimated causal effect using a variety of covariates obtained both within the open data and the US Department of Transportation’s General Estimates System accident database, and we comment on the applicability of new and traditional identification strategies and sensitivity analysis. Final estimates are then calculated via poststratification.