Association for Public Policy Analysis & Management

The growing interest for behavioral interventions in the energy domain has captured the attention of governmental and private agencies, increasing the importance of rigorous evaluations. Even when random assignment is assured, another problem with many interventions is the potential confounding artifact of people changing their behavior because of participation in the program per se, rather than because of their response to the substance of the intervention. This has been referred to as the Hawthorne Effect. In fact, the Hawthorne Effect has been reported as a limitation in both large and small scale energy-saving interventions.

Here, we conducted a large randomized controlled trial to examine the Hawthorne Effect with residential electricity customers. After this field study, we also conducted a survey with a subsample of the participants to further examine changes in people’s perceptions in response to the intervention. Finally, we examine potential Hawthorne effects in a natural experiment - installing household smart meters that track hourly electricity usage, prior to activation. The results of these different methodological approaches (field experiment, survey, and natural experiment) shed light on the extent and sources of Hawthorne Effects in energy programs, and other domains.

In the field experiment, participants (N = 6,350) were randomly assigned to treatment and control groups. Households in the treatment group received five weekly postcards notifying, and then reminding, them of their participation in a one-month study of electricity use. No incentives or energy information were provided. The control group did not receive any notification. Monthly electricity usage was collected before, during, and after the experimental period for both treatment and control groups.

We found that the in average households used 2.2% - 2.9% less electricity during the month of the study compared to the control group – a significant amount compared to the annual conservation goal currently mandated by any state in the U.S. Interestingly, this treatment effect vanished when the intervention ended. Responses to the follow-up survey indicated that people perceived the study as having heightened their awareness of electricity usage. This interpretation is consistent with the field experiment's lack of post-treatment effect. Finally, we found little evidence of a smart meters installation effect. This suggests that not all interventions may increase energy awareness as the field experiment did. We also discuss some methodological issues of this last result as this comes from a natural experiment.

Our research suggests some policy recommendations for programs designed to reduce energy consumption; although typically treated as a confounding artifact, the Hawthorne Effect can also be treated as an intervention that is important in its own right. Indeed, in health research, the Hawthorne Effect has been proposed as a tool for improving health outcomes based on awareness of external observation. In the current research, our results suggest that the effect of study participation, or the mere participation in an intervention or program, increases saliency of a focal behavior, such as energy consumption, inducing savings. We discuss ways to maintain awareness in novel interventions.