Assistant Professor Casey Wichman's paper "RCTs Against the Machine: Can Machine Learning Prediction Methods Recover Experimental Treatment Effects?" was accepted by the Journal of the Association of Environmental and Resource Economists.
"We use various machine learning algorithms to replicate treatment effects from a randomized electricity pricing and information experiment," Wichman said. "We show that machine learning tools can be used to generate counterfactual outcomes to recover treatment effects from a randomized controlled trial (RCT). This work suggests that researchers and policymakers, particularly in energy demand settings, can use observational data and machine learning tools to generate causal treatment effects even when running a full-blown RCT is not feasible."
Abstract
We investigate how successfully ML prediction algorithms can be used to estimate causal treatment effects in electricity demand applications with nonexperimental data. We use three prediction algorithms—XGBoost, random forests, and LASSO — to generate counterfactuals using observational data. Using those counterfactuals, we estimate nonexperimental treatment effects and compare them to experimental treatment effects from a randomized experiment for electricity customers who faced critical-peak pricing and information treatments. Our results show that nonexperimental treatment effects based on each algorithm replicate the true treatment effects, even when only using data from treated households. Additionally, when using both treatment households and nonexperimental comparison households, standard two-way fixed effects regressions replicate the experimental benchmark, suggesting little benefit from ML approaches over standard program evaluation methods in that setting.
Read more: https://www.journals.uchicago.edu/doi/10.1086/724518
Discover more featured research from the School of Economics, and follow us on Facebook, Twitter, Instagram, and LinkedIn to keep up with our students, school news, and upcoming events.