This website uses cookies.

By clicking the "Accept" button or continuing to browse our site, you agree to first-party and session-only cookies being stored on your device to enhance site navigation and analyze site performance and traffic. For more information on our use of cookies, please see our Privacy Policy.

« Back to Results

Econometrics of Policy Evaluation

Paper Session

Sunday, Jan. 6, 2019 10:15 AM - 12:15 PM

Atlanta Marriott Marquis, A601
Hosted By: American Economic Association
  • Chair: Jeffrey Wooldridge, Michigan State University

When Should You Adjust Standard Errors for Clustering?

Alberto Abadie
,
Massachusetts Institute of Technology
Susan Athey
,
Stanford University
Guido Imbens
,
Stanford University
Jeffrey Wooldridge
,
Michigan State University

Abstract

In empirical work in economics, it is common to report standard errors that account for clustering of units. Typically, the motivation given for the clustering adjustments is that unobserved components in outcomes for units within clusters are correlated. However, because correlation may occur across more than one dimension, this motivation makes it difficult to justify why researchers use clustering in some dimensions, such as geographic, but not others, such as age cohorts or gender. It also makes it difficult to explain why one should not cluster with data from a randomized experiment. In this paper, we argue that clustering is in essence a design problem, either a sampling design or an experimental design issue. It is a sampling design issue if sampling follows a two stage process where in the first stage, a subset of clusters were sampled randomly from a population of clusters, while in the second stage, units were sampled randomly from the sampled clusters. In this case, the clustering adjustment is justified by the fact that there are clusters in the population that we do not see in the sample. Clustering is an experimental design issue if the assignment is correlated within the clusters. We take the view that this second perspective best fits the typical setting in economics where clustering adjustments are used. This perspective allows us to shed new light on three questions: (i) when should one adjust the standard errors for clustering, (ii) when is the conventional adjustment for clustering appropriate, and (iii) when does the conventional adjustment of the standard errors matter.

Optimal Estimation when Researcher and Social Preferences are Misaligned

Jann Spiess
,
Microsoft Research

Abstract

Econometric analysis typically focuses on the statistical properties of fixed estimators and ignores researcher choices. In this article, I approach the analysis of experimental data as a mechanism-design problem that acknowledges that researchers choose between estimators, sometimes based on the data and often according to their own preferences. Specifically, I focus on covariate adjustments, which can increase the precision of a treatment-effect estimate, but open the door to bias when researchers engage in specification searches. First, I establish that unbiasedness is a requirement on the estimation of the average treatment effect that aligns researchers’ preferences with the minimization of the mean-squared error relative to the truth, and that fixing the bias can yield an optimal restriction in a minimax sense. Second, I provide a constructive characterization of all treatment-effect estimators with fixed bias as sample-splitting procedures. Third, I show how these results imply flexible pre-analysis plans for randomized experiments that include beneficial specification searches and offer an opportunity to leverage machine learning.

Deep Inference: AI for Structural Estimation

Tetsuya Kaji
,
University of Chicago
Elena Manresa
,
New York University
Guillaume Pouliot
,
University of Chicago

Abstract

We propose a new estimation method for structural models based on Artificial Intelligence tools. The approach leverages on the availability of modern pattern recognition methods, discriminators, that can accurately distinguish between real data from generated data using a fully specified model and a particular choice of structural parameter values. The estimator is defined as the value of the structural parameters for which the discriminator is unable to distinguish between the true data and the corresponding generated data. Different types of discriminators define different estimators and we show that when using a logit model as a discriminator the estimator is asymptotically equivalent to the popular optimally weighted simulated method of moments (e.g. Gourieroux, Monfort, and Renault (1993)). Discriminators based on Neural Networks with one or multiple hidden layers can provide more efficient estimators. We showcase the good properties of the proposed method using simulated data from a two-period Roy Model where heterogeneous individuals can choose to work between two locations in exchange of wages, and there are returns to experience. Compared to an indirect inference estimator with optimal weighting, our estimator, when using a logit as a discriminator, has smaller standard deviation at no additional computational cost.
JEL Classifications
  • C2 - Single Equation Models; Single Variables