This website uses cookies.

By clicking the "Accept" button or continuing to browse our site, you agree to first-party and session-only cookies being stored on your device to enhance site navigation and analyze site performance and traffic. For more information on our use of cookies, please see our Privacy Policy.

  1. Home
  2. Journals
  3. Journal of Economic Literature
  4. September 2021
  5. Automated Linking of Historical Data

Journal of Economic Literature

ISSN 0022-0515 (Print) | ISSN 2328-8175 (Online)

Automated Linking of Historical Data

  • Ran Abramitzky
  • Leah Boustan
  • Katherine Eriksson
  • James Feigenbaum
  • Santiago Pérez
Journal of Economic Literature
vol. 59, no. 3, September 2021
(pp. 865–918)
Download Full Text PDF

Abstract

The recent digitization of complete count census data is an extraordinary opportunity for social scientists to create large longitudinal datasets by linking individuals from one census to another or from other sources to the census. We evaluate different automated methods for record linkage, performing a series of comparisons across methods and against hand linking. We have three main findings that lead us to conclude that automated methods perform well. First, a number of automated methods generate very low (less than 5 percent) false positive rates. The automated methods trace out a frontier illustrating the trade-off between the false positive rate and the (true) match rate. Relative to more conservative automated algorithms, humans tend to link more observations but at a cost of higher rates of false positives. Second, when human linkers and algorithms use the same linking variables, there is relatively little disagreement between them. Third, across a number of plausible analyses, coefficient estimates and parameters of interest are very similar when using linked samples based on each of the different automated methods. We provide code and Stata commands to implement the various automated methods.

Citation

Abramitzky, Ran, Leah Boustan, Katherine Eriksson, James Feigenbaum, and Santiago Pérez. 2021. "Automated Linking of Historical Data." Journal of Economic Literature, 59 (3): 865–918. DOI: 10.1257/jel.20201599

Additional Materials

JEL Classification

  • C81 Methodology for Collecting, Estimating, and Organizing Microeconomic Data; Data Access
  • C83 Survey Methods; Sampling Methods
  • N01 Development of the Discipline: Historiographical; Sources and Methods
  • N31 Economic History: Labor and Consumers, Demography, Education, Health, Welfare, Income, Wealth, Religion, and Philanthropy: U.S.; Canada: Pre-1913
  • N32 Economic History: Labor and Consumers, Demography, Education, Health, Welfare, Income, Wealth, Religion, and Philanthropy: U.S.; Canada: 1913-