« Back to Results
Marriott Marquis, Rancho Santa Fe 2
Hosted By:
American Economic Association
Achievement Tests I: On the Validity of Comparisons across Cohort, Grade, and Subject
Paper Session
Saturday, Jan. 4, 2020 8:00 AM - 10:00 AM (PDT)
- Chair: Kevin Lang, Boston University
Knowledge, Tests, and Fadeout in Educational Interventions
Abstract
Educational interventions are often evaluated and compared on the basis of their impacts on test scores. Decades of research have produced two empirical regularities: interventions in later grades tend to have smaller effects than the same interventions in earlier grades, and the test score impacts of early educational interventions almost universally "fade out" over time. This paper explores whether these empirical regularities are an artifact of the common practice of rescaling test scores in terms of a student's position in a widening distribution of knowledge. If a standard deviation in test scores in later grades translates into a larger difference in knowledge, an intervention's effect on normalized test scores may fall even as its effect on knowledge does not. We evaluate this hypothesis by fitting a model of education production to correlations in test scores across grades and with college-going using both administrative and survey data. Our results imply that the variance in knowledge does indeed rise as children progress through school, but not enough for test score normalization to fully explain these empirical regularities.Is Intervention Fadeout a Scaling Artefact?
Abstract
To determine whether fadeout of cognitive impacts in early education interventions might reflect scaling decisions, we reanalyze a well-known RCT of an early mathematics intervention which showed almost complete fadeout over the study period. We examine how various order-preserving transformations of the scale affect how the relative mathematics achievement of the control and experimental groups changes with age. A fadeout-eliminating scale requires math achievement at the upper-end of the achievement distribution to be much more important than elsewhere in the scale. While we cannot rule this out, it would have substantial implications for interpreting the effects of educational interventions.Closing the SES Achievement Gap: Trends in U.S. Student Performance
Abstract
Concerns about limited intergenerational mobility have led to a focus on educational achievement gaps by socio-economic status (SES). Using intertemporally linked assessments from NAEP, TIMSS, and PISA, we trace the achievement of U.S. student cohorts born between 1954 and 2001. Achievement gaps between the top and bottom quartiles of the SES distribution have been large and remarkably constant for a near half century. These unwavering gaps have not been offset by overall improvements in achievement levels, which have risen at age 14 but remained unchanged at age 17 for the most recent quarter century. The long-term failure of major educational policies to alter SES gaps suggests a need to reconsider standard approaches to educational mitigating disparities.Discussant(s)
Dev Patel
,
Harvard University
Drew H. Bailey
,
University of California-Irvine
Jeffrey Wooldridge
,
Michigan State University
Markus Broer
,
American Institutes for Research
JEL Classifications
- I2 - Education and Research Institutions
- C1 - Econometric and Statistical Methods and Methodology: General