What Did We Learn?

Two articles in this issue of the AEJ Applied reported results from similar analyses that investigated whether having a medical professional (physician) in the family affects health of family members. Chen et al. examined this question for Sweden and Artmann et al. examined this question in Netherlands. Both studies leverage an as good as random assignment process that governs admission to medical schools. Therefore, estimates of the effect of having a physician in the family are plausibly interpreted as causal.

Chen et al. conclude: “we show that having a doctor in the family raises preventive health investments throughout the life cycle, improves physical health, and prolongs life.” Artmann et al. conclude: “Our findings indicate that informal access to medical expertise and services is not an important cause of differences in health care use and mortality.”

Unfortunately, both studies lack statistical power, which makes the findings arguably uninformative. Consider results from Chen et al. (Table 2). The precision of the instrumental variable estimates indicate that the analysis can reliably detect effect sizes (relative to the control complier mean) of 88% (heart attack); 70% (heart failure); 367% (lung cancer); and 81% (Type 2 diabetes). Given this lack of power, it seems imprudent to conclude that a physician in the family improves health (see Gelman and Carlin 2014). The precision of estimates in Artmann et al., is better, but still sufficient to detect relatively large effects. The precision of the instrumental variable estimates (Table 7) indicate that the analysis can reliably detect effect sizes (relative to the control complier mean) of 18% and 25% for father’s and mother’s mortality, respectively. Is it plausible that a physician in the family can reduce mortality by around 20% among a relatively wealthy, well-insured population? Of note, the better powered study (Artmann et al.) finds no statistically significant effects, which is consistent with the criticism of Gelman and Carlin (2014).

It is easy to see why these studies passed editorial review. Both studies were based on a research design that, if powered sufficiently, had the potential to produce informative results. However, Chen et al was clearly grossly under powered and Artmann et al. was, in my opinion, under powered given my priors about a plausible effect size. The lack of reviewer attention to statistical power is a common occurrence in economics that leads to a less informative body of research with often misleading conclusions.