What Did We Learn?

Two articles in this issue of the AEJ Applied reported results from similar analyses that investigated whether having a medical professional (physician) in the family affects health of family members. Chen et al. examined this question for Sweden and Artmann et al. examined this question in Netherlands. Both studies leverage an as good as random assignment process that governs admission to medical schools. Therefore, estimates of the effect of having a physician in the family are plausibly interpreted as causal.

Chen et al. conclude: “we show that having a doctor in the family raises preventive health investments throughout the life cycle, improves physical health, and prolongs life.” Artmann et al. conclude: “Our findings indicate that informal access to medical expertise and services is not an important cause of differences in health care use and mortality.”

Unfortunately, both studies lack statistical power, which makes the findings arguably uninformative. Consider results from Chen et al. (Table 2). The precision of the instrumental variable estimates indicate that the analysis can reliably detect effect sizes (relative to the control complier mean) of 88% (heart attack); 70% (heart failure); 367% (lung cancer); and 81% (Type 2 diabetes). Given this lack of power, it seems imprudent to conclude that a physician in the family improves health (see Gelman and Carlin 2014). The precision of estimates in Artmann et al., is better, but still sufficient to detect relatively large effects. The precision of the instrumental variable estimates (Table 7) indicate that the analysis can reliably detect effect sizes (relative to the control complier mean) of 18% and 25% for father’s and mother’s mortality, respectively. Is it plausible that a physician in the family can reduce mortality by around 20% among a relatively wealthy, well-insured population? Of note, the better powered study (Artmann et al.) finds no statistically significant effects, which is consistent with the criticism of Gelman and Carlin (2014).

It is easy to see why these studies passed editorial review. Both studies were based on a research design that, if powered sufficiently, had the potential to produce informative results. However, Chen et al was clearly grossly under powered and Artmann et al. was, in my opinion, under powered given my priors about a plausible effect size. The lack of reviewer attention to statistical power is a common occurrence in economics that leads to a less informative body of research with often misleading conclusions.

We appreciate that Professor Kaestner has commented on the apparently contradicting findings from our paper and the closely related paper of Chen et al. Professor Kaestner concludes first that “both studies lack statistical power” after which he states “The precision of estimates in Artmann et al., is better, but still sufficient to detect relatively large effects.” and “Artmann et al. was, in my opinion, under powered given my priors about a plausible effect size.” We think that these remarks do not do justice the strength of the evidence presented in our paper.

Professor Kaestner infers that our empirical analysis can reliably detect effect sizes (relative to the control complier mean) of 18% and 25% for father’s and mother’s mortality, respectively. The 18% for fathers is based on a comparison of a point estimate of -0.0042 minus twice a standard error of 0.0151 and a control complier mean of 0.1708. For mothers, the respective numbers are: 0.0004, 0.0122 and 0.0965. We argue that estimates and standard errors of this magnitude are typically regarded as precisely estimated zeros. Furthermore, estimates from a proportional hazard model exclude reductions of the hazard rate larger than 6% for fathers and 7.5% for mothers with 95% probability. The Chi-square value from a Wilcoxon rank-sum test for equality of the survivor functions of parents of lottery winners and lottery losers equals 0.00 (p=0.98) for fathers and 0.45 (p=0.50) for mothers. The test-statistic for fathers states that distributions of mortality ages are completely identical, which holds also almost for mothers.

We complement our findings on parents’ mortality with estimates of the effects on a large number of variables related to health care costs and use. Many of these estimates are precisely estimated zeros. For example, the effect on GP visits by fathers is -0.0137 (s.e. 0.0091) and by mothers is -0.0023 (s.e. 0.0084), which should be compared to complier means of 0.8299 for fathers and 0.8592 or mothers. Finally, our study is more than sufficiently powered to rule out the estimates reported by Chen et al.