Indefinite social action indeed. The CDC states “No safe blood lead level in children has been identified.” Based on that logic, even if blood lead levels drop to ng/L or even pg/L, they’ll still be “too high.” Thank you for the very interesting post.
My impression is that in Southern California there is a negative correlation between the prestige of a school, public or private, and it's proximity to a freeway. In the San Fernando Valley, the well-known schools tend to be located near the 101 Freeway and the less prestigious schools are off in the hinterlands.
I'm fascinated that lead poisoning rates appear so low in the Hispanic population, which often isn't a high performer in things like wealth-poverty, academic achievement, etc.. There must be more of a story there...
So this data you refer to and, I would argue, successfully debunk appears to all be correlational data. What do you think then of Keyes et al. (2023) paper which uses the quasi-random variation in exposure to lead following the Clean Air Act in the 1970s? In the paper they control for a lot of these confounding factors you identify as well and still find positive results. I wouldn't attempt to defend any of the correlational data you cite above, but this paper seems to have a fairly robust identification strategy (although I'd love to hear if you think I am missing something).
Studies like Buser & Scinicariello (2017) also control for these confounders. The study doesn't have a (quasi) causal identification strategy like Keyes, but if they are controlling for this wide suite of confounders (as you suggest the problem is) then do you think these results have some significance and weight, or will there always be confounders not controlled for in your opinion?
The first study has a lot of bunching of p-values just below 0.05 and its strategy seemingly relies on rather than obviates confounding. I am curious about why they only used two of the available tests.
The second study makes no pretense about being causally informative and leaves plenty of room on the table for residual confounding, with respect to a different phenotype.
Thanks for your reply! I must admit I am a bit lost on your reasoning so I hope it's ok if I ask for some clarification.
First, can you explain why you think there is a bunching of p-values just below 0.05? I don't see that, most coefficient values seem to be valid at the 1% significance level. Only 3 out of 12 calculated coefficients are significant at only the 5% level, the rest are at the 1% level, and of those three significant at the 5% level I see only one which is close to the threshold.
Second, can you explain why you think their strategy relies on confounding?
Finally, which tests are you referring to when you say they use only two of the available tests? What other tests are available that you are referring to?
Only look at the relevant specifications, not all specifications and tests. In any case, this study does not really provide causal identification, hence it's relying on rather than obviating the issue of confounding. There's no measurement of BLLs, there's likely-confounded imputation.
I am referring to the fact that they only used two of the available subtests for some reason.
It is kind of crazy this literature doesn't seem to have more RDD or DID studies. It seems like this is the perfect area for this type of study. Like causal effects shouldn't be that difficult to discern here. I am unsure why so many studies have only pulled used correlational relationships
The physiological effects of lead may be complicated by co-factors that vary across different groups and that, when last I checked, have toxicologists genuinely puzzled. I guess all the toxicology studies which have found a link might also be the result of confounding + confirmation bias etc. and the ones that haven't aren't. Have to find out what methodologies the toxicology studies use. If they differ significantly from those used by the social scientists, then this may not be a case of 'either / or' but 'both, depending'.
When you look at both factors, I found a 0.99d gap using averaged scores across math, reading and science, and 1.13 composite d(between 3rd+ generation blacks and whites).
Indefinite social action indeed. The CDC states “No safe blood lead level in children has been identified.” Based on that logic, even if blood lead levels drop to ng/L or even pg/L, they’ll still be “too high.” Thank you for the very interesting post.
My impression is that in Southern California there is a negative correlation between the prestige of a school, public or private, and it's proximity to a freeway. In the San Fernando Valley, the well-known schools tend to be located near the 101 Freeway and the less prestigious schools are off in the hinterlands.
I'm fascinated that lead poisoning rates appear so low in the Hispanic population, which often isn't a high performer in things like wealth-poverty, academic achievement, etc.. There must be more of a story there...
So this data you refer to and, I would argue, successfully debunk appears to all be correlational data. What do you think then of Keyes et al. (2023) paper which uses the quasi-random variation in exposure to lead following the Clean Air Act in the 1970s? In the paper they control for a lot of these confounding factors you identify as well and still find positive results. I wouldn't attempt to defend any of the correlational data you cite above, but this paper seems to have a fairly robust identification strategy (although I'd love to hear if you think I am missing something).
Studies like Buser & Scinicariello (2017) also control for these confounders. The study doesn't have a (quasi) causal identification strategy like Keyes, but if they are controlling for this wide suite of confounders (as you suggest the problem is) then do you think these results have some significance and weight, or will there always be confounders not controlled for in your opinion?
Links?
Sorry yes
Keyes et al. (2023) https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4462722
Buser & Scinicariello (2017) https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8117117/pdf/nihms-1698543.pdf
The first study has a lot of bunching of p-values just below 0.05 and its strategy seemingly relies on rather than obviates confounding. I am curious about why they only used two of the available tests.
The second study makes no pretense about being causally informative and leaves plenty of room on the table for residual confounding, with respect to a different phenotype.
For a causally-informative estimate of the effect of measured blood lead levels, see, for example, this sibling analysis: https://x.com/cremieuxrecueil/status/1781821837950726575
Thanks for your reply! I must admit I am a bit lost on your reasoning so I hope it's ok if I ask for some clarification.
First, can you explain why you think there is a bunching of p-values just below 0.05? I don't see that, most coefficient values seem to be valid at the 1% significance level. Only 3 out of 12 calculated coefficients are significant at only the 5% level, the rest are at the 1% level, and of those three significant at the 5% level I see only one which is close to the threshold.
Second, can you explain why you think their strategy relies on confounding?
Finally, which tests are you referring to when you say they use only two of the available tests? What other tests are available that you are referring to?
Only look at the relevant specifications, not all specifications and tests. In any case, this study does not really provide causal identification, hence it's relying on rather than obviating the issue of confounding. There's no measurement of BLLs, there's likely-confounded imputation.
I am referring to the fact that they only used two of the available subtests for some reason.
Hey, just pinging you on this because I'm interested in what I am missing
It is kind of crazy this literature doesn't seem to have more RDD or DID studies. It seems like this is the perfect area for this type of study. Like causal effects shouldn't be that difficult to discern here. I am unsure why so many studies have only pulled used correlational relationships
The physiological effects of lead may be complicated by co-factors that vary across different groups and that, when last I checked, have toxicologists genuinely puzzled. I guess all the toxicology studies which have found a link might also be the result of confounding + confirmation bias etc. and the ones that haven't aren't. Have to find out what methodologies the toxicology studies use. If they differ significantly from those used by the social scientists, then this may not be a case of 'either / or' but 'both, depending'.
How to interpret that black-white gap is 0,8 d in PISA?
https://nces.ed.gov/surveys/pisa/pisa2018/index.asp#/
There are a few issues with just using the mean and sd for PISA(as well as other academic tests)
1. Generational effects, it seems like recent Black immigrants do better that legacy African Americans, for example a review in 2014 found a 0.99d for 3rd+ generation Blacks vs 2nd generation at 0.84d below 3rd+ generation Whites(https://www.researchgate.net/publication/301293927_EthnicRace_Differences_in_Aptitude_by_Generation_in_the_United_States_An_Exploratory_Meta-analysis), and recent data from PISA and the ABCD sample seems to indicate a gap of 0.6-0.7d.
2. Using averaged scores instead of sum scores. Because of the imperfect correlation between tests , composite scores are ideally used(fairly standard for IQ tests, see https://www.hmhco.com/~/media/sites/home/hmh-assessments/clinical/woodcock-johnson/pdf/wjiv/wjiv_asb_7.pdf), which are more extreme than the sum of their parts).
When you look at both factors, I found a 0.99d gap using averaged scores across math, reading and science, and 1.13 composite d(between 3rd+ generation blacks and whites).