After it... Therefore because of it

The study behind the newspaper article behind the fact check

Today I posted the first in what will be a series called Who Fact Checks the Fact Checkers (here). In that post I promised to return to the matter of the academic study that was the subject of the Daily Mail article.

The Study

The study described in the Daily Mail article is “Impacts of Climate Change and Air Pollution on Neurologic Health, Disease and Practice: A Scoping Review” by Louis et al. from the Journal Neurology.

In the paper, Louis et al. claim on the basis of a study of 364 human studies linking medical conditions with a climate or meteorological phenomena that irreversible environmental changes have already occured and that for the subjects of each of the studies - their neurological conditions are directly caused or exacerbated by the effects of man-made climate change.

The Authors

The lead author on the study was a neurology student (Shreya Louis, top right in the class photograph below which also shows that the earliest she will graduate is 2026). She graduated the MD medical program of a different academic institute in 2022 (here) and was immediately accepted to the neurology program at Stanford on the 14^th of March, 2022 (here) and the article was submitted to the journal Neurology on the 1^st of July, 2022. This suggests that in only around 90 days she led a study that reviewed and critically appraised 364 different clinical studies – at a rate that required qualitative and quantitative evaluation of more than four clinical studies per day. While perhaps not impossible, such a study would normally consume a year or more of an average PhD student’s time. However, and unlike Dr Louis, your average PhD student isn’t also working a normal full-time residency shift load requirement in a clinical setting.

Of the other authors, Abhilash Suresh has a Bachelor of Science and works for the National Institutes for Health (NIH). His academic mentor during his capstone research project in 2015 was Dr Akshay Gupte who was a global health Fellow of the NIH and who acknowledged the assistance of Fauci’s NIAID in her PhD dissertation (here). Joshua Rim lists the NIAID in his affiliations and as a primary funder for his work on other papers (here).

The Method

In reviewing their search terms I attempted to re-run their searches.

For the collection of what seemed to be the primary core terms: [“climate change”, “air pollution” and “neurological disorders”], Medline returned a seemingly reasonable 302 results. However, Google (because the authors describe having performed a general internet search) returned over 31,700. When we replaced “neurological disorders” with “stroke” Medline returned 874 and Google returned 22,100. A search for [“global warming”, “air pollution” and “neurological disorders”] returned 738 results, while replacing “neurological disorders” with “stroke” returned 13,800. It is therefore difficult to believe that the authors only found 3,060 in their initial search when they were using combinations composed from a larger pool of search terms, and that 20% (one in every five) were duplicates. Our searches also resolved that the authors assertion that no additional articles were identified by the addition of a general internet or grey literature search also seems incredible. We found the internet to be replete with articles and posts authored by academics and lay-people that assert links between some climate change topic and the health conditions listed in Louis et al’s paper.   

Inclusion Criteria

When I considered the inclusion criteria for Louis et al’s study, something interesting jumped out. For inclusion, a study had only to use basic quantitative incidence statistics for one of the identified diseases and contrast that to historic climate event or meteorological data. Most studies simply looked at the number of hospitalisations of people with exacerbation of existing medical conditions during periods of weather phenomena of interest. Without in-depth analysis, studies could easily end up being included that draw conclusions based on correlation (association) or even mere coincidence. There was no requirement for a study to actually demonstrate the climate or meteorological event was even a necessary factor in the clinical diagnosis (causation).

The Results

The paper goes on to describe how different studies describe an increase of particular health events in correlation to specific weather conditions. For example:

• For stroke they identified 24 studies. Six that reported the incidence of strokes increased with high temperatures and humidity, and two that reported strokes in low temperatures. However, it is notable that towards the end of the section on stroke they appear to report that only two of the 24 stroke studies considered climate and pollution (greenhouse gas emissions) and both did so from the perspective of projecting, or modelling, what might happen in the mid-to-late 21^st century (beyond 2050).

• For dementia they identified five studies. Two sought to test for an association between hospital admissions and temperature. Two others showed that people with dementia are often at higher risk of injury or death due to extreme heat events such as bushfire. The final study only suggested that heat and particulate matter may increase the risk of hospitalisation, again for people with dementia.

The evidence and analysis for the remaining health conditions continues along the same lines.

Conclusion

Even if we ignore the issues with: (i) lax inclusion criteria; (ii) mixing of predictive/modelling studies with retrospective clinical case reviews; and (iii) that somehow they performed and refined a literature search that began with over 3,000 papers and resolved that down to a collection of 364 papers that they read, reviewed and wrote up - all in 90 days, there are logical fallacies that simply cannot be overlooked.

Each study is treated as an example of (x) an integrated form of representativeness heuristic called prototype theory wrapped in (y) post hoc ergo proctor hoc.

That because exacerbation of the medical condition followed change in a phenomena under observation (temperature, bushfire, humidity, smoke or particulates from the bushfire, chemical exposure etc.):

x: the phenomena is a component of or related to climate change, and

y: therefore climate change caused the medical condition.

Ultimately the Louis et al. study is garbage - and was entirely unworthy of the headlines that were generated on its behalf.