Michael Ejercito
2024-09-08 15:34:26 UTC
https://www.reddit.com/r/LockdownSkepticism/comments/1fasuje/always_check_the_denominator_no_the_risk_of/
Always Check the Denominator
No, the risk of critical Covid disease is not 4% among healthy children.
Kelley K
May 30, 2024
I’ve been very busy lately with my actual job and some other things, so
I haven’t been writing much lately. I paused paid subscriptions for the
summer to try and get caught up. I’m not taking the summer off - I plan
to start writing again even while payments are paused. Hopefully you’ll
be hearing more from me soon! Today seemed like a good day to start…
I woke up to this outrageous claim from CIDRAP in my Twitter feed:
This is not the first time that I’ve written about CIDRAP amplifying
misleading claims from a Covid study to amp up fears about Covid and
kids. They also misreported a study about Long Covid, claiming that 84%
of Covid survivors had Long Covid symptoms 2 years later. Like that
ridiculous Long Covid claim, the claim from their current tweet and
article on Covid risks among kids just doesn’t pass a basic sanity
check. Michael Mina did some basic math to show how absurd this claim is:
Unfortunately, the reporters at CIDRAP (along with many other Covid
journalists and even some scientists) seem to lack the common sense to
do these kinds of sanity checks on the data — because this keeps happening.
A Meta-Analysis of “Risk Factors for Pediatric Critical COVID-19”
The study, “Risk Factors for Pediatric Critical COVID-19: A Systematic
Review and Meta-Analysis” is primarily about which comorbidities are
risk factors for severe Covid — it’s focused on severe cases and kids
with comorbidities. However, the study does include the 4% claim cited
by CIDRAP. I can’t access the full text of the published paper from the
Journal of the Pediatric Infectious Diseases Society, but the study’s
abstract does state: “In previously healthy children, the absolute risk
of critical disease from COVID-19 was 4% (95% CI, 1%-10%).” I wanted
more details of the underlying data, so I found the pre-print of the
study, and the following section of the Results is where this 4%
statistic comes from:
Figure 4 from the study
This small section is the only mention of outcomes in children without
comorbidities, and it doesn’t include nearly enough caveats about the
underlying data. It is a poor conclusion to draw and shouldn’t have been
included in the abstract at all in my opinion. The abstract even says
that “the absolute risk for critical COVID-19 in children and
adolescents without underlying health conditions is relatively low,”
which doesn’t seem consistent with a 4% “absolute risk of critical
disease” from their results. In reality, the absolute risk among
previously healthy children is at least an order of magnitude lower than
that.
Underlying Study Data
It should be obvious to the casual observer that the 4% was calculated
from a population that doesn’t represent typical healthy children. So I
looked up each of the 6 studies included in this subset of studies from
the meta-analysis. Of the 6 studies that looked at medical complexity, 4
were among hospitalized children (3 of the 4 in Brazil)1, one was among
12-17-year-olds with symptomatic Covid recorded in medical records2, and
one was a large study of children with documented Covid in Mexico3.
Outcomes for Children without Comorbidities (data extracted from Figure
4 above, population descriptions from underlying studies)
Also, all of the studies were from early in the pandemic (2020 and part
of 2021), so they were primarily first infections, they were primarily
pre-vaccine, and all were pre-Omicron. So they don’t tell us much about
the current situation, even among children who may be hospitalized with
Covid in 2024.
The large study in Mexico included over 130,000 children and found 0.3%
of children with documented Covid infections and no comorbidities died
between March 2020 and mid-June 2021. And that is still biased to more
severe cases, because many asymptomatic and mild infections aren’t
documented in medical records. Despite the overwhelming findings from
this large study of children from the broader community (not just
hospitalized children), the study authors calculated an absolute risk
from a small subset of children from these 6 studies (less than 16,000
patients). They don’t provide any further details the subset that they
used for their calculations, but clearly the studies of hospitalized
children and symptomatic teens skewed the results significantly.
Denominators Matter
There’s been much discussion about the importance of denominators
throughout the pandemic, but scientists, journalists, and others keep
making the same mistakes. When we use percentages to talk about Covid
outcomes or other things, we’re talking about a fraction of the
population. But it’s super important to understand WHICH population
we’re talking about. Outcomes among hospitalized patients, or among the
elderly, differ greatly from outcomes in the general population.
Just like the percentage of adults who ride mountain bikes is much
higher among attendees at a mountain biking race than among all adults,
the percentage of children with critical Covid is obviously much higher
among children who are hospitalized for Covid than among all children.
We see this same error with Long Covid studies, where the findings are
percentages among patients at a Long Covid clinic, or among patients who
already have Long Covid, but are often reported as if they are
percentages among a representative sample of the population.
The percentage of children with critical Covid outcomes is obviously
much higher among children who are hospitalized for Covid than among all
children.
The Sad State of Covid Science and Journalism
The authors of this study were irresponsible to include the claim that
“the absolute risk of critical disease from COVID-19 was 4%” among
healthy children without comorbidities, without explaining that their
data was heavily biased toward patients already hospitalized with COVID.
Without that caveat, the 4% claim is a gross misrepresentation of the
underlying data. And this study isn’t from some obscure scientist at a
no-name institution — the senior author, Dr. Carlos Oliveira, is an
Assistant Professor of Pediatrics, Biostatistics, and Biomedical
Informatics & Data Science at Yale! Surely he knows that 4% of American
children didn’t have severe Covid outcomes. Why include that misleading
statistic in the paper?!?
I also think the writers at CIDRAP needs to do a better job of
understanding and vetting the studies they report on to make sure their
reporting is accurate. They are a trusted voice for many people, and
their studies often get amplified by the Zero Covid crowd on Twitter. It
is profoundly unhelpful to continue scaring people with these misleading
and unrealistic statistics. Journalists need to understand the harm they
cause by publishing these irresponsible claims.
By the way… I’m already seeing this same error with H5N1 (bird flu) in
the media as well, and will probably write more on that later. Many news
reports refer to a 50% “fatality rate”, but that’s only among known
cases that were previously documented. Among the 3 cases identified so
far in the US (one in 2022 and two this year), none have died. The two
known cases this year have only experienced eye infections.
UPDATE: The moment I published this, I saw that another case of H5N1 has
been identified in the US. That patient did have respiratory symptoms
but is recovering. (5/30/24)
Thanks for reading Check Your Work! Subscribe for free to receive future
posts.
Type your email...
Subscribe
1
(A) Ward JL, Harwood R, Smith C, Kenny S, Clark M, Davis PJ, et al. Risk
factors for PICU admission and death among children and young people
hospitalized with COVID-19 and PIMS-TS in England during the first
pandemic year. Nat Med. 2022 Jan;28(1):193–200.
(B) Oliveira EA, Colosimo EA, Simoes ESAC, Mak RH, Martelli DB, Silva
LR, et al. Clinical characteristics and risk factors for death among
hospitalised children and adolescents with COVID-19 in Brazil: an
analysis of a nationwide database. Lancet Child Adolesc Health. 2021
Aug;5(8):559–68.
(C) Hendler JV, Miranda do Lago P, Muller GC, Santana JC, Piva JP, Daudt
LE. Risk factors for severe COVID-19 infection in Brazilian children.
Braz J Infect Dis. 2021 Nov-Dec;25(6):101650
(D) Horta M, Ribeiro GJC, Campos NOB, de Oliveira DR, de Almeida
Carvalho LM, de Castro Zocrato K, et al. ICU Admission, Invasive
Mechanical Ventilation, and Mortality among Children and Adolescents
Hospitalized for COVID-19 in a Private Healthcare System. Int J Pediatr.
2023;2023:1698407.
2
Campbell JI, Dubois MM, Savage TJ, Hood-Pishchany MI, Sharma TS, Petty
CR, et al. Comorbidities Associated with Hospitalization and Progression
Among Adolescents with Symptomatic Coronavirus Disease 2019. J Pediatr.
2022 Jun;245:102–10 e2.
3
Sanchez-Piedra C, Gamino-Arroyo AE, Cruz-Cruz C, Prado-Galbarro FJ.
Impact of environmental and individual factors on COVID-19 mortality in
children and adolescents in Mexico: An observational study. Lancet Reg
Health Am. 2022 Apr;8:100184.
Always Check the Denominator
No, the risk of critical Covid disease is not 4% among healthy children.
Kelley K
May 30, 2024
I’ve been very busy lately with my actual job and some other things, so
I haven’t been writing much lately. I paused paid subscriptions for the
summer to try and get caught up. I’m not taking the summer off - I plan
to start writing again even while payments are paused. Hopefully you’ll
be hearing more from me soon! Today seemed like a good day to start…
I woke up to this outrageous claim from CIDRAP in my Twitter feed:
This is not the first time that I’ve written about CIDRAP amplifying
misleading claims from a Covid study to amp up fears about Covid and
kids. They also misreported a study about Long Covid, claiming that 84%
of Covid survivors had Long Covid symptoms 2 years later. Like that
ridiculous Long Covid claim, the claim from their current tweet and
article on Covid risks among kids just doesn’t pass a basic sanity
check. Michael Mina did some basic math to show how absurd this claim is:
Unfortunately, the reporters at CIDRAP (along with many other Covid
journalists and even some scientists) seem to lack the common sense to
do these kinds of sanity checks on the data — because this keeps happening.
A Meta-Analysis of “Risk Factors for Pediatric Critical COVID-19”
The study, “Risk Factors for Pediatric Critical COVID-19: A Systematic
Review and Meta-Analysis” is primarily about which comorbidities are
risk factors for severe Covid — it’s focused on severe cases and kids
with comorbidities. However, the study does include the 4% claim cited
by CIDRAP. I can’t access the full text of the published paper from the
Journal of the Pediatric Infectious Diseases Society, but the study’s
abstract does state: “In previously healthy children, the absolute risk
of critical disease from COVID-19 was 4% (95% CI, 1%-10%).” I wanted
more details of the underlying data, so I found the pre-print of the
study, and the following section of the Results is where this 4%
statistic comes from:
Figure 4 from the study
This small section is the only mention of outcomes in children without
comorbidities, and it doesn’t include nearly enough caveats about the
underlying data. It is a poor conclusion to draw and shouldn’t have been
included in the abstract at all in my opinion. The abstract even says
that “the absolute risk for critical COVID-19 in children and
adolescents without underlying health conditions is relatively low,”
which doesn’t seem consistent with a 4% “absolute risk of critical
disease” from their results. In reality, the absolute risk among
previously healthy children is at least an order of magnitude lower than
that.
Underlying Study Data
It should be obvious to the casual observer that the 4% was calculated
from a population that doesn’t represent typical healthy children. So I
looked up each of the 6 studies included in this subset of studies from
the meta-analysis. Of the 6 studies that looked at medical complexity, 4
were among hospitalized children (3 of the 4 in Brazil)1, one was among
12-17-year-olds with symptomatic Covid recorded in medical records2, and
one was a large study of children with documented Covid in Mexico3.
Outcomes for Children without Comorbidities (data extracted from Figure
4 above, population descriptions from underlying studies)
Also, all of the studies were from early in the pandemic (2020 and part
of 2021), so they were primarily first infections, they were primarily
pre-vaccine, and all were pre-Omicron. So they don’t tell us much about
the current situation, even among children who may be hospitalized with
Covid in 2024.
The large study in Mexico included over 130,000 children and found 0.3%
of children with documented Covid infections and no comorbidities died
between March 2020 and mid-June 2021. And that is still biased to more
severe cases, because many asymptomatic and mild infections aren’t
documented in medical records. Despite the overwhelming findings from
this large study of children from the broader community (not just
hospitalized children), the study authors calculated an absolute risk
from a small subset of children from these 6 studies (less than 16,000
patients). They don’t provide any further details the subset that they
used for their calculations, but clearly the studies of hospitalized
children and symptomatic teens skewed the results significantly.
Denominators Matter
There’s been much discussion about the importance of denominators
throughout the pandemic, but scientists, journalists, and others keep
making the same mistakes. When we use percentages to talk about Covid
outcomes or other things, we’re talking about a fraction of the
population. But it’s super important to understand WHICH population
we’re talking about. Outcomes among hospitalized patients, or among the
elderly, differ greatly from outcomes in the general population.
Just like the percentage of adults who ride mountain bikes is much
higher among attendees at a mountain biking race than among all adults,
the percentage of children with critical Covid is obviously much higher
among children who are hospitalized for Covid than among all children.
We see this same error with Long Covid studies, where the findings are
percentages among patients at a Long Covid clinic, or among patients who
already have Long Covid, but are often reported as if they are
percentages among a representative sample of the population.
The percentage of children with critical Covid outcomes is obviously
much higher among children who are hospitalized for Covid than among all
children.
The Sad State of Covid Science and Journalism
The authors of this study were irresponsible to include the claim that
“the absolute risk of critical disease from COVID-19 was 4%” among
healthy children without comorbidities, without explaining that their
data was heavily biased toward patients already hospitalized with COVID.
Without that caveat, the 4% claim is a gross misrepresentation of the
underlying data. And this study isn’t from some obscure scientist at a
no-name institution — the senior author, Dr. Carlos Oliveira, is an
Assistant Professor of Pediatrics, Biostatistics, and Biomedical
Informatics & Data Science at Yale! Surely he knows that 4% of American
children didn’t have severe Covid outcomes. Why include that misleading
statistic in the paper?!?
I also think the writers at CIDRAP needs to do a better job of
understanding and vetting the studies they report on to make sure their
reporting is accurate. They are a trusted voice for many people, and
their studies often get amplified by the Zero Covid crowd on Twitter. It
is profoundly unhelpful to continue scaring people with these misleading
and unrealistic statistics. Journalists need to understand the harm they
cause by publishing these irresponsible claims.
By the way… I’m already seeing this same error with H5N1 (bird flu) in
the media as well, and will probably write more on that later. Many news
reports refer to a 50% “fatality rate”, but that’s only among known
cases that were previously documented. Among the 3 cases identified so
far in the US (one in 2022 and two this year), none have died. The two
known cases this year have only experienced eye infections.
UPDATE: The moment I published this, I saw that another case of H5N1 has
been identified in the US. That patient did have respiratory symptoms
but is recovering. (5/30/24)
Thanks for reading Check Your Work! Subscribe for free to receive future
posts.
Type your email...
Subscribe
1
(A) Ward JL, Harwood R, Smith C, Kenny S, Clark M, Davis PJ, et al. Risk
factors for PICU admission and death among children and young people
hospitalized with COVID-19 and PIMS-TS in England during the first
pandemic year. Nat Med. 2022 Jan;28(1):193–200.
(B) Oliveira EA, Colosimo EA, Simoes ESAC, Mak RH, Martelli DB, Silva
LR, et al. Clinical characteristics and risk factors for death among
hospitalised children and adolescents with COVID-19 in Brazil: an
analysis of a nationwide database. Lancet Child Adolesc Health. 2021
Aug;5(8):559–68.
(C) Hendler JV, Miranda do Lago P, Muller GC, Santana JC, Piva JP, Daudt
LE. Risk factors for severe COVID-19 infection in Brazilian children.
Braz J Infect Dis. 2021 Nov-Dec;25(6):101650
(D) Horta M, Ribeiro GJC, Campos NOB, de Oliveira DR, de Almeida
Carvalho LM, de Castro Zocrato K, et al. ICU Admission, Invasive
Mechanical Ventilation, and Mortality among Children and Adolescents
Hospitalized for COVID-19 in a Private Healthcare System. Int J Pediatr.
2023;2023:1698407.
2
Campbell JI, Dubois MM, Savage TJ, Hood-Pishchany MI, Sharma TS, Petty
CR, et al. Comorbidities Associated with Hospitalization and Progression
Among Adolescents with Symptomatic Coronavirus Disease 2019. J Pediatr.
2022 Jun;245:102–10 e2.
3
Sanchez-Piedra C, Gamino-Arroyo AE, Cruz-Cruz C, Prado-Galbarro FJ.
Impact of environmental and individual factors on COVID-19 mortality in
children and adolescents in Mexico: An observational study. Lancet Reg
Health Am. 2022 Apr;8:100184.