Critically Examining COVID Epidemiologic Data

Interpreting COVID Cases, COVID Hospitalizations, and COVID Deaths

An Exercise in Critical Thinking and Healthy Dialogue

On November 10, 2020 CNN reported that, during the preceding week, the following was occurring in the USA [1]:

• 119,238 new cases of COVID were occurring per day (on average).
• “Soon, there will likely be 200,000 new cases occurring per day.”
• 59,000 new COVID hospitalizations were occurring per day.
• More than 1000 new COVID deaths were occurring per day.

CRITICAL EXAMINATION OF DATA:

COVID-19 can be life-threatening and life-taking, especially in the elderly and frail. A tragic number of people have died from COVID-19. The above CNN report further heightens fears.

However, in the tradition of rigorous science and disciplined practice of medicine, it is imperative to critically examine such data— to determine the scientific quality of the data and whether they understate, accurately state, or overstate the threat we face. It is not scientific to accept data at face value, without thorough critical analysis. Included in this critical examination is the need to pay attention to language and insist on careful, accurate use of language.

This article is intended to facilitate healthy, respectful, evidence-based dialogue about COVID-related epidemiologic data. The purpose of the article is to emphasize what questions need to be asked about the quality of the data being collected and reported. The goal of the article is to raise the necessary questions, not necessarily answer them. However, it is hoped that these questions will be answered by those who are leading our nation’s response to this epidemic (e.g., the USA Task Force on COVID, the CDC, Johns Hopkins University Bloomberg School of Public Health, WHO, the University of Washington Institute for Health Metrics and Evaluation, and the Gates Foundation).

To determine the quality and meaning of reported data, such as that reported by CNN on 11/10/20, it is necessary for physicians, public health officials, politicians, and the public to ask the following questions:

• What definition of a “new COVID case” is being used in this reportage? What criteria must be fulfilled for an event to be declared a “new COVID case”?
• Do these “119,238 new SARS-CoV-2 infections” occurring per day represent people who have been confirmed to be currently and definitely infected with the SARS-CoV-2 virus? Or do these 119,238 “new COVID cases,” more accurately, represent simply the number of “positive COVID tests” that have been recorded? This distinction is important, because a “positive COVID test” may or may not mean that the tested person has truly been infected with the COVID-19 virus [2-34]; and, even in the case of a true positive, may or may not mean that the person is capable, at the time, of infecting others [3, 8, 24-26].
• Is a “positive COVID test,” alone, sufficient to be counted as a “new case?” That is, can asymptomatic people who happen to have a “positive COVID test” be counted as “new cases?” Or to be counted as a new case, must a person be both “COVID test positive” and have COVID symptoms? If so, what minimum number and types of symptoms must be present (and when, chronologically) for a person to be considered to have “COVID symptoms?”
• What criteria must be fulfilled to declare a person’s COVID test to be positive? Does a “positive COVID test” need to be a PCR test, or could it be one of the less reliable rapid antigen tests?
• For PCR tests used, what cycle threshold (Ct) “cutoff” is being used for declaration of a positive test?” For a detailed explanation of Ct values, see companion article entitled, The Importance of Knowing the Ct Value at which a COVID PCR Test is Positive—LONG VERSION, and REFERENCES [2-34]
• For PCR tests that are positive only after 37 or more amplification cycles (i.e., at a Ct of 37 or higher), are we sure that that positivity is truly and always due to definite COVID infection, especially in asymptomatic or minimally and non-specifically symptomatic people who were tested as part of a screening (surveillance) process?
• What is the false positivity rate of COVID PCR tests? Is the false positivity rate higher when the test is used in a surveillance setting, as opposed to a hospital/ICU setting? Is the false positivity rate higher in people whose test is only weakly positive (Ct of 37 or 40, or 45), than in people whose test is strongly positive?
• Since the beginning of the COVID pandemic, results of COVID PCR tests have been reported only as positive or negative, without any mention of the Ct value at which the test was positive—i.e., without any mention of how strongly or weakly positive. Has that been a wise policy?
• Of the recent/current COVID test positive “new cases” what was the breakdown regarding the degree of test positivity—what percentage were positive only at a Ct of 37 or higher? What percentage were positive only at a Ct between 30-34? What percentage were positive at a Ct less than 30; less than 24; less than 20; less than 15; less than 10? Do these Ct statistics not matter? Does it not matter whether the test was strongly positive or only very weakly positive?
• Of the “new cases,” what is the breakdown regarding degree of symptoms—what percentage were asymptomatic, mildly symptomatic, moderately symptomatic, severely symptomatic, and specifically symptomatic of COVID?
• When a “positive” COVID test result is reported to the State Health Department and then to the CDC and the Johns Hopkins Bloomberg School of Public Health database, who has filled out a data record form on that person and how frequently and accurately does it contain all the above-mentioned clinical information, including details about symptoms and the Ct at which their test was positive? Or is there often little or no clinical information provided, other than the fact that the person “had a positive COVID test result?”
• The World Health Organization (WHO) and many government health ministries have encouraged diagnosis of SARS-CoV-2 infection (COVID) on the basis of a single positive PCR result, even in asymptomatic persons without any history of exposure. [7] For example, WHO has defined a “confirmed case” as a person with a positive test result, “irrespective of clinical signs and symptoms.” [35] Is that the policy that is being applied to the “new case” counts being currently reported in the USA?
• In the USA, the Council of State and Territorial Epidemiologists (CSTE) has developed a more appropriate and nuanced “COVID-19 Interim Case Definition,” which was approved by the CDC in August 2020. [36] To what extent are the criteria recommended by the CSTE being strictly applied in the current collection, determination, and reportage of “new COVID cases” in the USA? Of the 119,238 daily “new cases” reported on 11/10/20, what percentage of them fulfilled strict CSTE criteria for presence of “confirmatory” vs “presumptive” vs only “supportive” evidence for designation as a “new case.” If a person had a “positive COVID test,” but other details needed to make an accurate CSTE designation were missing, was such a person, nevertheless, entered as a “new case,” or were such people designated as “possible new case, but available information is insufficient” and not included in the case count, or included as a separately reported “possible case”?
• Bear in mind that even the more demanding “case definition” recommended by the CSTE does not include any mention of the extent to which a COVID test is positive—i.e., Ct values are not considered. According to the CSTE recommendation, a person whose PCR test for COVID is positive only at a Ct of 40, or even 45 (very weak and questionable positivity) would be considered to have “confirmatory laboratory evidence” of COVID. Does that represent a scientifically sound decision?
• Likewise, the case definitions recommended by the CDC and WHO do not include any mention of the extent of COVID test positivity (e.g., Ct values). Is this wise?
• To what extent have efforts been made to ensure that “new cases” are not counted more than once? For example, if a previously known “COVID positive” person has had several positive follow-up re-tests (to document possible ongoing infectivity) is each positive re-test included in the “new case” count? How is such duplicate counting prevented?
• If 119,238 “new cases” were noted on a particular day in the USA (on 11/9/20, e.g.), how many people were tested on that same day? It turns out that approximately 1.5 million tests were performed daily during the week before 11/10/20. What was the breakdown, regarding percentage of those 119,238 that were positive only at a Ct greater than 30? What percentage were positive at a Ct of 30 or less?
• How many false positives would be expected, if 1.5 million asymptomatic or minimally and non-specifically symptomatic people were tested, as part of a mass screening/surveillance campaign? Since we currently do not know what the false positivity rate is in the mass surveillance setting, we do not know how many of the 1.5 million would be expected to have a false positive result.
• During a given week, to what extent is a portion of a “rising daily new case count” simply a reflection of a rising number of people being tested?
• Regarding the reported number of “new COVID hospitalizations,” were all these patients hospitalized primarily because of COVID illness, or were some (what percentage?) simply patients who were hospitalized for other reasons and happened to have a positive COVID test (how strongly or weakly positive?) when routinely screened at the time of admission?
• Likewise, regarding new “COVID deaths,” what criteria were strictly applied in each case to accurately determine whether a patient truly died of COVID, as opposed to dying from some other cause, but having a “positive COVID test” (again, how strongly or weakly positive?) or even just COVID exposure? How complete and accurate has information on these “COVID death” certificates been? On a death certificate, does the mere mention of a positive COVID test, or exposure to someone with COVID positivity, mean that that death is automatically included in the national “COVID death” count? To what extent have those who complete death certificates been told that they must list “COVID test positivity” and “COVID exposure” on a death certificate (if such positivity and exposure have occurred), even if COVID was not considered the cause of death or even a minor contributing factor? In past years, have “influenza test positivity” or “exposure to influenza” been listed on death certificates, if influenza was not considered one of the main causes of death, or even a minor contributing cause? What have the rules been, regarding this issue, and how uniformly have rules been applied.
• Regarding people who definitely died of COVID, what treatment did they receive? Did they receive prompt, timely, appropriately aggressive immunosuppression, if they had evidence of hyperinflammation/cytokine storm? What percentage of true COVID deaths could have been prevented with application of a different, more aggressive and compulsive treatment approach? [37,38] (See companion articles on Treatment of Severe COVID Illness.)
• In short, do the above CNN-reported data represent solid, quality data that were collected and interpreted in a scientifically sound fashion and accurately reflect reality—and might even understate the threat we face?
• Or have the COVID data collected and reported (at least in the USA) been of much lower scientific quality than has been assumed.

Without answers to the above questions, it is difficult to know what a “new COVID case” means, what a “rising new case count” means, or what a “rise in COVID hospitalizations” and a “rise in COVID deaths” mean.

SUMMARY:

The quality of data, regarding “new COVID cases,” “new COVID hospitalizations,” new COVID ICU admissions,” and “new COVID deaths,” depends, fundamentally, on:

1. The quality, reliability, interpretation, and wise use of the COVID tests upon which these data are based; and
2. Disciplined and uniform use of carefully constructed diagnostic and classification/stratification criteria for “COVID case,” “COVID hospitalization,” and “COVID death.”
3. The extent to which patients with severe COVID illness are optimally treated.

Furthermore, the quality of COVID-related public health policy and public understanding of COVID depend on the quality of the above data.

If most COVID tests are performing 37 or more cycle amplifications in their effort to detect presence of viral material; if most positive tests (especially in surveillance testing) are positive only at a Ct of 35 or greater; if people with a positive test at a Ct of 37 or 40 are, in fact, not contagious; and if the currently unknown incidence of false positive tests (especially in the surveillance setting) is higher than heretofore appreciated; then, the currently reported COVID data may be greatly inaccurate, or at least excessively alarming—particularly “new case” data that have been generated primarily through mass screening of increasing numbers of asymptomatic and minimally (and non-specifically) symptomatic people.

Similarly, if Ct values and uniform criteria have not been strictly applied to reported data on COVID hospitalizations and COVID deaths, these data need to be viewed with caution. Finally, if treatment of patients with severe COVID illness has been suboptimal, COVID ICU and death data will be misleading.

We all have an obligation to make certain that fundamental principles of science, medicine, clinical care, and clinical/epidemiological research are being rigorously followed in the collection, interpretation, and reporting of COVID cases, COVID hospitalizations, and COVID deaths.

REFERENCES:

1. https://www.cnn.com/2020/11/10/health/us-coronavirus-tuesday/index.html
2. Binnicker MJ. 2020. Challenges and controversies to testing for COVID-19. J Clin Microbiol 58: e01695-20. https://doi.org/10 .1128/JCM.01695-20
3. Tom MR, Mina MJ. To Interpret the SARS-CoV-2 Test, Consider the Cycle Threshold Value. Clin Infect Dis. 2020 May 21: ciaa619. Published online 2020 May 21. doi: 10.1093/cid/ciaa619
4. Hosseini A, Pandey R, Osman E, et al. Roadmap to the Bioanalytical Testing of COVID-19: From Sample Collection to Disease Surveillance [published online ahead of print, 2020 Oct 30]. ACS Sens. 2020; acssensors.0c01377. doi:10.1021/acssensors.0c01377
5. Guo JJ, Yu YH, Ma XY, et al. A multiple-center clinical evaluation of a new real-time reverse transcriptase PCR diagnostic kit for SARS-CoV-2. Future Virol. 2020;10.2217/fvl-2020-0299. doi:10.2217/fvl-2020-0299
6. Wang M, Chen D, Wu W, et al. Analytical performance evaluation of five RT-PCR kits for severe acute respiratory syndrome coronavirus 2 [published online ahead of print, 2020 Oct 27]. J Clin Lab Anal. 2020; e23643. doi:10.1002/jcla.23643
7. Yüce M, Filiztekin E, Özkaya KG. COVID-19 diagnosis -A review of current methods [published online ahead of print, 2020 Oct 24]. Biosens Bioelectron. 2020; 172:112752. doi:10.1016/j.bios.2020.112752
8. TWiV 640: Test often, fast turnaround, with Michael Mina. https://youtu.be/kDj4Zyq3yOA
9. Your Coronavirus Test is Positive. Maybe it shouldn’t be. Interview with Michael Mina, MD. Published August 29, 2020; Updated September 17, 2020. https://www.nytimes.com/2020/08/29/health/coronavirus-testing.html
10. Vandenberg O, et al. Considerations for diagnostic COVID-19 tests. Nat Rev Microbiol. 2020 Oct 14: 1–13. doi: 10.1038/s41579-020-00461-z
11. Surkova E, et al. False-positive COVID-19 results: hidden problems and costs. Lancet Resp Med. published online, September 29, 2020. https://doi.org/10.1016/52213-2600(20)30453-7
12. Mayers C, Baker K. Impact of false-positives and false-negatives in the UK’s COVID-19 RT-PCR testing programme. June 3, 2020. https://assets. publishing.service.gov.uk/government/uploads/system/uploads/ attachment_data/file/895843/S0519_Impact_of_false_positives_and_ negatives.pdf (accessed Aug 8, 2020).
13. Skittrall JP, Wilson M, Smielewska AA, et al. Specificity and positive predictive value of SARS-CoV-2 nucleic acid amplification testing in a low-prevalence setting [published online ahead of print, 2020 Oct 14]. Clin Microbiol Infect. 2020; S1198-743X (20)30614-5. doi:10.1016/j.cmi.2020.10.003
14. Hellou MM, et al Nucleic-acid-amplification tests from respiratory samples for the diagnosis of coronavirus infections: systematic review and meta-analysis, Clinical Microbiology and Infection, https://doi.org/10.1016/j.cmi.2020.11.002.
15. Corman VM, Landt O, Kaiser M, et al. Detection of 2019 Novel Coronavirus (2019-NCoV) by Real-Time RT-PCR. Euro Surveill. 2020, 25 (3), 1−8.
16. Corman V B, leicker T, Brunink S, et al. Diagnostic Detection of Wuhan Coronavirus 2019 by Real-Time RT-PCR; World Health Organization, 2020; pp 1−12.
17. Francesca F, et al. Detection of SARS-COV N2 Gene: Very low amounts of viral RNA or false positive? J Clin Virol. 2020 Dec; 133: 104660. Published online 2020 Oct 14. doi: 10.1016/j.jcv.2020.104660.
18. Katz AP, et al. False positive reverse transcriptase polymerase chain reaction screening for SARS-CoV-2 in the setting of urgent head and neck surgery and otolaryngologic emergencies during the pandemic: Clinical implications, Head Neck 42 (7) (2020) 1621–1628, https://doi.org/10.1002/hed.26317
19. Wang Z, et al. External Quality Assessment for Molecular Detection of Severe Acute Respiratory Syndrome Coronavirus 2 in Clinical laboratories, The Journal of Molecular Diagnostics (2020), doi: https://doi.org/10.1016/j.jmoldx.2020.10.008.
20. Cohen AN, et al. Diagnosing SARS-CoV-2 infection: the danger of over-reliance on positive test results; false positive test results impact clinical and policy decisions. medRxiv preprint doi: https://doi.org/10.1101/2020.04.26.20080911
21. Cohen AN, Kessel B, Milgroom MG. Analysis of expected false positive rates in SARS-CoV-2 testing: technical background, limitations and objections. https://doi.org/10.5281/zenodo.4035317.
22. Ruiz-Villalba A, et al. Amplification of nonspecific products in quantitative polymerase chain reactions (qPCR) Biomol Detect Quantif. 2017 Dec; 14: 7–18. Published online 2017 Nov 1. doi: 10.1016/j.bdq.2017.10.001
23. Willman D. Contamination at CDC lab delayed rollout of coronavirus tests. April 18, 2020. https://www.washingtonpost.com/investigations/ contamination-at-cdc-lab-delayed-rollout-of-coronavirustests/2020/04/18/fd7d3824-7139-11ea-aa80-c2470c6b2034_story.html. (accessed Aug 16, 2020).
24. Bullard J, et al. Predicting infectious SARS-CoV-2 from diagnostic samples. Clin Infect Dis. 2020 May 22: ciaa638. Published online 2020 May 22. doi: 10.1093/cid/ciaa638
25. Singanayagam A, Patel M, Charlett A, et al. (2020). Duration of infectiousness and correlation with RT-PCR cycle threshold values in cases of COVID-19, England, January to May 2020. Euro surveillance: bulletin Europeen sur les maladies transmissibles = European communicable disease bulletin, 25(32), 2001483. https://doi.org/10.2807/1560-7917.ES.2020.25.32.2001483
26. Jaafar R, Aherfi S, Wurtz N, et al. Correlation Between 3790 Quantitative Polymerase Chain Reaction–Positives Samples and Positive Cell Cultures, Including 1941 Severe Acute Respiratory Syndrome Coronavirus 2 Isolates, Clinical Infectious Diseases, ciaa1491, https://doi.org/10.1093/cid/ciaa1491
27. https://www.sciencemag.org/news/2020/09/one-number-could-help-reveal-how-infectious-covid-19-patient-should-test-results
28. Magleby R, Westblade LF, Trzebucki A, et al. Impact of Severe Acute Respiratory Syndrome Coronavirus 2 Viral Load on Risk of Intubation and Mortality Among Hospitalized Patients with Coronavirus Disease 2019, Clinical Infectious Diseases; ciaa851, https://doi.org/10.1093/cid/ciaa851
29. He X, Lau EHY, Wu P, et al. Temporal dynamics in viral shedding and transmissibility of COVID-19. Nat Med 2020; 26:672-5. Doi:http://dx.doi.org/10.1038/s41591-020-0869-5.
30. Yagci AK, Sarinoglu RC, Bilgin H, et al. Relationship of the cycle threshold of SARS-C0V-2 polymerase chain reaction and total severity score of computerized tomography in patients with COVID-19. International Journal of Infectious Diseases 101 (2020) 160-165.
31. Bryan A, Fink SL, Gattuso MA, et al., SARS-CoV-2 viral load on admission is associated with 30-day mortality. Open Forum Infect Dis. 2020 Dec; 7(12): ofaa535. Published online 2020 Nov 3. doi: 10.1093/ofid/ofaa535
32. Perchetti GA, Nalla AK, Huang ML, et al. Validation of SARS-CoV-2 detection across multiple specimen types. J Clin Virol. 2020; 128:104438. doi: 10.1016/j.jcv.2020.104438
33. Long QX., Liu BZ., Deng HJ, et al. Antibody responses to SARS-CoV-2 in patients with COVID-19. Nat Med 26, 845–848 (2020). https://doi.org/10.1038/s41591-020-0897-1
34. Rhoads D,  Peaper DR,  She RC, et al.  College of American Pathologists (CAP) Microbiology Committee Perspective: Caution must be used in interpreting the Cycle Threshold (Ct) value. Clin Infect Dis. 2020 Aug 12; ciaa1199. doi: 10.1093/cid/ciaa1199. Online ahead of print.
35. World Health Organization. Global surveillance for COVID-19 caused by human infection with COVID-19 virus. Interim guidance. 20 Mar 2020. https://www.who.int/emergencies/diseases/novelcoronavirus-2019/technical-guidance/laboratory-guidance.
36. https://wwwn.cdc.gov/nndss/conditions/coronavirus-disease-2019-covid-19/case-definition/2020/08/05/ (CSTE Criteria)
37. Rennebohm RM.  Analysis of the COVID-19 epidemic: an additional narrative; an alternative response. Pediatrician (St. Petersburg). 2020;11(3):23-40. https://doi.org/10.17816/PED11323-40
38. Rennebohm RM. Has undertreatment of severe COVID illness been widespread? A pediatric rheumatologist’s perspective. Russia Biomedical Research, 2020, Vol 5, No 3, p. 3-13.

Rob Rennebohm, MD

1/7/21

Email: rmrennebohm@gmail.com

Website: https://notesfromthesocialclinic.org/

About the Author: https://notesfromthesocialclinic.org/about-me/