COVID-19. Part 3 — What’s with the high fatality ratio?

There were two indicators that formed the basis for ‘locking down’ and other drastic actions: spread and severity.

In Parts 1 and 2, we saw that the knee-jerk strategy for monitoring the virus’s spread was so flawed that those monitoring it had no idea when, where, or even whether it was spreading.  In this instalment, we’ll take a look at attempts to estimate its severity.  But first, let me tell you a story.

The speckled-duck story

Jones has a duck farm with two varieties of ducks.  To keep things simple, we’ll call them white and speckled.  The white ducks are all white; the speckled ducks are also white but they have speckles to varying degrees.  The two varieties are difficult to distinguish until you actually pick them up and take a close look.

One morning Jones felt that something was killing the speckled variety.  His careful tally of the 100-odd duck carcasses collected each month showed that, on average, half of them (50) were speckled.  Of course, this would be expected if speckled ducks made up half of his flock, but Jones was quite certain that only around 10% of his flock looked speckled.  He decided to check.

In order to check, he had to count them.  But there was a problem.  There were roughly 10,000 ducks on the farm — all in one large pen — and the variety of each duck couldn’t be distinguished without catching and examining it.  How would he do this?  

There were three ways.  The first was to catch each and every duck and tally how many were speckled.  This is what’s known as taking a census.  Jones had neither the time nor the inclination for such a tedious task.  

The second was to catch a smaller number of ducks — say 100 (which amounts to 100th of the total flock) — and count how many of these were speckled.  Then, provided he’d randomly selected the ducks (and not simply targeted the birds with darker or lighter shading) he could multiply the count by 100 to get an estimate of the total number of speckled birds in the flock: a survey.  But Jones knew nothing about surveying, so he didn’t choose this option.

The third was to simply catch as many as possible in the time that he had available that morning — about two hours — examine them for speckles, and tally the results.  However many speckled ducks he could find during that time would then become his best estimate of the total number of speckled ducks on the farm.  

Obviously the last option isn’t methodologically sound (no one should ever make an important estimate in this way), but it’s included here because it’s the one that appealed to Jones, and it’s the one that he adopted.  After morning tea, he caught and counted furiously and didn’t stop until lunch.  But Jones worked smart.  He didn’t attempt to catch and examine all the ducks; that would take him days.  He’d worked with ducks for many years, and reckoned he could pick those most likely to be speckled by their slightly darker shading — a kind of off-white.  Using this duck-sense, he would knock the job over quickly.

At any rate, by lunchtime Jones had a grand tally of 1000 speckled ducks and, from that moment, declared that there were 1000 speckled ducks on the farm.

Now it was time for some calculations.  The normal rate of loss of farm ducks in Jones’s location was 1% per month.  In other words, provided conditions were normal, duck farmers expected 1% of their flock to die each month due to old age, attack, etc.  That’s one death per hundred ducks, or 10 per 1000.  But Jones was losing 50 of his 1000 speckled ducks each month — which was 5% of his total speckled ducks!

Jones immediately called in an expert, who alerted the authorities, who in turn placed his farm in quarantine amid fears of a mystery illness.  Production ground to a halt, workers were laid off, and Jones started reaching into his savings to pay for what seemed to be never-ending and ever-conflicting advice.  

Finally, after nearly two months and exhaustive forensic efforts, it was decided that the speckled ducks did in fact have a mystery illness and needed to be destroyed.  Furthermore, the destruction had to be carried out by the authorities at a cost of one dollar per duck.  Reluctantly, Jones gave the go-ahead and drove to the bank to withdraw the last $1000 from his savings account to pay for destruction of every speckled duck.

Alas, on his return from the bank, he was handed a bill for $5000!  

It turned out that the authorities carefully examined each and every duck and found that 5000 were speckled.

The moral of the story?  It’s important to count your ducks carefully, especially when the farm depends on it.  Jones had been careful in calculating how many of the dead ducks were speckled.  That was easy.  The problem arose because he didn’t make the same effort in calculating the speckled proportion of the rest of the flock.

Of 5000 speckled ducks, 50 of them dying each month represented a 1% per month death rate — completely normal.

There had been no problem to begin with!

COVID-19  

When governments world wide declared states of emergency, closed businesses, and ordered people to stay apart or at home, they did so due to a perception that people with the virus were dying at a high rate.  The primary indicator behind this perception was the fatality ratio.

A virus’s fatality ratio is simply the proportion who died, of those infected with it.  It is commonly expressed as a percentage, and calculated — as it was in the duck example above — via a fraction, with the number of deaths on the top and the total number infected on the bottom.

In order to understand what went wrong with the fatality ratio for COVID-19, we need to look at how these numbers were arrived at.

First, the bottom number was, as we highlighted in Parts 1 and 2, a  simple count of those who had so far tested positive for infection with the virus.  The top number was the number of deaths in those who had tested positive either before or after they died.  The ratios calculated in this way were published in relation to the world as a whole; to various countries; and to communities within countries.

For example, if a community counted 100 deaths and a total of 10,000 infected people, the fatality ratio was 1%.

On the surface, a fatality ratio calculates the average chance of dying if infected.  But the calculation’s legitimacy rests on a very important condition: that the effort to examine the living for evidence of the virus was equal, proportionally, to the effort to examine the dead.  

That rings a bell, doesn’t it?  Jones carefully examined the dead ducks for speckles, but his troubles occurred because he neglected to carefully examine the living ducks.  

Unequal testing effort

Italy attracted attention early on due to what appeared to be a high death toll, so other parts of the world looked to it for an estimate of the fatality ratio.  Here’s how it was calculated.

By March 20, as Australia was phasing in its shelter-in-place measures, Italy had recorded just over 4000 COVID-19 deaths.  These formed the top number of the fatality ratio.  At that time, the country had tested just over 200,000 individuals for the virus.  The simple count of positive results from these tests (47,000) formed the bottom number.  

A quick calculation yields a fatality ratio of more than 8 per cent.  This is a very large fatality ratio, and certainly ample justification for drastic action.  But let’s check for duck-farm blunders.  Remember, Jones carefully examined all the dead ducks but made a very poor effort to examine the living ducks.  He didn’t even conduct a survey!  That’s what got him into trouble.  

What did they do in Italy?

By March 20, the authorities had tested around 200,000 people for the virus, as mentioned.  As the calculation below shows, this represents roughly 0.33% of the Italian population. 

The virus-positive results from this miniscule effort formed the bottom number of the fraction (47,000).  To put that effort into perspective, consider Jones selecting 33 ducks from his 10,000 flock, counting how many of them were speckled (16 or 17 if they were chosen randomly; 33 if he really did have an eye for the speckles), and using that for his total count of speckled ducks on the farm.  As you can imagine, no matter how hard he’d tried to choose the birds most likely to be speckled, such a plan would be pure folly.  Yet medical authorities around the world obtained their infection figures in exactly this way.

The next question we need to ask is whether this proportion (0.33%) of the entire population was comparable with the proportion of deaths examined (tested) for the virus.  Unfortunately the proportion of deaths tested isn’t published, but we can still answer our question.

Under normal circumstances, an average of around 12,000 people die each week in Italy.  Therefore, over the four weeks that it took to accumulate the 4000 COVID-19 deaths, we would expect 48,000 deaths to have occurred.  Let’s assume for a moment that the official assumption is correct, and that the 4000 COVID-19 deaths were additional to the expected deaths from other causes, giving a total of 52,000 deaths from all causes.

We’d like to know how many of these 52,000 were or had been tested for COVID-19.  As already mentioned, we can’t determine that.  What we do know, though, is that 4000 of them were declared COVID-19 deaths.  So, at a bare minimum, those 4000 were tested.  We can, and probably should, surmise that many more than this were tested — possibly all, or nearly all — but we do know for certain that at least 4000 were tested.  Taking 4000 as the minimum, then, we can confidently say:

Comparing the testing rate of the dead (at least 7.69%) with that in the general population (0.33%), we can see that the effort to locate infected people proceeded in the dead with at least 23 times the intensity that it did in the living.

This is understandable: Italy had a policy of testing all hospital admissions, and those in the process of dying are commonly admitted to hospital.

Mind you, Jones too had good reasons for focussing his tallying efforts on the dead ducks.  Unfortunately however it meant that he failed to properly tally the living ones, and that’s what led to his problems.

Italy was not the only country for which a fatality ratio was estimated, of course.  There were many of them — and all calculated their fatality ratios as we did above.  At the start of March, the World Health Organisation (WHO) announced that the global average fatality rate was 3.4 per cent.  They contrasted it with seasonal flu, which they said “generally kills far fewer than 1% of those infected”.

Such fatality ratios were quoted frequently in the media, drumming up intense alarm and seeming to justify the drastic actions that we were about to take.  The ratios varied considerably between communities, and even within communities over time; but, until recently, all of them arose from this flawed methodology.

Which fatality ratio?

Some will think that I’m conflating two different fatality ratios.  This section is particularly for them.

There are two types of fatality ratio frequently discussed: the Case Fatality Ratio (CFR) and the Infection Fatality Ratio (IFR).  The CFR is the number of deaths divided by the number of confirmed cases, and the IFR is the number of deaths divided by the number infected. 

Note that they share the same numerator (top number): the number of deaths.  Their difference lies solely in their denominators (bottom number).  One uses confirmed cases, and the other uses infections.  Of most illnesses, a confirmed case is defined as an infection accompanied by a specific set of clinical manifestations.  That’s not the case with COVID-19.  The WHO defines a confirmed case of COVID-19 as a lab-confirmed infection irrespective of clinical manifestations.  That means that, in the case of COVID-19, the denominators for the CFR and the IFR are practically identical — one is a lab-confirmed infection; the other is an infection.  Hence, for all practical purposes, the two rates are the same; the only difference being ascertainment.  Put simply, any difference reported between the CFR and the IFR for COVID-19 is nothing more than a measure of how well we ascertained infection — again, regardless of clinical manifestations.

Having read Parts 1 and 2, you may guess that case ascertainment for COVID-19 was poor.  Indeed, the results of recent studies, which I will discuss in the next instalment, suggest that it was woeful.  

At any rate, the question on everyone’s lips is: what’s the danger if I get infected?  It’s not: what’s the danger if I get infected, and someone notices that I’m infected, and I happen to be tested at the right time, and that test happens to turn up positive?

How to count properly

There are ways to accurately estimate the true number of infected people, even when it’s impractical to count them all.  In the duck-farm story, Jones should have chosen option two — taken a random sample of the ducks, counted the speckled ones amongst them, and extrapolated that proportion to the entire farm.

A similar approach can be taken with infections in a population.  By conducting a serological survey (serosurvey for short), we can estimate how many individuals have been infected.  Serosurveys use a test that’s different from the COVID-19 test we’ve been using to ascertain cases: rather than test people’s mucus for presence of the virus, it tests their blood for antibodies to the virus, indicating whether they have been exposed to it at some point.  Testing a representative sample of the population provides a gauge of the number in that population who have been infected.

It has been quite some time since I published Part 2 in this series, and I promised that I’d follow it with a post about deaths.  I’ve been awaiting the results of these serosurveys.  They have started trickling in, but, as you may have foreseen, they have met with considerable resistance.  I can understand why, as they correct the flaw in the “method” described above and thereby demand a very uncomfortable rewrite of our recent history.

In the next instalment, we’ll take a look at serosurveys.  These represent our first real attempt to estimate a relevant denominator and thence a legitimate fatality ratio.

Later, we will explore all-cause mortality, which refers to all deaths in a population.  This measure follows a surprisingly consistent pattern throughout the year.  It deviates from that pattern during crises such as war, famine, economic depression, and epidemic illness.  All-cause mortality is great in that it can tell us exactly how many have died, free from the various problems of undercounting, sampling bias, and distinguishing who died from what.  And, although it can’t tell us what caused any deviations, it can help us to identify possible causes, by examining the deviations’ timing and extent.

3 thoughts on “COVID-19. Part 3 — What’s with the high fatality ratio?”

  1. Love it. Well done. You have perfectly articulated the bleeding obvious…..or rather, for our illustrious leaders, what should have been bleeding obvious!

  2. Rather than relying solely on medical experts they should be complemented with the expertise of statisticians and actuaries.??

    1. If only!

      There were statisticians involved in making these decisions. There were also statisticians trying to tell them that they were making big mistakes.

      I think the problem had more to do with fear. Being such a compelling emotion, fear leads us to abandon our education, our objectivity, and our common sense.

      Apart from individual fear of the virus, politicians feared being blamed and shamed for not acting on advice, and their advisers feared the same. And of course the ferocious media ensured that they felt this fear.

      So I think that the solution, when analysing data, is to have people who can keep their heads while everyone else is losing theirs. 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *