3rd February 2012
Yesterday I received permission from the Department of Health and Ageing to publish data which I received late last year (along with my comment, which had to be ‘approved’). It concerns whooping cough notifications in Australia for the under-5 years age group. There were conditions attached to the publication: caveats and a link for assistance in interpreting it, all of which I have included below.
As many will already be aware, Australia has seen a meteoric rise in the number of notified cases of whooping cough in the past few years. Most media commentators relate this to a lack of vaccination, however, the reports don’t tell us who is getting whooping cough. Is it the un-vaccinated, the vaccinated, or is it a combination?
Well, the Department now has a system in place for collecting this information in under-5-year-olds, so I put in a request and was kindly supplied with the following. It wasn’t instantaneous, mind you. The request took more than a year to fulfill, and required a considerable amount of hoop-jumping and justification, although the staff I dealt with were very helpful.
Here’s the data
Here’s what we see (my ‘approved’ comment):
For the years 2008-2010, there were a total of 9333 cases of whooping cough reported in children aged 0-4 years. Among them were 754 who were either too young or otherwise not eligible to have been vaccinated, and 1497 for whom vaccine status was not known.
Of the remaining 7082 cases whose vaccine status was known, and who were eligible to have received the vaccine, 5296 (75%) were fully vaccinated for their age. A further 986 (14%) were partly vaccinated. Only 800 (11%) were un-vaccinated.
As part of the conditions for publishing this data, I have been asked by CDNA to include the following.
The following caveats apply to these data:
Note that notifications for this disease are available online athttp://www9.health.gov.au/cda/Source/CDA-index.cfm
- Notification data from NNDSS are supplied by States and Territories and are subject to retrospective revision.
- The current Australian national notifiable disease case definition for measles and pertussis can be found online:http://www.health.gov.au/casedefinitions. Please note that some jurisdictions may not follow the case definition strictly and adherence may vary between jurisdictions. Case definitions are also reviewed and updated over time resulting in the potential for variation when comparing current to historical data.
- Notifications to NNDSS rely on a largely passive surveillance system which may result in an underestimation of the total number of cases in the community at any given time.
For assistance in interpreting the data please visit http://www.immunise.health.gov.au/internet/immunise/publishing.nsf/Conte…
Update 4th February 2012
Wow! Reaction plus!
I’ve had numerous emails in the past 24 hours. It seems this page has gone viral. Some have alerted me to group discussions about it where the creativity has been astounding. One commenter referred to me as an “innumerate dingbat”.
Anyway, I’ve decided to open this page to comments for those who wish to discuss it. Please note, I will be moderating the comments to ensure the discussion is civil and on subject. I will NOT remove or edit comments for ANY other reason.
Update 7th February
The ‘innumerate dingbat’ strikes again! I found a blog piece here: http://www.mycolleaguesareidiots.com/archive/2012/02/05/How-not-to-use-m… and decided to respond.
In the blog, a chap by the name of Jason Brown (presumably the original ‘innumerate dingbat’ claimant) goes to town on the information I’ve published, bemoaning my alleged poor maths, and wrapping it up with:
“Yes, Greg, you’re an Innumerate Dingbat. You’re just going to have to learn to live with it.”
I’ve posted a copy of my response below in case Jason chooses to not publish it.
Actually, I was unable to post my response on Jason’s website. Tried many times and got the following message:
“Sorry, but we cannot accept this comment.
Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.”
I then tried to contact him via the ‘Contact’ link on his site, but got the following error message:
“Due to problems currently beyond my control, the email form that would normally appear here, doesn’t.”
Anyway, here it is…
After a long day out yesterday, I returned to find an alert from a friend. They directed me to your blog and, it seems, the source of the ‘innumerate dingbat’ call. I see you’ve had a go at my maths and concluded “Yes, Greg, you’re an Innumerate Dingbat. You’re just going to have to learn to live with it.”
Well, live with it I will, Jason. After all, you seem intent on sticking with the name, and you’re free to do so. I did however detect a hint of hostility in your analysis, so in my response I’ll be trying to ‘spread the love’ so to speak, which is something I think is really needed in this debate.
The first thing I need to draw your attention to is that none of the issues you raise actually have anything to do with numeracy. Not one of them. Did you realise that? You’re actually having a poke at my comprehension, not my maths. So the ‘innumerate’ part doesn’t fit (neither does the comprehension, but we’ll get to that). I just wanted to point that out. It doesn’t mean I think you’re ‘illiterate’ though. Perhaps you initially thought of “not-so-good-at-comprehension dingbat”, but found it not rolling off the tongue as nicely. Mind you, you claimed it was all about maths:
“Greg is no smarter than Meryl Dorey when it comes to numbers… The maths here are not hard at all. They’re barely high-school level…”
It’s just that NONE of your issues were about maths, Jason.
There are many more problems with your piece so I suggest you get a drink. Next problem (and this is worse) is that my ONLY comment on the data was actually approved by the custodians of the data. If you think I didn’t comprehend it properly then you’re obviously taking issue with them. Did you realise this when you wrote your piece? Read the page again and ask yourself, where did he say that this data showed us ANYTHING about the vaccine being good or bad? Nowhere. So again, this doesn’t mean I think you don’t know how to read, but I wonder whether you should be commenting on someone else’s comprehension. You’ve clearly inferred something that was not there.
Now, you’re possibly wondering why I didn’t draw any conclusions from the data. After all, I published it. Why do that and not infer anything? Anyone familiar with my writings will know the answer to this. I don’t regard notifications as valid indicators of disease incidence. Chapter 4 of my latest book goes into some detail as to my reasons for this, but you can get a small taste of them here: https://groups.google.com/group/vaccination-respectful-debate/msg/e2463b…
Why did I publish it? To generate discussion. It is of immense public interest. Reported cases of whooping cough have increased roughly 80-fold in the past 20 years, and during this time vaccination against it has also INCREASED. Ironically, this has led to damnation of unvaccinated children in the media. They are being blamed for the situation. So I acquired the data and threw it out there for discussion. And I’m indebted to you for taking it up.
If I were going to be drawn into making any meaningful comment on the data it would have to be that I’m astonished that a system which is biased toward reporting whooping cough in unvaccinated children, actually shows up so many in the vaccinated. The system predisposes doctors to suspecting whooping cough in children who are unvaccinated, and not suspecting it so much in the vaccinated. How much of an influence this is we don’t know, but it will inevitably skew the data to some extent. Doctors assume the vaccine is protective, and are therefore less likely to report the disease in the vaccinated. So, I am very surprised each time I see examples like this (and they are common) showing that such a high proportion of their reports are in the vaccinated.
It occurs to me that there could be large numbers of vaccinated kids out there, running around with whooping cough but not reported to the system because their doctor was inclined to think they had bronchitis, croup, or whatever (because they should be ‘immune’ to whooping cough). On the other hand there might only be a small group who fit into this category. Who knows?
Anyway, let’s get to the details of your complaints. If I’m not wrong your concerns are focussed on the semantics of terms: unvaccinated, fully vaccinated for age, partly vaccinated for age.
First, you feel kids who have been partly vaccinated, but not received the full number of doses for their age should have been included in the unvaccinated group. I have no problem with you feeling that way Jason, but the data was in three columns – fully, partially, and not. This is how it was reported, and also how my summary treated it. You want to combine the partially with the not, presumably to make things look better for the vaccine. That’s OK. I didn’t combine any of them. I reported them separately which is the most open approach (i.e. not trying to push a barrow).
Next, the ineligible. Many kids who became sick were too young to have been vaccinated. Again, you feel they should have been included with the unvaccinated group. And again, that’s presumably to make things look better for the vaccine. But by definition, babies who are too young to have been vaccinated are in fact appropriately vaccinated for their age. Hence, they belong to both the fully vaccinated for agegroup, and the not vaccinated group. Although it’s most appropriate to include them in the former (when comparing with a control cohort, as you did) I left them out altogether. Again, I was trying to not take sides. People can include them in whatever way they wish, as you have done.
Now, the messy part. You take the figures and compare them with the vaccination rate in the community (as a control cohort). That’s a perfectly valid approach IF notifications are an accurate reflection of true incidence. In my opinion this is not likely, for the reasons I’ve already mentioned. Also, when we get up around 90% it only takes a few percent out to completely change our assessment. But let’s follow through with your approach for a moment.
The rates in the community can be found here: http://www.health.gov.au/internet/main/publishing.nsf/content/cda-cdi350… (tables 1, 2, and 3)
The ~95% rate you quote is correct for 2-year-olds. For 1-year-olds however it is 91.8%, and for 5-year-olds 89.9% (both of which put a serious dent in your calculations). These percentages are of “appropriately vaccinated for age”. Of course the rate in the community of the ineligible group is practically zero (and you wanted to include this group in the comparison, mind you… comprehension, again). So, if the data I published were a valid representation of true incidence there is still a ‘please explain’ in order. Vaccine efficacy appears to be markedly lower than we’ve been told. If, on the other hand, there are a disproportionate number of kids in the community who had whooping cough that wasn’t reported (because their doctor thought they were ‘immune’ to it), we have a much greater problem for the vaccine.
Anyway, as I’ve already stated, I didn’t draw any conclusions from the data because it is of poor quality. I merely presented it. You drew conclusions, and so did Meryl. Both of them are plausible and of course they are opposite.
Finally, the really interesting part. You said:
“put simply, if you’re vaccinated, you can still catch an infection, but it’ll be far less severe than in a non-vaccinated individual. You’ll most likely recover quicker, and you’ll be less likely to develop severe complications, which in the case of pertussis include death.
Add in that factor and who knows where the dice will fall? One thing’s for sure, they won’t fall in Greg Beattie’s favour.”
Hallelujah! Jason, I’ve been arguing for ages that deaths are a more valid way of estimating a vaccine’s usefulness. Death data isn’t perfect but it’s a hell of a lot closer to perfect than notifications. At the very least, it is complete. Every death is recorded. We don’t have to guess how many deaths are reported… they all are! And the bonus is they’re an indicator of severity also, as death is the most severe outcome. I carried out a very thorough examination of death data in my recent book and found little if any evidence of the whooping cough vaccine saving any lives whatsoever. That’s another subject, though.
All the best, and thanks for your comments.
Update – 13th February
It seems Jason Brown didn’t appreciate me calling him out over the ‘innumerate dingbat’ rant. But instead of telling me he wrote another blog piece about it… one to which I was alerted the other day by a fellow debater at https://groups.google.com/group/vaccination-respectful-debate/topics?hl=…
His new piece can be read here: http://www.mycolleaguesareidiots.com/archive/2012/02/08/The-innumerate-d…
Unfortunately, I’ve been unable to respond to him as my comments keep getting rejected, as you can see below.
At first I thought this could be because my comments were too large, and may exceed a character limit. But when I try to contact him I get the following:
So I’m left with no option other than to present my response here. I apologise in advance for its complexity. Things shouldn’t be so confusing. It only gets like this when you have to correct people’s errors. If anyone is able to copy it and post to his site please do so (and let me know you have done it, as his is not a site on my ‘most visited’ list).
After skating along on wafer-thin reasoning to deliver some fine vitriol, you suggest I shouldn’t have an arse-kicking contest with a centipede, and that you’re a centipede by virtue of the fact that you’re a computer programmer. For one, I’m not into arse kicking (my arse is a long way from your foot, anyway). I used to be a computer programmer as well, back when graphics wasn’t thought of and processing large amounts of meaningful data was the order of the day. Back then computer programmers generally had a passion for logic and algorithms, and although this is probably still the case I suspect many today just have a passion for games.
Either way, you’re obviously an intelligent person. However, there was a glaring problem with logic in your response. In order to justify the ‘innumerate’ part of the name, you had only one task – to point out where I was innumerate. And you didn’t do that… because you couldn’t. Because it isn’t possible. I displayed the data, and accompanied it with the following:
“For the years 2008-2010, there were a total of 9333 cases of whooping cough reported in children aged 0-4 years. Among them were 754 who were either too young or otherwise not eligible to have been vaccinated, and 1497 for whom vaccine status was not known.
Of the remaining 7082 cases whose vaccine status was known, and who were eligible to have received the vaccine, 5296 (75%) were fully vaccinated for their age. A further 986 (14%) were partly vaccinated. Only 800 (11%) were un-vaccinated.”
This is factually and numerically correct. On the basis of this and this alone you called me an ‘innumerate dingbat’. Obviously this was a stupid thing to do as it displayed your true abuse-at-all-costs colours. And you were understandably annoyed about being called out for it. So you’ve gone back to your computer and written another blog, taking my response apart bit-by-bit. You’ve explained how you’ve drawn the conclusions you have from the data (that you don’t understand, and therefore don’t know how to interpret). And you did all this obviously thinking you were teaching me something I didn’t know.
Being a computer programmer you would no doubt have a good understanding of the GIGO principal (garbage in… garbage out). In this case, the ‘garbage in’ is notification data. The ‘garbage out’ is your conclusion. How do we know it’s ‘garbage in’? That’s easy. There’s an obvious confounder… actually, there are many of them but there’s an obvious one which works to overestimate the value of the vaccine. Doctors assume the vaccine prevents the illness.
This confounder is something which every reasonable person on the planet would recognise. It’s so glaringly obvious that a child would pick up on it. But you (and your mates who are busy protecting the pro-vaccine message) are claiming it doesn’t exist! Mind you, you contradict yourself by admitting that it does exist:
“Assumption is not needed. The figures clearly demonstrate a protective effect.”
But still you think that, assumption or not, doctors are NOT predisposed to under-diagnosing the disease in the vaccinated, or over-diagnosing it in the not vaccinated! What planet are you from? You and I both grew up believing the vaccine prevented the illness, didn’t we? Right, everybody did. Doctors had it reinforced even more. Then they were tasked with marketing the thing and, finally, injecting it into the babies of wary mothers. Do you really think all this doesn’t have an effect on their decisions where the diagnosis is not clear (which is common with whooping cough)? Do you have any idea what the “double-blind” is for, in a randomised double-blind placebo-controlled trial? Have you ever heard of the term ‘investigator bias’?
You ask for citations for the bleeding obvious. I’m asking for citations to demonstrate that the data is safeguarded from it (yes I know data is plural… I hover between using the grammatically correct and the colloquial. Replace it with data-set if it helps). I can provide you with documentation showing official recognition of it but tell me, honestly, do I need to?
As you indicated above, the bias exists. You just think that it is understandable because the ‘figures’ support it. But THAT JUST SUPPORTS THE FACT THAT IT EXISTS. And the effectiveness of the vaccine is what’s in dispute (in fact, it’s what you’re trying to demonstrate). You can’t demonstrate it by assuming it’s true. That’s called self-fulfilling prophecy.
But getting back to GIGO. The notification data is only part of the ‘garbage in’ in your case (there’s that plural again). The other part is the community vaccination rates… the 95% etc. Actually, these aren’t ‘garbage in’. It’s here that your method falls apart because you don’t understand the nature of the data. You’ve compared the two data-sets, but I don’t think you realise how inappropriately you’ve done it. The surveyed rates are ‘snapshots’ – quite different to the notifications. Snapshots show the vaccine status of children at a single point in their life (e.g. their second birthday). If you want to meaningfully compare vaccination rates in the community to those in the notifications data you have to use proper case-control methodolgy and age-match EACH case. Sounds tedious, but it’s the only valid approach. And let’s face it, that’s what computers are for.
Let me give you an example to illustrate why your approach is invalid. You said you would like to include the ineligibles in the calculation as ‘unvaccinated’. OK – lets do that. You also said you didn’t think the 5-year-old rate should be included because the cases were ‘under-5’. OK – lets get rid of that. Now we are left with a survey rate of 95% for 2-year-olds and 91.8% for 1-year-olds. Let me throw in one more. For one-month-olds… it’s 0%. Perfectly legitimate because there are roughly the same number of children having a one-month-old birthday as there are having a two-year-old birthday. It’s just that none of them are vaccinated.
Next, using your ‘method’ we add the three estimates and divide by three: (95+91.8+0) / 3 = 62% (rounded). Do you now see the problem? Just to explain a little further, let’s look at what these snapshots mean. The day a child turns 6 mths old they are supposed to have three doses of whooping cough vaccine under their belt. If they don’t, they’re considered not appropriately vaccinated. However, the snapshot taken of them on their first birthday informs us as to whether they have caught up to the expected 3 doses yet. And that snapshot tells us that 91.8% of kids have indeed caught up with their 3rd dose by this time… six months after it was due. The second birthday snapshot tells us that 95% have caught up with this same third dose… 18 mths after it was due! But during that 18 months had one of these children developed whooping cough BEFORE getting that catch-up shot they would have been placed in the ‘not fully vaccinated’ group.
For example, take two kids. We’ll call them Tim and Rudolph. Both are 9 mths old and both are behind in their shots (this is apparently common as many parents like to delay as long as they can). The following week, poor Tim develops whooping cough but Rudolph doesn’t. You with me? Both not up-to-date but only Tim gets sick. He becomes a notification and is classed as not up-to-date. A couple of months later Rudolph gets his shot, and becomes part of your snapshot of up-to-date kids. Two kids, same vaccine status but recorded differently in the two data-sets.
I know it’s complex, but as a computer programmer I know you will be able to follow it. The bottom line is you can’t compare the vaccination rate in community snapshots to the vaccination rate in cases (unless they are the same snapshot). Well… you CAN compare them coarsely but only as a guide. And if you do, you must understand that the rate in the snapshot will ALWAYS be higher than that in the cases.
Note: for simplicity, I used the example of a one day snapshot. The government cut a slightly wider path. Their methodology used a three month birth cohort. In other words, on the assessment day, their query ran through the medicare database and picked up all kids who turned two in the last three months. (This addition of three months serves to push the estimate even higher, allowing a further three months for catch-up.)
The only correct way to use the community snapshot figures is the way Meryl Dorey has. I realise that’s not what you want to hear, but all the snapshots show us is that vaccination has increased substantially. Meryl has correctly pointed out that THIS HAS OCCURRED ALONGSIDE A METEORIC INCREASE IN NOTIFICATIONS.
So in summary:
1. With case notifications there is a bias (toward unvaccinated) which, as far as I’m aware, has never been quantified… and for which there are no controls (unless you can give me a citation which shows there are).
2. The snapshots show us vaccination has increased, but provide no suitable data for allowing fine comparison with cases (whose vaccine status is determined at time of illness).
GIGO. If you use these sets of data in the manner you have you will be drawing poor and misleading conclusions. Both of the above issues favour the vaccine.
The only reasonable conclusion from the data is what I originally quoted when I published it, and my subsequent statement of surprise that so many notifications are in the vaccinated despite doctors believing the vaccine prevents the illness. My personal thoughts are in line with Meryl’s statement that findings such as this add weight to the notion that we are dealing with a useless vaccine.
I know you appreciate quality-of-data issues because I read your assessment of mortality data… viz:
“You cannot judge the effectiveness of a vaccine based on death rates alone because, for one thing, death rates are too heavily impacted by other factors. Antibiotic use, timeliness of diagnosis, complicating infections, quality of medical care, ideological objections to conventional treatment – all of these become confounding variables which alter the death rate without altering the incidence rate, and it becomes correspondingly difficult – I would say almost impossible – to tease out exactly which factor is which. This is why control of variables is so important in epidemiology, and in fact in all scientific fields. Messy datasets are datasets from which one cannot draw strong conclusions, and trying to figure out vaccine efficacy from death rates is messy.”
That’s all reasonable comment. I have no issue with it. But my personal opinion is that, where deaths are still numerous, they are a more suitable proxy for incidence than are notifications. Notifications are just way too sloppy. Incidence (real incidence) would be great, especially if it were collected in a blinded fashion.
PS – your comment…
“All cases less than 8 weeks are ineligible. Those who received the correct does for age are in the vaccinated category. They’re not fully vaccinated, but they’re already included there. Didn’t you accuse me of a lack of comprehension earlier, Greg?”
I think you’ve misinterpreted this. Cases less than 8 weeks old were coded as ‘ineligible’. There were a few who received the vaccine at 6-8 weeks old due to a special response to an outbreak in two states. They were included as appropriately vaccinated for age.