Eminence vs. evidence

posted on: 2012-11-30 06:57

Alan Cassels is the author of the recently published Seeking Sickness: Medical Screening and the Misguided Hunt for Disease, and is currently writing a social history of The Cochrane Collaboration. Here he presents the essential differences – with illustrative examples – between ‘eminence-based medicine’ and ‘evidence-based medicine’.

Cross-posted with grateful acknowledgment from common ground, Western Canada's “biggest and best-loved monthly magazine dedicated to health, wellness, ecology and personal growth”.

What is ‘eminence-based medicine’? It means relying on the opinion of a medical specialist or other prominent health official when it comes to health matters, rather than relying on a careful assessment of relevant research evidence. You might be asking, “Who am I to question an ‘expert’, especially a physician, a specialist or a prominent medical researcher who knows so much more than me?” Here are a few recent examples of eminence-based medicine in the real world, which hopefully will leave you questioning.

Exhibit A
Recently, an Australian journalist shared with me the thoughts of a very prominent professor of medicine concerning cholesterol-lowering drugs. This national expert is one of the authors of Australia’s lipid guidelines – guidance for doctors on cholesterol-lowering or statin drugs – and he weighed in with his opinion around the benefits and harms of statins.

His answers sounded very authoritative, citing studies and reports that buttressed an unmistakable love of statins. He also cherry-picked his data, selecting and presenting research evidence that supported his love affair, concluding that most people, even people at moderate risk of heart disease, should be taking a statin. Saying that double-blind trials provide incontrovertible proof that statins reduce the risk of heart attacks and are among the “safest class of drug ever developed”, he maintained that more people needed to take these drugs. Risks? He said that the “risks of side effects are less than 0.1%”.

If you prefer evidence to expert testimony, you’d look for the most reliable research studies out there and compile them into an analysis that made sense, using methods that are clear and replicable. Above all, referring to single studies is bad, bad, bad because a single study only represents a small slice of research. If you present a study and I counter with a study that found the opposite, the resulting ping-pong game would leave us both exhausted, but no smarter. Instead of single studies, you need to demand, and depend upon, overviews of all available research in an area, such as a systematic review of the literature. This is the kind of stuff produced by The Cochrane Collaboration, and on statins it tells a very different story.

The Cochrane Review says that statins for primary prevention – drugs to prevent a person’s first heart attack – might have some benefit, but they can’t say for sure. The story is incomplete. Their overview of the best quality research is hampered by the fact that some companies refuse to release the full set of adverse event data. Other research suggests that 0.1% risk of side effects is a gross underestimate in the real world, and that as many as 20% of patients experience muscle weakening when taking statins. So whom do you trust – the expert claiming these are the safest drugs in the world, or the evidence that says we actually don’t have a full picture of the safety of these drugs? As one doctor once told me, “In God We Trust; all others must show data”. To which I say, “Amen”.

Exhibit B: ‘flu drugs
British Columbia (BC) launched a very aggressive anti-‘flu policy this fall, with healthcare workers being forced to get the ‘flu shot or wear a mask. The rationale is that anyone caring for patients shouldn’t also be passing on viruses to them and making them sick. Fair enough. But does the evidence support mandatory ‘flu shots for healthcare workers? Does the vaccine even work to prevent the spread of the virus?

BC Provincial Health Officer Dr. Perry Kendall thinks so and so do some of his colleagues, including Dr. Paul Van Buynder, Chief Medical Office of Fraser Health. A researcher friend of mine asked Dr. Van Buynder for the evidence used to support this new policy. He received a reference list that fits into the ‘shotgun’ category of literature reviews, containing dozens and dozens of references, some relevant studies as well as editorials, commentaries, eminent opinions, and other detritus. Hmmm. And this is somehow supposed to placate us as a reasonable ‘evidence base’?

When you use a shotgun, you’re likely to hit something – maybe. Included in the province’s ‘evidence’ to support the new policy was a whole range of studies and outcomes, including some deemed ‘biologically implausible’ - such as that vaccinating health workers reduces death by all causes, which is to say the ‘flu shot also prevents death by strangulation, gun shots, and zombies.

To counter this, I looked for the sniper rifle and spoke to Dr. Tom Jefferson, a Rome-based researcher who produces ‘flu vaccine reviews for The Cochrane Collaboration. He’s been doing systematic reviews of ‘flu vaccines and ‘flu drugs for over a decade, so he has more than a passing interest in the subject. Jefferson’s team examined four large cluster randomised trials and one cohort trial of nearly 20,000 healthcare workers. According to their findings, the ‘flu vaccine showed “no effect on specific outcomes: laboratory-proven influenza, pneumonia, or deaths from pneumonia”.

In other words, the ‘flu policy, while eminently agreeable, is unsupported by evidence that has been systematically collected, critically evaluated, and properly synthesised. BC is not the only jurisdiction to adopt a ‘trust us, we’re experts’ pose, but in my opinion, that response is only fit for underlings, not intelligent, responsible healthcare workers facing the pointy end of a syringe this season.

Exhibit C
When my book came out last year, the Vancouver Sun asked me if I’d like to publicly debate the prostate-specific antigen (PSA) test, a blood test used to detect signs of prostate cancer. “Whoopee”, I said. I jumped at the chance to step in the ring with a prominent urologist at University of British Columbia over the value of a very controversial test for a disease that, although occasionally fatal, mostly isn’t, while the PSA test can make many men incontinent or impotent due to unnecessary treatment. The motto for the PSA test, synthesised by the United States Preventive Services Task Force (USPSTF), which uses strong systematic evidence gathering and synthesis is this: ‘Just Don’t Do It’. That drove the urologists nuts.

I really wanted to hear the urologist stand up and tell me the USPSTF evidence is wrong and that we need to keep subjecting men to the PSA test. Two days before the debate, the urologist backed out and I understand why: eminence-based medicine can look very silly in a public debate. He probably felt I was going to wipe the floor with him, but if you know any urologists who would like to debate the PSA with me, please contact me via www.alancassels.com

A final note
A recent study from US Public Citizen found that, since 1991, there have been 239 legal settlements, totaling $30.2 billion in federal and state penalties, levied against US pharmaceutical companies. There’s a real laundry list of crimes, but defrauding the government, hiding drug safety information, and hawking drugs for purposes beyond which they are approved are the main ones. Drug companies have pledged to change, signed ‘corporate integrity agreements’ and indicated that they want to move on, promising a better future. We can be hopeful, but we also have to be realistic. Paying huge fines for illegal activity is one thing, but will they be still playing the eminence game? Will they continue to fund their own experts and do research that goes through a selective reporting of ‘the evidence’? Sadly, that’s probably going to be the case so you must immunise yourself: keep asking questions and questioning answers.

Comments

Cochrane's previous meta review of flu vaccine trials

Cochrane conducted a meta review of ~10 influenza vacination trail papers a few years ago. I had it bookmarked but then Cochrane went and changed their website and link is dead. I meantioned the study and someone online I mentioned it too was able to track it down but I didn't grab the book mark. Since then I've never been able to find it. Here's the grab from the abstract in my bookmark:

Wiley Online Library: Reference Work: The Cochrane Library 2009, Issue 4     

We included 50 reports. Forty (59 sub-studies) were clinical trials of over 70,000 people. Eight were comparative non-RCTs and assessed serious harms. Two were reports of harms which could not be introduced in the data analysis. In the relatively uncommon circumstance of vaccine matching the viral circulating strain and high circulation, 4% of unvaccinated people versus 1% of vaccinated people developed influenza symptoms (risk difference (RD) 3%, 95% confidence interval (CI) 2% to 5%). The corresponding figures for poor vaccine matching were 2% and 1% (RD 1, 95% CI 0% to 3%). These differences were not likely to be due to chance. Vaccination had a modest effect on time off work and had no effect on hospital admissions or complication rates. Inactivated vaccines caused local harms and an estimated 1.6 additional cases of Guillain-Barré Syndrome per million vaccinations. The harms evidence base is limited.

 

I recall the detailed conclusion actually seemed to say zero days saved off work in most studies which contradicts this abstract (anyhow).


Does anybody know how I can find this paper?

Re: Cochrane's previous meta review of flu vaccine trials

Hi Alastair,

I think  this is the link to the review you are refering to above: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD001269.pub4/abstract;jsessionid=4F319C07B0C728147A65BC538805DA16.f03t04 

However, this is not the latest version of the article; this one is: http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD001269.pub5/abstract

Hope that's helpful!

 

 

 

Re: Eminence vs. evidence

It must be understood that The type of flu virus changes from year to year, causing experts to create a new vaccination annually. Flu shots typically prevent against three different flu viruses to help insure that individuals are properly protected from the year’s new strain of virus.​ We should all now be in a position to view and initiate legistaltion to make flu vaccinations mandatory - 

Re: Eminence vs. evidence

Spot on. I see this same issue in my own work where industry trends and expertism continue to support the myth that proper manual handling prevents back pain, but the Cochrane review (http://www.ncbi.nlm.nih.gov/pubmed/22317058) tells the opposite story. We have a whole profession of manual handling trainers which is founded on thin air.

Re: Eminence vs. evidence

Reminds me too much of the cigarette safety studies  in recent years....

Re: Eminence vs. evidence

hmm nicely written, I would also love to re post the link on my page... keep the detailed and good work coming

  • These online gambling australia have a place for all types of people today, additionally a multilingual seasoned aid team that's surely obtainable previously mentioned the cellphone all times in the week.

Re: Eminence vs. evidence

Could I post the article and and link on my site?

Re: Eminence vs. evidence

You are welcome to post a link to the article, but please do not reprint the text in full without submitting a full request, detailing how and where it would be used, for the author's attention at techsupport@cochrane.org. Thank you.

Re: Eminence vs. evidence

I would like to be able to tweet the link and/or post on my Facebook page. Is this feasible?

Zbys

Re: Eminence vs. evidence

Zbys - yes, please feel free to publicise the link to this blog post.

Nancy, Cochrane Web Team

Re: Eminence vs. evidence

I'm not entirely sure I can agree with the implicit message of this post, which seems dangerously close to "anyone who disagrees with the conclusions reached by the Cochrane Collaboration is wrong, and likely a paid shill of industry".

Does systematic review eliminate subjectivity and the influence of personal bias?  A Pubmed search on almost any subject shows that the "ping pong" of dueling expert opinions has largely been replaced by the ping pong of dueling meta analyses.  Statins are useful in primary prevention or they aren't. Tiotropium increases mortality or it doesn't. Varenicline poses a severe CV risk to patients with no CV disease history or poses no risk at all in this group.  Reboxetine has no demonstrable anti-depressant activity or is similar to other antidepressants. 

While Cochrane Group and others have extensively discussed the potential for selective trial publication to bias comprehensive reviews, the subjective process of selecting studies for inclusion and choosing the most appropriate statistical analysis seems to have played a major role in the conflicting conclusions obtained in many of these recent studies.

 Ping pong is alive and well.

Re: Eminence vs. evidence

John, I think you missed the message. The author isn't saying that Cochrane Reviews and Meta-Analyses lack subjectivity. Neither did the author say that Cochrane authors balk at criticism, nor that anyone who disagrees with the Collaboration is a "paid shill of industry," as you claim. The author is simply alluding to the fact that "experts" funded by industry sometimes are unwilling to share adverse event  data because they are funded by industry. Or, when they do share some of their data, they cherry pick. In almost every research study, including systematic reviews and meta-analyses, we cherry pick somewhat. That is partly why we pick primary outcomes and secondary outcomes, or pursue research that is not overambitious. But the difference between a well-done systematic review versus an "expert" who simply speaks for the sake of speaking is that the systematic review or meta-analysis is subject to greater scrutiny. Further, Cochrane reviews are often updated and when evidence is weak, results are still shared.

NB: These are my personal views and do not necessarily represent the views of the Collaboration

Post new comment

The content of this field is kept private and will not be shown publicly.
If you do NOT enter a subject, the first few words of your response will be used.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <p> <br> <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>

More information about formatting options

Updated on: November 30, 2012, 6:57

The Cochrane Official Blog is curated and maintained by the Cochrane Web Team. To submit items for publication to the blog, please email techsupport@cochrane.org.

The Cochrane Blog presents commentary and personal opinion on topics of interest from a range of contributors to the work of The Cochrane Collaboration. Opinions posted on the Cochrane Blog are those of the individual contributors and do not necessarily reflect the views or policies of The Cochrane Collaboration.