Three Cochrane Reviews focussing on the prevention of influenza in healthy adults, healthy children, and in the elderly are long-running reviews under the same senior author team. The protocol for the oldest review was first published 20 years ago.

Over the years the reviews have progressively accumulated evidence leading to ever greater stability in their conclusions. ‘Stable’ is a publication flag that usually indicates that the results are unlikely to change with the inclusion of new studies, such is the certainty of the results. The influenza vaccine reviews present us with a partly different situation. Readers will notice important outcomes where we have little or no data. They may also see that for some measures of influenza and ‘influenza-like illness’ (ILI), we have low-certainty evidence. We have reached a point where the evidence is not showing anything different to what it has done for a number of years. We know with varying degrees of certainty about vaccination effects on influenza and ILI, but the gap in our understanding of how vaccines affect the consequences of influenza persist. For each review, the impact of single studies is documented in the summary table 1 "Studies included in the various versions of this review and their impact on our conclusions". This month the three reviews appear in their latest updated and stabilised format.^1,2,3 Whilst we do not believe that periodic updating will complete the picture, our decision to stabilise is conditional. The three reviews will not be updated again unless certain criteria are met.

First, a new trial that meets inclusion criteria becomes available. Few trials of interest have been conducted recently, as a comparison with an inactive control is considered by some to be unethical. In the elderly, the latest completed trial dates from nearly two decades ago. Our searches have failed to find relevant ongoing trials.

A second condition is the introduction of a new generation of vaccines, based on new technology. This is possible given that several new technologies are being developed, such as vaccines containing fragments of the haemagglutinin antigen “stalk” on the viral surface (so called stalk-specific vaccines).⁴

The third condition is more complex: the development and testing of a new causal paradigm for ILI and influenza. Currently, massive worldwide machinery is needed to produce new vaccines every year to address viral antigenic changes, and to address the poor persistence of the antibody response in individuals. However, the vaccination selection and production programmes are based on aetiological assumptions which are neither explanatory nor predictive, as shown in our reviews. Overall the largest dataset to have accumulated to date is from trials conducted in the population least likely to benefit from vaccines but most likely to produce immunity: healthy adults. In healthy adult trials a high serological response is matched by a very small clinical effect (71 healthy adults need to be vaccinated to prevent one of them experiencing influenza). This weak effect cannot be explained simply by the mismatch of vaccine antigens with wild virus ones. A larger effect is observed in children over the age of two (five children need to be vaccinated to prevent one case of influenza, although there is huge uncertainty around these estimates). There is little evidence on prevention of complications, transmission, or time off work. Other reviews have drawn similar conclusions.⁵

During stabilisation we updated the randomised evidence, but for the first time have decided against updating the large observational evidence base. The observational dataset still appears in the reviews, but only as a historical record of earlier versions. Observational studies were included in the reviews over a decade ago in the hope they could provide long-term and rare harms data and improve the external validity of the trial evidence. They turned out to be of such low quality that their conclusions were inconclusive or unreliable. The most important example is the case-negative study to assess influenza vaccine effectiveness post hoc (i.e. after an influenza season) by harvesting data from a surveillance programme. This study design, which is similar to a case-control study, selects influenza cases (cases of ILI which have tested positive for influenza) and controls (cases of ILI which have tested negative) and calculates the relevant odds ratio (OR) of exposure to that season’s vaccine. An estimate of vaccine effectiveness is derived from this OR using a standard formula (vaccine effectiveness = 1 - OR%). However, despite their institutional popularity,^6,7 case-negative designs have limited public health significance because the design does not test field effectiveness, but, rather, laboratory efficacy of the vaccine (the capacity of the vaccine to generate a negative polymerase chain reaction (PCR) result). Both cases and controls are symptomatic, so any prevention is solely focused on PCR negativity. In addition, no useful public health absolute measures of effect can be derived (such as absolute risk reduction (ARR) and its reciprocal number needed to vaccinate to prevent one case (NNV)) because the background rates of infection and viral circulation are not part of the calculation of the estimates of effect. There are also problems with the mathematical assumptions made in this design (for details see the reviews). Case-negative studies are an illustration of the narrow and retrospective focus on influenza viruses at the expense of overall ILI - the illness cluster of interest to patients and their clinicians. Retrospective calculation of relative estimates of laboratory efficacy can be of interest for future decisions on composition of vaccines, but their relevance to everyday decisions seems questionable.

The underlying assumption that influenza vaccination does not affect the risk of non-influenza is contradicted by a recent report from the follow up of a trial by Cowling et al.⁸ In 115 participants, those who received trivalent influenza vaccines had higher risk of acute respiratory infection associated with confirmed non-influenza respiratory virus infection (RR, 4.40; 95% CI, 1.31–14.8) compared to placebo recipients. The agents were mainly rhinoviruses and coxsackie/echoviruses; ILI episodes occurred shortly after a peak of influenza activity.

Current yearly registration of candidate influenza vaccines is based on their ability to trigger a good antibody response. But antibody responses are poor predictors of field protection. This is another example of the use of surrogate outcomes in biomedicine, where effects on clinically important outcomes remain unmeasured or unproven from randomised trials: complications and death by influenza.

The simple answer is that we do not understand what the target is. What is the threat of influenza, and what can we ever expect of the vaccines?

The WHO Global Influenza Programme(GIP) with its backbone Global Influenza Surveillance and Response System(GISRS) is a complex network of 143 national reference centres and specialist laboratories in 113 states carrying out surveillance of circulating influenza viruses. GISRS was devised and developed to guide annual influenza vaccine production, and the emphasis is mainly on influenza viruses, their variants, and emerging strains.

However there is no reliable system to monitor and quantify the epidemiology and impact of ILI, the syndrome that presents clinically. Few states produce reliable data on the number of physician contacts or hospitalised cases due to ILI, and none tie these data to the proportion of ILI caused by influenza. We do not know for certain what the impact of ILI is, nor the impact of the proportion of ILI caused by influenza. Prospective studies apportioning positivity to the scores of viruses probably causing ILI are rare, as interest is focused on influenza. The standard quoted figure of 36,000 yearly deaths in the US is based on the “respiratory and circulatory deaths” category including all types of pneumonia, including secondary to meconium ingestion or bacterial causes. More recently, the US Centers for Disease Control and Prevention (CDC) have proposed estimates of impact ranging between 3,000 and 49,000 yearly deaths. When actual death certificates are tallied, influenza deaths on average are little more than 1,000 yearly. So, the actual threat is unknown (but likely to be small) and so is the estimation of the impact of vaccination.

The uncertainty over the aetiology of ILI, its capricious nature and the weak correlation between immunity and protection, point to possible causal or concurrent factors in the genesis of both ILI and influenza. In other words, virus positivity may only be one of the factors necessary for a case of influenza or ILI to manifest itself.

We await to see whether anyone has the interest or the courage to develop effective ways to control upper respiratory viral syndromes. Meanwhile our reviews will remain as a testimonial to the scientific failure of industry and governments to address the most important clinical outcomes for patients.

Tom Jefferson
Senior Associate Tutor
University of Oxford and Centre for Evidence Based Medicine
Oxford OX2 6GG
 

Alessandro Rivetti
Dipartimento di Prevenzione - S.Pre.S.A.L, ASL CN2 Alba Bra, Alba, Italy
 

Vittorio Demicheli
Cochrane ARI Group

Tom Jefferson is Senior Associate Tutor at the University of Oxford and Centre for Evidence Based Medicine. He and his co-authors are long-time Cochrane authors and contributors. In this post they have shared their personal interpretation of the findings and relevance of three recently updated Cochrane Reviews on the effectiveness of influenza vaccines on various populations. Please also note the standard disclaimer for all Cochrane Blog posts at the bottom of this page.
 

Acknowledgments

Chris Del Mar, Peter Doshi, Mark Jones
 

References

1. Demicheli V, Jefferson T, Ferroni E, Rivetti A, Di Pietrantonj C. Vaccines for preventing influenza in healthy adults. Cochrane Database of Systematic Reviews 2018, Issue 2. Art. No.: CD001269. DOI: 10.1002/14651858.CD001269.pub6.

2. Jefferson T, Rivetti A, Di Pietrantonj C, Demicheli V. Vaccines for preventing influenza in healthy children. Cochrane Database of Systematic Reviews 2018, Issue 2. Art. No.: CD004879. DOI: 10.1002/14651858.CD004879.pub5.

3. Demicheli V, Jefferson T, Di Pietrantonj C, Ferroni E, Thorning S, Thomas RE, Rivetti A. Vaccines for preventing influenza in the elderly. Cochrane Database of Systematic Reviews 2018, Issue 2. Art. No.: CD004876. DOI: 10.1002/14651858.CD004876.pub4.

4. Treanor J. Influenza vaccination. New England Journal of Medicine 2016;375(13):1261-8. [DOI: 10.1056/NEJMcp1512870]

5. Osterholm MT, Kelley NS, Sommer A, Belongia EA. Efficacy and effectiveness of influenza vaccines: a systematic review and meta-analysis. Lancet Infectious Diseases 2012;12(1):36-44.

6. Foppa IM, Haber M, Ferdinands JM, Shay DK. The case test-negative design for studies of the effectiveness of influenza vaccine. Vaccine 2013;31(30):3104-9.

7. Valenciano M, Ciancio BC, I-MOVE study team. I-MOVE: a European network to measure the effectiveness of influenza vaccine. Eurosurveillance 2012;17(39):pii=20281.

8. Cowling B, Fang V, Nishiura H, Chan K-H, Ng S, Ip DKM, et al. Increased risk of noninfluenza respiratory virus infections associated with receipt of inactivated influenza vaccine. Clinical Infectious Diseases 2012;54(12):1778–83.
 

Disclosure

TJ was a recipient of a UK National Institute for Health Research grant for a Cochrane Review of neuraminidase inhibitors for influenza. In addition, TJ receives royalties from his books published by Il Pensiero Scientifico Editore, Rome and Blackwells. TJ is occasionally interviewed by market research companies about phase I or II pharmaceutical products. In 2011-13, TJ acted as an expert witness in litigation related to the antiviral oseltamivir, in two litigation cases on potential vaccine-related damage and in a labour case on influenza vaccines in healthcare workers in Canada. He has acted as a consultant for Roche (1997-99), GSK (2001-2), Sanofi-Synthelabo (2003), and IMS Health (2013). In 2014 he was retained as a scientific adviser to a legal team acting on oseltamivir. TJ has a potential financial conflict of interest in the drug oseltamivir. In 2014-16, TJ was a member of three advisory boards for Boerhinger Ingelheim. He is holder of a Cochrane Methods Innovations Fund grant to develop guidance on the use of regulatory data in Cochrane Reviews. TJ was a member of an independent data monitoring committee for a Sanofi Pasteur clinical trial on an influenza vaccine. Between 1994 and 2013, TJ was the coordinator of the Cochrane Vaccines Field. TJ is a co-signatory of the Nordic Cochrane Centre Complaint to the European Medicines Agency (EMA) over maladministration at the EMA in relation to the investigation of alleged harms of HPV vaccines and consequent complaints to the European Ombudsman. TJ is co-holder of a John and Laura Arnold Foundation grant for development of a RIAT support centre (2017-2020) and Jean Monnet Network Grant, 2017-2020 for The Jean Monnet Health Law and Policy Network.

Demicheli and Rivetti: none.