William (Bill) Silverman (1924-2004) was one of the founders of American neonatal medicine. He was honoured repeatedly as one of the pioneers in his specialty; however, he often evoked somewhat contradictory responses amongst his colleagues because he was in the habit of raising troubling questions about the scientific basis and ethics of his and their practices. Like many of the people who have helped to establish Cochrane, Bill Silverman could be regarded as a 'troublemaker'. As he reiterated frequently, however, criticism is a form of troublemaking that can help to drive progress. Furthermore, criticism should not be limited to examining the work of others, but should also include self-criticism.
Every year Cochrane recognizes the outstanding work of Bill Silverman with its own award. It is offered annually and explicitly acknowledges Cochrane's value of criticism, with a view to helping to improve its work, and thus achieve its aim of helping people make well-informed decisions about health care by providing the best possible evidence on the effects of healthcare interventions.
Paula Willamson won for her paper; ‘Assessing the relevance and uptake of core outcome sets (an agreed minimum collection of outcomes to measure in research studies) in Cochrane systematic reviews: a review' We asked her about this award and what it means to her.
What made you decide to enter for this prestigious Cochrane award?
Five years ago, Mike Clarke and I wrote an article where we argued for greater involvement of systematic reviewers in the development and implementation of core outcome sets (COS). A COS is an agreed standardised set of outcomes that should be measured and reported, as a minimum, in all clinical trials in specific areas of health or health care. Because systematic reviews, especially Cochrane reviews, include clinical trials, we suggested that considering COS might help with planning of systematic reviews, in order to provide evidence on the effects of healthcare interventions on outcomes that matter to patients and other decision-makers.
The team who came together to undertake this work – myself, Ricardo de Ávila Oliveira, Mike Clarke, Sarah Gorst, Karen Hughes, Jamie Kirkham, Tianjing Li, Ian Saldanha, and Jochen Schmitt - have had varied experience in conducting Cochrane reviews, developing COS and advancing their methods, assessing COS uptake, and designing clinical trials. We were all keen to find out whether Cochrane reviewers more generally were aware of and considering COS. When we saw the results, we thought there was the potential that our findings could have a positive impact on Cochrane reviews and decided to submit our nomination.
Can you tell us a little more about what made you decide to enter for the Prize and the process involved?
The paper that won us this year’s award examined whether Cochrane reviewers are following the MECIR standard relating to choosing outcomes for a review, namely that ‘Where available, established sets of core outcomes should be used’.
We examined 100 Cochrane reviews from 2019 and found that authors of 40% of reviews noted problems due to outcome inconsistency across the included studies in the review. However, a COS was referenced in relation to the choice of outcomes in just 7 reviews, despite a relevant COS existing for a further 27. We described how we identified relevant COS by searching the Core Outcome Measures for Effectiveness Trials (COMET) database, applying an approach used previously to assess the relevance of a COS to a review in terms of the scope of each, defined in terms of the health condition, population and healthcare interventions covered.
The paper that won us this year’s award showed that whilst Cochrane reviewers are starting to use core outcome sets to inform their reviews, there is potential for much greater uptake of COS in Cochrane reviews. We made several recommendations on how to achieve this. We suggest that Cochrane reviewers follow a similar approach to ours to identify COS that they might use in their new reviews and updates of existing reviews. We provided to each Cochrane Review Group (CRG) Editor a reference list of the completed and ongoing COS that were within the scope of their CRG, to enable greater consideration going forward.
Using high quality relevant COS will help to provide evidence on the effects of healthcare interventions on outcomes that matter to patients and other decision-makers. We thought entering our paper for the Bill Silverman Prize could increase visibility of our findings. It could raise awareness of the benefits of considering COS when choosing the outcomes to assess in a review, and which of these to include in Summary of Findings (SoF) tables to summarise the results for the most important outcomes and to show the quality of this evidence.
How did it feel to win? What does this award mean to you, personally?
It is always encouraging to receive praise for the work one is involved in, but to receive this award from Cochrane is special. All of our study team have huge respect for the work that Cochrane does and many have been actively involved in conducting Cochrane reviews and related methodological work. It is so rewarding to know that the panel considered our paper important and relevant enough to Cochrane’s work to have given us this award. If our publication is read by systematic reviewers, who then look for and consider COS for their next review or review update, we will be delighted. If those same systematic reviewers find challenges or limitations with existing COS, then we will be even more delighted to hear about their experiences. We also encourage systematic reviewers to get engaged in COS development work because their perspective, as those who summarise the evidence, is very important.
We are proud to join the list of winners of the Bill Silverman Prize. Thank you, from myself and my co-authors, for this award.