This is part of a series of Cochrane case studies about prioritization work by Cochrane Review Groups. You can read more by visiting the priority setting case study page.
The Cochrane Tobacco Addiction Group (CTAG) has a longstanding commitment to prioritization, largely through informal discussions at the editorial base and by monitoring emerging research. In 2016, the year of their 20th anniversary, the Group decided to undertake a more in-depth prioritization exercise as part of the birthday celebrations.
As their starting point the group started looked at the well-established and successful prioritization framework developed by James Lind Alliance (JLA) in the UK. The JLA Priority Setting Partnership approach focuses heavily on clinicians and consumers, and the Cochrane CTAG wanted to ensure that the voices of researchers, commissioners, funders, and policymakers were also heard. Moreover the JLA approach would have taken more time than CTAG had to ensure completion before the 20th anniversary celebrations, so ultimately the group decided to use a modified version of the JLA framework to fit their circumstances.
The first task was to develop an online survey using Survey Monkey, which was disseminated on social media and through stakeholder mailing lists, and aimed to identify the stakeholder's most pressing unanswered questions about interventions and policies for the prevention and treatment of tobacco addiction. This survey allowed each person to propose up to four questions and explain why each was important to them. It was completed by several groups, including people who smoked/had smoked in the past, healthcare professionals, policymakers, service commissioners, research funders, and tobacco addiction researchers. In total this survey produced more than 600 questions - far more than had been anticipated!
Next, the process of narrowing down the questions to a more manageable number began. The group started by removing duplicate questions, using a dual decision making process to determine whether the question posed had already been answered by an existing, good quality systematic review or other reliable guidance. An unexpected but very useful consequence of this exercise is that it identified where Cochrane Review findings had not been adequately publicized. This knowledge was carried forward to the workshop stage of the process, when participants were asked to contribute ideas about improving evidence dissemination.
After all the sifting, 180 questions remained. This was far too many to ask people to rank individually, so the group sorted the questions into 15 categories. A second survey was sent to those who had already taken part and then the Delphi method used. Participants were asked to pick their top three of the 15 categories and then rank only those questions that appeared in their chosen categories.
The final piece of the process was a workshop attended by members of the public, guideline developers, stop smoking advisers, clinicians, and funders. They were invited to the CTAG 20th anniversary celebration to decide on the final list of high priority questions. These workshops were led by an independent facilitator who was paid through a grant that the CTAG received from the NIHR School for Primary Care Research. These funds also helped to support attendance at the workshop of members of the public, and covered venue costs, video, and photography. To ensure that participants understood the scope of the task, members of the CTAG editorial team spent the first half of the workshop talking about the history and work of CTAG, the history of tobacco research generally, and the steps in the process leading up to the workshop. The actual ranking exercise took place in the second half of the day and whilst it did focus on the results of the survey, the facilitator took care to allow new ideas to emerge.
The methodology of this process was presented at Evidence Live (UK) in June 2016, and even received some attention from a BioMed Central representative keen to see the exercise written up. The CTAG do plan to document their approach to prioritization, as it produced questions and ideas that are not only relevant for Cochrane, but also will be of great interest to others, such as the National Institute for Health Research (NIHR) in the UK.
Despite attracting some financial support, the CTAG were not permitted to use these funds to hire staff to assist with the process, so all of the work leading up to the day of the workshop had to be absorbed at the editorial base. The fact that they all had extensive knowledge in the field of tobacco addiction meant that the de-duplication and categorization work that occurred ahead of the workshops was comparatively easy; however not all Cochrane Groups will have the resources to be able to process such a large quantity of data.
Despite the cost and resource challenges, the CTAG felt the approach worked well – it was rigorous yet relatively short, it engaged a broad range of stakeholders, identified gaps in the CTAG portfolio, generated a list of questions that are relevant both in and outside of Cochrane, and galvanized the group into creating a more formal dissemination plan for their reviews.
For any questions or comments about this blog series or if you’d like to tell us about your prioritization work, please contact Ruth Foxlee at the Cochrane Editorial Unit – rfoxlee@cochrane.org
Ruth Foxlee - Information Specialist, Cochrane Editorial Unit
Lydia Wilson – Intern, Cochrane Editorial Unit