Bronwen Merner, Dianne Lowe, Louisa Walsh and Sophie Hill from Cochrane Consumers and Communication and stakeholder panel members, Anne Mussared and Cheryl Wardrope share their top tips for involving stakeholders in Cochrane Review screening.
Stakeholder involvement in health research, including Cochrane Reviews, is beneficial in improving research relevance and implementation (Pollock et al 2018). Reflecting this, the Cochrane Knowledge Translation Framework now includes co-production as a core theme. Despite the growing prominence of stakeholder involvement in research at the policy level, it remains an enigma for many researchers on the ground (Concannon et at 2018). To help remedy this, this blog describes the process and experience of co-producing the full-text screening stage of a Cochrane qualitative evidence synthesis (QES) on person-centred care with a stakeholder panel
Previous stakeholder involvement in the review
An Australian-based, 18-member stakeholder panel involving consumers, health practitioners and health decision-makers was convened by the researchers in the early stages of this review. In 2017, the panel met twice via teleconference to shape the topic (consumers and providers working in partnership to promote person-centred health services), review type (qualitative evidence synthesis) and conceptual development of the protocol. Following these teleconferences, the researchers sought the panel members’ interest in having a face-to-face meeting to co-produce the full-text screening step of the Cochrane Review. Following a positive response, a face-to-face meeting day took place in November 2018.
The full-text screening training day
As many Cochrane authors will know, screening can be an overwhelming task. Applying selection criteria to complex interventions and working out what should be ‘in’ or ‘out’ often needs detailed discussion. This complexity is potentially magnified when the screeners are relatively new to the research process, and more specifically, the Cochrane Review process. Therefore, the research team began the face-to-face meeting with a practical explanation of what screening is, why it’s important and where it fits within the Cochrane Review process. The researchers then gave some examples of how to apply the inclusion and exclusion criteria.
After this brief training session, the researchers and stakeholders worked in small groups to apply the inclusion criteria to a range of full-text qualitative research articles. Neither the research team nor the stakeholders had read the articles prior to the meeting, so the decisions about inclusion or exclusion were shared after extensive discussion and problem-solving. The process was aided through the research team’s prior development of a screening sheet which proved very useful (for stakeholders and researchers alike!) in making inclusion or exclusion decisions.
After two 45 minute sessions of screening, each small group had screened three or four full-text articles. Stakeholders expressed interest in continuing to screen articles after the sessions ended, so we sent an email invitation to follow-up after the meeting. Four stakeholders have since contributed their new skills to full-text screening of further articles.
From our experience, we would highly recommend involving stakeholders in co-producing your Cochrane Review methods. The research team’s earlier engagement with stakeholders helped to ensure the scope of the research question was of maximum interest and importance. Stakeholders also found this earlier involvement beneficial as they could help fill gaps they were aware of in the field of person-centred care.
Stakeholder involvement in the full-text screening verified that the research team’s interpretations of the stakeholders’ views were “on the right track”. The researchers also benefited from the expertise of the stakeholders in brainstorming solutions to “sticky” screening issues.
A post-event evaluation showed stakeholders felt their views were valued during the screening and that their understanding of the process of a Cochrane Review had improved. The process also helped stakeholders understand the valuable contribution their own projects within health services could be making to the field of person-centred care research, and that further emphasis was needed in the field to publish about innovations and best practice.
Whilst acknowledging the success of this experience of co-production, it is important to note the stakeholder panel members had strong English language skills and many had tertiary level qualifications. A more diverse group of stakeholders may have found this task more challenging and potentially less engaging.
Tips for involving stakeholders in screening:
- Engage in face-to-face stakeholder meetings as much as possible, so people have ample opportunity to ask questions and clarify their understanding.
- Provide a sound explanation of what screening is, why it’s important and where it fits within the Cochrane Review process. Our stakeholders found the funnel diagram of the process especially useful.
- Use the inclusion criteria your stakeholders have helped develop, and provide examples of definitely excluded, likely included and ‘uncertain’ articles based on title and abstract screening.
- The authors and stakeholders should ideally screen full-text articles for the first time together to promote genuine involvement. A simple screening sheet can help both stakeholders and authors to simplify complex screening decisions.
- Provide opportunities for people to conduct further screening if they want to, arranging to feedback the results of the “second screen” and requesting their consensus on screening decisions when needed.
Next steps:
Of course, a review doesn’t end with full-text screening, and neither does the involvement of stakeholders! Stay tuned for future blogs on the experiences of our researchers and stakeholders, and more tips on how to co-produce a review, as we continue down this #CochraneCoPro road.
Acknowledgements: Thank you to all of our stakeholder panel members, and co-authors who contributed directly, or from afar, to the screening day.