New guidance to making Cochrane blogshots is now available, comprising a quick checklist and a detailed document with examples. Cochrane UK Knowledge Broker, Sarah Chapman, who has helped produce the document, explains how this has come about.

At Cochrane UK, we first experimented with blogshots four years ago and have been making and sharing them in pretty much the same way since then, apart from some early refinements to the format, including versions for the health professional and non-medical audiences of our ‘Evidence for Everyday’ series. Others in Cochrane took this up, making or translating blogshots, and there are now many available in languages other than English.

As part of Cochrane’s Knowledge Translation work, we were asked to produce some guidance for those in Cochrane wishing to make blogshots, with the aims of improving consistency, accuracy and quality. In the process of doing this, our scrutiny of current practice revealed the extent to which the blogshots vary and highlighted both the need for guidance and also the many uncertainties about what ‘best practice’ might look like.

Blogshots – quick to read and share

Blogshots have great potential. They give brief information in a format that is quick to read and share on social media, with a link to the full review for those who want more information, and they are easily updated when the Cochrane Review is updated.  Anecdotally, people seem to like them. But we know little about people’s understanding and use of them and there is a lack of evidence to guide most aspects of blogshot production and dissemination. What’s more, the production, translation, storing and updating of blogshots across Cochrane is unregulated and uncoordinated.

Moving from innovation to evaluation

As an organization dealing in evidence, we think we need to do better than base practices on anecdote and best guesses! We also need to ensure that we avoid duplication of effort, and that innovation isn’t at the expense of quality. It’s good to experiment, but now we need to evaluate how well our new products and practices are working and how we could improve them. At worst, they might be misinforming people. Like any intervention, we need to consider the potential benefits and harms.

Making improvements

Blogshots may look like ‘quick wins’, but they are not a simple undertaking. Something we quickly discovered was the difficulty of imposing uniformity on blogshots when they are based on Cochrane Reviews in all their variety! However, we hope that the guidance will go some way towards providing some basic presentation guidelines whilst allowing flexibility to ensure the information included is relevant to the target audience and adapted to the Cochrane Review. We have also consulted with colleagues working on the Core Principles for Dissemination to ensure the guidance aligns with these.

We have amended our own practice and made changes to the template, which is now available to be used by everyone in Cochrane making blogshots. There are also changes in the way they are being stored.

Sharing evidence – blogshots or something else?

A key question worth asking if you are currently making, or considering making, blogshots to share Cochrane evidence, is what is your primary aim? At Cochrane UK, it is to give people a nugget of evidence, with the expectation that this might be useful and that they might or might not chose to go to the full review. For Cochrane Review Groups, the primary aim might be to increase click-throughs to the reviews. If so, Cochrane’s pilot study (https://visuallycochrane.net/2016/11/30/tweet-tweet-blogshot-or-tweet-picture-results-from-a-twitter-pilot-study/) conducted as part of the deep dive’ on blogshots suggests that increasing click-throughs may be more effectively achieved through sharing a tweet with a picture and link to the review. This is also less resource intensive.

If you do choose to make blogshots, we hope you find the new guidance useful. Our thanks are due to colleagues who have helped us produce this guidance, including those who tried out a draft version and provided valuable feedback.

• Read the guidance for producing, translating, storing, and sharing Blogshots

The UK National Institute for Health Research receive the first Cochrane-REWARD prize in 2017 for its Adding Value in Research programme, which promoted a range of activities tackling waste at every stage of research. We asked Matt Westmore, NIHR lead for this work and Operations Director at NIHR’s Evaluation, Trials and Studies Coordinating Centre about how things have developed receiving the prize.

Could you introduce NIHR’s Adding Value in Research framework?

Adding Value in Research (AViR) has been a long-running initiative for us. It goes back to 2009, with the Lancet paper on research waste by Iain Chalmers and Paul Glasziou. We started talking about our role in reducing research waste as a funder and have been working on it since.

We developed and introduced an AViR framework to our organisation. Some of it is deliberate work – looking across our activities and thinking about actions we should take. At the same time, it is important to note that we have also been successful in embedding the AViR values within the different areas of our work in NIHR. For example, we recently launched a funding and projects website. While not directly linked to AViR, it speaks to the culture of transparency in the organisation.

There has never really been resistance within NIHR. It’s been more about how to make the necessary changes and working through the practical barriers.

How has winning the prize impacted your work in this area?

Despite the lack of resistance to these ideas within NIHR, some of the implications are challenging. Winning the prize made a big difference there. The external endorsement gave credibility to our internal discussions. We also used the prize money to support the spread of this agenda within NIHR. We established our own internal award to support earlier career colleagues attend Ensuring Value in Research (EViR) meetings.

How has the Ensuring Value in Research Funders’ Forum developed in the last two years?

A big area of work has been the growth of the Ensuring Value in Research (EViR) Funders’ Collaboration and Development Forum. NIHR was an initiator of EViR, and we have been working on it with other funders, particularly the Netherlands Organisation for Health Research and Development (ZonMW) and the Patient-Centered Outcomes Research Institute (PCORI). At the time of winning the prize, we had had a couple of meetings, but work has really intensified since then.

The forum is a good space for us to challenge ourselves and each other. Together, we have developed 10 Guiding Principles, which give clarity to the key areas where we as funders can add value and reduce waste through the research we fund. These principles are helping refine our focus and give us a list of things to work through together.

There is a real variety in terms of the size and focus of different members of the forum. For example, we have members who fund preclinical research – something we at NIHR don’t fund – and it is valuable for us to work together and learn from them.

What else have you been working on?

Since winning the Cochrane prize, we have revised two points of the AViR framework. This incorporates more of the direction being taken by EViR with the 10 Guiding Principles – looking at types of research wider than that which NIHR funds, and with more attention on dissemination.

After the World Health Organization published their joint statement on public disclosure of results from clinical trials in 2017, we spent a year working on an NIHR policy and implementation strategy in response.

We are also a part of EVBRES (EVidence Based RESearch), an EU-funded network bringing together lots of initiatives with the aim of making evidence-based research a reality. Within EVBRES, I am working on a stream which is focusing on how this work impacts funders and how it can be put into practice. This will be a consensus process, involving many different stakeholders, and I will be looking to members of EViR for their input.

Is there anything else you would like to add?

I want to stress the importance of collaboration and partnership in this work – both internally within NIHR, globally, and across the sector. Partnerships can be tricky but are essential – no one part of the system alone can solve all the problems associated with research waste.

The Cochrane-REWARD prize – 2019 nominations open

The annual Cochrane-REWARD prize recognizes good local or pilot initiatives that have the most potential to reduce research waste if scaled up globally.

Nominations for this year’s prize are open until 5 June 2019.

• Find out more about the prize and how to submit a nomination
• Read our interview with Hans Lund, Chair of EVBRES

Following the development of the Risk of Bias 2 (RoB 2) tool, Cochrane will be rolling it out on new reviews in 2020. Here, Ella Flemyng, Methods Implementation Coordinator at Cochrane, highlights feedback from the Cochrane community on its implementation during the ‘Risk of Bias 2 Strategic Session’ at the mid-year Cochrane Governance Meetings last month.

The mid-year Cochrane Governance Meetings in Krakow was a great opportunity for us to present plans for implementing Risk of Bias 2 (RoB 2) - see the post ‘Update on the implementation of Risk of Bias 2 in Cochrane‘ - and to hear from the Cochrane community about what is going to help them work with this new method. Given the collaborative approach that we are taking with RoB 2 roll-out, we needed to get your feedback on our plans for its implementation.

We had some really useful discussions about the RoB 2 roll-out at our strategic session. The session began with Julian Higgins (University of Bristol and Bias Methods Group Co-Convener) introducing RoB 2 and the differences compared to original tool. This was followed by Rachel Richardson (Network Research Fellow, Abdomen and Endocrine Network) who gave a case study on how she used RoB 2 in an ongoing Cochrane Review. Kerry Dwan (Statistical Editor, CET) then highlighted proposed changes to the data extraction form, Rebecka Hall (Product Owner of RevMan) showcased changes for RevMan Web, and Toby Lasserson (Senior Editor, CET) highlighted plans for the pilot, wider roll-out and proposals for the Cochrane Library. The slides have been made available here.

Questions arose around the expectations of when RoB 2 should be applied in the context of updates, the role of funding source, and stipulating the effect of interest. We are currently developing these questions into a FAQ.  

The final section of the session was a breakout, where those attending gave feedback on what tools, guidance, training and support they feel are needed, for us to build in to our plans. Full details on the feedback can be found here and the following were the most frequently mentioned:

• Guidance = Quick-start, bite-sized guides (such as on each domain) using diagrams, ‘how-to’ YouTube clips, etc. that would support full guidance.
• Training = Virtual sessions, including offline, webinars, recorded webinars, and interactive e-learning tools.
• Support = Centralised, ongoing, expert support team for editorial teams and authors including ‘Help’ Slack channel or clinics.
• Tools = Integration of all tools with RevMan Web, including the RoB 2 Excel tool, Covidence, updates to GRADE, Archie, and signalling questions.

Cochrane Common Mental Disorders recently asked Karen Morley if she would give consumer peer review a go. In this blog, first published here,  Karen provides an insightful and honest reflection of how she found the experience and notes important challenges that Cochrane and Cochrane groups can improve on.  

Would I, as a new consumer volunteer for Cochrane Common Mental Disorders, have jumped in at the deep end and attempted a peer review if I hadn’t been asked? I don’t think so. But now I’m here, unsteadily dogpaddling, with two protocols and a systematic review under my belt, I thought I’d share some reflections and questions with you.

First came the Fear. I felt quite tiny in the huge, complex Cochrane organisation. And ‘peer reviewing’ sounded so imposing.  I know there are consumers who are themselves researchers or health professionals, but how could I be considered a ‘peer’?

Jess Hendon, the Managing Editor, was warm, kind and welcoming and assured me my lived experience was what counted. But did I have the knowledge, understanding and skills for the task? I had some experience of close reading and critical evaluation from my years as a secondary English teacher, and a basic understanding of systematic reviews from contributing to some Cochrane consumer learning resources that are currently in development, but would that be enough? I was afraid of being judged and afraid what I had to say might not be useful. Well, it was too late for dithering; a deadline was approaching: I took the plunge.

There was a checklist that gave some focus and structure. There was also some linked online support. At the moment this is quite brief and packed with detail and assumes some prior knowledge in the reader (for instance, ‘interventions’, ‘outcomes’, ‘comparator’ and ‘settings’ have specialist meanings that might be unfamiliar to a complete newcomer) and the language is rather formal. I was lucky to have already explored the subject a little and I wondered what it must feel like to be starting from scratch.

Despite the checklist I was still full of doubt. What exactly was expected of me? How should I express myself and in how much detail? A worked example would have been helpful. When I later saw the feedback from the authors on all the reviewers’ comments I had a much clearer idea of the process and the kind of contributions that were acceptable. I felt strangely detached and lonely as I looked at my screen. It would have been nice to have the luxury of a mentor, or some buddies to bounce ideas off, or maybe to have had a group practice run. It would also be interesting to hear an author’s view. (I’ve since had the pleasure of meeting Lindsay Robertson, which I found reassuring – thanks, Lindsay!) Last year I took part in a James Lind Alliance Priority Setting Partnership and I found it was illuminating to hear the comments of other patients, carers and clinicians: we have lots to learn from each other.

The protocol I looked at first would certainly be challenging for a consumer to read. I had to check my own understanding of ‘monoamine neurotransmission’, for instance. (That was quite a ride!)

I’d be interested to know whether, when consumer reviewers write about language and style, the same issues crop up frequently, and whether it might be desirable or feasible to collect them or to have some sort of glossary. Making suggestions for explanations or alternative expressions throws up a host of knotty problems. Who is the reader I’m writing for? What level am I pitching at? Which consumers read protocols – and systematic reviews - and why? How much can something be simplified without losing important nuances or even completely misrepresenting its meaning? Is there in fact a conflict between several different audiences and purposes? Which ones do the authors have at the forefront of their minds? Why doesn’t a protocol have some kind of Plain Language Summary? 

I suppose I felt the most confident about working on the language and less so about commenting on things like objectives and the criteria for considering studies. I knew there would be other contributors but I had a strong sense of responsibility: a real desire to write something that would actually be helpful to the authors. I have enormous respect for them and the huge, intricate, demanding task they have carried out. I don’t want to waste their time.

But now to the heart of the matter: representation. ‘Does the background address the hopes and concerns of people considering the treatment?’ asks one of the prompts. My task is to put myself into the shoes of the most important end users of systematic reviews and protocols: the consumers. This is the biggest responsibility of all. I can do my best but I’m aware that, alone, I’m not up to the task. I am one old, white, middle class woman, with one experience of one manifestation of a wide-ranging mental health condition: one tiny part of a huge jigsaw. Cochrane needs many consumer peer reviewers to represent us in all our diversity. And not just reviewers. I appreciate my opportunity to make changes to the review before it’s published, but this is the end of a very long journey. Have other consumers been involved before I see the review – in setting priorities or determining the research question or outcomes, say? How would I know? Shouldn’t it be mentioned in the review somewhere?

So, consumers, you are needed! If you feel the stirrings of an inclination to get involved, please have a go at something - anything. You might feel reticent, but come and help lead the way until it becomes commonplace for us to be included and to shape research around all our hopes and concerns.

I am sometimes shocked when I think how comparatively young the modern evidence based medicine movement is. Similarly, it’s easy to forget that the meaningful inclusion of consumers is really only beginning. I am delighted that Cochrane’s revised 2018 Peer Review Policy means Cochrane Review Groups must now aim to have at least one consumer peer reviewer for each Cochrane Review. (If there were more of us, fellow consumers, it could be more than an aim!) I’m delighted and daunted in equal measure to be one of those peer reviewers, and enthusiastic about CCMD’s plans to take this forward. It has been a tremendous pleasure to meet Rachel Churchill, Jess Hendon, Lindsay Robertson and Noortje Uphoff (well, I popped into her office!). The reception they have given me has been affirming and encouraging. I feel the burden, but I can see a great opportunity. 

Karen Morley

• Visit the Cochrane Consumer Network website
• View the Consumer Referee Training materials
• Read the Cochrane Peer Review Policy

‘Cochrane at the WHO' is a blog series that highlights the partnership between Cochrane and the World Health Organization. If you would like to share a story about how your Cochrane Group is working with WHO, please contact Emma Thompson, Advocacy and Partnership Officer.

In this blog post Professor Liliya Eugenevna Ziganshina, Director of Cochrane Russia, talks about her involvement in two WHO/Europe initiatives that aim to promote evidence-based health policy: EVIPNet and the European Advisory Committee on Health Research. (Read this blog post in Russian)

Could you give a quick introduction to yourself and your work within Cochrane?

I am currently the Director of Cochrane Russia. I have been with Cochrane since 2003, starting with a workshop on review protocol development with Professor Paul Garner at the Cochrane Infectious Diseases Group, which ended up being a life-changing event. After that, we developed and published the review, ‘Fluoroquinolones for treating tuberculosis’, with three updates over 10 years.

I learned about Cochrane at a WHO workshop on problem-based pharmacotherapy teaching in 1997. There, I met Dr Hans Hogerzeil, who introduced the concept of essential medicines for health. I started looking into getting access to Cochrane Library, and Dr William Stones would kindly send floppy disks, and then CD-ROMs, of the Cochrane Library from the UK to Kazan until I got author access in 2005. We would use Cochrane Systematic Reviews in the selection of essential medicines for the Tatarstan Republic Formulary.  

Can you tell us about the WHO Evidence-Informed Policy Network?
 
The WHO Evidence-Informed Policy Network (EVIPNet) is a global initiative to promote the systematic use of the best available research evidence in health policy making and implementation. It focuses on low- and middle-income countries and aims to build capacity in the development of policy briefs as an established mechanism of evidence/knowledge translation.

EVIPNet was established in 2005 and was launched in WHO/Europe in 2012. I got involved with EVIPNet Europe in autumn 2016 as part of Cochrane’s ongoing partnership with WHO, when I had a chance to apply to serve as a Cochrane trainer.

How have you been involved in EVIPNet Europe so far?

Initially there was a series of webinars on evidence for policy and policy briefs for EVIPNet members, organised by Tanja Kuchenmüller, who leads WHO/Europe’s Knowledge Management, Evidence and Research for Policy-Making Unit, and her team. The webinars were very helpful, with assignments and tasks to be completed by deadlines. This was quite a challenge, but hugely helpful for my learning process and professional development. 

I was then privileged to participate in the fifth annual EVIPNet Europe multicountry meeting in Bratislava, in June 2017, where I met Dr Claudia Stein, Director of the Division of Information, Evidence, Research and Innovation at WHO/Europe and her vibrant, enthusiastic team. This was a very stimulating, eye-opening experience with a lot of positive energy!   

The next big milestone was helping to prepare and deliver a one-day workshop for WHO/Europe staff on Cochrane systematic reviews with Livia Puljak from Cochrane Croatia and Tanja’s fantastic team. The training was based on the Cochrane Handbook for Systematic Reviews of Interventions, and we designed it to be as interactive as possible. The participants were enthusiastic and knowledgeable – and keen to learn more about Cochrane Systematic Reviews and share their experiences with using evidence. I learned a lot that day too.
 
I hope that I will continue to have much more engagement with EVIPNet Europe in the future.

You are also a member of the WHO European Advisory Committee on Health Research (EACHR). Could you tell us more about this?
 
The EACHR was established in 2011 to promote and strengthen the use of research evidence in health policy making and to inform policies for further development of health research in the European Region. It functions as a regional equivalent of the WHO Global Advisory Committee of Health Research and reports directly to the WHO Regional Director for Europe. Its membership consists of 20-24 public health research experts from across the European Region, who serve for three years.

I had the honour to be invited to apply to a call for expression of interest to serve as a member of EACHR and was successful – I think my Cochrane background helped. My first face-to-face EACHR meeting took place in May 2018 in Copenhagen. It was great to meet the other Committee members and Dr Zsuzsanna Jakab, WHO Regional Director for Europe, who led the meeting. I learned about the huge amount of work being done and volunteered to review the upcoming European Health Report. There was also a lot of discussion around big data and I was able to share the Cochrane approach, as presented at the 2015 Colloquium.

How has Cochrane evidence impacted on the different WHO processes you are involved in?
 
A lot! Cochrane evidence is the solid basis for EVIPNet Europe work. The Cochrane methodology and intervention reviews are much appreciated by staff within various units and public health fields at WHO/Europe. When reading various WHO documents, I am happy to see frequent mentions of Cochrane Review Groups and Cochrane methodology. Cochrane’s vision and mission are in complete accord with and contribute to furthering the work of the EACHR. EACHR is all about advising on health research programmes – and Cochrane Systematic Reviews can help identify gaps where this research is needed.

Would you recommend that other members of the Cochrane community participate in WHO activities?

Yes, certainly! Keep strong and keep going!

Is there anything else you would like to add?

I would like to give heartfelt thanks to my colleagues both at Cochrane and at WHO, who have enlightened, inspired, empowered, supported and shaped me professionally and personally over many happy years! 

Do you work with WHO?

We would love to hear more about how members of the Cochrane community are working with WHO so that we can help publicise these activities.

Please email Emma Thompson, Cochrane Advocacy and Partnership Officer, and let us know.

Many members of the Cochrane community have been engaged in the Evidence-Based Research Network (EBRNetwork) and will be participating in EVBRES (Evidence Based RESearch). In this interview, as part of our REWARD series, chair of EVBRES Hans Lund gives an overview of what this new initiative sets out to achieve in promoting evidence-based research and reducing research waste.

Could you tell us about EVBRES and how it started?
EVBRES (EVidence Based RESearch) is a European Cooperation in Science and Technology (COST) Action – an EU-funded network that brings scientists together to collaborate around a topic – that started in October 2018 and will run until October 2022. We launched with a meeting in Bergen, Norway, in February 2019, with 85 people from Europe and beyond coming along and actively participating, which was fantastic.

It is a project under the Evidence-Based Research Network (EBRNetwork), which I established together with a group of people interested in the concept of evidence-based research in 2014 in Bergen, Norway. The EBRNetwork is made up of very enthusiastic people, but we have not always had the energy, resources or time to follow up in addition to doing our usual research work. We had always just been doing the best we could with our spare time – and I think we managed a lot, with a paper, ‘Towards evidence-based research’, published in the BMJ, and several more to come.

The COST Action funding will be a huge help, making it possible for us to come together and work on the issue. However, when the COST Action funding comes to an end, we know that the work of the EBRNetwork will continue.

How did you get interested in evidence-based research?
During my PhD in 1994, I was introduced to three concepts: evidence-based medicine, systematic reviews and Cochrane. I was hooked immediately and attended the Colloquium in Oslo in 1995 and was very inspired. While my personal circumstances meant I couldn’t be very active within Cochrane in the following years, I promoted systematic reviews and evidence-based medicine through writing, courses and lectures, and was very enthusiastic about my profession, physiotherapy, being evidence-based.

Later, I realised that systematic reviews were not only useful for informing evidence-based medical practice. I would talk with colleagues who were researchers and wonder why they didn’t use systematic reviews. I had a feeling that there was an opportunity to use systematic reviews for new studies, to justify them, come up with a design or place the research into context.

I stumbled on a presentation by Iain Chalmers who articulated this much more clearly, with some meta-research results that showed that when people were not using systematic reviews to justify a new study, there was a great risk for doing redundant or unnecessary studies. I found lots of studies and became much clearer about this. Among the papers I found were a number by Karen Robinson, the Director of The Johns Hopkins Evidence-based Practice Center. We met in April 2014, and together established the EBRNetwork that December.

What is EVBRES aiming to achieve?
We have four Working Groups (WGs) looking at different areas:

• WG1 is focused on the key stakeholders that can make doing evidence-based research difficult or easy. This could be ethics committees, funding agencies, patient groups or medical journals. We want to identify the implications of taking an evidence-based approach.
• WG2 is about training and developing a course for health researchers on how to be evidence-based and why it is important
• WG3 is about supporting international efforts towards a more efficient production and update of systematic reviews without compromising quality.
• WG4 is about meta-research – research on research. We need to standardise meta-research to follow developments. Now we can see only very few are using systematic reviews to justify research and referring to earlier similar studies, but how will we, in the future, know if we have accomplished what we aim for?

What amazed me the most during our meeting in Bergen is that not only 85 people attended and showed interest, but that they worked so hard to figure out how to fulfil the intentions laid out in the application for EVBRES. In 2 or 3 days we made so much progress because people engaged so strongly. We may even be able to achieve some of our goals sooner than expected.

We are currently applying for funding. The COST Action supports our meetings, but we have no funding for the work we are doing between those meetings – right now, people are doing it in their spare time because they think it is so important.

How do you think EVBRES can help to tackle research waste?
Research waste goes beyond the different scientific disciplines and medical specialities. Many of us are dealing with our own challenges in our work – we don’t often have the time to consider how we are doing the research; we have to publish. It is ‘easy’ to say that one should use systematic reviews, but how to do that is the real challenge. EVBRES is bringing people together to consider this and to listen to the experience of researchers, funders, ethics committees, journal editors and other stakeholders and to consider the practicalities. It is unique to bring together so many people to evaluate research practice.

We are fully aware that we are talking about something that has been a part of the scientific approach since the 15th or 16th century. It is just that now, for the first time in history, we have the technology to help address it. And we would very much like to help!  

How does this tie in with Cochrane’s work?
I think the concept of research waste, as written about by Iain Chalmers and Paul Glasziou in 2009 and explored in the 2014 Lancet series, has been an eye opener for many people. There is a very close relationship between the thinking behind Cochrane and the concept of research waste, and particularly evidence-based research. Whenever you do a systematic review, you show the problems with original studies – you pull the curtains back and show the good and bad parts. Many of the people engaged in our network are also from Cochrane.

Why do you think this work is so essential right now?
Validity of research is very important, but it is not enough. What is the use of a study that is fantastic in every way, but has no relevance to what patients and carers need and is completely unnecessary? Redundancy is an ethical issue.
The research community could end up in a situation where the public, funders and politicians tell us that we have been wasting their money, health and time for decades, so will no longer support us. We need be transparent and identify and repair our errors – we can use science to improve science.

The Cochrane-REWARD prize – 2019 nominations open
The annual Cochrane-REWARD prize recognizes good local or pilot initiatives that have the most potential to reduce research waste if scaled up globally.

Nominations for this year’s prize are open until 5 June 2019

Find out more about the prize and how to submit a nomination

'Cochrane at the WHO' is a blog series that highlights the partnership between Cochrane and the World Health Organization. If you would like to share a story about how your Cochrane Group is working with WHO, please contact Emma Thompson, Advocacy and Partnership Officer.

In this blog post Dr Tari Turner from Cochrane Australia talks about the WHO Department of Reproductive Health and Research’s Scientific and Technical Advisory Group (STAG) meeting in Geneva.

Could you give a quick introduction to yourself and your work within Cochrane?
I am a Senior Research Fellow at Cochrane Australia where I work on TaskExchange, Living Systematic Reviews and Living Guidelines. I also work on knowledge translation, particularly in maternal and child health in low resource settings, which I really love.

Can you tell us about the WHO process you are involved in?    
STAG provides guidance to the WHO Department of Reproductive Health and Research, and the Human Reproduction Programme (HRP). Essentially, the HRP presents their draft workplan for the year, along with key strategic questions about their work or challenges they are facing, and we provide suggestions and advice.

What were the key conclusions?
As well as reviewing the 2019 workplan, we discussed the importance of sexual and reproductive health (SRH) within the global focus on universal health coverage, how to ensure effective evidence-based SRH in emergency settings and primary health care, and how implementation research fits within the work of the department. I am particularly interested in this latter bit, and it is so great to be in a room of clever people who get the value and challenges of implementation research, and who are committed to doing it well.

The next step is for HRP to continue with their great work. The STAG meeting is an annual event, so I now step back and watch with great pleasure as HRP achieves great things over the next 12 months.

How has Cochrane evidence impacted on the process?
There is a lot of respect for Cochrane at RHR, which has a close relationship with the Cochrane Pregnancy and Childbirth Review Group, and many of the WHO staff have long histories of supporting and authoring Cochrane reviews. Cochrane evidence underpins many of the guidelines that the department produces, and evidence synthesis is core business.

Being a part of the STAG is a fantastic privilege and I would definitely recommend people take up similar opportunities. It is fascinating – a real learning experience – and great fun.

Is there anything else you would like to add?
Just my huge thanks to WHO and the RHR/HRP team for their fantastic work in these vital areas, and for the opportunity to contribute, it is humbling.

Do you work with WHO?
We would love to hear more about how members of the Cochrane community are working with WHO so that we can help publicise these activities.

Please email Emma Thompson, Cochrane Advocacy and Partnership Officer, and let us know.

Muriah Umoquit, Communications and Analytics Officer with Cochrane’s Knowledge Translation Department, talks about one solution that Cochrane is using to help break down information silos -  the Communications Network which shares information across groups weekly.

A few years ago when I started at Cochrane, I knew the community was doing great things. We’d hear about an event on Facebook, see a tweet or posting on social media, or learn about a group’s dissemination activities at the Colloquium. But hearing about these successes in the community was sporadic, and it was hard to capture them in a way that we could share further. While Cochrane is a collaboration of many working parts, it was clear that information silos were keeping us from hearing - and sharing - the successes across Cochrane.

It was with this in mind that we started the Cochrane Communications Network in 2015, with its main focus being a weekly digest of all the news and translation work happening within Cochrane. The idea was that those doing communication activities for their Cochrane group could share items in the digest further, and translation teams could better coordinate their activities and promote their work. We started by identifying 23 people who we knew were doing communication work in Cochrane. Now, we’ve grown to 270 members, many joining based on a Cochrane colleague recommending it – a testament to how well the digest is working! You can watch a video how Cochrane Russia is using the digest here.

Why has the CommsNetwork digest worked so well? Given the feedback we’ve gotten, these features stand out as contributing to its success:

• Regular digest: Every Tuesday you get an email digest; many users build in time into their Wednesday schedule to review. If you miss a week, items with time-sensitive information and items with deadlines are carried over to the next week’s digest.
• Broad coverage: I monitor a lot of information so you don’t have to! I  pull content from: RRS feeds of all our Cochrane websites; any mentions of @cochranecallab or @cochranelibrary on twitter; Cochrane.org and the  Cochrane Community website; dissemination recommendations from KTD, the Cochrane Library Editorial Team, and our publisher Wiley; the Review Group and Methods Digest; the Centre and Fields Digest; tons of Cochrane group newsletters; and submissions from members. Given the space limitations, only a small slice of what goes into the digest makes it into the Cochrane Community and Cochrane Connect newsletters - the Comms Digest gives you the full coverage!
• Easy to scan: If you only have a minute, at the very top of the newsletter we’ll highlight one or two important items. The newsletter is then organized under the same headings each week: internal membership audience news, external supporter audience news, news about the Colloquium, and date specific items.
• Easy to share: We’ve listened to feedback and made sharing news items easier - most external news items have ‘suggested tweets’ that you can use on your own accounts. Having it in an email format, rather than a 'nicer looking' newsletter, makes it easier for your to simply copy-and-paste these tweets and use.
• Easy to submit news: The CommsNetwork Digest is a great platform to share your successes – just email me your news! News items that start in digest could evolve into larger postings, such as community blog posts or features on Cochrane.org.

What’s the Comms Digest? - 0:33 What does it include? – 0:52 How is it put together? – 1:43 How do you use it? – 3:42 (example at 4:25) How do you sign up? – 8:41

If you do any communications and knowledge translation work for your Cochrane group, come join the Comms Network and get the weekly digest! It will make content creation work easier and give you a platform for sharing your news and successes with a larger audience!

• Join the CommsNetwork
• See other KT Resources on the Training website

Muriah Umoquit
Communications and Analytics Officer
Cochrane Knowledge Translation

Bronwen Merner, Dianne Lowe, Louisa Walsh and Sophie Hill from Cochrane Consumers and Communication and stakeholder panel members, Anne Mussared and Cheryl Wardrope share their top tips for involving stakeholders in Cochrane Review screening.

Stakeholder involvement in health research, including Cochrane Reviews, is beneficial in improving research relevance and implementation (Pollock et al 2018). Reflecting this, the Cochrane Knowledge Translation Framework now includes co-production as a core theme. Despite the growing prominence of stakeholder involvement in research at the policy level, it remains an enigma for many researchers on the ground (Concannon et at 2018). To help remedy this, this blog describes the process and experience of co-producing the full-text screening stage of a Cochrane qualitative evidence synthesis (QES) on person-centred care with a stakeholder panel

Previous stakeholder involvement in the review
An Australian-based, 18-member stakeholder panel involving consumers, health practitioners and health decision-makers was convened by the researchers in the early stages of this review. In 2017, the panel met twice via teleconference to shape the topic (consumers and providers working in partnership to promote person-centred health services), review type (qualitative evidence synthesis) and conceptual development of the protocol. Following these teleconferences, the researchers sought the panel members’ interest in having a face-to-face meeting to co-produce the full-text screening step of the Cochrane Review. Following a positive response, a face-to-face meeting day took place in November 2018.

The full-text screening training day
As many Cochrane authors will know, screening can be an overwhelming task. Applying selection criteria to complex interventions and working out what should be ‘in’ or ‘out’ often needs detailed discussion. This complexity is potentially magnified when the screeners are relatively new to the research process, and more specifically, the Cochrane Review process. Therefore, the research team began the face-to-face meeting with a practical explanation of what screening is, why it’s important and where it fits within the Cochrane Review process. The researchers then gave some examples of how to apply the inclusion and exclusion criteria.

After this brief training session, the researchers and stakeholders worked in small groups to apply the inclusion criteria to a range of full-text qualitative research articles. Neither the research team nor the stakeholders had read the articles prior to the meeting, so the decisions about inclusion or exclusion were shared after extensive discussion and problem-solving. The process was aided through the research team’s prior development of a screening sheet which proved very useful (for stakeholders and researchers alike!) in making inclusion or exclusion decisions.

After two 45 minute sessions of screening, each small group had screened three or four full-text articles. Stakeholders expressed interest in continuing to screen articles after the sessions ended, so we sent an email invitation to follow-up after the meeting. Four stakeholders have since contributed their new skills to full-text screening of further articles.

From our experience, we would highly recommend involving stakeholders in co-producing your Cochrane Review methods.  The research team’s earlier engagement with stakeholders helped to ensure the scope of the research question was of maximum interest and importance. Stakeholders also found this earlier involvement beneficial as they could help fill gaps they were aware of in the field of person-centred care.

Stakeholder involvement in the full-text screening verified that the research team’s interpretations of the stakeholders’ views were “on the right track”. The researchers also benefited from the expertise of the stakeholders in brainstorming solutions to “sticky” screening issues.

A post-event evaluation showed stakeholders felt their views were valued during the screening and that their understanding of the process of a Cochrane Review had improved. The process also helped stakeholders understand the valuable contribution their own projects within health services could be making to the field of person-centred care research, and that further emphasis was needed in the field to publish about innovations and best practice.

Whilst acknowledging the success of this experience of co-production, it is important to note the stakeholder panel members had strong English language skills and many had tertiary level qualifications. A more diverse group of stakeholders may have found this task more challenging and potentially less engaging.

Tips for involving stakeholders in screening:

1. Engage in face-to-face stakeholder meetings as much as possible, so people have ample opportunity to ask questions and clarify their understanding.
2. Provide a sound explanation of what screening is, why it’s important and where it fits within the Cochrane Review process. Our stakeholders found the funnel diagram of the process especially useful.
3. Use the inclusion criteria your stakeholders have helped develop, and provide examples of definitely excluded, likely included and ‘uncertain’ articles based on title and abstract screening.
4. The authors and stakeholders should ideally screen full-text articles for the first time together to promote genuine involvement. A simple screening sheet can help both stakeholders and authors to simplify complex screening decisions.
5. Provide opportunities for people to conduct further screening if they want to, arranging to feedback the results of the “second screen” and requesting their consensus on screening decisions when needed.

Next steps:
Of course, a review doesn’t end with full-text screening, and neither does the involvement of stakeholders! Stay tuned for future blogs on the experiences of our researchers and stakeholders, and more tips on how to co-produce a review, as we continue down this #CochraneCoPro road.

Acknowledgements: Thank you to all of our stakeholder panel members, and co-authors who contributed directly, or from afar, to the screening day.

'Cochrane at the WHO' is a blog series that highlights the partnership between Cochrane and the World Health Organization. If you would like to share a story about how your Cochrane Group is working with WHO, please contact Emma Thompson, Advocacy and Partnership Officer.

In this blog post Emma Carter, Selena Ryan-Vig, and Anne Eisinga from Cochrane UK talk about the WHO Model List of Essential Medicines.

At Cochrane UK our programme of work includes identifying and charting the impact of Cochrane evidence. Beyond auditing the use of Cochrane Reviews in guidelines and other evidence-based guidance, we are also interested in tracking policy documents where Cochrane Reviews have contributed evidence to decision making for preventing harm or for promoting health and wellbeing.

One such project involves monitoring how Cochrane evidence is being used to inform decisions made by the World Health Organization (WHO) in devising the WHO Model List of Essential Medicines. This list is designed to guide nations in their selection of medicines to be made accessible in order to address the priority healthcare needs of their population. The medicines are selected on the basis of relevance to public health, evidence on efficacy and harms, and comparative costs.

Methods
We examined the WHO Expert Committee Reports on the Selection and Use of Essential Medicines, from 2000 to 2017, which are published every two years and chart the considerations and justifications for changes made to the WHO Model List of Essential Medicines and (from 2007) the WHO Model List of Essential Medicines for Children.  

We searched the full text of each report for the word “Cochrane” and noted the section and subsection in which evidence from Cochrane Reviews was considered, extracted the relevant text on the review’s findings, and categorized how Cochrane evidence influenced any resulting recommendations and decisions by the Committee.

Results
In total, 234 Cochrane Reviews from 45 Cochrane Review Groups (range 1 to 22 reviews) have been used to inform all ten reports of the WHO Expert Committee on the Selection and Use of Essential Medicines (range 2 to 67 reviews per report) published between 2000 and 2017.

• 2000:    2 reviews from 2 Cochrane Review Groups
• 2003:    11 reviews from 8 Cochrane Review Groups
• 2005:    30 reviews from 10 Cochrane Review Groups
• 2007*:   29 reviews from 13 Cochrane Review Groups
• 2009:    41 reviews from 18 Cochrane Review Groups
• 2011:    25 reviews from 15 Cochrane Review Groups
• 2013:    30 reviews from 13 Cochrane Review Groups
• 2015:    41 reviews from 16 Cochrane Review Groups
• 2017:    67 reviews from 25 Cochrane Review Groups

* Two separate WHO Expert Committee Reports were published in 2007, both informed by Cochrane Reviews – the combined totals are given here.

How has Cochrane evidence been used?
Cochrane evidence has been used to inform decisions by the WHO Expert Committee to:

• reject applications to add or delete medicines from the core Essential Medicines List (evidence includes highlighting the paucity of existing evidence; uncertainties due to the low- and very low-quality evidence available; ongoing research; potential risk of harms) (n=59);
• recommend adding new medicines to the list or additional footnotes to clarify uses for the existing medicines listed (evidence includes comparative efficacy with minimal harms) (n=104)
• recommend deletions from the list (evidence includes avoidance of harms or identification of more effective alternatives now available) (n=31)
• recommend moving items from the Essential Medicines List to the Complementary Medicines List* or adding items to the Complementary Medicines List only (n=32)
• defer decisions pending more information (evidence includes highlighting lack of existing evidence; complexities identified requiring further review; ongoing trials identified) (n=4)
• keep entries on the list unchanged (evidence endorses current listing; quality of evidence not robust enough to overturn existing recommendations at this time) (n=49)
• other: Cochrane evidence noted as important but did not lead to any specific recommendations (provided context-specific background) (n=24)

* The Complementary Medicines List presents essential medicines for priority diseases, for which specialized diagnostic or monitoring facilities, and/or specialist medical care, and/or specialist training are needed. Medicines may also be listed on the basis of consistent higher costs or less favourable cost-effectiveness in particular settings.

Value of Cochrane evidence
These data indicate that Cochrane’s policy of regularly updating reviews to incorporate newly identified evidence, to locate unpublished research, to track ongoing studies, and to include rigorous assessments of the quality of evidence and risk of bias, seems to have been valuable to the WHO Committee’s decision making over the years.  For example, newly updated versions of previously used reviews have led to the original decisions by the Committee being altered in the light of review conclusions that have changed. Or their decisions have been deferred, awaiting the completion of randomized controlled trials, identified as ongoing studies in Cochrane Reviews.

Cochrane Review authors’ assessment of the quality of evidence using GRADE (Grading of Recommendations, Assessment, Development and Evaluation), a systematic approach to rating the certainty of evidence, has also enabled the Committee to gain a more nuanced confidence in the certainty of the findings and present an evidence-based rationale for their decisions on inclusion, deletion or rejection.

Responsible use of antibiotics
The current list, published in June 2017, aims, in particular, to optimize antibiotic use and reduce antibiotic resistance without restricting access to essential antibiotics. We found that Cochrane evidence on antibiotic efficacy and harms (in 17 Cochrane Reviews from seven Cochrane Review Groups) has helped to inform the Committee’s decision making on which antibiotics to use for common infections and which to preserve for severe conditions, to endorse current practice or to add new medicines to the list.

What next?
Our work is ongoing and we will be integrating data from the next report of the WHO Essential Medicines List due in May 2019.

We have also extended our project to include identifying the use of Cochrane Diagnostic Test Accuracy Reviews in informing the new WHO Model List of Essential In Vitro Diagnostics, published for the first time in May 2018 by the WHO Strategic Advisory Group on In Vitro Diagnostics (SAGE-IVD), covering tests for specific infectious diseases including HIV, tuberculosis, malaria, hepatitis B and C, syphilis and human papillomavirus as well as general laboratory tests. The second edition is expected to be released in May 2019 and is likely to cover tests for antimicrobial resistance, neglected tropical diseases, noncommunicable diseases, outbreaks/emergencies and sepsis.

Our plan is to send detailed data to Cochrane Networks and Review Groups on how evidence from their intervention and diagnostic test accuracy reviews has been used to inform decisions made to compile the respective WHO Essential Medicines and Essential Diagnostics Lists. We hope this might prove helpful to them when they are seeking to describe aspects of the impact of their research in their specialized areas of health care.

Emma Carter, Selena Ryan-Vig and Anne Eisinga.

Emma Carter, Cochrane UK Information and Administration Support Officer
Selena Ryan-Vig, Cochrane UK Communications and Engagement Officer
Anne Eisinga, Cochrane UK Information Specialist