By Ella Flemyng (Methods Implementation Manager)

We are delighted to announce that the first Cochrane Review using Risk of Bias (RoB 2) as part of the formal pilot has published in the Cochrane Library. 

This Review, from Craig Williams, University of Exeter, and colleagues, examined the effects of physical activity interventions for people with congenital heart disease. The RoB 2 tool was used to examine results of the included trials in relation to five outcomes: maximal cardiorespiratory fitness, submaximal cardiorespiratory fitness, health related quality of life, physical activity and muscular strength. 

With the publication of this first Review using RoB 2 from the pilot, we have compiled a list of the key resources for the community on lessons learnt so far and what to expect going forward:

• The Cochrane Editorial Board has posted a position statement on RoB 2 in Cochrane and the expectations for Cochrane Review Groups going forward – a must read for any Cochrane Review Group staff
• An Editorial from the RoB 2 Pilot Team on RoB 2 has published in the Cochrane Library that reflects on this published Review and the phased approach for introducing a new methodology in Cochrane
• A ‘What we’ve learnt from Cochrane’s Risk of Bias 2 Pilot’ post collates developments since the beginning of the pilot and how we addressed concerns from our community as part of the development
• A podcast with William Curtis, lead author of this Review using RoB 2, on the author team’s experience with RoB 2 is available via the Cochrane Heart website
• An introductory leaflet with pre-recorded videos, a detailed Starter Pack, including protocol and review reporting checklists, and RoB 2 FAQs can be found via Cochrane Methods
• Editorial checklist for protocols and reviews are available for Cochrane Review Groups, please contact support@cochrane.org
• And of course, see what RoB 2 looks like in a published Cochrane Review, including the new designs for interactive, results-level tables for these assessments (found in the new ‘Risk of Bias’ section, after the Characteristics of Studies section)

The phased approach to implementing RoB 2 in Cochrane is a new way of introducing a method and was highlighted as a must by the Cochrane Review Groups at the start of this process. It has been insightful (and sometimes eventful!) and we would not have been able to do it without the wonderful people we’ve worked with. I’d like to thank the developers of the RoB 2 tool, Jonathan Sterne and Julian Higgins, their team at the University of Bristol, as well as the Convenors of the Bias Methods Group (Julian Higgins, Isabelle Boutron, Matt Page and Asbjørn Hróbjartsson) for their expertise and insights. We especially want to thank Cochrane Review Group editors and staff with Reviews using RoB 2 in the pilot and their authors, who have informed developments over the last 18 months to improve the experience of future review teams. In particular, Nicole Martin, Managing Editor for Cochrane Heart, who managed this first Review in the pilot. I would also like to thank my colleagues across the Cochrane Central Executive team who have coordinated developments in their areas to support RoB 2 in Cochrane Reviews. And last, but by no means least, I’d like to thank Kerry Dwan, who is co-lead of the Pilot with me, and Tess Moore, who has been an incredible deputy for the project!

What next?

The decision on whether a Cochrane Review should use RoB 2 is still that of the Cochrane Review Group. The position statement from the Editorial Board outlines the expectations for RoB 2 in Cochrane Reviews going forward:

1. RoB 2 is recommended as the preferred tool for assessing RCTs in new intervention reviews as per the Handbook, but RoB 1 may still be used;
2. Reviews using RoB 2 must be prepared in RevMan Web because the functionality to record and display RoB 2 will not be added to RevMan 5;
3. The decision of which tool to use should be made between authors and CRG staff before the protocol has been drafted (switching to RoB 2 after protocol publication is discouraged);
4. There is no expectation that reviews already in development will adopt the method retrospectively;
5. There is no expectation that updates will adopt the method at any point, and authors and CRG staff can decide whether to switch to RoB 2 at the point of update.

The following diagram showcases the process and support available for Cochrane Reviews using RoB 2, from title registration to publication of the Review (steps in darker color highlight the additional support available for the first Review in a Cochrane Review Group that uses RoB 2): (click here to view larger)

 
We are continuing to learn from those using RoB 2 in Cochrane and building this feedback into our plans. There are certain conditions that must be met before Cochrane considers making RoB 2 mandatory for all newly registered Cochrane Intervention Reviews, which are detailed in the position statement, and we’ll keep you in the loop with further posts as they develop.

If anyone has any questions about RoB 2 or the phased implementation, please contact support@cochrane.org.