What you need to know about Risk of Bias 2 (RoB 2) in Cochrane

An image of a graph and magnifying glass in shades of blue

By Ella Flemyng (Methods Implementation Manager)

We are delighted to announce that the first Cochrane Review using Risk of Bias (RoB 2) as part of the formal pilot has published in the Cochrane Library. 

This Review, from Craig Williams, University of Exeter, and colleagues, examined the effects of physical activity interventions for people with congenital heart disease. The RoB 2 tool was used to examine results of the included trials in relation to five outcomes: maximal cardiorespiratory fitness, submaximal cardiorespiratory fitness, health related quality of life, physical activity and muscular strength. 

With the publication of this first Review using RoB 2 from the pilot, we have compiled a list of the key resources for the community on lessons learnt so far and what to expect going forward:

The phased approach to implementing RoB 2 in Cochrane is a new way of introducing a method and was highlighted as a must by the Cochrane Review Groups at the start of this process. It has been insightful (and sometimes eventful!) and we would not have been able to do it without the wonderful people we’ve worked with. I’d like to thank the developers of the RoB 2 tool, Jonathan Sterne and Julian Higgins, their team at the University of Bristol, as well as the Convenors of the Bias Methods Group (Julian Higgins, Isabelle Boutron, Matt Page and Asbjørn Hróbjartsson) for their expertise and insights. We especially want to thank Cochrane Review Group editors and staff with Reviews using RoB 2 in the pilot and their authors, who have informed developments over the last 18 months to improve the experience of future review teams. In particular, Nicole Martin, Managing Editor for Cochrane Heart, who managed this first Review in the pilot. I would also like to thank my colleagues across the Cochrane Central Executive team who have coordinated developments in their areas to support RoB 2 in Cochrane Reviews. And last, but by no means least, I’d like to thank Kerry Dwan, who is co-lead of the Pilot with me, and Tess Moore, who has been an incredible deputy for the project!

What next?

The decision on whether a Cochrane Review should use RoB 2 is still that of the Cochrane Review Group. The position statement from the Editorial Board outlines the expectations for RoB 2 in Cochrane Reviews going forward:

  1. RoB 2 is recommended as the preferred tool for assessing RCTs in new intervention reviews as per the Handbook, but RoB 1 may still be used;
  2. Reviews using RoB 2 must be prepared in RevMan Web because the functionality to record and display RoB 2 will not be added to RevMan 5;
  3. The decision of which tool to use should be made between authors and CRG staff before the protocol has been drafted (switching to RoB 2 after protocol publication is discouraged);
  4. There is no expectation that reviews already in development will adopt the method retrospectively;
  5. There is no expectation that updates will adopt the method at any point, and authors and CRG staff can decide whether to switch to RoB 2 at the point of update.

The following diagram showcases the process and support available for Cochrane Reviews using RoB 2, from title registration to publication of the Review (steps in darker color highlight the additional support available for the first Review in a Cochrane Review Group that uses RoB 2): (click here to view larger)

Diagram of the processes and support available for RoB2 use
We are continuing to learn from those using RoB 2 in Cochrane and building this feedback into our plans. There are certain conditions that must be met before Cochrane considers making RoB 2 mandatory for all newly registered Cochrane Intervention Reviews, which are detailed in the position statement, and we’ll keep you in the loop with further posts as they develop.

If anyone has any questions about RoB 2 or the phased implementation, please contact support@cochrane.org

19 November 2020

The Cochrane Official Blog is curated and maintained by the Development Directorate. To submit items for publication to the blog or to add comments to a blog, please email mumoquit@cochrane.org.

The Cochrane Blog presents commentary and personal opinion on topics of interest from a range of contributors to the work of Cochrane. Opinions posted on the Cochrane Blog are those of the individual contributors and do not necessarily reflect the views or policies of Cochrane.