MECIR Manual
Assessing risk of bias in included studies (C52-C60)

Cochrane Training resources: assessing RoB and RoB 2.0 webinar

Cochrane Interactive Learning (CIL): module 5 - introduction to study quality and risk of bias

  Standard Rationale and elaboration Resources
C52 Assessing risk of bias  Mandatory  
  Assess the risk of bias for each study result contributing to an outcome in the ‘summary of findings’ table. For randomized trials, the RoB 2 tool should be used, involving judgements and support for those judgements across a series of domains of bias, as described in the Handbook.

Risk of bias in individual study results for the included studies should be explicitly considered to determine the extent to which findings of the studies can be believed. Risks of bias might vary by result. It may not be feasible to assess the risk of bias in every single result available across the included studies, particularly if a large number of studies and results are available. Review author should therefore assess risk of bias in the results of outcomes included in their ‘summary of findings’ tables, which present the findings of seven or fewer outcomes that are most important to patients. The RoB 2 tool – as described in the Handbook– is the preferred tool for all randomized trials in new reviews. The Cochrane Evidence Production and Methods Directorate is, however, aware that there remain challenges in learning and implementation of the tool, and use of the original Cochrane risk of bias tool is acceptable for the time being.

See Handbook Section 7.1.2; Chapter 8
C53 Assessing risk of bias in duplicate Mandatory  
  Use (at least) two people working independently to apply the risk-of-bias tool to each included study, and define in advance the process for resolving disagreements. Duplicating the risk-of-bias assessment reduces both the risk of making mistakes and the possibility that assessments are influenced by a single person’s biases.

 
See Handbook Section 7.3.2; Chapter 8
C54 Supporting judgements of risk of bias Mandatory  
  Justify judgements of risk of bias (high, low and some concerns) and provide this information in the risk-of-bias tables (as ‘Support for judgement’). Providing support for the judgement makes the process transparent. See Handbook Section 7.3.2; Chapter 8
C55 Providing sources of information for risk of bias assessments Mandatory  
  Collect the source of information for each risk of bias judgement (e.g. quotation, summary of information from a trial report, correspondence with investigator etc.). Where judgements are based on assumptions made on the basis of information provided outside publicly available documents, this should be stated. Readers, editors and referees should have the opportunity to see for themselves from where supports for judgements have been obtained. See Handbook Section 7.3.2; Chapter 8
C56 Summarizing risk-of-bias assessments. Highly desirable  
  Summarize the risk of bias for each key outcome for each study This reinforces the link between the characteristics of the study design and their possible impact on the results of the study and is an important prerequisite for the GRADE approach to assessing the certainty of the body of evidence. See Handbook Section 7.5; Chapter 8
C57 Addressing risk of bias in the synthesis. Highly desirable  
  Address risk of bias in the synthesis (whether quantitative or non-quantitative). For example, present analyses stratified according to summary risk of bias, or restricted to studies at low risk of bias. Review authors should consider how study biases affect results. This is useful in determining the strength of conclusions and how future research should be designed and conducted. See Handbook Section 7.6.1; Chapter 8
C58 Incorporating assessments of risk of bias. Mandatory  
  If randomized trials have been assessed using one or more tools in addition to the RoB 2 tool, use the RoB 2 tool as the primary assessment of bias for interpreting results, choosing the primary analysis, and drawing conclusions. For consistency of approach across Cochrane Intervention Reviews, the RoB 2 tool should take precedence when two or more tools are used for assessing risk of bias in randomized trials. The RoB 2 tool also feeds directly into the GRADE approach for assessing the certainty of the body of evidence. See Handbook Section 7.6.1; Chapter 8 
C59 Addressing conflicts of interest in included trials. Highly desirable  
  Address conflict of interests in included trials, and reflect on possible impact on: a) differences in study design; b) risk of bias in trial result, and c) risk of bias in synthesis result Review authors should consider assessing whether they judge a trial to be of “notable concern about conflicts of interest”. This assessment is useful for exploration of possible heterogeneity between trials (e.g. in a subgroup analysis), and for reflection on relevant mechanisms for how conflict of interest may have biased trial results and synthesis results. Concerns about conflicts of interest can be reported in the ‘Characteristics of included studies’ table. See Handbook Section 7.8.6; Chapter 8
C60 Not applicable    

 

 

 

Section info
Contact
support@cochrane.org
Describe change
C52-C60 changed as an update of the RoB chapter of the Handbook resulted in them no longer being applicable.
Details: Previously C56 and C57 advised authors to assess risk of bias due to lack of blinding and risk of bias due to incomplete outcome data for different outcomes. These recommendations were necessary for users of the 2011 Cochrane risk of bias tool, but are no longer needed, because users of RoB 2 or ROBINS-I are asked explicitly to assess risk of bias on a per-result basis for all domains. In the updated RoB chapter C56 & C57 have been deleted and two new standards added: C56 Ensuring results of outcomes included in 'Summary of findings' tables are assessed for risk of bias, and C60: Addressing conflicts of interest in included trials. As two standards have been deleted and directly the numbering of all subsequent standards stays the same.
C55 changed from Highly Desirable to Mandatory.

C52, column 2: "in this Handbook" changed to "in Handbook version 6"; column 4: "See Handbook 8.2.1, 8.5, 8.9 to 8.15" to "See Handbook version 6 (Chapter 8)
C59, column 4: "See Handbook 8.8.1" to "See Handbook version 6 (Chapter 8)"

8/8/2019
C52 column 3: Recommendations for assessing bias in randomized studies included in Cochrane Reviews are now well established. -changed to- DELETED
C52 column 3: as described in this Handbook -changed to- as described in Handbook version 6
C52 column 4: See Handbook version 6 (Chapter 8) -changed to- See Handbook (version 6), Section 7.1.2; Chapter 8
C53 column 2: risk of bias tool -changed to- risk-of-bias-tool
C53 column 3: the risk of bias assessment -changed to- the risk-of-bias assessment
C53 column 4: See Handbook 8.3.4 -changed to- See Handbook (version 6), Section 7.3.2; Chapter 8
C54 column 2: risk of bias tables -changed to- risk-of bias tables
C54 column 3: Items that are judged to be at an unclear risk of bias but are without accompanying information supporting the judgment appear as empty cells in the graphical plots based on the ‘Risk of bias’ tool in the published review. -changed to- DELETED
C54 column 4: Handbook 8.5.2 -changed to- Handbook (version 6), Section 7.3.2; Chapter 8
C55 column 2; risk of bias judgement -changed to- risk-of-bias judgement
C54 column 3: judgments -changed to- judgements
C55 column 4: Handbook 8.5.2 -changed to- Handbook (version 6), Section 7.3.2; Chapter 8
C56 column 4: Handbook 8.5.1, 8.11.2, 8.12.2 -changed to- Handbook (version 6), Section 7.3.2; Chapter 8
C57 title: Summarizing risk of bias assessments changed to- Summarizing risk-of-bias assessments
C57 column 4: Handbook 8.5.1, 8.13.2 -changed to- Handbook (version 6), Section 7.5; Chapter 8
C58 column 4: Handbook 8.7 -changed to- Handbook (version 6), Section 7.6.1; Chapter 8
C59 column 4: Handbook version 6 (Chapter 8) -changed to- Handbook (version 6), Section 7.6.1; Chapter 8
C60 column 3: “notable concern of conflicts of interest” -changed to- “notable concern about conflicts of interest”
C60 column 4: Handbook 8.8.1 -changed to- Handbook (version 6), Section 7.8.6; Chapter 8

26/09/2019
Inserted links to Handbook (version 6)

22/10/2020
Update pending: C56 Highly desirable -will change to- C56 Mandatory

24/02/2021
C56 Highly desirable -changed to- C56 Mandatory

04/03/2021
Removed (version 6) from all references to the Handbook as all references are to the most current version of the Handbook
Change date
4 March 2021