MECIR Manual
Assessing risk of bias in included studies (C52-60)

Assessing risk of bias in included studies

Cochrane Training resources: assessing RoB and RoB 2.0 webinar

Cochrane Interactive Learning (CIL): module 5 - introduction to study quality and risk of bias

  Standard Rationale and elaboration Resources
C52 Assessing risk of bias Mandatory  
  Assess the risk of bias in at least one specific result for each included study. For randomized trials, the RoB 2 tool should be used, involving judgements and support for those judgements across a series of domains of bias, as described in this Handbook The risk of bias in at least one specific result for every included study must be explicitly considered to determine the extent to which its findings can be believed, noting that risks of bias might vary by result. Recommendations for assessing bias in randomized studies included in Cochrane Reviews are now well established. The RoB 2 tool – as described in this Handbook – must be used for all randomized trials in new reviews and all newly included randomized trials in updated reviews. This does not prevent other tools being used. See Handbook 8.2.1, 8.5, 8.9 to 8.15
C53 Assessing risk of bias in duplicate Mandatory  
  Use (at least) two people working independently to apply the risk of bias tool to each included study, and define in advance the process for resolving disagreements. Duplicating the risk of bias assessment reduces both the risk of making mistakes and the possibility that assessments are influenced by a single person’s biases.

See Handbook 8.3.4
C54 Supporting judgements of risk of bias Mandatory  
  Justify judgements of risk of bias (high, low and some concerns) and provide this information in the risk of bias tables (as ‘Support for judgement’). Providing support for the judgement makes the process transparent. Items that are judged to be at an unclear risk of bias but are without accompanying information supporting the judgment appear as empty cells in the graphical plots based on the ‘ Risk of bias’ tool in the published review. See Handbook 8.5.2
C55 Providing sources of information for risk of bias assessments Mandatory  
  Collect the source of information for each risk of bias judgement (e.g. quotation, summary of information from a trial report, correspondence with investigator etc.). Where judgements are based on assumptions made on the basis of information provided outside publicly available documents, this should be stated. Readers, editors and referees should have the opportunity to see for themselves from where supports for judgments have been obtained. See Handbook 8.5.2
C56 Ensuring results of outcomes included in ‘Summary of findings’ tables are assessed for risk of bias. Highly desirable  
  Ensure that assessments of risk of bias cover the outcomes included in the ‘Summary of findings’ table. It may not be feasible to assess the risk of bias in every single result available across the included studies, particularly if a large number of studies and results are available. Review author should strive to assess risk of bias in the results of outcomes that are most important to patients. Such outcomes will typically be included in ‘Summary of findings’ tables, which present the findings of seven or fewer patient-important outcomes. See Handbook 8.5.1, 8.11.2, 8.12.2
C57 Summarizing risk of bias assessments. Highly desirable  
  Summarize the risk of bias for each key outcome for each study This reinforces the link between the characteristics of the study design and their possible impact on the results of the study and is an important prerequisite for the GRADE approach to assessing the certainty of the body of evidence. See Handbook 8.5.1, 8.13.2
C58 Addressing risk of bias in the synthesis. Highly desirable  
  Address risk of bias in the synthesis (whether quantitative or non-quantitative). For example, present analyses stratified according to summary risk of bias, or restricted to studies at low risk of bias. Review authors should consider how study biases affect results. This is useful in determining the strength of conclusions and how future research should be designed and conducted. See Handbook 8.7
C59 Incorporating assessments of risk of bias. Mandatory  
  If randomized trials have been assessed using one or more tools in addition to the RoB 2 tool, use the RoB 2 tool as the primary assessment of bias for interpreting results, choosing the primary analysis, and drawing conclusions. For consistency of approach across Cochrane Intervention Reviews, the RoB 2 tool should take precedence when two or more tools are used for assessing risk of bias in randomized trials. The RoB 2 tool also feeds directly into the GRADE approach for assessing the certainty of the body of evidence. See Handbook 8.8.1
C60 Addressing conflicts of interest in included trials. Highly desirable  
  Address conflict of interests in included trials, and reflect on possible impact on: a) differences in study design; b) risk of bias in trial result, and c) risk of bias in synthesis result Review authors should consider assessing whether they judge a trial to be of “notable concern of conflicts of interest”. This assessment is useful for exploration of possible heterogeneity between trials (e.g. in a subgroup analysis), and for reflection on relevant mechanisms for how conflict of interest may have biased trial results and synthesis results. Concerns about conflicts of interest can be reported in the ‘Characteristics of included studies’ table. See Handbook 8.8.1




Section info
Describe change
C52-C60 changed as an update of the RoB chapter of the Handbook resulted in them no longer being applicable.
Details:Previously C56 and C57 advised authors to assess risk of bias due to lack of blinding and risk of bias due to incomplete outcome data for different outcomes. These recommendations were necessary for users of the 2011 Cochrane risk of bias tool, but are no longer needed, because users of RoB 2 or ROBINS-I are asked explicitly to assess risk of bias on a per-result basis for all domains. In the updated RoB chapter C56 & C57 have been deleted and two new standards added: C56 Ensuring results of outcomes included in 'Summary of findings' tables are assessed for risk of bias, and C60: Addressing conflicts of interest in included trials. As two standards have been deleted and directly the numbering of all subsequent standards stays the same.
C55 changed from Highly Desirable to Mandatory.
Change date
26 February 2019