See Handbook Section III.3.3.3 and Section 4.4.10

Cochrane Training resource: selecting studies

CIL: module 4 - selecting studies and collecting data

PR23

MECIR conduct standard 43: Use a data collection form that has been piloted.

MECIR conduct standard 45: Use (at least) two people working independently to extract study characteristics from reports of each study, and define in advance the process for resolving disagreements

See Handbook Section III.3.3.3, Section 5.4.1 and Section 5.5.2

Cochrane Training resource: collecting data

CIL: module 4 - selecting studies and collecting data

See Handbook Section III.3.3.3,and Section 5.2.3

Cochrane Training resource: collecting data

CIL: module 4 - selecting studies and collecting data

This information is a useful basis for the design of data collection forms and also indicates what sort of information about the included studies readers might anticipate seeing in the full text of the review. Detailed lists are not necessary. Instead, a broad outline of the summary information that authors might collect will suffice, for example:

“We will collect information on study design and setting, participant characteristics (including disease severity and age), study eligibility criteria, details of the intervention(s) given, the outcomes assessed, the source of study funding and any conflicts of interest stated by the investigators.”  

MECIR conduct standard 44: Collect characteristics of the included studies in sufficient detail to populate a table of ‘Characteristics of included studies’.

See Handbook Section III.3.3.3 and Section 5.3.1

Cochrane Training resource: collecting data

CIL: module 4 - selecting studies and collecting data

Briefly describe any planned strategies that will be used to address missing data. This might include imputation of missing outcome data for individuals within studies (such as worst-case or best-case scenarios), or imputations of missing standard deviations. Note that standard deviations can sometimes be computed from other reported statistics.

MECIR conduct standard 47: Collect and utilize the most detailed numerical data that might facilitate similar analyses of included studies. Where 2×2 tables or means and standard deviations are not available, this might include effect estimates (e.g. odds ratios, regression coefficients), confidence intervals, test statistics (e.g. t, F, Z, Chi2) or P values, or even data for individual participants.

MECIR conduct standard 64: Consider the implications of missing outcome data from individual participants (due to losses to follow-up or exclusions from analysis).

See Handbook Section III.3.3.3, Section 5.3.6 and Section 10.12.1

Cochrane Training resources: collecting data; analysing dichotomous outcomes; analysing continuous outcomes; assessing RoB included studies

Different tools are likely to be appropriate for different types of studies (e.g. randomized trials and non-randomized studies). If the current Handbook guidance for undertaking ‘Risk of bias’ assessments will be followed in its entirety, then a reference to the Handbook is sufficient to provide the criteria used to assign judgements. Justify any intended deviations from the tool.

MECIR conduct standard 20: Plan in advance the methods to be used for assessing risk of bias in included studies, including the tool(s) to be used, how the tool(s) will be implemented, and the criteria used to assign study results to judgements of low risk, high risk and unclear risk of bias.

MECIR conduct standard 52: Assess the risk of bias for each study result contributing to an outcome in the ‘summary of findings’ table. For randomized trials, the RoB 2 tool should be used, involving judgements and support for those judgements across a series of domains of bias, as described in the Handbook. 

Also MECIR conduct standards 53–60

See Handbook Section III.3.3.3, Section 7.1.2 and Chapter 8.

Cochrane Training resources: assessing RoB included studies and Rob 2.0 webinar

CIL: module 5 - introduction to study quality and risk of bias

MECIR conduct standard 53: Use (at least) two people working independently to apply the ‘Risk of bias’ tool to each included study, and define in advance the process for resolving disagreements.

If the Risk of Bias 2 tool (see Handbook (version 6) Chapter 8) is to be used, state whether interest will be in the effect of assignment to intervention or the effect of adhering to intervention, and explain how results will be selected to be assessed for risk of bias (i.e. for which outcome domains, outcome measures, time points and analyses).

See Handbook Section III.3.3.3, Section 7.3.2 and Chapter 8

Cochrane Training resources: assessing RoB included studies and Rob 2.0 webinar

CIL: module 5 - introduction to study quality and risk of bias

See Handbook Section III.3.3.3

Cochrane Training resources: analysing dichotomous outcomes and analysing continuous outcomes

CIL: module 6 - analysing the data

In some circumstances, specific study designs are likely to be identified in which unit-of-analysis errors might arise. This includes cluster-randomized trials, cross-over trials, trials involving multiple body parts and non-randomized studies with clustered designs.

MECIR conduct standard 70: Consider the impact on the analysis of clustering, matching or other non-standard design features of the included studies.

See Handbook Section III.3.3.3 and Section 6.2.1

Cochrane Training resource: non-standard data and study designs

CIL: module 6 - analysing the data

Note that it is mandatory to describe these methods in the full version of the review if studies with more than one arm are identified and included.

MECIR conduct standard 66: If multi-arm studies are included, analyse multiple intervention groups in an appropriate way that avoids arbitrary omission of relevant groups and double-counting of participants.

See Handbook Section III.3.3.3, Section 6.2.9 and Chapter 11.

Cochrane Training resource: non-standard data and study designs

CIL: module 6 - analysing the data

In the majority of reviews, most of this information is located under the subheading ‘Data synthesis’. Note, however, that additional subheadings should be used to provide details of subgroup analyses, assessment of heterogeneity and sensitivity analysis.  

MECIR conduct standard 21: Plan in advance the methods to be used to synthesize the results of the included studies, including whether a quantitative synthesis is planned, how heterogeneity will be assessed, choice of effect measure (e.g. odds ratio, risk ratio, risk difference or other for dichotomous outcomes), and methods for meta-analysis (e.g. inverse variance or Mantel Haenszel, fixed-effect or random-effects model).

MECIR conduct standard 62: Undertake (or display) a meta-analysis only if participants, interventions, comparisons and outcomes are judged to be sufficiently similar to ensure an answer that is clinically meaningful.

MECIR conduct standard 63: Assess the presence and extent of between-study variation when undertaking a meta-analysis.

See Handbook Section III.3.3.3, Section 1.5 and Section 10.10.2.

Cochrane Training resources: introduction to meta-analysis and exploring heterogeneity

CIL: module 6 - analysing the data

See Handbook Chapter 12

Cochrane Training resources: Cochrane Consumers & Communication Data Synthesis and Analysis

CIL: module 8 - reporting the review

See Handbook Chapter 13

Cochrane Training resources: small study effects and reporting biases

CIL: module 7 - interpreting the findings

Several options are available for addressing risk of bias in a synthesis, including reporting separate syntheses for studies at different risks of bias, restricting analysis to studies at low (or low and unclear) risk of bias only, and undertaking sensitivity analysis to examine the impact of risks of bias on the conclusions.  An understanding of the impact of risks of bias is important to inform GRADE assessments.

MECIR conduct standard 57: Address risk of bias in the synthesis (whether quantitative or non-quantitative). For example, present analyses that are stratified according to summary risk of bias, restricted to studies at low risk of bias or restricted to low-and-some-concerns of risk of bias

See Handbook Section 7.6.1 and Chapter 8 

Cochrane Training resources: assessing RoB included studies and RoB 2.0 webinar

See Handbook Section III.3.3.3

Cochrane Training resource: exploring heterogeneity

CIL: module 6 - analysing the data

See Handbook Chapter 20

CIL: module 6 - analysing the data

See Handbook Chapter 21

Cochrane Training resource: Webinar - GRADE CERQual

State the methods to be used to assess the certainty of the body of evidence (using the five GRADE considerations).

If the current GRADE guidance for these assessments will be followed in its entirety (see Handbook Chapter 14), then a reference to this is sufficient to provide the criteria used to make judgements.

MECIR conduct standard 74: Use the five GRADE considerations (risk of bias, consistency of effect, imprecision, indirectness and publication bias) to assess the certainty of the body of evidence for each outcome, and to draw conclusions about the certainty of evidence within the text of the review.

See Handbook Section III.3.3.3 and Section 14.2.1

Cochrane Training resource: GRADE approach to evaluating evidence quality

Incorporating GRADE in Cochrane Reviews.

A maximum of seven important outcomes should be prespecified for inclusion in a ‘Summary of findings’ table (see Handbook Chapter 14).  If possible, sources of any assumed risks to be presented in a ‘Summary of findings’ table should be explained.  

MECIR conduct standard 23: Plan in advance the methods to be used for assessing the certainty of the body of evidence, and summarizing the findings of the review. 

See Handbook Section III.3.3.3 and Section 1.5

Cochrane Training resource: Summary of Findings tables.

Planning GRADE and Summary of Findings tables

CIL: module 8 - reporting the review