MECIR Manual
Data collection & analysis (PR22-PR40)

Data collection & analysis 

  Standard Rationale and elaboration Resources
PR22 Inclusion decisions Mandatory  
  State how inclusion decisions will be made (i.e. from search results to included studies), clarifying how many people will be involved and whether they will work independently. MECIR conduct standard 39: Use (at least) two people working independently to determine whether each study meets the eligibility criteria, and define in advance the process for resolving disagreements.

Cochrane Training resource: selecting studies

CIL: module 4 - selecting studies and collecting data

PR23 Data collection process Mandatory  
  State how data will be extracted from reports of included studies, clarifying how many people will be involved (and whether they will work independently), and how disagreements will be resolved.

MECIR conduct standard 43: Use a data collection form that has been piloted.

MECIR conduct standard 45: Use (at least) two people working independently to extract study characteristics from reports of each study, and define in advance the process for resolving disagreements

Cochrane Training resource: collecting data

CIL: module 4 - selecting studies and collecting data

PR24 Requests for data Highly desirable  
  Describe what attempts will be made to obtain or clarify data from individuals or organizations. MECIR conduct standard 49: Seek key unpublished information that is missing from reports of included studies.

Cochrane Training resource: collecting data

CIL: module 4 - selecting studies and collecting data

PR25 Data items Mandatory  
  State the types of information that will be sought from reports of included studies.

This information is a useful basis for the design of data collection forms and also indicates what sort of information about the included studies readers might anticipate seeing in the full text of the review. Detailed lists are not necessary. Instead, a broad outline of the summary information that authors might collect will suffice, for example:

“We will collect information on study design and setting, participant characteristics (including disease severity and age), study eligibility criteria, details of the intervention(s) given, the outcomes assessed, the source of study funding and any conflicts of interest stated by the investigators.”  

MECIR conduct standard 44: Collect characteristics of the included studies in sufficient detail to populate a table of ‘Characteristics of included studies’.

Cochrane Training resource: collecting data

CIL: module 4 - selecting studies and collecting data

PR26 Missing data Highly desirable  
  Comment on how missing data will be addressed.

Briefly describe any planned strategies that will be used to address missing data. This might include imputation of missing outcome data for individuals within studies (such as worst-case or best-case scenarios), or imputations of missing standard deviations. Note that standard deviations can sometimes be computed from other reported statistics.

MECIR conduct standard 47: Collect and utilize the most detailed numerical data that might facilitate similar analyses of included studies. Where 2×2 tables or means and standard deviations are not available, this might include effect estimates (e.g. odds ratios, regression coefficients), confidence intervals, test statistics (e.g. t, F, Z, Chi2) or P values, or even data for individual participants.

MECIR conduct standard 64: Consider the implications of missing outcome data from individual participants (due to losses to follow-up or exclusions from analysis).

See Handbook 7.7

Cochrane Training resources: collecting data; analysing dichotomous outcomes; analysing continuous outcomes; assessing RoB included studies

 

PR27 Tools to assess risk of bias in individual studies Mandatory  
  State and reference the tool(s) that will be used to assess risk of bias for included studies, how the tool(s) will be implemented, and the criteria that will be used to assign study results to judgements of low risk, high risk and unclear risk of bias

Different tools are likely to be appropriate for different types of studies (e.g. randomized trials and non-randomized studies). If the current Handbook guidance for undertaking ‘ Risk of bias’ assessments will be followed in its entirety, then a reference to the Handbook is sufficient to provide the criteria used to assign judgements (see Handbook Sections 8.9 to 8.15). Justify any intended deviations from the tool.

MECIR conduct standard 20: Plan in advance the methods to be used for assessing risk of bias in included studies, including the tool(s) to be used, how the tool(s) will be implemented, and the criteria used to assign study results to judgements of low risk, high risk and unclear risk of bias.

MECIR conduct standard 52: Assess the risk of bias for each included study. For randomized trials, the Cochrane 'Risk of bias' tool should be used, involving judgements and supports for those judgements across a series of domains of bias, as described in Chapter 8 of the Handbook (version 5 or later).

Also MECIR conduct standards 53–60

Cochrane Training resources: assessing RoB included studies and Rob 2.0 webinar

CIL: module 5 - introduction to study quality and risk of bias

PR28 ‘Risk of bias’ assessment process Mandatory  
  State how risk of bias will be assessed, clarifying how many people will be involved (and whether they will work independently), and how disagreements will be resolved. MECIR conduct standard 53: Use (at least) two people working independently to apply the ‘Risk of bias’ tool to each included study, and define in advance the process for resolving disagreements.

Cochrane Training resources: assessing RoB included studies and Rob 2.0 webinar

CIL: module 5 - introduction to study quality and risk of bias

PR29 Measures of effect Mandatory  
  State the effect measures that will be used to describe effect sizes in any included studies or meta-analyses, or both (e.g. risk ratio or odds ratio, mean difference or standardized mean difference).  

Cochrane Training resources: analysing dichotomous outcomes and analysing continuous outcomes

CIL: module 6 - analysing the data

PR30 Unit of analysis issues Mandatory  
  If designs other than individually randomized, parallel-group randomized trials are likely to be included, describe any methods that will be used to address clustering, matching or other design features of the included studies

In some circumstances, specific study designs are likely to be identified in which unit-of-analysis errors might arise. This includes cluster-randomized trials, cross-over trials, trials involving multiple body parts and non-randomized studies with clustered designs.

MECIR conduct standard 70: Consider the impact on the analysis of clustering, matching or other non-standard design features of the included studies.

Cochrane Training resource: non-standard data and study designs

CIL: module 6 - analysing the data

PR31 Studies with more than two groups Highly desirable  
  If multi-arm studies are likely to be included, explain how they will be addressed and incorporated into syntheses.

Note that it is mandatory to describe these methods in the full version of the review if studies with more than one arm are identified and included.

MECIR conduct standard 66: If multi-arm studies are included, analyse multiple intervention groups in an appropriate way that avoids arbitrary omission of relevant groups and double-counting of participants.

Cochrane Training resource: non-standard data and study designs

CIL: module 6 - analysing the data

PR32 Quantitative synthesis Mandatory  
  Describe any intended statistical methods for combining results across studies (e.g. meta-analysis, subgroup analysis, meta-regression, sensitivity analysis), including methods for assessing heterogeneity (e.g. I2, Tau2, statistical test).

In the majority of reviews, most of this information is located under the subheading ‘Data synthesis’. Note, however, that additional subheadings should be used to provide details of subgroup analyses, assessment of heterogeneity and sensitivity analysis.  

MECIR conduct standard 21: Plan in advance the methods to be used to synthesize the results of the included studies, including whether a quantitative synthesis is planned, how heterogeneity will be assessed, choice of effect measure (e.g. odds ratio, risk ratio, risk difference or other for dichotomous outcomes), and methods for meta-analysis (e.g. inverse variance or Mantel Haenszel, fixed-effect or random-effects model).

MECIR conduct standard 63: Undertake (or display) a meta-analysis only if participants, interventions, comparisons and outcomes are judged to be sufficiently similar to ensure an answer that is clinically meaningful.

MECIR conduct standard 64: Assess the presence and extent of between-study variation when undertaking a meta-analysis.

Cochrane Training resources: introduction to meta-analysis and exploring heterogeneity

CIL: module 6 - analysing the data

PR33 Non-quantitative synthesis Mandatory  
  Describe any intended non-statistical methods for synthesizing findings across studies (sometimes referred to as narrative or qualitative synthesis). It may be apparent that a meta-analysis is unlikely, in which case methods should be prespecified for how the findings of the included studies will be compared and contrasted.

Cochrane Training resources: Cochrane Consumers & Communication Data Synthesis and Analysis

CIL: module 8 - reporting the review

PR34 Risk of reporting bias across studies Highly desirable  
  Describe any methods that will be used for assessing the risk of reporting biases such as publication bias.  

Cochrane Training resources: small study effects and reporting biases

CIL: module 7 - interpreting the findings

PR35 Addressing risk of bias Mandatory  
  Describe how studies with high or variable risks of bias will be addressed in the synthesis.

Several options are available for addressing risk of bias in a synthesis, including reporting separate syntheses for studies at different risks of bias, restricting analysis to studies at low (or low and unclear) risk of bias only, and undertaking sensitivity analysis to examine the impact of risks of bias on the conclusions.  An understanding of the impact of risks of bias is important to inform GRADE assessments.

MECIR conduct standard 59: Address risk of bias in the synthesis (whether quantitative or non-quantitative). For example, present analyses that are stratified according to summary risk of bias, or restricted to studies at low risk of bias.

Cochrane Training resources: assessing RoB included studies and RoB 2.0 webinar
PR36 Subgroup analyses Mandatory  
  If subgroup analysis (or meta-regression) are planned, state the potential effect modifiers with rationale for each. MECIR conduct standard 22: Predefine potential effect modifiers (e.g. for subgroup analyses) at the protocol stage, restrict these in number, and provide rationale for each.

Cochrane Training resource: exploring heterogeneity

CIL: module 6 - analysing the data

PR37 Methods for economic evidence Mandatory  
  If health economics evidence is to be reviewed, state the methods to be used to assess and synthesize this evidence.  

See Handbook 15.2-15.9

CIL: module 6 - analysing the data

 

PR38 Methods for qualitative research evidence Mandatory  
  If qualitative research evidence is to be reviewed, state the methods to be used to assess and synthesize this evidence.  

See Handbook 20.3 - 20.3.2

Cochrane Training resource: Webinar - GRADE CERQual

 

PR39 Certainty of the evidence Mandatory  
 

State the methods to be used to assess the certainty of the body of evidence (using the five GRADE considerations).

If the current GRADE guidance for these assessments will be followed in its entirety (see Handbook Chapter 12), then a reference to this is sufficient to provide the criteria used to make judgements.

MECIR conduct standard 74: Use the five GRADE considerations (risk of bias, consistency of effect, imprecision, indirectness and publication bias) to assess the certainty of the body of evidence for each outcome, and to draw conclusions about the certainty of evidence within the text of the review.

Cochrane Training resource: GRADE approach to evaluating evidence quality

Incorporating GRADE in Cochrane Reviews.

PR40 'Summary of findings’ table Mandatory  
  State which outcomes and comparisons it is planned will be included in a ‘Summary of findings’ table.

A maximum of seven important outcomes should be prespecified for inclusion in a ‘Summary of findings’ table.  If possible, sources of any assumed risks to be presented in a ‘Summary of findings’ table should be explained.  

MECIR conduct standard 23: Plan in advance the methods to be used for assessing the certainty of the body of evidence, and summarizing the findings of the review. 

Cochrane Training resource: Summary of Findings tables.

Planning GRADE and Summary of Findings tables

CIL: module 8 - reporting the review

Section info
Contact
methods@cochrane.org
Describe change
30/09/2019
PR39 column 2 and 3: quality -changed to- certainty (x4)
PR40 column 3: quality -changed to- certainty
Change date
30 September 2019