MECIR Manual
Data collection & analysis (R39-55)

Data collection & analysis

  Standard Rationale and elaboration Resources
R39 Inclusion decisions Mandatory  
  State how inclusion decisions were made (i.e. from search results to included studies), clarifying how many people were involved and whether they worked independently. MECIR conduct standard 39: Use (at least) two people working independently to determine whether each study meets the eligibility criteria, and define in advance the process for resolving disagreements.

Cochrane Training resource: selecting studies

CIL: module 4 - selecting studies and collecting data

R40 Data collection process Mandatory  
  State how data were extracted from reports of included studies, clarifying how many people were involved, whether they worked independently, and how disagreements were resolved. Describe data collection process for any reports requiring translation.

MECIR conduct standard 43: Use a data collection form that has been piloted.

MECIR conduct standard 45: Use (at least) two people working independently to extract study characteristics from reports of each study, and define in advance the process for resolving disagreements.

Cochrane Training resource: collecting data

CIL: module 4 - selecting studies and collecting data

R41 Requests for data Highly desirable  
  Describe attempts to obtain or clarify data from individuals or organizations. MECIR conduct standard 49: Seek key unpublished information that is missing from reports of included studies.

Cochrane Training resource: collecting data

CIL: module 4 - selecting studies and collecting data

R42 Data items Mandatory  
  State the types of information that were sought from reports of included studies. MECIR conduct standard 44: Collect characteristics of the included studies in sufficient detail to populate a table of ‘Characteristics of included studies’.

Cochrane Training resource: collecting data

CIL: module 4 - selecting studies and collecting data

R43 Transformations of data Mandatory  
  Explain any transformations of reported data prior to presentation in the review, along with any assumptions made. Explain any procedures for extracting numeric data from graphs. MECIR conduct standard 47: Collect and utilize the most detailed numerical data that might facilitate similar analyses of included studies. Where 2×2 tables or means and standard deviations are not available, this might include effect estimates (e.g. odds ratios, regression coefficients), confidence intervals, test statistics (e.g. t, F, Z, Chi2) or P values, or even data for individual participants.

Cochrane Training resource: collecting data

CIL: module 4 - selecting studies and collecting data

R44 Missing outcome data Highly desirable  
  Explain how missing outcome data were addressed. Describe how assumptions are applied for missing data, e.g. last observation carried forward, or assumptions of particular values such as worst-case or best-case scenarios.

Cochrane Training resource: collecting data

CIL: module 4 - selecting studies and collecting data

R45 Tools to assess risk of bias in individual studies Mandatory  
  State and reference the tool(s) used to assess risk of bias for included studies, how the tool(s) was implemented, and the criteria used to assign studies to judgements of low risk, high risk and unclear risk of bias.

If the Handbook guidance for undertaking ‘ Risk of bias’ assessments was followed in its entirety, then a reference to the Handbook is sufficient to provide the criteria used to assign judgements (see Handbook Sections 8.9 to 8.15). Justify any deviations from the tool.

MECIR conduct standard 52: Assess the risk of bias for each included study. For randomized trials, the Cochrane 'Risk of bias' tool should be used, involving judgements and supports for those judgements across a series of domains of bias, as described in Chapter 8 of the Handbook (version 5 or later).

MECIR conduct standards 53 – 61

Cochrane Training resources: assessing RoB included studies and RoB 2.0 webinar

CIL: module 5 - introduction to study quality and risk of bias

R46 Effect measures Mandatory  
  State the effect measures used by the review authors to describe effect sizes (e.g. risk ratio, mean difference) in any included studies or meta-analyses, or both.  

Cochrane Training resources: analysing dichotomous outcomes and analysing continuous outcomes

CIL: module 6 - analysing the data

R47 Non-standard designs Mandatory  
  If designs other than individually randomized, parallel-group randomized trials are included, describe any methods used to address clustering, matching or other design features of the included studies. MECIR conduct standard 70: Consider the impact on the analysis of clustering, matching or other non-standard design features of the included studies.

Cochrane Training resource: analysing non-standard data & study designs

CIL: module 6 - analysing the data

R48 Studies with more than two groups Mandatory  
  If multi-arm studies are included, explain how they were addressed and incorporated into syntheses. MECIR conduct standard 66: If multi-arm studies are included, analyse multiple intervention groups in an appropriate way that avoids arbitrary omission of relevant groups and double-counting of participants.

Cochrane Training resource: analysing non-standard data & study designs

CIL: module 6 - analysing the data

R49 Assessing heterogeneity Mandatory  
  Describe the methods used to identify the presence of heterogeneity between the studies in the review (e.g. non-quantitative assessment, I2, Tau2 or statistical test).

MECIR conduct standard 69: Take into account any statistical heterogeneity when interpreting the results, particularly when there is variation in the direction of effect.

MECIR conduct standard 62: Undertake (or display) a meta-analysis only if participants, interventions, comparisons and outcomes are judged to be sufficiently similar to ensure an answer that is clinically meaningful.

MECIR conduct standard 63: Assess the presence and extent of between-study variation when undertaking a meta-analysis.

Cochrane Training resource:  exploring heterogeneity

CIL: module 6 - analysing the data

R50 Risk of reporting bias across studies Highly desirable  
  Describe any methods used for assessing the risk of reporting biases such as publication bias.  

Cochrane Training resource: small study effects & reporting biases

CIL: module 7 - interpreting the findings

R51 Data synthesis Mandatory  
  Describe any methods used for combining results across studies. Where data have been combined in statistical software external to RevMan, reference the software, commands and settings used to run the analysis.

Decisions to depart from intended methods, for example an alternative statistical model, should be reported and justified.

MECIR conduct standard 62: Undertake (or display) a meta-analysis only if participants, interventions, comparisons and outcomes are judged to be sufficiently similar to ensure an answer that is clinically meaningful.

Cochrane Training resource: intro to meta-analysis

CIL: module 6 - analysing the data

R52 Subgroup analyses Mandatory  
  If subgroup analysis (or meta-regression) was performed, state the potential effect modifiers with rationale for each, stating whether each was defined a priori or post hoc and how they were compared (e.g. statistical tests).

MECIR conduct standard 22: Predefine potential effect modifiers (e.g. for subgroup analyses) at the protocol stage, restrict these in number, and provide rationale for each.

MECIR conduct standard 67: If subgroup analyses are to be compared, and there are judged to be sufficient studies to do this meaningfully, use a formal statistical test to compare them.

Cochrane Training resource: exploring heterogeneity

CIL: module 6 - analysing the data

R53 Addressing risk of bias Mandatory  
  Describe how studies with high or variable risks of bias are addressed in the synthesis. MECIR conduct standard 59: Address risk of bias in the synthesis (whether quantitative or non-quantitative). For example, present analyses that are stratified according to summary risk of bias, or restricted to studies at low risk of bias.

Cochrane Training resources: assessing RoB included studies and RoB 2.0 webinar

CIL: module 6 - analysing the data

R54 Sensitivity analysis Mandatory  
  State the basis for any sensitivity analyses performed. MECIR conduct standard 71: Use sensitivity analyses to assess the robustness of results, such as the impact of notable assumptions, imputed data, borderline decisions and studies at high risk of bias.

Cochrane Training resources: assessing RoB included studies and exploring heterogeneity

CIL: module 6 - analysing the data

R55 Summary of findings Highly desirable  
 

State any methods for summarizing the findings of the review, including the assessment of the quality of the body of evidence for each outcome. 

MECIR conduct standard 75 (Include a ‘Summary of Findings’ table according to recommendations described in Chapter 10 of the Cochrane Handbook (version 5 or later). Specifically:

  • include results for one population group (with few exceptions);
  • indicate the intervention and the comparison intervention;
  • include seven or fewer patient-important outcomes;
  • describe the outcomes (e.g. scale, scores, follow-up);
  • indicate the number of participants and studies for each outcome;
  • present at least one baseline risk for each dichotomous outcome (e.g. study population or median/medium risk) and baseline scores for continuous outcomes (if appropriate);
  • summarize the intervention effect (if appropriate); and
  • include a measure of the quality of the body of evidence)

MECIR conduct standard 76 (Use the five GRADE considerations (study limitations, consistency of effect, imprecision, indirectness and publication bias) to assess the quality of the body of evidence for each outcome, and to draw conclusions about the quality of evidence within the text of the review.) [PRISMA item 12]

Common issues in Summary of Findings tables.

Incorporating GRADE in Cochrane Reviews.

CIL: module 7 - interpreting the findings

CIL: module 8 - reporting the review

 

Section info
Change date
27 June 2016