Data collection and analysis
| Standard | Rationale and elaboration | Resources | |
| R39 | Inclusion decisions | Mandatory | |
| State how inclusion decisions were made (i.e. from search results to included studies), clarifying how many people were involved and whether they worked independently. | MECIR conduct standard 39: Use (at least) two people working independently to determine whether each study meets the eligibility criteria, and define in advance the process for resolving disagreements. |
See Handbook Section III.3.3.3 and Section 4.4.10 Cochrane Training resource: selecting studies |
|
| R40 | Data collection process | Mandatory | |
| State how data were extracted from reports of included studies, clarifying how many people were involved, whether they worked independently, and how disagreements were resolved. Describe data collection process for any reports requiring translation. |
MECIR conduct standard 43: Use a data collection form that has been piloted. MECIR conduct standard 45: Use (at least) two people working independently to extract study characteristics from reports of each study, and define in advance the process for resolving disagreements. |
See Handbook Section III.3.3.3, Section 5.4.1 and Section 5.5.2 Cochrane Training resource: collecting data |
|
| R41 | Requests for data | Highly desirable | |
| Describe attempts to obtain or clarify data from individuals or organizations. | MECIR conduct standard 49: Seek key unpublished information that is missing from reports of included studies. |
See Handbook Section III.3.3.3 and Section 5.2.3 Cochrane Training resource: collecting data |
|
| R42 | Data items | Mandatory | |
| State the types of information that were sought from reports of included studies. | MECIR conduct standard 44: Collect characteristics of the included studies in sufficient detail to populate a table of ‘Characteristics of included studies’. |
See Handbook Section III.3.3.3 and Section 5.3.1 Cochrane Training resource: collecting data |
|
| R43 | Transformations of data | Mandatory | |
| Explain any transformations of reported data prior to presentation in the review, along with any assumptions made. Explain any procedures for extracting numeric data from graphs. | MECIR conduct standard 47: Collect and utilize the most detailed numerical data that might facilitate similar analyses of included studies. Where 2×2 tables or means and standard deviations are not available, this might include effect estimates (e.g. odds ratios, regression coefficients), confidence intervals, test statistics (e.g. t, F, Z, Chi2) or P values, or even data for individual participants. |
See Handbook Section 5.3.6 Cochrane Training resource: collecting data |
|
| R44 | Missing outcome data | Highly desirable | |
| Explain how missing outcome data were addressed. | Describe how assumptions are applied for missing data, e.g. last observation carried forward, or assumptions of particular values such as worst-case or best-case scenarios. |
See Handbook Section III.3.3.3, Section 5.3.6 and Section 10.12.1 Cochrane Training resource: collecting data |
|
| R45 | Tools to assess risk of bias in individual studies | Mandatory | |
| State and reference the tool(s) used to assess risk of bias for included studies, how the tool(s) was implemented, and the criteria used to assign studies to judgements of low risk, high risk and unclear risk of bias. |
If the Handbook guidance for undertaking ‘Risk of bias’ assessments was followed in its entirety, then a reference to the Handbook is sufficient to provide the criteria used to assign judgements. Justify any deviations from the tool. MECIR conduct standard 52: Assess the risk of bias for each study result contributing to an outcome in the ‘summary of findings’ table. For randomized trials, the RoB 2 tool should be used, involving judgements and support for those judgements across a series of domains of bias, as described in the Handbook. |
See Handbook Section III.3.3.3, Section 7.1.2 and Chapter 8 Cochrane Training resources: assessing RoB included studies and RoB 2.0 webinar CIL: module 5 - introduction to study quality and risk of bias |
|
| R46 | Effect measures | Mandatory | |
| State the effect measures used by the review authors to describe effect sizes (e.g. risk ratio, mean difference) in any included studies or meta-analyses, or both. |
See Handbook Section III.3.3.3 Cochrane Training resources: analysing dichotomous outcomes and analysing continuous outcomes |
||
| R47 | Non-standard designs | Mandatory | |
| If designs other than individually randomized, parallel-group randomized trials are included, describe any methods used to address clustering, matching or other design features of the included studies. | MECIR conduct standard 70: Consider the impact on the analysis of clustering, matching or other non-standard design features of the included studies. |
See Handbook Section 6.2.1 Cochrane Training resource: analysing non-standard data & study designs |
|
| R48 | Studies with more than two groups | Mandatory | |
| If multi-arm studies are included, explain how they were addressed and incorporated into syntheses. | MECIR conduct standard 66: If multi-arm studies are included, analyse multiple intervention groups in an appropriate way that avoids arbitrary omission of relevant groups and double-counting of participants. |
See Handbook Section III.3.3.3, Section 6.2.9 and Chapter 11 Cochrane Training resource: analysing non-standard data & study designs |
|
| R49 | Assessing heterogeneity | Mandatory | |
| Describe the methods used to identify the presence of heterogeneity between the studies in the review (e.g. non-quantitative assessment, I2, Tau2 or statistical test). |
MECIR conduct standard 69: Take into account any statistical heterogeneity when interpreting the results, particularly when there is variation in the direction of effect. MECIR conduct standard 62: Undertake (or display) a meta-analysis only if participants, interventions, comparisons and outcomes are judged to be sufficiently similar to ensure an answer that is clinically meaningful. MECIR conduct standard 63: Assess the presence and extent of between-study variation when undertaking a meta-analysis. |
See Handbook Section 10.10.2 and Section 10.10.3 Cochrane Training resource: exploring heterogeneity |
|
| R50 | Risk of reporting bias across studies | Highly desirable | |
| Describe any methods used for assessing the risk of reporting biases such as publication bias. |
See Handbook Chapter 13 Cochrane Training resource: small study effects & reporting biases |
||
| R51 | Data synthesis | Mandatory | |
| Describe any methods used for combining results across studies. Where data have been combined in statistical software external to RevMan, reference the software, commands and settings used to run the analysis. |
Decisions to depart from intended methods, for example an alternative statistical model, should be reported and justified. MECIR conduct standard 62: Undertake (or display) a meta-analysis only if participants, interventions, comparisons and outcomes are judged to be sufficiently similar to ensure an answer that is clinically meaningful. |
Cochrane Training resource: intro to meta-analysis |
|
| R52 | Subgroup analyses | Mandatory | |
| If subgroup analysis (or meta-regression) was performed, state the potential effect modifiers with rationale for each, stating whether each was defined a priori or post hoc and how they were compared (e.g. statistical tests). |
MECIR conduct standard 22: Predefine potential effect modifiers (e.g. for subgroup analyses) at the protocol stage, restrict these in number, and provide rationale for each. MECIR conduct standard 67: If subgroup analyses are to be compared, and there are judged to be sufficient studies to do this meaningfully, use a formal statistical test to compare them. |
See Handbook Section III.3.3.3 and Section 10.11.3.1 Cochrane Training resource: exploring heterogeneity |
|
| R53 | Addressing risk of bias | Mandatory | |
| Describe how studies with high or variable risks of bias are addressed in the synthesis. | MECIR conduct standard 57: Address risk of bias in the synthesis (whether quantitative or non-quantitative). For example, present analyses that are stratified according to summary risk of bias, restricted to studies at low risk of bias or restricted to low-and-some-concerns of risk of bias. |
See Handbook Section 7.6.1 and Chapter 8 Cochrane Training resources: assessing RoB included studies and RoB 2.0 webinar |
|
| R54 | Sensitivity analysis | Mandatory | |
| State the basis for any sensitivity analyses performed. | MECIR conduct standard 71: Use sensitivity analyses to assess the robustness of results, such as the impact of notable assumptions, imputed data, borderline decisions and studies at high risk of bias. |
See Handbook Section 10.14 Cochrane Training resources: assessing RoB included studies and exploring heterogeneity |
|
| R55 | Summary of findings | Highly desirable | |
|
State any methods for summarizing the findings of the review, including the assessment of the certainty of the body of evidence for each outcome. |
MECIR conduct standard 75: Justify and document all assessments of the certainty of the body of evidence (for example downgrading or upgrading if using GRADE). MECIR conduct standard 74: Use the five GRADE considerations (risk of bias, consistency of effect, imprecision, indirectness and publication bias) to assess the certainty of the body of evidence for each outcome, and to draw conclusions about the certainty of evidence within the text of the review. |
See Handbook Section 14.2.1 Common issues in Summary of Findings tables. Incorporating GRADE in Cochrane Reviews. |