MECIR Manual
Effects of interventions (R76-R99)

Effects of interventions

Cochrane Interactive Learning: module 8 - reporting the review

  Standard Rationale and elaboration Resources
R76 Use of ‘Data and analysis’ headings Mandatory  
  Ensure appropriate use of the heading hierarchy of Comparisons, Outcomes, Subgroups and Study data in the ‘Data and analysis’ section. Appropriate use of the hierarchy ensures consistency of structure across reviews. It is confusing for the user if outcomes are listed against the heading ‘Comparison’ and interventions listed against the heading ‘ Outcome or subgroup’.  
R77 Presenting data Highly desirable  
  Ensure that simple summary data for each intervention group, as well as estimates of effect size (comparing the intervention groups), are available for each study for each outcome of interest to the review. These appear by default when dichotomous or continuous outcome data are analysed within RevMan. Simple summaries such as numbers of events, means and standard deviations should be presented for each treatment group when available. This is achieved primarily by using the ‘Data and analyses’ section of the review, for dichotomous and continuous outcomes. For other outcomes, these should typically be presented in tables labelled ‘Other data’. When data for each separate intervention group are available for outcomes analysed as ‘generic inverse-variance’ data, these might be presented in Additional tables.  
R78 Number of studies and participants Mandatory  
  State how many studies and how many participants contributed data to results for each outcome, along with the proportion of the included studies and recruited participants potentially available for the relevant comparison.  It is unlikely that the same number of studies will contribute data to every outcome of interest. Specific studies may contribute different numbers of participants for different outcomes. Therefore, for each comparison, it is helpful to indicate to readers what proportion of the relevant included studies and recruited participants contribute data to each outcome. Failure to disclose this may be misleading.   
R79 Source of data Highly desirable  
  State the source of all data presented in the review, in particular, whether it was obtained from published literature, by correspondence, from a trials register, from a web-based data repository, etc. Transparency of data source enables validation or verification of data by others, including editors or readers of the review.   
R80 Multiple outcome data Mandatory  
  Describe any post hoc decisions that might give rise to accusations of selective outcome reporting, for example when there were multiple outcome measures (e.g. different scales), multiple time points or multiple ways of presenting results.

Transparent disclosure of post hoc decisions will enable readers of the review to assess the credibility of the results of the review for themselves. Post hoc decisions to change the definition or priority of outcome measures must be reported and justified under ‘Differences between the protocol and review’.

MECIR conduct standard 16: Define in advance details of what are acceptable outcome measures (e.g. diagnostic criteria, scales, composite outcomes).

MECIR conduct standard 17: Define in advance how outcome measures will be selected when there are several possible measures (e.g. multiple definitions, assessors or scales).

MECIR conduct standard 18: Define in advance the timing of outcome measurement.

 
R81 Ordering of results and ‘Data and analysis’ section Highly desirable  
  Organize results to follow the order of comparisons and outcomes specified in the protocol, following in particular the distinction between primary and secondary outcomes.  Review authors must avoid selective reporting of analysis results in a way that depends on the findings. The best way to achieve this is to follow a well-structured protocol and present results as outlined in that protocol. However, sometimes a pragmatic decision needs to be made that an alternative arrangement is preferable, particularly with regard to comparisons. This choice should be explicitly justified.   
R82 Prespecified outcomes Mandatory  
  Report synthesis results for all prespecified outcomes, irrespective of the strength or direction of the result. Indicate when data were not available for outcomes of interest, and whether adverse effects data were identified.  To avoid selective outcome reporting (in truth or in perception), the review should address all outcomes specified in the protocol.  
R83 Statistical uncertainty Mandatory  
  Accompany all effect size estimates with a measure of statistical uncertainty (e.g. a confidence interval with a specified level of confidence such as 90%, 95% or 99%). Confidence intervals are the preferred method for expressing statistical uncertainty.

 

Cochrane Training resource: intro to meta-analysis

R84 P values Highly desirable  
  If reporting P values, provide exact P values (e.g. P = 0.08 rather than P > 0.05). Effect estimates with confidence intervals are the preferred method of presenting numeric results. P values should not be used as an alternative to confidence intervals and should not be used to divide results into ‘significant’ or ‘non-significant’; exact P values portray the strength of evidence against the null hypothesis. See Handbook 12.4.2 
R85 Tables and Figures Mandatory  
  Link to each Table and Figure. All tables and figures should have a brief descriptive caption and must be referred to in numerical order in the review text.   
R86 Number of Tables and Figures Highly desirable  
  Keep the number of Tables and Figures low to convey key findings without affecting the readability of the review text.  Tables (typically implemented as Additional tables) and Figures (including RevMan flow charts, RevMan forest plots and imported graphics) may be added to reviews and included in the body of the text. Reviews should try and avoid including more than six such Tables and Figures in total. Further Tables and Figures can be included as supplementary material (e.g. as ‘Data and analysis’ forest plots or within Appendices).  
R87 Consistency of results Mandatory  
  Ensure that all statistical results presented in the main review text are consistent between the text and the ‘Data and analysis’ tables. Errors can be introduced, particularly when analyses are rerun.  
R88 Direction of effect Mandatory  
  State whether findings indicate a clear direction of benefit. Where results indicate that an intervention is better or worse than another intervention, it is important to make it clear which intervention is favoured. This is the case particularly when different scales are combined using standardized mean differences.

 

Cochrane Training resource: intro to meta-analysis

R89 Interpretability of results Mandatory  
  Ensure that key findings are interpretable, or are re-expressed in an interpretable way. For instance, they might be re-expressed in absolute terms (e.g. assumed and corresponding risks, NNTBs, group means), and outcomes combined with a standardized scale (e.g. standardized mean difference) might be re-expressed in units that are more naturally understood. If minimally important differences were prespecified or are available, these should be provided to aid interpretation. Absolute effects provide a useful illustration of the likely impact of an intervention, and are usually easier to understand than relative effects. They may need to be accompanied, however, with information about assumed baseline risks. Confidence intervals should be presented for NNTBs and similar summary measures. Re-expressing relative effects as absolute effects often requires the specification of assumed (e.g. untreated) risks, and the source of these should be provided. Results expressed as standardized mean differences reflect the number of standard deviations’ difference between mean responses. This is not intuitive to many readers who may be more familiar with specific scales. Ideally, minimally important effect sizes should be specified in the protocol. Cochrane Training resources: analysing dichotomous outcomes and analysing continuous outcomes
R90 Studies without usable data Mandatory  
  Comment on the potential impact of studies that apparently measured outcomes, but did not contribute data that allowed the study to be included in syntheses. 

There is good evidence of selective outcome reporting among clinical trials.  Outcomes that are believed to have been measured but are not reported in a usable format may therefore be systematically different from those that are usable, and introduce bias. ‘Usable’ in this sense refers both to incorporation in a meta-analysis and to consideration in non-statistical syntheses of findings. Authors might consider using a table to indicate which studies contributed data to the outcomes of interest in the review.

MECIR conduct standard 40: Include studies in the review irrespective of whether measured outcome data are reported in a ‘usable’ way.

 
R91 Missing outcome data Highly desirable  
  Discuss the implications of missing outcome data from individual participants (due to losses to follow-up or exclusions from analysis). MECIR conduct standard 64: Consider the implications of missing outcome data from individual participants (due to losses to follow-up or exclusions from analysis).  
R92 Skewed data Highly desirable  
  Discuss the possibility and implications of skewed data when analysing continuous outcomes. MECIR conduct standard 65: Consider the possibility and implications of skewed data when analysing continuous outcomes.

 

Cochrane Training resource: analysing continuous outcomes

CIL: module 6 - analysing the data 

R93 Forest plots Highly desirable  
  Present data from multiple studies in forest plots (using the 'Data and analyses' structure in RevMan) wherever possible, providing it is reasonable to do so. Presenting data in forest plots can be useful, even if the studies are not combined in a meta-analysis.

Cochrane Training resource: intro to meta-analysis

CIL: module 6 - analysing the data 

R94 Multiple subgroup analyses and sensitivity analyses Highly desirable  
  If presenting multiple sensitivity analyses or different ways of subgrouping the same studies, present these in summary form (e.g. a single Table or Figure) and not in multiple forest plots.  

Cochrane Training resource: exploring heterogeneity

CIL: module 6 - analysing the data 

R95 Labels on plots Mandatory  
  Label the directions of effect and the intervention groups in forest plots with the interventions being compared. By default, RevMan currently uses ‘experimental’ and ‘control’ within labels. It is helpful to replace these with more specific intervention names, and essential if the ordering is swapped (or for head-to-head comparisons). Directions of effect should be used as consistently as possible within a review.

Cochrane Training resource: intro to meta-analysis

CIL: module 6 - analysing the data 

R96 Risk of bias across studies Highly desirable  
  Present results of the assessment of risk of bias across studies (and across domains) for each key outcome, and state whether this leads to concerns about the validity of the review’s findings.  Considerations of risk of bias across studies are required for assessments of the certainty of the body of evidence (e.g. using GRADE).

Cochrane Training resources: assessing RoB included studies and RoB 2.0 webinar

CIL: module 5 - introduction to study quality and risk of bias

CIL: module 7 - interpreting the findings

R97 Reporting biases Highly desirable  
  Present results of any assessment of the potential impact of reporting biases on the review’s findings. MECIR conduct standard 73: Consider the potential impact of reporting biases on the results of the review or the meta-analyses it contains.

 

Cochrane Training resource: small study effects and reporting biases

CIL: module 7 - interpreting the findings

R98 Summary of findings’ table Mandatory  
  Present a ‘Summary of findings’ table according to recommendations described in Chapter 11 of the Handbook (version 5 or later). 

Specifically: include results for one clearly defined population group (with few exceptions); indicate the intervention and the comparison intervention; include seven or fewer patient-important outcomes; describe the outcomes (e.g. scale, scores, follow-up); indicate the number of participants and studies for each outcome; present at least one baseline risk for each dichotomous outcome (e.g. study population or median/medium risk) and baseline scores for continuous outcomes (if appropriate); summarize the intervention effect (if appropriate); and include a measure of the certainty of the body of evidence for each outcome.  

Efforts should be made to incorporate information presented in ‘Summary of findings’ tables (such as absolute effects, certainty ratings and downgrading decisions) in other parts of the review including the Abstract, Plain language summary, Effects of interventions, Discussion and Authors’ conclusions. 

Cochrane Training resource: GRADE approach to evaluating evidence quality

Incorporating GRADE in Cochrane Reviews

CIL: module 8 - reporting the review

R99 Assessments of the certainty of the body of evidence Mandatory  
  Provide justification or rationale for any measures of the certainty of the body of evidence for each key outcome. If a ‘Summary of findings’ table is used, use footnotes to explain any downgrading or upgrading according to the GRADE approach.

MECIR conduct standard 74: Use the five GRADE considerations (risk of bias, consistency of effect, imprecision, indirectness and publication bias) to assess the certainty of the body of evidence for each outcome, and to draw conclusions about the certainty of evidence within the text of the review.

MECIR conduct standard 75: Justify and document all assessments of the certainty of the body of evidence (for example downgrading or upgrading if using GRADE).

Cochrane Training resource: GRADE approach to evaluating evidence quality

Incorporating GRADE in Cochrane Reviews

CIL: module 7 - interpreting the findings

CIL: module 8 - reporting the review

 

Section info
Contact
methods@cochrane.org
Describe change
R98: Status changed from "Highly Desirable" to "Mandatory".

27/09/2019
R96: column 3: quality -changed to- certainty
R98: column 3: quality -changed to- certainty (x2)
R99 column 2 and 3: quality -changed to- certainty (x5)
Change date
27 September 2019