Effects of interventions
Cochrane Interactive Learning: module 8 - reporting the review
| Standard | Rationale and elaboration | Resources | |
| R76 | Use of ‘Data and analysis’ headings | Mandatory | |
| Ensure appropriate use of any heading hierarchy of Comparisons, Outcomes, Subgroups and Study data in the ‘Data and analysis’ section. | Appropriate use of the hierarchy in RevMan5 ensures consistency of structure across reviews. It is confusing for the user if outcomes are listed against the heading ‘Comparison’ and interventions listed against the heading ‘Outcome or subgroup’. This standard will not be required when using the study-centric data structure of RevMan Web. | ||
| R77 | Presenting data | Highly desirable | |
| Ensure that simple summary data for each intervention group, as well as estimates of effect size (comparing the intervention groups), are available for each study for each outcome of interest to the review. These appear by default when dichotomous or continuous outcome data are analysed within RevMan. | Simple summaries such as numbers of events, means and standard deviations should be presented for each treatment group when available. This is achieved primarily by using the ‘Data and analyses’ section of the review, for dichotomous and continuous outcomes. For other outcomes, these should typically be presented in tables labelled ‘Other data’. When data for each separate intervention group are available for outcomes analysed as ‘generic inverse-variance’ data, these might be presented in Additional tables. | See Handbook Section III.3.4.4. | |
| R78 | Number of studies and participants | Mandatory | |
| State how many studies and how many participants contributed data to results for each outcome, along with the proportion of the included studies and recruited participants potentially available for the relevant comparison. | It is unlikely that the same number of studies will contribute data to every outcome of interest. Specific studies may contribute different numbers of participants for different outcomes. Therefore, for each comparison, it is helpful to indicate to readers what proportion of the relevant included studies and recruited participants contribute data to each outcome. Failure to disclose this may be misleading. | ||
| R79 | Source of data | Highly desirable | |
| State the source of all data presented in the review, in particular, whether it was obtained from published literature, by correspondence, from a trials register, from a web-based data repository, etc. | Transparency of data source enables validation or verification of data by others, including editors or readers of the review. | ||
| R80 | Multiple outcome data | Mandatory | |
| Describe any post hoc decisions that might give rise to accusations of selective outcome reporting, for example when there were multiple outcome measures (e.g. different scales), multiple time points or multiple ways of presenting results. |
Transparent disclosure of post hoc decisions will enable readers of the review to assess the credibility of the results of the review for themselves. Post hoc decisions to change the definition or priority of outcome measures must be reported and justified under ‘Differences between the protocol and review’. MECIR conduct standard 16: Define in advance details of what are acceptable outcome measures (e.g. diagnostic criteria, scales, composite outcomes). MECIR conduct standard 17: Define in advance how outcome measures will be selected when there are several possible measures (e.g. multiple definitions, assessors or scales). MECIR conduct standard 18: Define in advance the timing of outcome measurement. |
See Handbook Section 3.2.4.1 and Section 5.4.1 | |
| R81 | Ordering of results and ‘Data and analysis’ section | Highly desirable | |
| Organize results to follow the order of comparisons and outcomes specified in the protocol, following in particular the distinction between primary and secondary outcomes. | Review authors must avoid selective reporting of analysis results in a way that depends on the findings. The best way to achieve this is to follow a well-structured protocol and present results as outlined in that protocol. However, sometimes a pragmatic decision needs to be made that an alternative arrangement is preferable, particularly with regard to comparisons. This choice should be explicitly justified. | ||
| R82 | Prespecified outcomes | Mandatory | |
| Report synthesis results for all prespecified outcomes, irrespective of the strength or direction of the result. Indicate when data were not available for outcomes of interest, and whether adverse effects data were identified. | To avoid selective outcome reporting (in truth or in perception), the review should address all outcomes specified in the protocol. | ||
| R83 | Statistical uncertainty | Mandatory | |
| Accompany all effect size estimates with a measure of statistical uncertainty (e.g. a confidence interval with a specified level of confidence such as 90%, 95% or 99%). | Confidence intervals are the preferred method for expressing statistical uncertainty. |
Cochrane Training resource: intro to meta-analysis |
|
| R84 | P values | Highly desirable | |
| If reporting P values, provide exact P values (e.g. P = 0.08 rather than P > 0.05). | Effect estimates with confidence intervals are the preferred method of presenting numeric results. P values should not be used as an alternative to confidence intervals and should not be used to divide results into ‘significant’ or ‘non-significant’; exact P values portray the strength of evidence against the null hypothesis. | See Handbook Section 15.3.2 | |
| R85 | Tables and Figures | Mandatory | |
| Link to each Table and Figure. | All tables and figures should have a brief descriptive caption and must be referred to in numerical order in the review text. | ||
| R86 | Number of Tables and Figures | Highly desirable | |
| Keep the number of Tables and Figures low to convey key findings without affecting the readability of the review text. | Tables (typically implemented as Additional tables) and Figures (including RevMan flow charts, RevMan forest plots and imported graphics) may be added to reviews and included in the body of the text. Reviews should try and avoid including more than six such Tables and Figures in total. Further Tables and Figures can be included as supplementary material (e.g. as ‘Data and analysis’ forest plots or within Appendices). | ||
| R87 | Consistency of results | Mandatory | |
| Ensure that all statistical results presented in the main review text are consistent between the text and the ‘Data and analysis’ tables. | Errors can be introduced, particularly when analyses are rerun. | ||
| R88 | Direction of effect | Mandatory | |
| State whether findings indicate a clear direction of benefit. | Where results indicate that an intervention is better or worse than another intervention, it is important to make it clear which intervention is favoured. This is the case particularly when different scales are combined using standardized mean differences. |
Cochrane Training resource: intro to meta-analysis |
|
| R89 | Interpretability of results | Mandatory | |
| Ensure that key findings are interpretable, or are re-expressed in an interpretable way. For instance, they might be re-expressed in absolute terms (e.g. assumed and corresponding risks, NNTBs, group means), and outcomes combined with a standardized scale (e.g. standardized mean difference) might be re-expressed in units that are more naturally understood. If minimally important differences were prespecified or are available, these should be provided to aid interpretation. | Absolute effects provide a useful illustration of the likely impact of an intervention, and are usually easier to understand than relative effects. They may need to be accompanied, however, with information about assumed baseline risks. Confidence intervals should be presented for NNTBs and similar summary measures. Re-expressing relative effects as absolute effects often requires the specification of assumed (e.g. untreated) risks, and the source of these should be provided. Results expressed as standardized mean differences reflect the number of standard deviations’ difference between mean responses. This is not intuitive to many readers who may be more familiar with specific scales. Ideally, minimally important effect sizes should be specified in the protocol. | Cochrane Training resources: analysing dichotomous outcomes and analysing continuous outcomes | |
| R90 | Studies without usable data | Mandatory | |
| Comment on the potential impact of studies that apparently measured outcomes, but did not contribute data that allowed the study to be included in syntheses. |
There is good evidence of selective outcome reporting among clinical trials. Outcomes that are believed to have been measured but are not reported in a usable format may therefore be systematically different from those that are usable, and introduce bias. ‘Usable’ in this sense refers both to incorporation in a meta-analysis and to consideration in non-statistical syntheses of findings. Authors might consider using a table to indicate which studies contributed data to the outcomes of interest in the review. MECIR conduct standard 40: Include studies in the review irrespective of whether measured outcome data are reported in a ‘usable’ way. |
See Handbook Section 4.6.3 | |
| R91 | Missing outcome data | Highly desirable | |
| Discuss the implications of missing outcome data from individual participants (due to losses to follow-up or exclusions from analysis). | MECIR conduct standard 64: Consider the implications of missing outcome data from individual participants (due to losses to follow-up or exclusions from analysis). | See Handbook Section 10.2.1 | |
| R92 | Skewed data | Highly desirable | |
| Discuss the possibility and implications of skewed data when analysing continuous outcomes. | MECIR conduct standard 65: Consider the possibility and implications of skewed data when analysing continuous outcomes. |
See Handbook Section 10.5.3 Cochrane Training resource: analysing continuous outcomes |
|
| R93 | Forest plots | Highly desirable | |
| Present data from multiple studies in forest plots (using the 'Data and analyses' structure in RevMan) wherever possible, providing it is reasonable to do so. | Presenting data in forest plots can be useful, even if the studies are not combined in a meta-analysis. |
Cochrane Training resource: intro to meta-analysis |
|
| R94 | Multiple subgroup analyses and sensitivity analyses | Highly desirable | |
| If presenting multiple sensitivity analyses or different ways of subgrouping the same studies, present these in summary form (e.g. a single Table or Figure) and not in multiple forest plots. |
Cochrane Training resource: exploring heterogeneity |
||
| R95 | Labels on plots | Mandatory | |
| Label the directions of effect and the intervention groups in forest plots with the interventions being compared. | By default, RevMan currently uses ‘experimental’ and ‘control’ within labels. It is helpful to replace these with more specific intervention names, and essential if the ordering is swapped (or for head-to-head comparisons). Directions of effect should be used as consistently as possible within a review. |
Cochrane Training resource: intro to meta-analysis |
|
| R96 | Risk of bias across studies | Highly desirable | |
| Present results of the assessment of risk of bias across studies (and across domains) for each key outcome, and state whether this leads to concerns about the validity of the review’s findings. | Considerations of risk of bias across studies are required for assessments of the certainty of the body of evidence (e.g. using GRADE). |
Cochrane Training resources: assessing RoB included studies and RoB 2.0 webinar CIL: module 5 - introduction to study quality and risk of bias |
|
| R97 | Reporting biases | Highly desirable | |
| Present results of any assessment of the potential impact of reporting biases on the review’s findings. | MECIR conduct standard 73: Consider the potential impact of reporting biases on the results of the review or the meta-analyses it contains. |
See Handbook Section 13.4 Cochrane Training resource: small study effects and reporting biases |
|
| R98 | ‘Summary of findings’ table | Mandatory | |
| Present a ‘Summary of findings’ table according to recommendations described in the Handbook (version 5 or later). |
Specifically: include results for one clearly defined population group (with few exceptions); indicate the intervention and the comparison intervention; include seven or fewer patient-important outcomes; describe the outcomes (e.g. scale, scores, follow-up); indicate the number of participants and studies for each outcome; present at least one baseline risk for each dichotomous outcome (e.g. study population or median/medium risk) and baseline scores for continuous outcomes (if appropriate); summarize the intervention effect (if appropriate); and include a measure of the certainty of the body of evidence for each outcome. Efforts should be made to incorporate information presented in ‘Summary of findings’ tables (such as absolute effects, certainty ratings and downgrading decisions) in other parts of the review including the Abstract, Plain language summary, Effects of interventions, Discussion and Authors’ conclusions. |
Cochrane Training resource: GRADE approach to evaluating evidence quality |
|
| R99 | Assessments of the certainty of the body of evidence | Mandatory | |
| Provide justification or rationale for any measures of the certainty of the body of evidence for each key outcome. If a ‘Summary of findings’ table is used, use footnotes to explain any downgrading or upgrading according to the GRADE approach. |
MECIR conduct standard 74: Use the five GRADE considerations (risk of bias, consistency of effect, imprecision, indirectness and publication bias) to assess the certainty of the body of evidence for each outcome, and to draw conclusions about the certainty of evidence within the text of the review. MECIR conduct standard 75: Justify and document all assessments of the certainty of the body of evidence (for example downgrading or upgrading if using GRADE). |
See Handbook Section 14.2.1 Cochrane Training resource: GRADE approach to evaluating evidence quality Incorporating GRADE in Cochrane Reviews |