Issue

What to do

PICO

Setting of the research question often not specified.

Provide a brief description of the setting of the research question (e.g. community, hospital, outpatient, inpatient, country).

Outcomes

Adverse effects or harms often not presented.

Time of outcome measurement, scale of measurement, or range of scores often not specified.

Important outcomes often omitted if not measured or reported by included studies.

Present most important outcomes for patients and decision makers:

• Include at least one adverse effect outcome (helps ensure a balanced assessment of treatment effect).
• Specify time point of outcome measurement (e.g. Follow-up: 3 to 6 months).
• Specify scale of measurement and explain scores (e.g. Pain measured by VAS instrument; scale 0 to 100, 0 = no pain, 100 = worst possible pain
• For outcomes measured at multiple time points, present one valid time point (short- or long-term as appropriate).
• Include key outcomes even if data not available from included studies (i.e. outcomes not measured or reported). Knowledge about lack of data is important, highlights gap in the available evidence.

Illustrative example: See Methods section (Summary of Findings tables), and Summary of Findings table for the Main comparison & Additional Summary of Findings tables in the review: Needle size for vaccination procedures in children and adolescents.

Assumed (Control) risk or score

Sources of assumed (control) risks or scores often not specified.

Present at least one assumed risk or score for each outcome.

Provide information about source of assumed risk or score (this provides the basis for translating relative effects into absolute effects; and also informs inference about applicability of review findings).

Potential sources of assumed risks or scores:

• Well-conducted observational studies with representative participants and interventions.
• Median risk or score among control groups in included studies (rather than weighted average).
• Control group risk or score from one well conducted study among included studies.
• If considerable variation in control risks or scores exists across included studies, present a range of control risks or scores derived from these studies.
• If no meaningful estimate for control score can be derived (e.g. from standardized mean difference measure), this should be stated.

Illustrative example: See Summary of Findings table for the Main comparison & Additional Summary of Findings tables in the review: Needle size for vaccination procedures in children and adolescents.

Corresponding risk or score

Standardized mean differences (SMDs) often not translated into easily understood measures (most readers less familiar with results expressed in standard deviation units).

Present at least one corresponding intervention risk or score, including confidence interval, using either of the following options:

Option 1: Provide an interpretation of SMD in the Comments column (e.g. based on Cohen’s effect sizes where appropriate):

• 0.2 represents a small effect.
• 0.5 represents a moderate effect.
• 0.8 represents a large effect.

“A standard deviation of 0.2 represents a small difference between groups”

Option 2: Convert SMD into Mean difference (MD); provide a comparison of the MD with Minimum Important Difference (MID) if known (e.g. MID derived from relevant literature or based on effect sizes used in sample size calculations in the included studies).

“The mean difference did not reach a clinically important improvement of 50 points”

“Differences of less than 10 points on the VAS may not be clinically important”

To convert SMD into MD:

1. Select one well-conducted study with representative participants and intervention from the meta-analysis.
2. Multiply the standard deviation of the control group by the pooled SMD.

The resulting number is the MD and can be presented in the SoF table in the usual way (the original pooled SMD should be presented in the Comments column).

Illustrative example:

See Methods section (Identification and definitions of minimum important difference), and Summary of Findings table for the Main comparison & Additional Summary of Findings tables in the review: Needle size for vaccination procedures in children and adolescents.

See Cochrane Handbook: Chapter 12.6

Explanatory footnotes

Lack of clarity on quality assessment criteria, in particular GRADE factors involved and levels of downgrading or upgrading quality.

Upgrading quality of randomized trials.

Specify GRADE factor and number of levels of downgrading or upgrading quality: “Downgraded one level for serious risk of bias (due to high risk of selection bias in the majority of included studies)”

Explain decisions for not downgrading quality where relevant:

• May not downgrade for imprecision if the confidence interval for the relative effect translates into a small difference in absolute effect.
• May not downgrade for inconsistency if the direction of effect is consistent across studies, despite evidence of statistical heterogeneity.

Upgrading criteria should only be applied to well conducted observational studies.

Illustrative example: See Summary of Findings table for the Main comparison & Additional Summary of Findings tables in the below reviews:

Needle size for vaccination procedures in children and adolescents.

Improving GRADE evidence tables part 3: detailed guidance for explanatory footnotes supports creating and understanding GRADE certainty in the evidence judgments.

Number of comparisons

Choice of comparisons presented in SoF tables often not explained where multiple comparisons addressed (e.g. only comparisons with the most data presented).

Prioritize comparisons covered in tables (i.e. focus on the most important comparisons for decision makers).

Present comparison most important to users as Main SoF table; other relevant comparisons can be included as Additional SoF tables.

Illustrative example: See Methods section (Summary of Findings tables), and Summary of Findings table for the Main comparison & Additional Summary of Findings tables in the review: Needle size for vaccination procedures in children and adolescents.

Narrative data

Key outcomes often omitted from tables if data not meta-analyzed

Present key outcomes irrespective of the synthesis method (it's feasible to present numerical data alongside narrative data in SoF tables).

Illustrative example: See Summary of Findings table for the Main comparison in the review: Interventions for improving outcomes in patients with multimorbidity in primary care and community settings.

Subgroup and sensitivity analysis

Where reported, no clear rationale for inclusion of data from subgroup or sensitivity analysis.

Inclusion of findings from subgroup or sensitivity analysis need to be justified:

• Only present subgroup findings from the analysis of pre-defined subgroups if they are reliable enough to provide critical information for decision making.
• Only present results of sensitivity analysis if effect sensitive to assumption, for instance, to choice of meta-analytic model. However, it is useful to draw on the results from sensitivity analysis to justify downgrading decisions.