Early Career Professionals Network: Learn more about GRADE

Dr. Luis Enrique Colunga-Lozano is both an internal medicine and a critical care physician from Guadalajara, México. He currently works at the COVID-19 intensive care unit at Hospital Civil de Guadalajara - Dr. Juan I Menchaca. He is also a junior professor at Universidad de Guadalajara, where he teaches evidence-based medicine. Dr. Colunga-Lozano received his Master’s degree in Health Research Methods from McMaster University under Prof. Gordon Guyatt’s supervision. Furthermore, he has been a member of Cochrane since 2013 and has been involved in the production of systematic reviews as part of various Cochrane Groups including the Metabolic and Endocrine Disorders Group, the Heart Group, and the Schizophrenia Group. He is also a member of the Geographical Group in Guadalajara, Mexico. Dr. Colunga-Lozano works closely with members of the Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group on guideline development projects.

The assessment of the certainty (quality) of evidence is an important task during evidence synthesis, with implications on evidence applicability in decision-making. Several methods are available to evaluate the certainty of evidence. However, the GRADE approach has become the preferred method due to its systematic process and transparency. This Cochrane Blog provides a summary for researchers that are not familiar with the GRADE approach. A more detailed description is available in the GRADE series published in the Journal of Clinical Epidemiology. Useful resources are also available on the GRADE working group’s website. Please visit: https://www.gradeworkinggroup.org/

Grading of Recommendations Assessment, Development and Evaluation (GRADE)

What is GRADE?

The GRADE working group is a collaboration that began in 2000, with an interest of addressing the limitations of grading systems in health care. The GRADE working group developed a sensible and transparent approach to assess the certainty of evidence and strength of recommendations. To date, more than 110 organizations have adopted the GRADE approach as part of their methodology (e.g., WHO, UpToDate, Cochrane, DynaMed, the National Health Service, and the Centers for Disease Control and Prevention).

The first article describing the GRADE approach was published in 2004 in the British Medical Journal (BMJ). In 2008, the BMJ published a series containing guidance for clinicians to understand the GRADE approach, and from 2011 to the present day, more than 30-parts of a series expanding the GRADE concepts have been published in the Journal of Clinical Epidemiology, which aims to provide guidance on systematic reviews and clinical guideline development.

What are we grading?

There are two components that we rate when we are using the GRADE approach. Firstly, the certainty (quality) of the evidence, and secondly, the strength of recommendations. GRADE divides the certainty of the evidence as very low, low, moderate, and high confidence. It is important to remark that GRADE does not intent to assess single studies, it is about assessing bodies of evidence, ideally produced from systematic reviews. GRADE also classifies the x

What are the GRADE criteria for assessing the certainty of evidence?

Currently, there are specific criteria for assessing certain bodies of evidence (e.g., treatment (pairwise), network, diagnosis, and prognosis). For example, in the assessment of treatment effects, randomized controlled trials will start as high confidence, and observational studies will start as low confidence. However, confidence in the evidence can change based on the following domains. Evidence certainty will be downgraded by the following: risk of bias (-1, -2), inconsistency (-1, -2), indirectness (-1, -2), imprecision (-1, -2) and publication bias (-1, -2), and upgraded with the following: large effect (+1), dose response (1+), all plausible confounding (1+).

What are the GRADE criteria for assessing the strength of recommendations?

Recommendations are informed by the evidence and a strong recommendation implies that benefits outweigh the risk/hassle/cost. There are several determinants of the strength of a recommendation. We can consider three as very important: Firstly, the balance between desirable and undesirable effects. For example, if the desirable consequences outweigh the undesirable consequences, there is more likely to be a strong recommendation, or if the desirable and undesirable consequences are closely balanced, there is more likely to be a weak/conditional recommendation. Secondly, the higher the quality of the evidence, the higher the likelihood that a strong recommendation is warranted. Finally, if the guideline panel is very certain of the trade-off between the desirable and undesirable consequences of treatment, that means that the preference is similar between patients, there is more likely to be a strong recommendation. If the guideline panel is uncertain about patient values and preference, and they believe that this will be different across patients, there is more likely to be a weak recommendation. Other considerations include cost, importance of the problem, acceptability, feasibility, and equity (Evidence to Decision Frameworks).

What is the interpretation of a strong vs weak recommendations?

With a strong recommendation, the variability of patient preference will be less (i.e., almost all (>90%) of informed patients will decide on the same choice); whereas with a weak recommendation, informed patients will select different choices. With a strong recommendation (based on supporting evidence), interaction with the patient is to ‘inform’ the suggestion (e.g., I consider this should be the course of action, because of this…); whereas with a weak recommendation, the right choice may differ between patients, highlighting the need for a shared decision-making process to identify the right choice for each patient. Strong recommendations are considerations for quality of care, and weak considerations are not, because the right choice differs from patient to patient.

Conclusion

Healthcare professionals and policy makers need evidence summaries to help them during their decision-making, these summaries should report the quality of the evidence and the strength of recommendations. The GRADE framework provides explicit guidance on the assessment of the certainty of evidence and strength of recommendations. The GRADE approach is transparent and systematic, which may be one of the reasons it has been adopted by many international institutions.

28 January 2022