Cochrane Exercise and ME/CFS Review Update: July 2021

Report by Hilda Bastian, Independent Advisory Group Lead

Posted online: 4 August 2021

Note: Background information on this project is here. The views expressed in this report are those of the author, and are not necessarily shared by the Independent Advisory Group (IAG), the review team, or Cochrane.

This is the third monthly report on progress with this project. You can see a list of these reports here. This month, there are updates on:

Along with this report, we’ll be updating bios and disclosures of interest at the end of the month as needed. This month, there is an update for me.

And finally, a reminder that you can contact us directly at Cochrane.IAG@gmail.com

Hilda Bastian
31 July 2021

Progress on the first document for consultation and other recent major reviews

In previous reports, I’ve discussed the process we have used to engage with criticisms made of the Cochrane review on exercise since 2008. Along with documents submitted to the IAG for consideration, they were analysed to identify specific points for consideration: there were over 100. This collection of points will form the basis for discussion, consultation, and reporting as the review progresses.

The raw data of this content analysis are brief summaries of each point, with an even shorter explanatory heading. To make this list of points at least somewhat more accessible for wider consultation, we are developing a discussion paper to accompany them. That paper explains the process used to develop the list, as well as explaining how it will be used, and how people can contribute their thoughts and concerns. To facilitate this, the points have been organised into themes for inclusion in the discussion paper, with some brief introductory information for context.

Alongside this process, we have been working on a technical document comparing the specific methods used in the current version of the Cochrane review and the draft systematic reviews released by National Institute for Health and Care Excellence (NICE) in the UK and the Centers for Disease Control and Prevention (CDC) in the US. That is going through quality control, and will be shared after that is complete.

We will be considering whether including the relevant segments of this comparison provide useful context within the discussion paper, too. As we get closer to the consultation on the draft protocol for the update of the Cochrane review, seeing the similarities and areas of difference between the methods chosen by different groups of systematic reviewers may be useful to many people.

Some differences in choices of methods are unlikely to have an influence on the major outcomes for the review. For example, to be included in the NICE and CDC reviews, a trial has to have been published in English. If there are no trials published in a language other than English, then that makes no difference to the outcome.

However, some differences in methods lead to important differences in the results the authors will have as the basis for their conclusions. In the case of exercise-related trials for ME/CFS, for example, results could be reported at various time points – at the end of the course of exercise-related sessions and much later. Those specific results can be very different. When a review’s protocol sets different choices about approaching this question, it can have a direct impact on the review’s outcome. Identifying key areas like this is an important part of assessing the protocol of any systematic review.

Membership of the IAG

When the original makeup of the IAG was decided on, the brief to me as independent lead included a focus on developing partnerships with patient-advocacy groups across the world.

At this point, nine members of the IAG have been appointed including me – you can see their bios and disclosures of interest here. A tenth member is to be appointed by the IAG, and that has resulted in considerable debate and lobbying. Cochrane’s editor-in-chief, Karla Soares-Weiser, and I met to discuss this, and agreed to expand the IAG to add an additional position for person who has recovered from ME/CFS. Suggestions for this 11th position are welcome by August 22 to: Cochrane.IAG@gmail.com

Processes and policies for consultation and engagement

Engagement beyond the membership of the IAG will be a major focus of this project. This will include a range of opportunities for participation. A key focus will be consultations, the first of which will focus on the discussion paper built on criticisms of the Cochrane review since 2008.

This will begin with the release of the list of discussion points and the accompanying discussion paper. The IAG will be testing out a process for online submission of comments shortly, as well as discussing additional options for engagement around this process.

We will also be developing a policy for the IAG on engagement broadly, covering aspects typically addressed by policies people develop, for example, to moderate discussion forums, or to run consultations for public agencies.