Stakeholder engagement in high-profile reviews pilot

Introduction

Cochrane is conducting a pilot project for engaging stakeholders (such as consumers, clinicians and researchers) in the development of high-profile Cochrane reviews. The pilot will develop and test a process for involving stakeholders in updating the Cochrane review on Exercise therapy for chronic fatigue syndrome. This webpage provides information and updates on the progress of the pilot and the update of the Cochrane review.

Progress reports:

Frequently asked questions (FAQs) about the update of the Cochrane review on ‘Exercise therapy for chronic fatigue syndrome’

FAQs last updated 10 November 2023

Why is the review being updated?

Cochrane reviews can be updated because a review’s methods or included evidence may be out of date, or in response to concerns. This review is being updated for all those reasons.

Cochrane’s Editor in Chief, Karla Soares-Weiser, considered the range of concerns expressed about this review and concluded it needs a patient-focused, contemporary perspective on exercise therapy for myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS). The current review was amended in 2019, but still approaches the issue with a research question and a set of methods from the early 2000s, reflecting evidence from studies that applied definitions of ME/CFS from the 1990s. In addition, the results of any relevant studies available after May 2014 are not included.

Why was the Cochrane review amended in October 2019?

The amendment by the authors in October 2019 aimed to address some of the concerns raised about the review in the short term, pending the lengthier process of a complete update.

The changes in that amended review related to the limited applicability of the evidence given the definitions of ME/CFS in the studies, limited data on long-term effects on fatigue, and limited assessment of potential harms. An editorial note was also attached to the review, linking it to a statement announcing the amendment and planned update process.

Why is the Cochrane review being updated instead of retracted?

In the past, Cochrane reviews could be withdrawn for being out-of-date alone. However in 2019, the editorial policy for withdrawing Cochrane reviews was brought into line with the retraction guidelines of the Committee on Publication Ethics (COPE). Withdrawal is now considered where there is serious error, retraction of an included study has made the Cochrane review unreliable, or misconduct in the review has been established, including serious breach of Cochrane’s conflict of interest policy.

An investigation of whether or not the 2019 version Cochrane review contained serious errors justifying retraction was conducted following requests for withdrawal of this review on that ground. That investigation was conducted by a sub-group of Cochrane’s Editorial Board and a Cochrane Coordinating Editor, and included consideration of the consultation version of the updated NICE guideline and evidence review. In 2021, they agreed with the Editor in Chief’s determination that the review did not contain serious error.

The other potential reasons for retraction – retraction of an included study or a determination that there was misconduct in the review – do not apply.

Can Cochrane do anything else to address concerns while waiting for the update?

Finalising the update of the review will take time. So a new editorial note is being developed to attach to the current, 2019 version, of the review. The note will address some concerns about use of that review by clarifying its limitations and the need for an update.

Why was there such a long delay in this update, and how long will the new Cochrane review take?

The updating process for the review was interrupted first by the pandemic and the Cochrane Editorial Unit’s focus on COVID-19, including substantial time from the Cochrane-supported authors of the updated review. Secondly, in mid-2021 Cochrane’s major public funder announced its support would end in early 2023, including for the Cochrane Review Group that was the editorial base for the update. The funding change triggered an extensive re-organisation of Cochrane and its editorial processes, including disbanding the previous structure of Cochrane Review Groups and Networks.

Through this time, the author group continued to progress work on the protocol for an updated review (more on this process below). The Editorial Unit is currently planning future steps for this review.

Is the review update being done by the authors and editorial review group of the current review?

No. The author team of the current review have stepped down. The biographies and declarations of the new author team are here.

At the beginning of this project, a new editorial "home" was established for the review, with the Cochrane Musculoskeletal, Oral, Skin and Sensory Network (MOSS) Network, in collaboration with the Cochrane Pain, Palliative and Supportive Care (PaPaS) group. While both these structures were discontinued in Cochrane’s re-organisation, some individuals remain in ongoing editorial structures and will have continued involvement.

Why is there an Independent Advisory Group?

The goals of the Independent Advisory Group (IAG) are to:

  • ensure and facilitate responsiveness to stakeholders' concerns; and
  • improve the relevance, accessibility, and credibility of the revised and updated review.

The involvement of representatives of advocacy groups for people living with ME/CFS from different parts of the world is intended to help ensure a patient-focused perspective on the review’s questions. 

The lead for the IAG, Hilda Bastian, was chosen by the Cochrane Editor in Chief. Hilda has been a health consumer advocate and researcher, and has held many roles with healthcare bodies. She was chosen for the broad support seen for her writing on the specific concerns of people with ME/CFS related to this review, as well as her understanding of the methodology of systematic reviews.

The IAG resumed working in August 2023, and the IAG lead resumed public reporting on the project in November 2023. Now that the IAG has resumed work, reports are expected to usually be every month or two until the protocol is finalised. The reporting plan may be revised after that.

You can see the biographies and declarations of interests of the IAG members here and read about how they were chosen here.

More on the background: announcement of the appointment of the lead for the IAG 

More on Cochrane’s position on consumer involvement: The statement of principles for consumer involvement in Cochrane.

Will there be ways for the views of people other than members of the IAG to be heard?

Yes. The first step in the update process will be a new protocol. The protocol will set out, in detail, the proposed scope and methods for conducting the review. A draft will be peer reviewed, and then be available for a brief public consultation period. Further opportunities for engagement in the update will be announced.

When protocols and full reviews are published, Cochrane has a commenting function that can be used to submit comments.

People can communicate directly with the IAG lead via the email address, Cochrane.IAG@gmail.com. The email account is monitored by the IAG lead, not members of Cochrane’s staff. When the IAG finishes its work, the contents of the account will be archived at Cochrane.

What will the IAG do?

The IAG will provide advice at each stage of the review’s update process. It is anticipated that the IAG will be:

  • Advising on the language used to describe ME/CFS;
  • Advising on key issues for the review’s new protocol, such as diagnostic criteria;
  • Reviewing and providing an IAG response to the drafts of the protocol and review;
  • Suggesting peer reviewers;
  • Reviewing the revisions made by the author team following peer review;
  • Considering issues raised by peer reviewers, stakeholder groups, and other readers about the protocol, the process for updating the review, and the review.

The IAG will shape its own role, consider gaps in its membership, and determine communication processes and confidentiality in the update of the review. Meetings will be conducted by teleconference and most communication will likely be over email.

The IAG lead will recruit members in liaison with Cochrane, and is the IAG’s spokesperson. She will facilitate its discussions and chair its meetings, and coordinate its responses to the Cochrane team. The IAG lead received an honorarium in the early stages of this process.

What happens if the IAG and the review authors do not agree on an issue?

If that happens, Cochrane's Editor in Chief will have final decision-making responsibility.

Where can we get updates and answers to other questions?

Archive of previous versions of this webpage: