Cochrane Exercise and ME/CFS Review Update: November 2023

November 2023

Report by Hilda Bastian, Independent Advisory Group Lead

Posted online: 10 November 2023

Note: Background information on this project is here. The views expressed in this report are those of the author, and are not necessarily shared by the Independent Advisory Group (IAG), the review team, or Cochrane.

This is the fourth report on progress with this project. You can see a list of these reports here. This time, there are updates on:

In addition, the FAQs on the Cochrane website for this project have been expanded and updated.

This report is long overdue, and so comes with deep apologies for my long silence. I was out of action for some time after the sudden death of my son late in 2021. That coincided with the upheaval for this project caused by the organisational changes and complaints discussed in this report. The combination of extended uncertainties and confidentiality while some processes were underway made meaningful reporting impossible. Those uncertainties included how long the unplanned hiatus for the IAG might last. The authors’ work on the draft protocol continued, however.

Now that reporting has resumed, we are planning on reporting regularly – usually about once a month. You can subscribe to these updates by sending an email to exerciseforme-cfsupdates@cochrane.org

In the past, Twitter was a means of communicating with me publicly about this project. However, I am no longer active on that platform. So as a channel for communication, I have set up a personal project talk page, where I can be contacted there with public questions, suggestions, and to raise issues between reports. You can subscribe to see comments made there. You can also contact me privately at Cochrane.IAG@gmail.com

Finally, I’m very grateful to the project manager, Rachel Marshall, and the members of the IAG for their forbearance, kindness, and ongoing diligence throughout. Rachel has left Cochrane. Her support and impressive skills – both scientific expertise and interpersonal – were invaluable, and are missed.

Hilda Bastian

10 November 2023

 

Recent IAG meetings and drafting a proposed editorial note for the review

The IAG resumed working after the unplanned hiatus in August 2023. At the end of September, we had a joint meeting with Cochrane’s Editor in Chief, Karla Soares-Weiser, her Deputy, Toby Lasserson, and a representative from Cochrane’s communication team. Straight after that, we had a pair of IAG-only meetings across time zones, with substantial overlapping attendance.

At these meetings, we discussed the recent history of the project, and future steps for the review, including the ongoing availability of Editorial Unit resources. I hope these steps will be firmed up in coming weeks, ready for the next report.

As completing the review will take time, the joint Cochrane/IAG meeting discussed a proposal from a member of the IAG for adding a revised editorial note. Some uses of the Cochrane review do not reflect critical nuance and limitations in its conclusions, and the goal of a note would be to try to reduce such misinterpretation. There was unanimous agreement to this step, and the IAG has begun drafting a proposal for the editors’ consideration.
 

Upcoming consultations and progressing the review’s draft protocol

Two consultations are currently planned for this project. The first will be for feedback on a paper summarising the criticisms made of the current and previous versions of the Cochrane review.

That paper on criticisms was prepared early in the project. The first draft was provided to the author team, so that the issues people have raised in the past could inform their discussions about the development of the protocol – the roadmap for how the updated review will be done. The final version of this criticisms paper after the consultation will be part of the public historical record for this review.

The original review has been through some updates. So some of the original critiques will be redundant. However, many of the comments made about the old versions of the review will still be relevant for consideration in the development of the new review. In addition, the consultation will be a test run of our processes for collecting, analysing, and reporting on the feedback we receive.

The second planned consultation will be held on the draft protocol for the new, updated review. The protocol lays out the rules for conducting the review, explaining what studies will be eligible and why, the ways these studies will be searched for, and how they will be assessed.  The background and methods sections of the protocol will form part of the final review.

The first draft of the protocol is currently with the Cochrane Editorial Unit, and the editors are planning the next steps for the project. Once they sign off on a protocol, the IAG will be reviewing it. Then it will go to peer reviewers, some of whom will be proposed by the IAG. Responses to peer review will also be considered by the IAG.

It will be the version after all that review that will be released as a preprint. We will use our experience with the first consultation to seek, analyse, and report on feedback to the protocol. The consultation period is currently planned to be six weeks after the preprint of the protocol is released. Providing written comments won’t be the only mechanism for discussing the protocol, and we’ll let you know about other opportunities.

Although the consultation time for the draft protocol is limited, we hope that the first paper on the history of critiques will help, as people can see what points others have already articulated.
 

Organisational changes affecting this review

Progress on this review had already been complicated because of pandemic upheaval and priorities. This was intensified in the middle of 2021, when Cochrane’s priorities had to shift again. Cochrane had to unexpectedly focus on a fundamental re-organisation when the funder of the infrastructure for most Cochrane groups announced that this funding would end in March 2023.

Those groups included Cochrane Pain, Palliative and Supportive Care Review Group, which was the group managing the editorial processes of the ME/CFS review for Cochrane’s Musculoskeletal, Oral, Skin and Sensory Network. The original arrangements were explained in the May 2021 report. As part of Cochrane’s re-organisation, these structures were disbanded. However, some individuals remain in ongoing structures, providing editorial continuity as the project moves forward.
 

Complaints about the IAG and review processes

There have been multiple complaints to Cochrane’s Editor in Chief since she announced this review project. I had the opportunity to respond to them, including when they were complaints about me leading the IAG.

Concern had previously been expressed about the current review remaining online, and that was reiterated. The status of the review was flagged with an editorial note, which in turn raised concerns. However, other than attempting to increase its prominence, the note remained as it was.

There have also been several requests for Cochrane to withdraw the review, beginning in late 2020. Cochrane’s policy requires exceptional circumstances for this step, such as serious error. The editors determined that this review does not meet those requirements, and an investigation conducted by a sub-group of Cochrane’s Editorial Board and a Coordinating Editor agreed. This issue has been discussed in the updated FAQs on this website.

In addition, there was a complaint about me, requesting that I be removed as lead for the IAG because my then-PhD supervisor had been a co-author of a planned individual patient data Cochrane review on exercise and ME/CFS that had been withdrawn. That complaint was not upheld. This had been part of my disclosures from the first blog post I wrote about the Cochrane review on exercise, and the relationship had been considered before my appointment. All those disclosures have been included in my IAG declaration as well. My PhD was completed in 2020.

There have been long periods where I have not published progress reports, and that has of course raised widespread concern. After the major pandemic-induced delay in the project, I undertook to report monthly, but there was a further lengthy silence, for the reasons discussed in this current report.

The reasons included the lengthy and confidential process following a detailed complaint that called for the process to be discontinued, and for the review to remain in its current form. The basis for incorporating an IAG into the editorial processes of a Cochrane review was challenged, as well as the need for an update, and the proposed editorial process. The complaint also argued that too many of the people who were authors of the review or members of the IAG, including me, were too biased in favour of criticisms of the current Cochrane review.

Multiple committees within Cochrane’s organisation considered this complaint consecutively, which took around a year. The complaint was not upheld after thorough consideration by the Conflict of Interest Panel, a subgroup of the Editorial Board, and finally Cochrane’s Governing Board.

As well as complaints about the project, there have been concerns about the impact on participants of personalised, aggressive communication coming from several directions, including both critics and supporters of the current Cochrane review. While we remain keen to have open and broad discussions, we will be outlining standards of conduct for that engagement, based on similar policies in other organisations.
 

IAG membership

In July 2021, I reported that the group would be expanded, to include a person who has recovered from ME/CFS and called for nominations. We received two nominations of impressive individuals, and we are grateful to both people for their willingness to contribute.

We were delighted to welcome Katharine Cheston to the IAG in August 2023. Katharine is a doctoral student at Durham University in England, and she had ME/CFS for about seven years.

The IAG now has a final position to fill, and we will be considering how to fill it with the concerns about bias within the IAG in mind.

Geoffrey Hallman was previously a proxy for the IAG member from ME/CFS Australia. He has moved on from the organisation and is no longer involved with the IAG. We are grateful for his contribution.

Finally, the form for disclosures of interest for this group has been updated to reflect changes in Cochrane’s policy, and new updated forms for all members will be progressively added to the biography and declarations of interest page.