Working smarter for greater impact: the significance of changing the Cochrane review format

Fireworks

When Cochrane’s founders completed their first systematic reviews in pregnancy and childbirth over 30 years ago, they understood the importance of making evidence accessible to those who need it. Those early reviews were summarised in a book to help midwifes and others use that evidence in practice. This first collection of systematic reviews in pregnancy and childbirth ignited a series of events that led to the creation of the Cochrane Collaboration. Promoting access and usability of evidence to meet our users' needs was one of the founding principles on which the Cochrane Collaboration was built.

As well as accessibility, Cochrane has been at the forefront of innovation in many other areas. It has led the field in methodological rigor, research integrity, and publishing and platform functionality, to name a few. Today, Cochrane’s systematic reviews are a cornerstone in making health decisions as users know they can trust, and have confidence in, our evidence.

Since its inception in 1993, Cochrane used the same production model. Over this time, it grew organically, resulting in inconsistent, time consuming and cumbersome processes, leading to variable and sometimes poor author experience. Even with prioritisation processes in place to oversee development of Cochrane’s review writing software and information management systems, development cycles had to accommodate competing priorities of multiple stakeholders. 

During Cochrane’s growth, a major tension developed. Leading the way with online publications and structured reporting within RevMan (Cochrane's authoring tool) meant authors could report details to demonstrate the high rigour Cochrane reviews are renowned for. However, this often led to including detail that was not of relevance to many users of Cochrane evidence, resulting in very lengthy publications that were difficult to navigate. Because of this, Cochrane's reviews have been known to run to 100s of pages and take years to complete.

Cochrane needed to simplify how its systematic reviews were produced to improve author experience and return to the founding principle of ensuring accessibility and usability of its evidence. In addition, Cochrane needed to reposition itself competitively within the systematic review production field, reaffirm its position as an innovator in open science, research integrity and methods, and maximise the benefits of new technological developments in developing and using Cochrane evidence. Cochrane had to balance any changes against the simple requirement to preserve the rigour which inspired its earliest contributors.

In March 2022, Cochrane’s Governing Board approved plans to change its review production model. This included developing new structures and tools, and improving processes to ensure Cochrane remains viable, sustainable, and focused on major global health challenges now and into the future. Developing the format of Cochrane reviews was an important pillar of this work. 

Extensive work over the years has informed the recent developments to the format of Cochrane reviews. This included research from Sarah Rosenbaum and colleagues in the Cochrane team in Norway in 2018, which scoped a new format for intervention reviews to make them easier to navigate and understand, as well as more streamlined for write and edit. In addition, research from Nicole Skoetz and colleagues in the Cochrane team in Cologne, Germany, in 2022, validated specific solutions for challenges with the format through stakeholder interviews and a survey that received nearly 1000 responses from across the Cochrane community.  

In April 2023, a major milestone was met when new data structures launched in RevMan, called study centric data. This was followed in September 2023 by the launch of the focused review format. By encompassing changes in both the review and data formats, authors are able to work smarter in RevMan. Study centric data guides authors to invest more time earlier in the review process to streamline the later analyses stages. This facilitates better defined and more focused reviews, with clearer criteria for the review and planned syntheses. 

Cochrane also simplified reporting for authors by moving to publishing standard reporting guidelines, like PRISMA 2020 for intervention reviews, and introduced new templates embedded within RevMan to guide authors on what to report and where, ensuring complete reporting with more consistency. Moreover, by including new sections on equity, consumer involvement, and data transparency, Cochrane is positioned at the forefront of developments in open science, building on its commitment to research integrity. 

We envision greater impact of Cochrane evidence with these changes. By focusing the main article on the main objective and main outcomes, the review itself is more concise, and easier to use by a more general audience. Those details that demonstrate Cochrane’s high rigour will not be lost; this has moved to a second layer of supplementary materials within the review, which provides additional context or enhances the review’s findings, and is focused on more specialist readers. In addition, more of the underlying data associated with the review are now available to download from the publication in easy-to-use formats, promoting collaboration and reducing research waste. This is the start of Cochrane’s journey towards FAIR (Findability, Accessibility, Interoperability and Reuse) principles for data reuse, which will open new opportunities for how we can innovate with Cochrane evidence.

Today is an important milestone with the publication of the first Cochrane review using the focused format. This review by Tomiko Sunaga and colleagues, titled ‘Anti-vascular endothelial growth factor biosimilars for neovascular age-related macular degeneration’ aimed to understand whether anti-vascular endothelial growth factor biosimilars work similarly well and are similarly safe compared with the originally approved drugs for people with neovascular age-related macular degeneration. It is a notable example of focused reporting on the main objective and main outcomes of the review, around the recommended word count of 10,000 words, which has resulted in a review that will be much easier for users to understand and apply to their own needs. The downloadable data package includes data related to the included studies, risk of bias assessments, the review’s analyses and references. 

We appreciate that navigating change is difficult and would like to thank all our authors who have started using the new format. The aim of this change is to help them to focus their reviews so that their review question is clearer, their syntheses feel less overwhelming, and the main findings more evident to users. This should not only result in it taking less time to develop and write the review, but it should be easier to stay on top of the question and evidence. Once submitted, shorter and more consistently reported reviews will speed up quality assurance, peer review, and copyediting, resulting in faster publication times.

Format does not fix scope, but what we have developed should mean authors think more about comparisons and questions that matter most to end users. We are continuing to monitor queries sent to Cochrane Support about the format changes to identify areas for further guidance or training. In early 2024, a Cochrane community survey aimed to understand how these format changes addressed the original issues we were aiming to address. Satisfaction in developing or supporting Cochrane reviews improved between 2022 and 2024 (18% dissatisfied or very dissatisfied in 2022, n=393; compared with 5% in 2024, n=85) and the overall satisfaction rating increased from favorable in 2022 to excellent in 2024 when asked how likely they would recommend a Cochrane review to a peer or colleague (2022, n=393; 2024, n=84).

The 2024 survey will inform scoping work for the next phase of developments to Cochrane’s review and data formats, which will also align with priorities included in Cochrane’s Scientific Strategy (due Quarter 3 2024). Our aim is to be agile and iterate as we get feedback on the changes already made. With a continued focus on simplifying processes and improving author experience, key areas likely to be prioritized next involve improving the use of existing data structures across Cochrane platforms to improve evidence surveillance and review development, improving the development of reviews using more advanced methods, and evolving data extraction core fields and improving standardization for included studies. 

The changes so far have created a solid foundation to facilitate innovation in how we share and use Cochrane evidence, which will facilitate Cochrane’s work with its new Scientific Strategy. The three decades since our earliest systematic reviews have seen major advances in methodology and technology. Keeping pace with change helps our authors to ensure users can access and use up-to-date trustworthy synthesized evidence. 

This has been a huge project, informed by extensive research from across the community, and delivered by multiple teams and stakeholders. We would like to thank Sarah Rosenbaum, Andy Oxman, Newton Opiyo, Simon Lewin, Claire Glenton, Sasha Shepperd, Nandi Siegfried and the rest of the Cochrane team in Norway for their extensive research in 2018 and recommendations for changes to the Cochrane intervention review format. We would like to thank the Nina Kreuzberger, Caroline Hirsch, Claire Iannizzi, Nicole Skoetz and the rest of the Cochrane team in Cologne, Germany, for the research in 2022 to validate specific solutions for challenges with the format. We would like to thank the 998 Cochrane community members and users of the Cochrane Library who completed the 2022 survey. We would also like to thank the 91 Cochrane community members and users of the Cochrane Library who completed the 2024 survey. In addition, we would like to thank everyone who was involved in the user experience testing for changes to RevMan and the Cochrane Library, and those involved in the consultation, in particular, the Editorial Board, Methods Executive and Handbook editors and authors. Finally, we would like to thank the teams across the Central Executive and Wiley who helped deliver these changes. 

For more information, you can:

6 June 2024