Cochrane is committed to providing our users with the most up-to-date methodologies to produce more accurate and robust research results. Following the recommendations of the Cochrane Statistical Methods Group, RevMan (Cochrane's bespoke software for writing Cochrane Reviews) is introducing a new set of statistical methods to be used along with the random-effects model in systematic reviews of interventions. 

What is changing?  

Previously, RevMan relied on the Wald-type confidence interval and the DerSimonian and Laird method. We are now updating the software and authors will be able to use these alternative methods.  

• Restricted Maximum Likelihood (REML): This method offers a more reliable estimation of heterogeneity variance.
• Confidence interval for heterogeneity variance: use of the Q-profile statistics is helpful for conveying the uncertainty in the estimate and for providing an indication as to whether there truly is heterogeneity
• Hartung-Knapp-Sidik-Jonkman (HKSJ) Method: Designed to deliver more accurate confidence intervals for the summary effect.
• Prediction Intervals: Utilizes t-distribution or normal distribution to offer clearer insights into review findings. 
   

These methods have been shown to provide more reliable effect estimates in systematic reviews of interventions.  

What impact will this have on Authors and RevMan users? 

• The availability of these advanced statistical methods will empower authors and users by enhancing their confidence in conducting analyses and research.
• Key review figures, such as forest plots, will now provide richer insights, helping authors and readers understand the analysis process better.
• Additional data, like prediction intervals, will further assist in comprehending review outcomes.
• The interpretation of known measures, such as I-square, remains unchanged for consistency. 
   

Ella Flemyng, Head of Editorial Policy & Research Integrity, explained:  "We are immensely grateful to the Cochrane Statistical Methods Group and its convenors for their invaluable support throughout the implementation of these new statistical methods in RevMan. These advances represent a pivotal step in enhancing the integrity and reliability of our systematic reviews, ensuring that our research continues to uphold the highest standards of excellence." 

When are these changes happening? 

These new statistical methods will be available in RevMan for all users starting 23 January 2024. The PRISMA 2020 expanded checklist and the Cochrane Handbook for Systematic Reviews of Interventions already include guidance on using these new methods, but we will update the relevant Cochrane chapters and develop other author guidance to provide more context and guidance. 

From 1 July 2025, all new protocol submissions to the Central Editorial Service should adopt these new recommended methods. For new review or update submissions, whichever methods authors have used is acceptable, though the statistical approach needs to be consistent across all analyses (not a mix of both in different approaches within a single review). 

We recommend that all authors consider using these new methods as soon as they are available in RevMan.  

I’m a Cochrane author. What does it mean for my protocol or review if I adopt these new methods? 

• Settings of your RevMan analyses: If you have a new protocol, the new default statistical methods of Restricted Maximum Likelihood (REML) will be selected in the Statistical Settings Tab for each random-effects meta-analysis using inverse variance methods. If you have already set up your analyses using DerSimonian and Laird (DL) but want to switch to using these new methods, you’ll need to update the statistical methods in the Statistical Settings Tab for each random-effects meta-analysis.
• Your Methods section in the review: The statistics selected in the settings of your RevMan analyses should be reflected in your written "Synthesis Methods" section. If you have changed the statistical setting since the last publication, detail this change in methods in the relevant “Methods” section. Please note that advice from the Statistical Methods Group is that it is okay to switch without prospectively reconsidering the methods.
• Your Results and Discussion sections in the review: Summarise the numerical information to reflect the methods used, such as the "Synthesis of Results" and "Summary of Main Results" sections. 
   

Top tip: RevMan now supports dynamic analysis results reporting. This means that if you add or remove studies from an analysis, for change the statistical setup (such as moving to adopt these new random effects statistics), RevMan would automatically update reporting of the relevant analysis results throughout the whole review. For more information, see the RevMan Knowledge Base. 

Learn More: 

• Visit the Cochrane Methods website for comprehensive information.
• Participate in the Methods Support Unit web clinic scheduled on January 28, 2025, for additional support and catch up on the recording of an introduction to new random effect methods in RevMan.
• Access the RevMan Knowledge Base for further guidance.