Search peer review

Contents:

Introduction to search peer review at Cochrane

Organizing pre-submission search QA

Assigning post-submission search peer review

Conducting a search peer review

Recommended forms integrating MECIR and PRESS elements

Peer reviewer resources

Join the Cochrane IS Search Peer Review Team

Search QA/peer review training resources and pilot report

Introduction

This page supports information specialists (IS) conducting pre-submission search quality assurance ("search QA") and post-submission search peer review in Editorial Manager (EM). This page also supports editors, authors and IS to address search peer review recommendations:

Search QA and search peer review

Cochrane IS participate in two activities to assure the quality of the search methods and strategies in Cochrane reviews: pre-submission search QA and post-submission search peer review.

 Search QASearch Peer Review
What

Search QA is an iterative, collaborative process between IS colleagues to design, test and refine search approaches and strategies. 

Search QA can take many hours or days. The same searches may be revisited multiple times over a period of months of even years.

Search peer review is an independent editorial check of the key PRESS elements of the primary databases strategy (see more about PRESS below), and Cochrane conduct and reporting standards for search. 

Search peer review should take between 30 and 90 minutes per submission.

WhenDuring the pre-submission process of review productionDuring the post-submission process of editorial peer review
WhereFace-to-face, via email, IM, phone, etc. In Editorial Manager (EM)
HowOrganized informally by IS colleaguesAssigned by an editor 

Both search QA and search peer review and strongly encouraged and recommended in Cochrane. 

Organizing Pre-Submission Search QA

Search QA is organized informally between IS colleagues in Cochrane. Requests for search QA may also be sent to the Cochrane Information Specialists' email discussion list. 

Assigning Post-Submission Search Peer Review

Editors assign the IS Reviewer task in EM. This task can be assigned to the Cochrane IS Search Peer Review Team (a dedicated team who perform search peer review for any editorial team), or to an individual Cochrane IS. Please see the EM Knowledge Base for guidance on assigning this task in EM. 

All submissions to the Central Editorial Service will receive independent search peer review by a member of the Cochrane IS Search Peer Review Team. It is not necessary to request peer review prior to submission.

If you have an established IS to peer review a submission in EM, this individual may be assigned the task by an editor. Please note: it is recommended that peer review in EM be conducted by a different IS than the individual who provided search QA.  

Conducting a Peer Review

Cochrane follows available guidance that an evidence-based checklist or appraisal form, such as the tools developed for the PRESS Peer Review of Electronic Search Strategies 2015 Guideline Statement, should be used to focus the review on the key elements of the strategy that are most likely to impact search retrieval. Use of an appraisal form has also been shown to reduce the time it takes to conduct search peer review (Relevo and Paynter 2012, Spry et al 2013). 

Using a standardized form in EM, the Cochrane IS search peer reviewer checks that:

  1. the search methods meet Cochrane's standards for conduct and reporting
  2. the key PRESS elements of the primary database strategy do not contain first order problems (errors that are likely to result in eligible studies being missed)

The search peer reviewer will recommend or suggest revisions (if required). The search peer reviewer does not re-write or re-run searches for author teams. 

Please also see our FAQs for information specialist peer reviewers.

Recommended forms integrating MECIR standards and PRESS elements

The following Word forms contain the same questions/elements that are presented as part of the IS Reviewer task in EM. 

Sample peer review assessment forms:

Helpful resources for Cochrane search peer reviewers

Resources for intervention reviews:

Resources for non-intervention reviews:

Resources for all review types:

Join the Cochrane IS Search Peer Review Team

Any information specialist working for Cochrane may apply to join the IS Search Peer Review Team. To request to join or leave the team, please contact the Cochrane Community Support Team: support@cochrane.org

Current members:

  • Yuan Chi         
  • Anne Cleeves    
  • Elise Cogo           
  • Irma Klerings     
  • Steve McDonald              
  • Maria-Inti Metzendorf    
  • Ina Monsef         
  • Jo Platt 
  • Doug Salzwedel
  • Sheila Wallace   
  • Valerie Wells      
  • Joost Daams      
  • Heidrun Janka   
  • Camila Escobar Liquitay 
  • Samantha Cox   
  • Anna Noel-Storr 

Search QA/Peer Review Training Resources & Pilot Report 

Peer review training is available to members of the IS Search Peer Review Team. 

The following slides are from assignment review sessions:

Please also see the accompanying quick reference guide for submitting recommendations in EM:

The Cochrane Information Specialist Support Team hosted a webinar on peer review of search strategies:

The Cochrane Musculoskeletal, Oral, Skin and Sensory (MOSS) Network conducted a pilot search QA project during 2018. A report on the pilot is available to download.

Last update: Aug 2022

FAQs for IS Search Peer Reviewers
FAQs for IS Search Peer Reviewers