Search peer review

Contents:

Introduction to search peer review at Cochrane

Organizing pre-submission search QA

Assigning post-submission search peer review

Conducting a search peer review

Recommended forms integrating MECIR and PRESS elements

Peer reviewer resources

Join the Cochrane IS Search Peer Review Team

Search QA/peer review training resources and pilot report

Introduction

This page presents resources to support information specialists (IS) conducting post-submission search peer review in Editorial Manager (EM) and participating in the pilot Editorial Independence and Efficiency Project. These resources also support IS conducting pre-submission search quality assurance tasks ("search QA") and recommendations:

Search QA and search peer review

Cochrane IS participate in two activities to assure the quality of the search methods and strategies in Cochrane reviews: pre-submission search QA and post-submission search peer review.

  Search QA Search Peer Review
What

Search QA is an iterative, collaborative process between IS colleagues to design, test and refine search approaches and strategies. 

Search QA can take many hours or days. The same searches may be revisited multiple times over a period of months of even years.

Search peer review is an independent editorial check of the key PRESS elements of the primary databases strategy (see more about PRESS below), and Cochrane conduct and reporting standards for search. 

Search peer review should take between 30 and 90 minutes per submission.

When During the pre-submission process of review production During the post-submission process of editorial peer review
Where Face-to-face, via email, IM, phone, etc.  In Editorial Manager (EM)
How Organized informally by IS colleagues Assigned by an editor 

Both search QA and search peer review and strongly encouraged and recommended in Cochrane. 

Organizing Pre-Submission Search QA

Search QA is organized informally between IS colleagues in Cochrane. Requests for search QA may also be sent to the Cochrane Information Specialists' email discussion list

Assigning Post-Submission Search Peer Review

Editors assign the Information Specialist Reviewer task in EM. This task can be assigned to the Cochrane IS Search Peer Review Team (a dedicated team who perform search peer review for any editorial team), or to an individual Cochrane IS. Please see the EM Knowledge Base for guidance on assigning this task in EM

All submissions to the Central Editorial Service will receive independent search peer review by a member of the Cochrane IS Search Peer Review Team. It is not necessary to request peer review prior to submission.

If you have an established IS you want to peer review a submission in EM, this individual may be assigned the task by an editor. Please note: it is recommended that peer review in EM be conducted by a different IS than the individual who provided search QA on the submission.  

Conducting a Peer Review

The search peer reviewer checks the key PRESS elements of the primary database strategy and will check that the search methods meet Cochrane's standards for conducting and reporting a search. They will then recommend or suggests revisions if required.

The search peer reviewer does not re-write or re-run searches for author teams. 

An evidence-based checklist or appraisal form, such as the tools developed for the PRESS Peer Review of Electronic Search Strategies 2015 Guideline Statement, should be used to focus the review on the key elements of the strategy that are most likely to impact search retrieval. Use of an appraisal form has also been shown to reduce the time it takes to conduct search peer review (Relevo and Paynter 2012, Spry et al 2013). 

Please also see our FAQs for information specialist peer reviewers.

Recommended forms integrating MECIR standards and PRESS elements

The following Word forms contain the same questions/elements that are presented as part of the Information Specialist Reviewer task in EM. 

Sample peer review assessment forms:

Helpful resources for Cochrane search peer reviewers

Resources for intervention reviews:

Resources for non-intervention reviews:

Resources for all review types:

Join the Cochrane IS Search Peer Review Team

Any information specialist working for Cochrane may apply to join the IS Search Peer Review Team. To request to join or leave the team, please contact the Cochrane Community Support Team: support@cochrane.org

Search QA/Peer Review Training Resources & Pilot Report 

Peer review training is available to members of the IS Search Peer Review Team. 

The following slides are from assignment review sessions:

Please also see the accompanying quick reference guide for submitting recommendations in EM:

The Cochrane Information Specialist Support Team hosted a webinar on peer review of search strategies:

The Cochrane Musculoskeletal, Oral, Skin and Sensory (MOSS) Network conducted a pilot search QA project during 2018. A report on the pilot is available to download.

Last update: May 2022