Contents:
Introduction to search peer review at Cochrane
Organizing pre-submission search QA
Assigning post-submission search peer review
Conducting a search peer review
Recommended forms integrating MECIR and PRESS elements
Join the Cochrane IS Search Peer Review Team
Search QA/peer review training resources and pilot report
Introduction
This page supports information specialists (IS) conducting pre-submission search quality assurance ("search QA") and post-submission search peer review in Editorial Manager (EM). This page also supports editors, authors and IS to address search peer review recommendations:
- Information Specialists Executive 2019 White Paper on incorporating peer review of search methods and strategies
- Section 4.4.8 Peer review of search strategies, of Chapter 4: Searching for and selecting studies in the Cochrane Handbook for Systematic Reviews of Interventions
Search QA and search peer review
Cochrane IS participate in two activities to assure the quality of the search methods and strategies in Cochrane reviews: pre-submission search QA and post-submission search peer review.
Search QA | Search Peer Review | |
What | Search QA is an iterative, collaborative process between IS colleagues to design, test and refine search approaches and strategies. Search QA can take many hours or days. The same searches may be revisited multiple times over a period of months of even years. | Search peer review is an independent editorial check of the key PRESS elements of the primary databases strategy (see more about PRESS below), and Cochrane conduct and reporting standards for search. Search peer review should take between 30 and 90 minutes per submission. |
When | During the pre-submission process of review production | During the post-submission process of editorial peer review |
Where | Face-to-face, via email, IM, phone, etc. | In Editorial Manager (EM) |
How | Organized informally by IS colleagues | Assigned by an editor |
Both search QA and search peer review and strongly encouraged and recommended in Cochrane.
Organizing Pre-Submission Search QA
Search QA is organized informally between IS colleagues in Cochrane. Requests for search QA may also be sent to the Cochrane Information Specialists' email discussion list.
Assigning Post-Submission Search Peer Review
Editors assign the IS Reviewer task in EM. This task can be assigned to the Cochrane IS Search Peer Review Team (a dedicated team who perform search peer review for any editorial team), or to an individual Cochrane IS. Please see the EM Knowledge Base for guidance on assigning this task in EM.
All submissions to the Central Editorial Service will receive independent search peer review by a member of the Cochrane IS Search Peer Review Team. It is not necessary to request peer review prior to submission.
If you have an established IS to peer review a submission in EM, this individual may be assigned the task by an editor. Please note: it is recommended that peer review in EM be conducted by a different IS than the individual who provided search QA.
Conducting a Peer Review
Cochrane follows available guidance that an evidence-based checklist or appraisal form, such as the tools developed for the PRESS Peer Review of Electronic Search Strategies 2015 Guideline Statement, should be used to focus the review on the key elements of the strategy that are most likely to impact search retrieval. Use of an appraisal form has also been shown to reduce the time it takes to conduct search peer review (Relevo and Paynter 2012, Spry et al 2013).
Using a standardized form in EM, the Cochrane IS search peer reviewer checks that:
- the search methods meet Cochrane's standards for conduct and reporting
- the key PRESS elements of the primary database strategy do not contain first order problems (errors that are likely to result in eligible studies being missed)
The search peer reviewer will recommend or suggest revisions (if required). The search peer reviewer does not re-write or re-run searches for author teams.
Please also see our FAQs for information specialist peer reviewers.
Recommended forms integrating MECIR standards and PRESS elements
The following Word forms contain the same questions/elements that are presented as part of the IS Reviewer task in EM.
- PROTOCOL: Search methods and strategy peer review assessment form for Cochrane intervention protocols
- FULL REVIEW: Search methods and strategy peer review assessment form for Cochrane intervention reviews
- UPDATE: Search methods and strategy peer review assessment form for Cochrane intervention updates
Sample peer review assessment forms:
- PROTOCOL sample assessment - Major revision
- FULL REVIEW sample assessment - Minor revision
- UPDATE sample assessment - Accept
Helpful resources for Cochrane search peer reviewers
Resources for intervention reviews:
- Reporting standards for protocols - MECIR PR17-PR21
- Conduct standards for intervention reviews - MECIR C24-C38
- Reporting standards for intervention reviews - MECIR R33-R38
- Conduct standards specific to updates - MECIR U6
- Reporting standards specific to updates - MECIR UR3
- Cochrane Handbook for Systematic Reviews of Interventions version 6.3, Chapter 4: Searching for and selecting studies
- PRISMA for Searching
Resources for non-intervention reviews:
- DTAs: Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy
- LSRs: Guidance for the production and publication of Cochrane living systematic reviews
- Methodology reviews: Guide to the contents of a Cochrane Methodology protocol and review
- Prognosis reviews: Resources for Prognosis Reviews
- Qualitative Evidence Synthesis (QES): Chapter 21.7 Searching for Qualitative Evidence
- Rapid reviews: Interim Guidance from the Cochrane Rapid Reviews Method Group
- Scoping reviews: PRISMA extension for scoping reviews
Resources for all review types:
- Cochrane Style Manual - includes template text and guidance for reporting search sources
- Reporting search dates in Cochrane reviews - Cochrane Editorial and Publishing Policy Resource guidance for reporting search dates
- PRISMA 2020
Join the Cochrane IS Search Peer Review Team
Any information specialist working for Cochrane may apply to join the IS Search Peer Review Team. To request to join or leave the team, please contact the Cochrane Community Support Team: support@cochrane.org
Current members:
- Yuan Chi
- Anne Cleeves
- Elise Cogo
- Irma Klerings
- Steve McDonald
- Maria-Inti Metzendorf
- Ina Monsef
- Jo Platt
- Doug Salzwedel
- Sheila Wallace
- Valerie Wells
- Joost Daams
- Heidrun Janka
- Camila Escobar Liquitay
- Samantha Cox
- Anna Noel-Storr
Search QA/Peer Review Training Resources & Pilot Report
Peer review training is available to members of the IS Search Peer Review Team.
The following slides are from assignment review sessions:
- Search peer review essentials slides (April 2022)
Please also see the accompanying quick reference guide for submitting recommendations in EM:
The Cochrane Information Specialist Support Team hosted a webinar on peer review of search strategies:
- Peer review of search strategies webinar recording (Feb 2019)
- Peer review of search strategies slides (Feb 2019)
The Cochrane Musculoskeletal, Oral, Skin and Sensory (MOSS) Network conducted a pilot search QA project during 2018. A report on the pilot is available to download.
Last update: Aug 2022