Editorial and Publishing Policy Resource
Comments on Cochrane Reviews

Cochrane welcomes comments on Cochrane Reviews and protocols published in the Cochrane Database of Systematic Reviews (CDSR).

Funders, other stakeholders and the scientific community expect transparency. Seeking and responding to comments, and the transparency of that process, are important parts of the scientific process and publication ethics, as reflected in guidance from the International Committee of Medical Journal Editors.

It is essential that efficient processes are in place for responding to new evidence, valid comments, and amending Cochrane Reviews or protocols if warranted. To achieve this, Cochrane is committed to timely updating of reviews and concurrent publishing of comments.

The management of comments received on Cochrane Reviews and protocols is part of the Cochrane Editorial Unit’s (CEU) remit. The CEU is available for support in dealing with comments of any level of complexity, and John Hilton (jhilton@cochrane.org) should be the contact in the first instance.

In this section:

Submitting comments on Cochrane Reviews and protocols

The 'Submit Comments' tool, available on the Cochrane Library website (www.cochranelibrary.com) located next to each Review, allows contributors to submit comments on Cochrane Reviews and protocols. Comments must be concise, relevant to the content of the review, supported by references where appropriate and not libellous, defamatory or in breach of confidentiality or copyright. Real or potential conflicts of interest must be declared.

When submitting a comment, contributors agree to the following terms:

  1. You grant Cochrane a non-exclusive licence to publish the comment, and to identify you as its author, as part of an amendment or update to a Cochrane Review.
  2. Cochrane will decide whether to publish the comment and may edit the comment for publication.
  3. John Wiley & Sons Ltd (‘Wiley’) will not accept comments that may be libellous, abusive, obscene, in breach of any obligation of privacy or confidentiality, or otherwise unlawful.
  4. You must have copyright ownership of all material that you submit. Publication of the comment must not infringe the copyright, trademark, trade secret, right of privacy or publicity, or any other rights of any third party.
  5. You take responsibility for submissions under your identification and use the information provided here at your own risk. Cochrane and Wiley will not be liable for the content of comments.
  6. Neither Cochrane nor Wiley can reply to requests for clinical or personal advice submitted as comments.

There is guidance for submitting comments on the Cochrane Library website.

Receiving and responding to comments

Comments on a Cochrane Review or Protocol can be received at any time after publication and will be sent to the Feedback Editor and the Managing Editor of the responsible Cochrane Group (CRG), following initial screening by Wiley editors. Screening by Wiley editors content of a Cochrane Review (e.g. comments about the website functionality, access, or display), comments that are offensive, nonsensical, or inconsequential, and comments that arise from system errors or system testing (e.g. duplicate entries). These comments are dealt with and replied to (or discarded) by Wiley editorial or production staff.

The CRG should appoint a Feedback Editor and establish processes for handling comments they receive, track the progress of each comment and any decisions made on how the comment is handled. The Feedback Editor's main task is to assess and edit comments, advise the CRG on a course of action. The Managing Editor's main task is to publish the comment and any associated changes to the Review or Protocol, and assist the Feedback Editor in communication with the review authors.

The Feedback Editor will ensure that the comment and language are appropriate and refer it to review authors for a response. The incoming comment should be sent to all review authors. The Contact Person for the review has primary responsibility for corresponding with the Feedback Editor and for responding to the comment.

When responding to comments, review authors are asked to:

  • Confine the response to the points made in the comment;
  • Reply to every substantive point, explicitly stating whether the review author agrees or disagrees with the comment and providing supporting evidence where necessary;
  • Describe any changes made to the review in response to the comment; and
  • Reply in clear and plain language.

If the Feedback Editor is an author of the review in question or if they have a conflict of interest in the particular topic, another Editor should be recruited to take on the role of Feedback Editor for that review.

Process for dealing with submitted comments

1. Screening by Wiley

Comments submitted through the Cochrane Library are received by Wiley editorial staff who will act within 2 days of receipt. Non-approved comments will be dealt with by Wiley and a response sent to the contributor. Approved comments will be sent to the FE and ME of the relevant CRG.

2. Initial handling by the CRG editorial base and the Feedback Editor

The Feedback Editor assesses the comment and decides:

  1. Is the comment valid (coherent, sensible, and relevant to the review)? If not, no further action is needed beyond informing the contributor of the decision.
  2. Does the comment make a substantive comment on the content of the review or a minor comment?
    (A) Substantive comments (see action below)
    (B) Minor comment (see action below).

The Feedback Editor and the Managing Editor should agree the appropriate course of action and share this and the comment with the review authors.

The Feedback Editor should respond promptly to the comment contributor within two weeks of receipt, acknowledging receipt, thanking the contributor, and explaining how the comment will be dealt with and a timeline for action.

The review may or may not be amended or updated, or in extreme cases, withdrawn, in response to a comment.

If the comment does prompt a change to the review, the Managing Editor will make relevant changes as agreed with the review authors and the Feedback Editor. The contributor should be informed of the outcome.

3. Action: dealing with invalid comments

The Feedback Editor responds to the contributor informing them of the decision and giving a brief explanation.

4. Action: dealing with minor comments

Minor comments could include spelling or typographic errors, make very general statements, or may be requests about the review or its status. Minor comments are not normally published unless the comment (or its response) offers insight or interest to readers, or an opportunity to provide useful clarification.

The Feedback Editor responds to the contributor informing them of the action taken.

It may be appropriate to thank the contributor in the Acknowledgements section of the Review.

5. Action: dealing with substantive comments

It is important that substantive comments are published as soon as possible, unless the CRG has a compelling reason not to do so, or if the contributor does not wish the comment to be published. Examples of comments that might not be published are those where it becomes apparent that the contributor is clearly mistaken, or when the CRG has a reason to believe that the contributor has a significant, undeclared conflict of interest. The FE should raise concerns about conflict of interest with the contributor. All substantive comments should be dealt with within three months.

A response from the review authors is sought and should be published promptly, either at the same time as the comment is published, or subsequently. Review authors' responses should be confined to comments about the review itself and should relate directly to the content of the comment.

The steps are:

  • Comment published (as soon as possible after receipt) (see below).
  • Response by review author published (or CRG response if author unavailable or unwilling).
  • Review amended (if necessary) and comments cited or acknowledged (as appropriate).
  • If the comment is to be dealt with in next update, this should be indicated in published response to the comment with a timeline on when the update is expected.
  • Contributor informed of outcome.

6. Publishing comments

The timing of comment publication, review author’s response, and review changes (if any), is up to the CRG and may depend on the status of the review or the nature of the comment. They can happen in sequence or simultaneously.

In some cases the comment may be edited or summarised (for example if the comment is particularly long with repetitious content) by the Feedback Editor, who will send the revised version to the comment contributor, informing them of the intention to publish the revised version. If the contributor had already consented to editing and publication of the comment, this is a courtesy and if the contributor does not respond within a specified time, the comment is published.

The comment is published in the 'Feedback' section of the review. The 'Contributors' section should include the name of the comment contributor and the name of those contributing to the response and their role in the CRG.

If the authors do not wish to respond or are unable to respond, the CRG should publish a response explaining that the author was unable to (or declined to) respond. The comment contributor should be informed.

Summary flowchart

This flowchart illustrates a simplified view of the typical process for handling comments.

Cochrane Library Comment management: flowchart

Handling comments in other languages

If a CRG receives a comment in a language other than English the Managing Editor or the Feedback Editor should make a reasonable effort to obtain a valid translation of the comment. The CRG should publish the translated comment alongside the original language version. The translator should be acknowledged. The comment contributor should be informed that the comment has been translated and the comment then actioned.

Comments on translations

Any comments relating to translations of reviews or protocols should be forwarded to Cochrane's Translations Support Officer, Hayley Hassan (hhassan@cochrane.org), who will liaise with the relevant translation team and help with addressing the comment.

Communication with the comment contributor would normally remain the responsibility of the CRG.

Comments on withdrawn reviews or reviews due to be withdrawn

If a CRG receives a comment on a Cochrane Review that has been or is soon to be withdrawn, there is usually no need to publish the comment. The comment contributor should be informed of this.

Comments on previous versions of reviews

Whether to retain or remove comments relating to previous versions depends on the nature of the comment. The usual approach is to leave previously published comments in place, because in most cases the comments remain relevant to the way the review was conducted or provides transparency on why changes were made (or not made). But if the comment is clearly no longer applicable (such that a reader would find it confusing or inexplicable) then the comment can be removed, but the contributor may be added to the acknowledgements if appropriate.

Comments submitted outside the Cochrane Library commenting system

Cochrane encourages submission of comments via the Cochrane Library, but comments can be submitted directly to review authors or CRGs, or published in other journals, blogs, social media or other fora.

If comments are received via one of these fora, the contributors should be encouraged to submit their comments via the Cochrane Library. This ensures that they are given the opportunity to declare any potential conflicts of interest and enables them to agree to the terms and conditions relating to publication of comments. If the comment contributor declines to submit their comment via the Cochrane Library, the comment can still be processed in the usual manner, but permission for publication should be sought from the contributor, making them aware of the terms and conditions of publication. The contributor should also be asked to disclose any potential conflicts of interest.

Comments posted on other journals, blogs, social media or other fora should be judged on merit and may prompt an amendment or update the review. In these cases the comments should be acknowledged or cited.

The Altmetric system can provide a useful source of comments on reviews. Comments posted on PubMed Commons (www.ncbi.nlm.nih.gov/pubmedcommons) or on PubPeer (pubpeer.com) are captured by Altmetric and recorded as ' Peer review' items. You can view the Altmetric data for any Cochrane Review by clicking on the Altmetric logo or score associated with the Review on the Cochrane Library.

Multiple comments submitted by one individual or long-running comments

It is possible for one contributor to submit comments on multiple reviews but for the relevant CRGs to be unaware of the submissions to other CRGs. In such cases, it can be useful for CRGs to be aware of the multiple submissions. The CEU monitors incoming feedback and will liaise with CRGs if such an event occurs.

Citing comments

Published comments form a section of a Cochrane Review itself, so to cite the comment elsewhere, you need to cite the review, but in this form:

(Johnson 2010, Feedback section, Comment by Smith 2013)

Reports and presentations

Report: Comments submitted during 2016 (June 2017): Report (PDF)

Poster: Feedback on Cochrane Reviews: how much is there and what happens to it? (September 2014): Poster (PDF) | Abstract

Workshop: Comments on Cochrane Reviews: approaches to managing feedback (September 2013): Presentation (PPT, 1.7 MB) | Abstract

Section info
Contact
John Hilton (jhilton@cochrane.org)
Describe change
Added (1) report of comments submitted during 2016 and (2) a flowchart diagram
Change date
16 June 2017