Cochrane welcomes comments on Cochrane Reviews and protocols published in the Cochrane Database of Systematic Reviews (CDSR).
Funders, other stakeholders and the scientific community expect transparency. Seeking and responding to comments, and the transparency of that process, are important parts of the scientific process and publication ethics, as reflected in guidance from the International Committee of Medical Journal Editors.
It is essential that efficient processes are in place for responding to new evidence, valid comments, and amending Cochrane Reviews or protocols if warranted. To achieve this, Cochrane is committed to timely updating of reviews and concurrent publishing of comments.
The management of comments received on Cochrane Reviews and protocols is part of the Cochrane Editorial & Methods Department. The Department is available for support in dealing with comments of any level of complexity, and John Hilton (email@example.com) should be the contact in the first instance.
Note: The new Cochrane Library platform, launched on 7 August, includes a new system for the submission and storage of comments. A new system to publish comments directly on the Cochrane Library will be introduced shortly after launch. See Changes to the Comments processes on the new Cochrane Library platform.
In this section:
- Submitting comments on Cochrane Reviews and protocols
- Changes to the Comments processes on the new Cochrane Library platform
- Receiving and responding to comments
- Process for dealing with submitted comments
- Handling comments in other languages
- Comments on translations
- Comments on withdrawn reviews or reviews due to be withdrawn
- Comments on previous versions of reviews
- Comments submitted outside the Cochrane Library commenting system
- Multiple comments submitted by one individual or long-running comments
- Citing comments
- Reports and presentations
Submitting comments on Cochrane Reviews and protocols
The 'Submit Comments' tool, available on the Cochrane Library website (cochranelibrary.com) located next to each Review, allows contributors to submit comments on Cochrane Reviews, Protocols, and Editorials. Comments must be concise, relevant to the content of the review, supported by references where appropriate and not libellous, defamatory or in breach of confidentiality or copyright. Real or potential conflicts of interest must be declared.
When submitting a comment, contributors agree to the following terms:
- You grant The Cochrane Collaboration ('Cochrane') a non-exclusive license to publish the Comment, and to identify you as its author.
- John Wiley & Sons, Ltd (‘Wiley’) will not accept Comments that may be libellous, abusive, obscene, in breach of any obligation of privacy or confidentiality, or otherwise unlawful.
- Cochrane will decide in its sole discretion whether to publish the comment, and may edit the comment for publication.
- You must have copyright ownership of all material that you submit. Publication of the Comment must not infringe the copyright, trademark, trade secret, right of privacy or publicity, or any other rights of any third party.
- You take responsibility for submissions under your identification and use the information provided here at your own risk. Cochrane and Wiley will not be liable for the content of Comments.
- Neither Cochrane nor Wiley can reply to requests for clinical or personal advice.
There is guidance for submitting comments on the Cochrane Library website.
Changes to the Comments processes on the new Cochrane Library platform
The new Cochrane Library platform, launched on 7 August, includes a new system for the submission and storage of comments. Cochrane Library users can (as previously) submit Comments on Cochrane Reviews and Protocols, and Cochrane Review Groups continue to receive notification of relevant new comments by email. The new system for publishing comments is not available to Cochrane Review Groups immediately at launch, and will be introduced soon after launch.
The new comments publication system, when introduced, involves three important changes:
Comments will be stored and managed in a new system that forms part of the Cochrane Library.
Comments will be published as independent items directly from the new system. There will be no need to republish the associated Review or Protocol in order to publish a Comment.
The Feedback section of Reviews and Protocols will no longer be used for new comments, although existing content will be retained in that section.
Receiving and responding to comments
Comments on a Cochrane Review or Protocol can be received at any time after publication and will be sent to the Feedback Editor and the Managing Editor of the responsible Cochrane Group (CRG), following initial screening by Wiley editors. Comments on Editorials are passed directly to the Cochrane Editorial & Methods Department. Screening looks for: comments that are not about the Cochrane Review (e.g. comments about the website functionality, access, or display); comments that are offensive, nonsensical, or spam; and comments that arise from system errors or system testing (e.g. duplicate entries). These comments are dealt with and replied to (or discarded) by Wiley editorial or production staff.
The CRG should appoint a Feedback Editor and establish processes for handling comments they receive, track the progress of each comment and any decisions made on how the comment is handled. The Feedback Editor's main task is to assess and edit comments, advise the CRG on a course of action. The Managing Editor's main task is to publish the comment and any associated changes to the Review or Protocol, and assist the Feedback Editor in communication with the review authors.
The Feedback Editor will ensure that the comment and language are appropriate and refer it to review authors for a response. The incoming comment should be sent to all review authors. The Contact Person for the review has primary responsibility for corresponding with the Feedback Editor and for responding to the comment.
When responding to comments, review authors are asked to:
- Confine the response to the points made in the comment;
- Reply to every substantive point, explicitly stating whether the review author agrees or disagrees with the comment and providing supporting evidence where necessary;
- Describe any changes made to the review in response to the comment; and
- Reply in clear and plain language.
If the Feedback Editor is an author of the review in question or if they have a conflict of interest in the particular topic, another Editor should be recruited to take on the role of Feedback Editor for that review.
Process for dealing with submitted comments
1. Screening by Wiley
Comments submitted through the Cochrane Library are received by Wiley editorial staff who will act within 2 days of receipt. Non-approved comments will be dealt with by Wiley and a response sent to the contributor. Approved comments will be sent to the FE and ME of the relevant CRG.
2. Initial handling by the CRG editorial base and the Feedback Editor
The Feedback Editor assesses the comment and decides:
- Is the comment valid (coherent, sensible, and relevant to the review)? If not, no further action is needed beyond informing the contributor of the decision.
Does the comment make a substantive comment on the content of the review or a minor comment?
(A) Substantive comments (see action below)
(B) Minor comment (see action below).
The Feedback Editor and the Managing Editor should agree the appropriate course of action and share this and the comment with the review authors.
The Feedback Editor should respond promptly to the comment contributor within two weeks of receipt, acknowledging receipt, thanking the contributor, and explaining how the comment will be dealt with and a timeline for action.
The review may or may not be amended or updated, or in extreme cases, withdrawn, in response to a comment.
If the comment does prompt a change to the review, the Managing Editor will make relevant changes as agreed with the review authors and the Feedback Editor. The contributor should be informed of the outcome.
3. Action: dealing with invalid comments
The Feedback Editor responds to the contributor informing them of the decision and giving a brief explanation.
4. Action: dealing with minor comments
Minor comments could include spelling or typographic errors, make very general statements, or may be requests about the review or its status. Minor comments are not normally published unless the comment (or its response) offers insight or interest to readers, or an opportunity to provide useful clarification.
The Feedback Editor responds to the contributor informing them of the action taken.
It may be appropriate to thank the contributor in the Acknowledgements section of the Review.
5. Action: dealing with substantive comments
It is important that substantive comments are published as soon as possible, unless the CRG has a compelling reason not to do so, or if the contributor does not wish the comment to be published. Examples of comments that might not be published are those where it becomes apparent that the contributor is clearly mistaken, or when the CRG has a reason to believe that the contributor has a significant, undeclared conflict of interest. The FE should raise concerns about conflict of interest with the contributor. All substantive comments should be dealt with within three months.
A response from the review authors is sought and should be published promptly, either at the same time as the comment is published, or subsequently. Review authors' responses should be confined to comments about the review itself and should relate directly to the content of the comment.
The steps are:
- Comment published (as soon as possible after receipt) (see below).
- Response by review author published (or CRG response if author unavailable or unwilling).
- Review amended (if necessary) and comments cited or acknowledged (as appropriate).
- If the comment is to be dealt with in next update, this should be indicated in published response to the comment with a timeline on when the update is expected.
- Contributor informed of outcome.
6. Publishing comments
The timing of comment publication, review author’s response, and review changes (if any), is up to the CRG and may depend on the status of the review or the nature of the comment. They can happen in sequence or simultaneously.
In some cases the comment may be edited or summarised (for example if the comment is particularly long with repetitious content) by the Feedback Editor, who will send the revised version to the comment contributor, informing them of the intention to publish the revised version. If the contributor had already consented to editing and publication of the comment, this is a courtesy and if the contributor does not respond within a specified time, the comment is published.
The comment is published in the 'Feedback' section of the review. The 'Contributors' section should include the name of the comment contributor and the name of those contributing to the response and their role in the CRG.
If the authors do not wish to respond or are unable to respond, the CRG should publish a response explaining that the author was unable to (or declined to) respond. The comment contributor should be informed.
This flowchart illustrates a simplified view of the typical process for handling comments.
Handling comments in other languages
If a CRG receives a comment in a language other than English the Managing Editor or the Feedback Editor should make a reasonable effort to obtain a valid translation of the comment. The CRG should publish the translated comment alongside the original language version. The translator should be acknowledged. The comment contributor should be informed that the comment has been translated and the comment then actioned.
Comments on translations
Any comments relating to translations of reviews or protocols should be forwarded to the Translations Co-ordinator, Andrea Cervera (firstname.lastname@example.org), who will liaise with the relevant translation team and help with addressing the comment.
Communication with the comment contributor would normally remain the responsibility of the CRG.
Comments on withdrawn reviews or reviews due to be withdrawn
If a CRG receives a comment on a Cochrane Review that has been or is soon to be withdrawn, there is usually no need to publish the comment. The comment contributor should be informed of this.
Comments on previous versions of reviews
Whether to retain or remove comments relating to previous versions depends on the nature of the comment. The usual approach is to leave previously published comments in place, because in most cases the comments remain relevant to the way the review was conducted or provides transparency on why changes were made (or not made). But if the comment is clearly no longer applicable (such that a reader would find it confusing or inexplicable) then the comment can be removed, but the contributor may be added to the acknowledgements if appropriate.
Comments submitted outside the Cochrane Library commenting system
Cochrane encourages submission of comments via the Cochrane Library, but comments can be submitted directly to review authors or CRGs, or published in other journals, blogs, social media or other fora.
If comments are received via one of these fora, the contributors should be encouraged to submit their comments via the Cochrane Library. This ensures that they are given the opportunity to declare any potential conflicts of interest and enables them to agree to the terms and conditions relating to publication of comments. If the comment contributor declines to submit their comment via the Cochrane Library, the comment can still be processed in the usual manner, but permission for publication should be sought from the contributor, making them aware of the terms and conditions of publication. The contributor should also be asked to disclose any potential conflicts of interest.
Comments posted on other journals, blogs, social media or other fora should be judged on merit and may prompt an amendment or update the review. In these cases the comments should be acknowledged or cited.
The Altmetric system can provide a useful source of comments on reviews. Comments posted on PubMed Commons (www.ncbi.nlm.nih.gov/pubmedcommons) or on PubPeer (pubpeer.com) are captured by Altmetric and recorded as ' Peer review' items. You can view the Altmetric data for any Cochrane Review by clicking on the Altmetric logo or score associated with the Review on the Cochrane Library.
It is possible for one contributor to submit comments on multiple reviews but for the relevant CRGs to be unaware of the submissions to other CRGs. In such cases, it can be useful for CRGs to be aware of the multiple submissions. The Cochrane Editorial & Methods Department monitors incoming feedback and will liaise with CRGs if such an event occurs.
Published comments form a section of a Cochrane Review itself, so to cite the comment elsewhere, you need to cite the review, but in this form:
(Johnson 2010, Feedback section, Comment by Smith 2013)
Reports and presentations
Report: Comments submitted during 2017 (February 2018): Report (PDF)