This Blog presents commentary and personal opinion on topics of interest from a range of contributors to the work of Cochrane. Opinions posted on the Cochrane Blog are those of the individual contributors and do not reflect the official views or policies of Cochrane.

It was sobering to reread the following blog written in March 2016.  On one hand, I recognised that all too human trait for dispensing more gung-ho advice than one would necessarily apply to oneself. And on the other hand, I realised that a zealous application could have prevented the publication of some high quality reviews on key topics. Even so, the underlying messages remain pertinent to this next phase of Cochrane. And I anticipate that reading this will be liberating, particularly in conjunction with the recently published Rejection of Cochrane Reviews policy. Ours is not the only CRG that has provided disproportionate support to authors of poor quality reviews. This approach has engendered running backlogs that have persisted for years. There have been disheartening endgames for several priority reviews that still did not make it to publication despite extensive feedback and support. Thus, I resolve to be stronger and tougher. I hope this following will help others avoid some of the pitfalls when editing Cochrane Reviews.

Helen Handoll
Co-ordinating Editor
Cochrane Bone, Joint and Muscle Trauma Group
January 2018                     

What to do on finding a review in need of attention.

1. Shout for help!

A good bellow is always worth a try. Although you may not find professional help (a statistician or even a content expert) readily available - demand greatly outstrips supply – making someone aware that you are about to edit a review is highly recommended (see tip 11).

2. Assess the situation

Do not plunge straight into the review. A few quick checks on the circumstances may prevent or lessen despair in the long run (see tip 11). Always check it is the most up to date version. Sometimes it might be worth someone asking the review authors if they had actually meant to submit their review. Beware though that the euphoric feeling upon discovering that the review does not need to be saved may be short-lived.

3. Check the protocol

Locating and looking at this key document is a crucial step. The attachment of review authors to their protocol varies but can be severely weakened when they are in the mixed and dangerous company of primary research reports. Carefully laid out inclusion criteria and methods can be cast aside.  Study selection may seem haphazard but sometimes you start to realise there has been a change, which may even be announced in the text, in the review’s scope. Pause before taking precipitative action, because it is possible the authors might be right (see tip 11). More commonplace is some mild forgetfulness of intent and purpose.  Perusal of the methods section may find superseded methods blithely recast in the past tense, and a source of false claims.

4. Out for the count (the numbers game)

An initial illusion that a review is fine may be chipped away with small discoveries of inaccuracies (how many trials were there again?) and inconsistencies. Be prepared for a plunge in your estimation of the authors’ ability, diligence and attention to detail when you check the information and data extraction from one of the included trials. The dismay is often renewed when you check a second trial report.  The old adage “why ruin a good story with the facts” comes to mind; but such discrepancies should not be taken lightly. There is no point in looking further than the results or providing a full or comprehensive feedback report in these circumstances.

5. Mind the language

When reading the text, always remember that you are there for the reader of the published review. If you do not understand it or find yourself lost in a busy stream of disorganised text rescued only by the next section heading, be assured you will not be alone. However achieved, a rewrite usually together with some restructuring will be necessary. Uneven quality in language may, regrettably, be a sign of plagiarism or poorly paraphrased text. Ironically, helping the authors improve such text may result in a greater similarity with the original copied but unattributed text. 

6. Try to avoid being taken for a ride

As an editor, you end up doing quite a lot of thinking and more for the authors. Naturally, there are times when you wonder why you are spending hours of your limited spare time providing feedback when it is abundantly clear that some if not most of the authors have contributed rather less than you have.  Indeed, one suspects the honest answer to “have all the authors actually read the review” will be “No!” It is likely there will be freeloaders, who were perhaps on nominal standby to arbitrate if necessary, in powerful positions that make it awkward for the working author(s) to call on their help or even content expertise.  You may even find that you are acting in a proxy PhD supervisor role, while the real supervisor is proving hard to get but might be spurred into action to ‘support their student’ where there are excessive editorial delays holding things up. One telltale sign of zilch involvement is where these aloof review authors are also trial investigators on included trials but have not resolved queries on their trials.  From experience this can stretch from failure to clarify trial methods to failure to notify on duplicate publications. It is inevitable that you will feel a mug from time to time but you should give yourself permission to throw in the towel where an extreme effort is required and just send it back to the authors. Ultimately an abject review should be their problem not yours.

7. Feedback – getting the message across.

Take advantage of proformas that help you structure your feedback. Never fool yourself that you will remember all the points you wished to make. Keep to the point in your feedback: review authors are rarely interested in lengthy explanations. Avoid giving more than one instruction at a time. Avoid sweeping assumptions of the authors’ diligence – for most authors addressing feedback is a necessary evil, not an end in itself, and authors will often comply with specifics not generalities. Authors generally don’t ‘get’ irony. Try to find something good to say about the review, perhaps at the start, and maybe, provided it does not stretch credulity, at the end. Give as realistic estimate of the work entailed as possible.

8. Temper the tone

Whatever your mood after your dispiriting ordeal in trying to rescue a review - whether sunk into despair, vexed to the point of screaming, or fermenting with barely restrained fury – the tone of your feedback needs to be calm and respectful. Remember, with rare exceptions, at least one person on the byline has worked hard to complete their review. Consider also whether there are redeeming features worthy of remark. Enlisting a diplomatic intermediary, to screen and smooth the prose, can be advisable. Otherwise, set aside and revisit before dispatch.

9. Wait

This can be the hardest part as by handing over your feedback and revised review to the review authors, you have relinquished all control. It is prudent to recognise that all your effort may be in vain – indeed the shock and outrage that anything was found wrong with their review can be too much for some authors – or that intensive remedial work may again be necessary. Whatever, you should be prepared for a long wait. On looking through authors’ response documents, it is especially important to avoid the illusion that authors understand the word “done” in response to direct instructions. Be wary also of finding much comfort in repetitive expressions of gratitude; “thank you” may instead anticipate the review authors’ perception of your future industry and continuing benevolence.

10. Time for reflection

A struggle as it may be, try to avoid getting cross with authors. Remember that some reviews cannot be saved and that, for these, ‘battling on’ is not the best option.  Also remember that editors have rights too.  Contemplations on this last sentiment will provide material for a future piece on ‘How I reclaimed my life’. The race is on…

11. Editors are not always right and sometime make mistakes

Helen Handoll
Co-ordinating Editor
Cochrane Bone, Joint and Muscle Trauma Group

(Originally published March 2016, updated January 2018)

Dr Helen Handoll is Co-ordinating Editor of Cochrane Bone, Joint and Muscle Trauma Group, one of four Groups in the Acute and Emergency Care Network. She shares with us some of her personal views on the work of her Group this year, and outlines the opportunities and challenges facing Cochrane Bone, Joint and Muscle Trauma in 2018.

This Blog presents commentary and personal opinion on topics of interest from a range of contributors to the work of Cochrane. Opinions posted on the Cochrane Blog are those of the individual contributors and do not reflect the official views or policies of Cochrane

Can you tell us a little about the main highlights from 2017? What have been the most impactful reviews or activities of work for Cochrane Bone, Joint and Muscle Trauma Group this year?

HH: I’m glad you asked about the highlights. These have to include the publication of our two largest reviews this year, both of which were a long time coming. I would like to pay tribute to the author teams of both reviews for their perseverance, including coping with the inevitable search updates and some hefty editorial feedback. One review (39 trials; 18 comparisons) was of special interest to our funder and it was rather nice to be able to report its publication: Injected corticosteroids for treating plantar heel pain in adults. The second review (50 trials; 1 comparison) featured as the highlighted review on the Cochrane Library, which is a cheerful way to end the year: Antioxidants for preventing and reducing muscle soreness after exercise.

Another highlight is the award by the NIHR (UK) of the Cochrane Programme Grant for hip fracture management reviews to our deputy Co-ordinating editor, Xavier Griffin. I very much like the approach that Xavier has taken with the proposal with its very strong focus on impact, directly linking with future guidelines, and the inbuilt collaboration with two other CRGs, Cochrane Anaesthesia, Critical and Emergency Care and Cochrane Dementia and Cognitive Improvement.

What have you been most proud of?

HH: See above.

What has been the most challenging piece of work or Review for you?

HH: In truth, they are all challenging. Happily, some make it to publication but others do not - it has been a grim task this year (2017) to reject several reviews on really key topics. I welcome the fact that Cochrane now has an official rejection policy in place to help manage the expectations of both CRGs and review teams.

The full SF Implementation plan was announced to the Cochrane Review Groups in August. Can you remember your first impressions of seeing the plan?

HH: Yes, I can. There was a long lead in time to this event during which I and core members of the editorial base had actively participated in the several consultations and provided feedback on previous drafts and presentations. Much of the plan was new material and our feedback appeared to have made rather little impact on the stuff that had been circulated previously. On the plus side it appeared a comprehensive and coherent plan; on the negative side we felt almost no part of it.

What did you like/ not like so well? What was the reaction from your colleagues and the Group?

HH: A mixture of resignation – it is happening anyway – and relief – it is always good to have a workable plan.

Your group will be part of the Acute and Emergency Care Network – what communication have you had with other CRGs within the Network?

HH:  Rather little mention so far. I recall a short email exchange with the Co-Ed of one Group – rewarded by a lovely photo of her and her one-year old son.

What thoughts have you had since understanding more about the formation of the Networks?

HH: So much depends on the character of the appointed Senior Editor. I hope they will be a strong advocate for the topic areas covered by our Groups. Otherwise, it is still early days and I really am waiting to see how it develops.

What do you see as the main opportunities?

HH: Perhaps opportunities for seeking solutions and support for editorial dilemmas and issues. Again, I want to see what happens.

What do you see as the biggest challenges in 2018?

HH: Delivering on the first set of reviews listed in the hip fracture management programme grant (see above) and on the reviews that are replacing the huge, highly accessed but now out of date prevention of falls in the community review.

As a Cochrane Co-ordinating Editor, what do you see as your key priority in 2018, and as the Network forms together?

HH: Delivering on high quality and impactful reviews.

What would be your message to other Co-Eds, or indeed other CRG staff, perhaps who share the same concerns/excitement?

HH: Run for the hills! Otherwise engage….

• For more information on the CRG Networks, visit our new CRG Networks Portal
• Visit the Cochrane Bone, Joint and Muscle Trauma Group website

The Covidence platform continues to evolve, with recent product updates advancing the tool towards the vision of a world where answers to questions about health and other fields of human wellbeing are accurate, up to date, and accessible. The Covidence team is committed to engaging with the Cochrane community and continuing to make improvements that are user-focused.

Recent updates

Data validation
Covidence allows for several types of data points to be collected: integers, decimals, percentages, and free text. The platform now automatically checks that the data entered to a cell matches the expected data type, saving time later on in double checking and cleaning data!

Manage customised tags
Did a few members of your team get a little too tag-happy? Did you make a typo when you added a tag? Fear no more, you can now delete those pesky superfluous tags from your settings page!

From your review dashboard, click on Settings, then on Study tags. You won't be able to delete the pre-populated tags (i.e. Awaiting classification and Ongoing study), but all others can be deleted from this page. You can also easily add tags from here!

Custom text fields now integrate to template
Covidence has a streamlined data extraction form with many of the standard elements pre-programmed, minimising time research teams have to spend on constructing their forms. You can see a walkthrough of creating a data extraction form in Covidence here.

Previously when a customised text field was added, it was limited to a study-by-study basis. Now, when added to the review template, the custom text field will be carried forward along with all the other customised elements of the template, including the baseline characteristics table, interventions characteristics table, and outcomes.

You can read more about custom text fields here.
 

Organisation accounts for Cochrane Review Groups
The Covidence team is in the process of creating organisational accounts for each Cochrane Review Group. These group accounts will enable editorial staff to be able to create reviews at any time in the process, including while the review is in editorial or prior to protocol registration. The editorial staff will also be able to monitor review activity within their own groups. Reviews will be clearly grouped according to their Review Group association.

Have some ideas on how to continue improving Covidence?

Thank you for voicing your feedback and user needs, these are an invaluable input into our product roadmap and prioritisation process. The Covidence team would love to hear more from you! Please contact Covidence Support at support@covidence.org if you’d like to share some ideas, have a chat, or just say hello!

We also welcome feedback via this form.

Some resources you might find helpful:

• YouTube
• Covidence Knowledge Base: English and Spanish

From little things, big things grow. Cochrane Crowd, Cochrane’s citizen science platform, now has 7500+ contributors who have notched over 1.5 million classifications. Here Anna Noel-Storr, Co-Lead of Cochrane Crowd, shares the story of Cochrane Crowd and how the platform may evolve in the future.

It often takes a combination of factors, including hard work and to some extent good fortune, to get a big project off the ground. Can you tell us a bit about how Cochrane Crowd came to be?

Good fortune, and the talent, enthusiasm, and vision of many individuals along the way, has got to us to where we are now. It’s quite hard to pinpoint when it all began. Many will remember the Embase Project that came before Cochrane Crowd. That work was instrumental, but actually there were events leading up to that project that rank as significant.

The first such moment was when my then boss, Rupert, introduced me to two medical students with the hope that I could find them something to do. We set them to work helping to extract some information about trials in the area of dementia (that was, and indeed still is, an area that I work in as an information specialist). They did a great job. Then an opportunity for some funding came along and again my boss popped his head round the door and wondered if this was a chance to see if we could scale this approach. It was very much feasibility work in those days but that grant was a huge learning experience for all involved. The project manager we recruited, Caroline, became a real driving force behind that early work. She was quite an inspiration to me.

I think that work helped demonstrate that:

• people often want to help but flexible opportunities are limited;
• the tasks you provide have to be doable and needed;
• and technology can be a huge enabler.

We’ve gone on to evolve and scale, first with the Trial Blazers study and then with the Embase Project.

And now Cochrane Crowd is 18 months old. It has been an incredible journey so far, and what’s exciting is that it is in no way over yet.

To the outsider, what was the platform like right at the start? What functionality did it have?

The early version used for the Embase Project was functional and easy to use. This was a critical factor in the project’s success. Much of that functionality still exists in Cochrane Crowd, but with Cochrane Crowd we’ve also added so much more.

In the early days you couldn’t prioritise the records you worked on (for example now you can prioritise by healthcare area such as dementia, or child health), and you couldn’t view your decisions against the final decision made on a record very easily. You also only had one task to choose from. Now there are three mainstream tasks, and very soon, there will be five!

Of course, I owe the project’s technical lead, Gordon, huge credit here. He works tirelessly to make the platform work and brings his amazing problem-solving brain to the many conundrums we have to solve.

There is an entire underbelly to Cochrane Crowd that not many people see. Can you tell us a little about that?

You’re right; there is a lot going on behind the scenes. The main thing is the ‘agreement algorithm’ that helps to ensure that the collective decision-making process works well. What this means is that for each task we need to work out how many decisions are needed for each record, as well as the configuration or ordering of those decisions to help make sure that the records end up with the correct classification. It sounds quite geeky, and I suppose it is, but it’s needed to ensure accuracy and efficiency.

To work out if we’ve got the algorithm right we run various evaluations: some are formal evaluations where we take a whole load of records and send them to the Crowd, and independently send them to experts so that we can compare the final classifications from each group. We’re about to do that with our latest task: CT identification. Even though this task is very similar to the RCT identification task, the records are different enough to warrant an evaluation. Based on our findings, we’ll either continue with the algorithm or tweak it and try again. Other evaluative activities are less formal and include things like random spot checks on records.

Cochrane Crowd is a citizen science platform, and yet there are some differences between Crowd and most other platforms in this genre. Can you tell us about that?

Cochrane Crowd is a bit different. The tasks on the whole are quite text-based. That does immediately make it more challenging in terms of wide appeal. I think I’d rather be out in the fresh counting bees, than reading a badly written abstract (oh dear, I’m not really selling the task am I?!). My point is, that when you have tasks like ours, you have to make everything else around the tasks as appealing as possible. For example, we are putting quite a bit of focus into the feedback we can provide contributors. Many citizen science projects don’t provide individuals with feedback related to performance. It is challenging to do this in a live system and I don’t think we’ve quite got it right yet, but it is something we’re very aware people want and we have some exciting new features coming very soon.

What kind of impact is Cochrane Crowd having?

Another tough question! Can’t you ask me something easier like how many cats do I have? [Ed.: Nope, sorry! Readers, visit Anna’s Twitter account for some all-important cat photos!] Ultimately, we want to achieve two things with Cochrane Crowd: first, we want to help in the effort to produce good evidence quickly, and second, we want to provide people with opportunities to be a part of that effort.

In our effort to help produce good evidence quickly, we are making an impact in several ways: we’re identifying thousands of reports of randomised trials for CENTRAL, helping to enrich that critical resource; we’ve helped develop machine learning classifiers that can now do over 70% of the work we humans were previously doing, meaning that we can focus our effort on finding RCTs from other sources; we are making good inroads into doing the same for other types of studies, such as diagnostic test accuracy (DTA) studies; and we’re creating new tasks aimed at describing health research in a consistent way which I feel confident will go a long way towards enhancing trial discovery in ways that are both fast and reliable.

In terms of providing people with the opportunity to contribute, I hope we are doing that too. Thousands of people have signed up, and the rate of sign up is increasing. However, it’s not just about sign up. We want people to feel they really can, and are, helping. Part of this comes down to offering a good range of tasks and making it easy to dive into those tasks; and part of it is about what we can give back in terms of rewards. Cochrane’s new membership scheme will certainly help here. In addition, we’re developing some new training materials that I hope will give those who want to know more, a chance to build up their knowledge and skills in understanding health evidence.

What do you foresee for Cochrane Crowd in the future?

I hope Cochrane Crowd will play an increasing role in the efficient identification, management and production of health evidence. I’d like to make it truly possible for anyone with an interest in health to be a part of this endeavour. We live in quite unsettling times. Whilst it has always been pretty easy to make bogus, non-evidence based claims about anything, it’s never been so easy to reach so many potential listeners or readers. That means that it’s more important than ever to counter those flimsy claims by challenging them and fighting back with evidence.

Systems like Cochrane Crowd give us a chance of keeping up with the information overload meaning that we can become more responsive to questions and claims as they arise, and quicker to answer them. What’s more, we need to stop starting again with every question we have, and instead make far better use of the intelligence that has already been generated.

I want Cochrane Crowd itself to be able to change and adapt as information needs inevitably change. This includes being able to create and scale new tasks quickly. I think all tasks on Cochrane Crowd will have a shelf life. In fact they should have. If we’re still struggling to identify RCTs in a few years from now, then we’re not doing enough to solve the underlying problems (such as poor reporting). Let’s face it, crowdsourcing RCT identification is a clever work-around for a problem that shouldn’t even exist anymore.

Looking back over these eventful 18 months with Crowd, what are the highlights for you? What have been the main challenges?

The last 18 months have seen many high points. The MedLitBlitz, our one-year birthday celebration where we teamed up with the wonderful Mark2Cure is certainly one. The pilot work on screening for individual reviews has been exciting with the results exceeding my expectations both in terms of uptake and quality. But probably the main, slightly unexpected outcome has been the work we’ve done with the machine-learning team. When we first started out, our aim was simply to identify reports of randomised trials for CENTRAL. Now, thanks to the Crowd, we’ve been able to build machine classifiers that can do much of the task. Human effort will always be needed but it should not be used on tasks that can be done in a fraction of the time by automation. [Ed.: I feel another blog coming on!]

Of course there have been challenging times too and many occasions when I’ve lain awake at 3am trying to figure out how to solve some issue or other. As the platform grows, both in size and role, it becomes increasingly important that it interacts and is integrated properly with other systems and processes, many of which are also being developed. We can’t, and we shouldn’t, operate in our own little silo anymore but system integration and development of workflows when one or more of those systems is still evolving, is not easy.

The biggest challenge of all though is time. We have so much we want to do. These first 18 months have been extremely busy; the next 12 months will be even busier.

And finally, how much tea do you actually drink?

Haha! Too much, but now you’ve put that thought in my head, I’m off to make another one! I think I’ve earned it after all those questions.

Sign up to Cochrane Crowd, follow us on Twitter, and contact us at crowd@cochrane.org.

Compiled by Emily Steele, Cochrane Crowd Community Engagement & Partnerships Manager

Support for Project Transform was provided by Cochrane and the National Health and Medical Research Council of Australia (APP1114605). The contents of the published material are solely the responsibility of the Administering Institution, a Participating Institution or individual authors and do not reflect the views of the NHMRC.

Dr. Sophie Hill, Co-ordinating Editor of the Consumers and Communication Group shares an overview of some of the changes in her Cochrane Group so that others in the Cochrane Community can learn from them. 

In the Consumers and Communication Group we have made some changes lately you might like to hear about as we are finding they greatly increase our control over work, enable us to be more responsive to policy makers’ issues, and are leading to a much more enjoyable time as Cochrane editors.

1.      Titles are open about two-three times a year. We advertise the opening times via our newsletter and website. Applications in each round are assessed together against rigorous criteria of relevance and quality. Recommended titles are then sent to editors for a final assessment. We set a very strict limit on how many are approved based on a forward assessment of our editorial workload. This is so we can be more timely in our comments. Check out our FAQs for potential new authors. 

2.      We have applied the Updating Classification System (UCS) in consultation with authors and it is likely that many of our older intervention reviews will not be updated. If they are, they will look very different and have new teams. To move to an update, a team has to meet very high standards in terms of vision for review, in-depth knowledge of evidence base, competence and resourced capability, and topic has to be a demonstrated priority. No team can update without approval. Read our detailed updating guide.  

3.      If context affects effectiveness, then increasingly reviews framed in 1990, 2000, or even 2010 are not necessarily relevant and evidence prior to these periods may not be relevant either. We will explore how this can affect updating of a small number of reviews where the social context or intervention has changed significantly.

4.      In-house, we are using sustained stakeholder engagement to re-shape and update a few priority reviews. We don’t just ‘consult’ our stakeholders: they are part of the decision making on what can go forward and how it goes forward. It is more interesting, produces more relevant reviews types and is of huge interest to our primary funder. Learn more about our priority reviews.

5.      We are changing the role of the external editorial group to be more strategic and no longer operational.

These changes are taking us over a year to implement and are building on earlier changes which increased the amount of funded technical editing time at the base. We also implemented policies around rejection, introduction of QESs, and non-acceptance of PhD students as lead authors. All policies are on our website.   
 
In the future, we will explore having:

• At least one review that includes use of ‘big data’;
• At least one living systemic review;
• A ‘QES’ of grey literature, such a consumer reports.

Check out our latest newsletter and if you want to stay informed with the editorial changes we are making, please subscribe, or email me and I will make sure you receive the next one.  

Dr. Sophie Hill
Co-ordinating Editor,
Cochrane Consumers and Communication 

• For more information on the CRG Networks, visit our new CRG Networks Portal
• Visit the Cochrane Consumers and Communication website

Cochrane Classmate, launched at the Global Evidence Summit in September, could change how evidence production is taught in classrooms around the world.

From the people that brought you Cochrane Crowd, we present Cochrane Classmate: a trainers’ toolkit that lets you create exciting, interactive tasks to help your students learn about evidence production.

Cochrane Classmate is available to everyone. Join the growing number of teachers and trainers using this innovative online learning environment.

“We ran another challenge earlier today…I told the teams that if any of them beat me I would buy them pizza. I was quite confident that I would win…they beat me, and I had to eat humble pie, and buy them dinner!” [Nai Ming Lai]

“Overall it’s a very straightforward interface to use” [Miranda Cumpston]

Who is Cochrane Classmate for?

Anyone who teaches on topics related to evidence production, such as evidence-based medicine, systematic review production, understanding PICO, study designs, citation screening or research reporting standards.

How does Cochrane Classmate work?

It’s very simple.

Step 1: Pick a training module from the Classmate library.

Step 2: Upload your class’s email addresses and Classmate will handle the rest, sending out instructions to each student.

Step 3: Make a cup of tea, sit back and relax.

What modules are available?

Three modules are available now:

1. Randomised controlled trial identification (RCT ID). Your students will learn how to identify RCTs and quasi-RCTs in a fun challenge environment.
2. Clinical trials identification (CT ID) This task is all about identifying randomised trials specifically from ClinicalTrials.gov: the world’s largest trial registry.
3. Diagnostic test accuracy identification (DTA ID). This is a more challenging task: learning how to identify studies that are about assessing the accuracy of diagnostic tests. These studies come in all shapes and sizes!

Will more modules be added?

Yes. The next modules, Introduction to Study Design and Introduction to CONSORT, are planned for late 2017.

Ready to get started?

Visit Cochrane Classmate now. Read the FAQs and start using Classmate as soon as you’re ready.

Want to learn more?

Please join us for an Introduction to Classmate webinar on Tuesday November 28th. Sign up here.

For further information about Cochrane Classmate please email us at classmate@cochrane.org. Follow us on Twitter for news and future module releases. And please share this blog using the buttons below.

Support for Project Transform was provided by Cochrane and the National Health and Medical Research Council of Australia (APP1114605). The contents of the published material are solely the responsibility of the Administering Institution, a Participating Institution or individual authors and do not reflect the views of the NHMRC.

From 6-12 November, Evidence Aid is organising the Humanitarian Evidence Week 2017 (HEW2017) – one week of blogs, webinars, training, debates, and other initiatives to promote a more evidence-based approach to humanitarian aid. For the HEW2017, Evidence Aid has brought together more than 20 organisations to create a platform highlighting their work related to the generation, use, and dissemination of evidence in the humanitarian sector. I, as Director of Evidence Aid, feel the HEW2017 provides that much needed space for organisations to promote initiatives and publications that otherwise would be relegated to shelves to collect dust and be forgotten.

Evidence Aid has highlighted more than once the need for a more evidence-based approach in the humanitarian sector and the many related evidence gaps. I feel that although there is clear need for additional evidence, there is also evidence and related initiatives out there that do not get the attention they deserve. Therefore, Evidence Aid developed this initiative, together with our partner the Centre for Evidence-Based Medicine (CEBM). For us it is important that it furthers our joined interest in promoting an evidence-based approach - something we have in common with Cochrane.

During HEW2017, Evidence Aid will be offering training and are also involved in several of the other events that you can read about on the HEW2017 webpage. I personally look forward to editorial on the IRIN news website by Save the Children on 6^th November that will kick off the HEW2017.

As part of HEW2017 there will be three exciting events in London. I look forward chairing the first one on Monday 6 November at the London School of Hygiene and Tropical Medicine (LSHTM) with Bayard Roberts (Professor of Health Systems and Policy at LSHTM) talking about the opportunities for the RECAP project to improve evidence-based decision-making and accountability in humanitarian response, with responses from Gareth Owen (Emergency Director at Save the Children UK) and Alice Obrecht (Research Fellow at the Secretariat for the Active Learning Network for Accountability and Performance, ALNAP).  The next day (Tuesday 7 November) our founder, Professor Mike Clarke, will run a 1-day Evidence Aid course placing systematic reviews in the context of disasters and the provision of humanitarian aid. Finally, on 9 November, I hope you will join me at the University College London Thomas Coram Research Unit Library at Woburn Square to discuss evidence gaps and research priorities in the humanitarian sector. We are proud to have the UNICEF Research Centre – Innocenti, 3ie, Action Against Hunger, International Rescue Committee, the Campbell Collaboration and the UCL EPPI Centre joining us to present some of their work and answer your questions.

The UNICEF Research Centre – Innocenti is also facilitating a webinar, along two other webinars by the Professionals in Humanitarian Assistance and Protection (PHAP) and the US National Library of Medicine – Disaster Information Management Research Center. These webinars will highlight issues often not discussed in relation to an evidence-based approach in the humanitarian sector. The UNICEF Research Centre – Innocenti will be addressing social protection in fragile contexts, the US National Library of Medicine – Disaster Information Management Research will address the librarian or information specialist perspective on access to reliable health information, and finally you can join PHAP in a discussion on the challenges and limits of evidence-based approaches to humanitarian action.

Aside from the events in London and webinars there will be several blogs, a vlog, podcast and an online discussion forum. We were able to secure blogs from UCL EPPI-Centre, CERAH Geneve, CENDEP Oxford Brookes University, CaLP, ReBuild Consortium and DFID/UKAID. The Karolinska Institute will contribute with a vlog (video blog), while the CEBM is interviewing our funder in a podcast. Finally, you can already contribute to the HEW2017 by joining HIFA (Healthcare Information For All) and take part in their Discussion Forum. It is providing humanitarians and others with the opportunity to join library and information professionals in exploring ways to improve the quality, usefulness, availability and use of healthcare information for humanitarian action.

With contributions from actors that are traditionally not associated with the humanitarian sector, I hope that we can attract others to assist the humanitarian sector in becoming more evidence-based. I therefore want to invite Cochrane contributors to take part in all of these events, as I am convinced that their views on evidence-based decision-making can often be applied to the decision making and policies applied in emergencies and can be useful for the humanitarian sector. The lessons learned from the unique approach towards evidence-based medicine that is spearheaded by Cochrane needs to be available to the humanitarian sector.

Registration information:

• Event at the London School of Hygiene and Tropical Medicine (LSHTM)
• Evidence Aid training
• Evidence gaps and research priorities in the humanitarian sector – at UCL: To register contact Jane Higgins
• PHAP Webinar – Contested Evidence
• HIFA Discussion Forum
   

Jeroen Jansen
Director, Evidence Aid

The Brazilian translation team have been modifying English language Cochrane podcast scripts before translating them into Portuguese, in order to make them more suitable for their local audience. Patricia Logullo, translation editor at Cochrane Brazil, tells us more about the changes made to the scripts, and how listeners have responded.

How did the initiative begin?

The Portuguese translation team at Cochrane Brazil felt that while Cochrane podcasts have the potential to be a great tool for disseminating health evidence, the podcast scripts were too long and too difficult for the average Brazilian listener to understand.  When translating from English to Portuguese, the Portuguese text can often end up being up to 30% longer than the original. Therefore, we decided to make tweaks to the scripts before we translated them and recorded the audio files in Portuguese.

What did we change?

Our main focus was finding a way to shorten the podcast script length significantly, by avoiding repetition of ideas and content. We established a rule that any podcast produced by us could be a maximum of 5 minutes in length. We believe that 10 to 12 minutes is too long for those using their mobiles to hear podcasts, especially on public transportation, and that these messages should get to the point quickly. We organized the podcast texts in the way radio journalists usually do, in segments, starting with a lead. In this way, it was possible to predict how long would a recording take, based on previous recordings used as a test. Then we produced a new text, divided into separate sentences (one per line), to make it easier for the journalist recording the podcast to read. We teamed up with radio journalist Tatiana Ferraz, who is also a volunteer at Cochrane Brazil, and responsible for recording all our podcasts.

Has the scheme been a success so far?

We tested two podcasts, recording the long, original script and also the new, shorter, reorganized script, and asked lay people to give their impressions, comparing them in a very informal way. These people felt that the short version was better than the original, and so we started to record the short, edited versions of the podcasts.

Now, when we publish an abstract translation on the Portuguese version of cochrane.org, and  The Cochrane Library, we also produce a dissemination package, comprising the recorded, short podcast, a short accompanying text for dissemination, and a post on Facebook. We publish the text and the podcast on the Cochrane Brazil website, and we also promote it on our Facebook page, with a link to the Cochrane Brazil webpage, where readers and listeners have access to all the dissemination material.

So far, we have seen a 42% rise in views of our website in August, compared to July, with 48% new visitors. It seems that people are getting familiar with our page and information, as there were around 20% repeat visitors in both months. We are still monitoring the statistics, and we hope we can present more details at future translation teams meetings. 

Example of a podcast produced by Cochrane Brazil: http://brazil.cochrane.org/news/leis-antifumo-melhoram-saúde-das-pessoas-que-não-fumam

Jennifer Dawson works with Cochrane’s Communications and External Affairs team as a Wikipedia Consultant. Her role includes maintaining and building further relations with Wikipedia, connecting new editors to the Wikipedia community, and supporting requests for engagement in Wikipedia work from the Cochrane community. This blog post shares two strategies for improving the quality of Wikipedia articles using Cochrane evidence.

Wikipedia is a powerful public knowledge-translation tool. Across all languages, health-related Wikipedia articles receive over 10 million visits per day from around the world.(1)

To improve the quality of Wikipedia articles, Cochrane and Wikipedia developed the Cochrane-Wikipedia Partnership in 2014. Presently, there are over 2000 uses of Cochrane Reviews in Wikipedia. Many Cochrane Groups are training Wikipedia editors and developing new ways to share high-quality evidence on Wikipedia. Wiley and the Cochrane Library have distributed over 85 free accounts to Wikipedians to support the sharing of Cochrane evidence on Wikipedia.

Cochrane-Wikipedia Update Project

Cochrane Reviews are updated regularly based on need and updated reviews receive a new citation on MedLine. Once these updates are published, the next step is to update the citation within the Wikipedia article and make sure that the new conclusions are reflected on Wikipedia. Out of date Cochrane Reviews are flagged automatically in Wikipedia with the “Cochrane-Update-Bot”, and this editing task is popular amongst Cochrane-Wikipedians. See our project page!

This volunteer task does not take a lot of time to perform, but the potential impact is very large. Between May 2017-October 2017, volunteers updated 330 Wikipedia articles and the articles have already received close to 32 million views. There are presently about 50 Cochrane Reviews that need to be updated on Wikipedia. If you have five minutes and want to stay up to date with Cochrane evidence, this volunteer task may be for you!

New Cochrane-Wikipedia Project!

Many content errors in Wikipedia articles are due to not enough skilled editors inserting new evidence.(2) A new Cochrane project will tackle this! The Cochrane Schizophrenia Group tested 200 Cochrane Reviews and found that approximately half were “Wiki-compatible”, meaning that their conclusions had a fairly obvious landing place within a current Wikipedia article.(3) In light of this, we have created a Wikipedia Project Page that includes a list of all the Cochrane Reviews not presently in Wikipedia. Volunteers will be directed to the project page, given Wikipedia-editing support, and encouraged to “be bold” (Wikipedia-style) and select Cochrane Reviews to insert into Wikipedia. There are over 5000 reviews on the list, and while not all of the reviews will have an obvious home in Wikipedia, it is our goal to work through the list over the next 12 months and add in new Cochrane content. We will be recruiting editors for this new task through Cochrane Engage or visit the project page directly and start editing!

References:

1. Heilman JM , West AG. Wikipedia and medicine: quantifying readership, editors, and the significance of natural language. J Med Internet Res 2015;17:e62.doi:10.2196/jmir.4069
2. Mesgari M , Okoli C, Mehdi M , et al. ‘The sum of all human knowledge’: A systematic review of scholarly research on the content of Wikipedia. J Assn Inf Sci Tec 2015;66:219–45 .doi:10.1002/asi.23172
3. Adams, C. What proportion of Cochrane Reviews are Wiki-compatible? http://community.cochrane.org/news/what-proportion-cochrane-reviews-are-wiki-compatible

Cochrane reviews have come a long way in twenty years. As we all work hard to maximise the quality and the usefulness of our reviews, Cochrane reviews now routinely incorporate more complex methods and address more complex questions. As an organization rooted firmly in our collaborative community and our volunteer ethos, how do we balance our dual aims of enabling new colleagues to get involved, and achieving that quality and complexity?

One of many highlights at this week’s Global Evidence Summit is the launch of the new Cochrane Interactive Learning, an online, introductory course on how to conduct a systematic review of interventions. This represents a major advance in our online learning approach, bringing together the world-leading expertise of Cochrane’s methodologist community with best practice in modern interactive learning design and pedagogy.

Over nine modules and 100+ bite-size pieces of self-directed learning, the course will serve as a comprehensive introduction to new authors, or a refresher and update for experienced authors and editors. The course covers framing a review, searching for studies, risk of bias assessment based on the new Cochrane Risk of Bias 2.0 tool, meta-analysis, GRADE and Summary of Findings tables, integrating economic evidence, drawing conclusions and reporting the review. The modules provide clear explanations with stimulating interactions, quizzes, animations, examples and reference resources, and enables authors to tailor their learning experience and complete assessments to demonstrate their learning.

For the first time, through Cochrane Innovations, our online learning will be accessible beyond the Cochrane community. Systematic review authors, educators, researchers, health professionals, policy makers and guideline and HTA organisations will be able to access Cochrane learning on a paid subscription basis, to support individual learning, professional development or as a resource integrated in a blended learning approach.

Cochrane authors, editorial teams and Group staff, along with residents of HINARI A and B countries, will be able to access the resources free of charge (see here for a complete list of eligible roles). Cochrane Members who don’t fall into those categories will qualify for a hefty discount.

So how does this introductory training connect to the balance we’re trying to strike, and the need to ensure that authors have the skills and knowledge to deliver complex methods and high level analysis?

There are two answers to that question. The first is that a solid grounding in the basics is always essential. As methods evolve over time, and as we reflect on our own common errors through the lens of the Cochrane Editorial Unit’s quality improvement programme, checking back with our foundational methods is never a waste of time. I’ve been teaching Cochrane methods for more than ten years, but I feel profoundly fortunate to continue learning from our generous methods experts (and continued to do so while building these modules, thank you Kerry Dwan, Julian Higgins, Carol Lefebvre, Matthew Page, Nancy Santesso, Jonathan Sterne and Luke Vale).

Providing increased access to this learning was also an important goal for this project – learners will be able to access content of this calibre from anywhere in the world that has an internet connection, when and where it suits them, using a laptop, tablet or phone.

The second part of the answer is that this course forms only part of a more complex picture. The knowledge conveyed in these modules is a grounding, but they cannot convey the full range and depth of methods and expertise that author teams must bring to many Cochrane reviews. It’s our hope that Cochrane’s extensive network of trainers and editorial teams will find these resources a useful tool to integrate with and complement their support for Cochrane authors. Cochrane Interactive Learning can complement face-to-face training workshops in a blended learning approach; be used as modular recommendations for authors with specific learning needs; and provide a foundation to enable limited training time and resources to focus on more personalized support or training in more advanced topics.

This is the culmination of two years of work, and my sincere thanks go to Dario Sambunjak and Chris Watts who have dedicated so much of their time and knowledge to this project, along with Giulia Boselli, Ruth Foxlee, Richard Hollis, Martin Janczyk, Toby Lasserson, Holly Millward, Charlotte Pestridge, Paolo Rosati, Zoë Rose, Roger Tritton, Gert van Valkenhoef, Maxine Whitton, and our technology and eLearning partners, City & Guilds Kineo and HT2.

The first four modules of Cochrane Interactive Learning are available now for free trial access for individuals and institutions until 5 November.

To find out more: W: interactivelearning.cochrane.org E: interactivelearning@cochrane.org

Miranda Cumpston

Head of Learning & Support

Cochrane Central Executive