Being able to identify randomized controlled trials (RCTs) is a critical part of most Cochrane Reviews that evaluate the effects of health treatments. In May last year Cochrane launched Cochrane Crowd. This platform enables anyone with an interest in health to dive into tasks that help identify and describe health research.
By late December 2016, more than 4,000 people had signed up to help and we were in sight of reaching one million individual classifications. For those of you new to Cochrane Crowd, each task is based on people answering a single question about a piece of research. Each piece of research is looked at multiple times by different people, and a final decision is made based on a certain number of decisions that are in agreement. We call each decision made a classification. We decided to try to reach this impressive milestone of one million classifications in style by running a 48-hour citation screening challenge. We also decided to make it a fundraiser, raising money for Medecins Sans Frontieres (MSF) and UNICEF.
Pre-challenge jitters
The challenge was set to start on Monday 19 December at 10am UK time, and finish at 10 am Wednesday, 21 December. As the date drew closer I began to worry: what if no one joins in because it’s so close to Christmas? What if we’ve totally miscalculated and don’t reach one million during the 48 hours? What if the whole site goes down, and never works again….
And we’re off
Then suddenly it was too late to do anything but make a cup of tea, get settled comfortably, and wonder if my first record would be an RCT. It wasn’t. In fact the first 100 or so records I screened during the challenge were all ‘rejects’ but it didn’t matter. The challenge was underway and it was thrilling to see so many people online and to watch the classifications counter steadily going up.
For the next few hours a good rhythm was established. Contributors in Australia, New Zealand, and neighbouring countries of the Asia-Pacific then went offline to get some kip, whilst others in the US and Canada woke up to join the fun.
24 hours later
At the halfway point we’d done 26,169 classifications, bringing our total to 985,362. Over 100 people had so far joined the collective effort. I began to allow myself to think we might actually do this. It seemed others were thinking the same, as suddenly the pace picked up considerably, driven on in equal measure by the approaching milestone and by Gordon’s brilliant and encouraging banner messages (Gordon is our IT maestro extraordinaire!).
By 4.20pm on Day Two we stood at 999,194 classifications. With my heart racing, and my fingers inaccurately trying to type frantic tweets whilst also screen, we crossed the one million mark just seven minutes later at 4.27pm.
I threw my headset off (why was I even wearing it?) and did a victory lap round my tiny study. As I sat back down, remembering my responsibilities to tweet and email, I felt a little emotional. To reach this milestone for Cochrane Crowd through the efforts of so many, not just during this challenge but throughout the last few years, felt significant.
No time to sit back and relax - we still had work to do!
We crossed the one million mark around 17 hours before the challenge was due to end. The rest of the time would therefore be focused on raising as much money as we could for MSF and UNICEF. More tea was made, the headset inexplicably put back on, and the pace picked up once more. Together we went on to raise a further £2,000, bringing our total to over £7,000.
And now, relax
At 9.59am on 21 December 2016 I wondered if my last record would be an RCT. It wasn’t, but it didn’t matter - we’d already found hundreds of them. As the challenge ended, I couldn’t help but feel we’d all become more than participants. We had been on an adventure together.
In numbers
Number of people who took part: 302
Number of countries: 48
Number of RCTs found: 838
Amount raised: £7,038
Number of individual classifications made: 54,110
Time at which we reached one million in total: 4.27pm GMT on Tuesday, 20 December 2016
Proportion of records screened in the first 24 hours: 48%
Proportion of records screened in the second 24 hours: 52%
Number of cups of tea made during challenge: too many to count!
In words
We received some really helpful feedback about what people felt worked well and what we could do to improve the experience next time. Here’s just a small sample of feedback that made me smile:
Why did you join the challenge?
"Erm, I actually simply joined Cochrane Crowd - and found myself in the middle of it :)”
Describe the experience:
“Great working with people around the world. Great sense of unity”
“I liked seeing that others were "out there" doing the same thing”
“The awareness that colleagues around the world were working on the same job at the same time”
“Very easy to do - even whilst eating a sandwich”
“It was serious fun”
Thank you
It most certainly was serious fun. If you took part in this event: thank YOU. There are lots of people that make Cochrane Crowd work - in fact, that’s exactly the point: lots of people, working together, can achieve brilliant things. We ended 2016 on a high. Let’s make 2017 even more exciting for Cochrane Crowd.
If you missed this event, don’t worry, there will be more challenges, and in the meantime why not pop over to Cochrane Crowd and get some practice in. And don’t forget to drop in and say hi on Twitter @cochrane_crowd.
Happy 2017 and happy citation screening.
From Anna, Emily, Gordon and the rest of the team
Support for Project Transform was provided by Cochrane and the National Health and Medical Research Council of Australia (APP1114605). The contents of the published material are solely the responsibility of the Administering Institution, a Participating Institution or individual authors and do not reflect the views of the NHMRC.