Can systematic reviews ever be the same again?

The definition of a systematic review is, according the Cochrane Handbook for Systematic Reviews of Interventions is that they:

“seek to collate all evidence that fits pre-specified eligibility criteria in order to address a specific research question”

But what happens when all the evidence is not available because results of trials are not published in full? Publication bias is not a new phenomena (eg papers from 1991 to 2010 and a nice review: http://www.trialsjournal.com/content/11/1/37 )

The BMJ this week use their editorial to highlight the difference that getting access to the unpublished trial results would have made to a systematic review of the evidence on the effectiveness of flu vaccines:


“From now on, they say, reviewers must have access to all unpublished data, not only from unpublished trials—the usual focus of concern about publication bias—but also from those that have been published in peer reviewed journals. Reviewers must assess entire trial programmes, and so new tools and methods are needed. If the trial reports are incomplete, reviewers should turn to reports from the drug regulators. As Tom Jefferson, the lead author for the Cochrane review, told me, “it’s goodbye PubMed, goodbye Embase.””

Watch the Channel 4 documentary “New doubts over Tamiflu”  
Read the BMJ editorial Goodbye pubmed, hello raw data  and article  Ensuring safe and effective drugs: who can do what it takes   Tom Jefferson and colleagues discuss how to improve the reliability of systematic reviews.
Read the Cochrane’s revised protocol on how to carry out a systematic review : this is a pre-publication draft of a protocol for a Cochrane Review, prepared by the Acute Respiratory Infections Cochrane Review Group

You may find this entry on the Trust the Evidence blog from the Centre from Evidence Based Medicine interesting.