Peer Review and the MMR Scandal

When a study is submitted to a scientific journal, it goes through a number of stages, at any of which, its publication could be rejected. The editorial team check the paper. They ensure it fits within the scope of their publication, that it is complete, free of obvious error, and would interest the journal’s readership. As many as 80% of submissions fail to pass this hurdle in the most widely read titles such as Nature. Should the paper survive this stage, it is sent to at least two, but often more, external reviewers to be examined more closely. Ideally, these “peers” are published scientists with expertise in the field that the submission concerns. They assess the quality of the study, its methodology, whether the results justify the conclusions, and as important, that it adds something new to the accumulated knowledge of the topic which it examines. The reviewers will then either recommend the study for publication or not. Generally, a consensus of the “peers” is required, but the final decision on publication lies solely with the journal’s editor.

The House of Commons Science and Technology Committee’s report, Peer Review in Scientific Publications, was released on 28th July, 2011. It focused on peer review to “see whether [the system] is operating effectively.” The report found the peer review system wanting, highlighting the MMR vaccine scare engineered by Andrew Wakefield at the behest of a law firm paying him to find a link between autism and vaccines.

In 1998, Dr Wakefield, then a researcher in gastroenterology at the Royal Free Hospital, London, and twelve colleagues submitted a paper entitled Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children to the hitherto well respected medical journal, The Lancet. They published it. At the press conference held on publication, Wakefield announced his belief that the MMR vaccine may cause autism (much to the surprise of many of his collaborators as the study found no such thing). He advised that the safest course of action was for children to receive separate shots against measles, mumps and rubella. Coincidentally, of course, the affable doctor had recently patented the measles vaccine he, himself, was developing.

MMR uptake fell sharply. Measles, once declared eradicated in the USA, returned as did its complications along with its triple jab siblings mumps and rubella. The tragedy was only compounded by America’s home-grown vaccine scare over so-called toxins. Measles epidemics broke out in Europe. Children have suffered, been injured, and some have died. Now, fifteen years on 700 children in South Wales have been infected with measles. At least sixty have been hospitalised and deaths are expected.

In January, The Lancet finally retracted Wakefield’s paper. By then, most of his collaborators had asked for their names to be withdrawn from the study and dozens of subsequent investigations had proved its findings impossible to reproduce. Wakefield, himself, had been struck off by the General Medical Council (GMC) who found against him on (among many others) twelve counts of abuse of developmentally impaired children and four of dishonesty – one of which related the statement that “investigations were approved by the Ethical Practices Committee of the Royal Free Hospital …” – A blatant lie.

Much of the evidence against Wakefield as a huckster and abuser of vulnerable children was uncovered by journalist Brian Deer. Writing for the Sunday Times, his tenacity and ingenuity made the outcome of the duplicitous doctor’s GMC hearing wonderfully predictable. Deer should, rightly, be lauded for his work. He is correct: scientists are only human and some do succumb to the temptation to embellish, cherry-pick, re-order or even make up their data. Disappointingly, the Wakefield incident is not unique. Nevertheless, Deer’s calls for a regulatory body overseeing scientific research, as the Science and Technology Committee’s report recommends, raises many objections.

Firstly, science is, essentially, if not self-policing then self-correcting. Fraud does not long go undetected because studies are falsifiable. This is intrinsic to the scientific method. If findings cannot be reproduced, a hypothesis fails to become a theory – as close to a fact as science allows. Science dogma does not stand or fall on the basis of a single study. Even the MMR-autism case substantiates this. An idea was proposed with assumed honest data to support it, but further and better conducted research around the world failed to find that which Wakefield’s group reported. This leads to the second objection: scientific research is an international pursuit. Safeguards and oversight in one country do not apply globally. One only has to look at the laws and conventions governing the use of animals in medical research to see the disparity from nation to nation. Imposing the requirement for monitoring could make international collaborations more bureaucratic, expensive and, potentially, infeasible.

Science, among few human endeavours, has the ability to transcend geographical borders, culture and ideology. Falsifying a hypothesis is not reliant on a single institution in a single country, so the Select Committee’s implications of cronyism fall at this hurdle. Look again at the Wakefield paper. Groups in Poland, Denmark, USA and more conducted research destroying that very hypothesis. Finding reviewers with the requisite level of very specialist knowledge often means drawing from a very small pool indeed. Finding overseers with multi-disciplinary skills and experience to monitor researchers would prove equally challenging.

Most science journals employ single-blind peer review, whereby the reviewer knows who made the submission, but the writers do not know who is reviewing it. This contrasts with the double-blind method favoured by social sciences publishers. Simply put, that group which is conducting a particular study or studies would likely already be known to reviewers who work in that area, too. In certain strands of the sciences, there are only a limited number labs (sometimes only one) from which the research could have emanated, be that because of equipment, the effects of a geographical variable being studied, etc. Double-blinding, therefore, becomes impractical and redundant. For these reasons, there are journals who impose no blinding whatsoever, giving researchers the potential to seek redress if they feel a particular reviewer has displayed bias towards them. The Committee on Public Ethics affirms, “it is probably impossible to eliminate all bias … but good editors endeavour to minimize it.”

That the cost to public health resulting from the publication Wakefield’s fabricated work was, and continues to be, so very high, is naturally worrisome. The fault for this, however, is not due to the scientific method or the peer review process. The reason this paper caused uproar and, ultimately, deaths is two-fold. That The Lancet, with so fine an international reputation, should ever have published it is contentious and the edition being heralded with a press conference by Wakefield et al was, undeniably, the spark that ignited the powder keg.

Professor Thomas T. MacDonald is a gastroimmunologist working at Bart’s hospital in London. His testimony at the Omnibus Autism Proceedings in the US Vaccine Courts was unambiguous. He called the 1998 study “probably the worst paper that’s ever been published in the history of [the Lancet].” Professor MacDonald has over 400 publications behind him. He questions Wakefield’s qualifications to conduct such a study and accurately interpret the results. “He is a surgeon,” MacDonald says. “He’s not a paediatrician. He’s not an immunologist. He’s not a histopathologist.”

Wakefield’s charisma has been commented on ad nauseum. He certainly seemed to have had The Lancet’s editor, Richard Horton, rapt. The 1998 study was not the first seriously flawed investigation of Wakefield’s to appear in the journal. Calling press conferences to hold measles (and subsequently, vaccine strain measles) responsible for inflammatory bowel disease was his M.O. Here was a debonair researcher with a penchant for controversy – an appealing combination to an editor like Horton, who clearly saw the role of editor as more journalistic than academic. Horton’s 2004 book, MMR: Science and Fiction – Exploring a Vaccine Crisis, is less the apology it should be, given the consequences of that single editorial decision, but page after page of excuses for letting a substandard study be published in what is (was?) arguably Britain’s premiere medical journal. Probably correctly, Horton assumes that the study may well have been published elsewhere had he not done so. There is a hierarchy of journals, at the top of which, in the UK at least, was The Lancet. Publication in its hallowed pages lends any study an air of validity. This unofficial ranking of journals is a criticism the Science and Technology committee levels at the peer review system. That this is understood, accepted and utilised by the scientific community is entirely lost on them.

The press love a good scare story. The fact that study after study that refuted Wakefield’s hypothesis barely caused a ripple after the tsunami of that initial study. It took Brian Deer’s muckraking and decrying of Wakefield to finally get the fact that MMR does not cause autism on to the front pages. In this context, muckraking is no slight on Deer, for he uncovered lie after filthy lie, abuse and despicable conduct as he investigated Wakefield’s methods and motives. The science may be difficult to comprehend, but Deer turned this into a  thriller-like story.

Had this travesty of a study not appeared in such a well respected journal, had it not been accompanied by a huge press conference laid on by Wakefield’s institution, the Royal Free, then the hypothesised MMR-autism link, that eventually met the fate of so many other poor and fraudulent pieces of work and drowned in its own irreproducability, would have long since faded into history. Too many commentators and politicians blame the press for the furore surrounding Wakefield’s study. Although partially culpable, this is a drastic oversimplification. Journalists, too few with science training or the ability to comprehend a scientific report, were spoon-fed this story by Wakefield and the Royal Free, who produced a video and a press pack to accompany the now infamous press conference.

The MMR-autism farrago was not facilitated by flaws in the peer review system, though they do exist. In fact, four of the six reviewers of the study rejected Wakefield’s paper. Were it not for an editor with the longing to be a sensationalist journalist, the lead researcher’s vested interest in clearing the way for his own vaccine, and the publicity required by a law firm hoping to win huge amounts of compensation for its clients, thus guaranteeing a big payday for themselves, the study would most likely have failed to reach such a wide audience. A poor editorial decision was made. Groups trying to reproduce the work failed. The hypothesis failed. This is what the scientific method does so well and has for centuries. That there are still people out there who, as if on an article of faith, believe that vaccines cause autism is the fault of media outlets and shady journals who make, at best, half-hearted attempts at peer review. Science is self-correcting; journalism and editorial egotism, unfortunately, are not.