|
Researchers ignore
'inconvenient' drug trial results
By
Robert Matthews, Science Correspondent
(Filed:
30/05/2004)
Scientists are routinely cherry-picking the results
of clinical trials so that they can present the findings that they
want, a study by academics at Oxford University shows.
The research, which assessed the published results of
more than 100 scientific trials, also found that inconvenient
findings were often not disclosed to the public. In several cases,
the stated purpose of the trial was altered as it progressed so that
acceptable findings, rather than inconvenient results, could be
published.
The manipulation, which contravenes official
guidelines on reporting medical research, was uncovered by academics
at Oxford University, led by Dr An-Wen Chan, a researcher on
clinical medicine.
Dr Chen warned that the findings called into question
the National Health Service's evidence-based approach to developing
medicine, in which clinical trials are used to determine whether to
introduce new treatments.
"The reporting of trial outcomes is not only
frequently incomplete but also biased and inconsistent with
protocols," said the team. "Published articles, as well as reviews
that incorporate them, may therefore be unreliable and overestimate
the benefits of an intervention."
Suspicion about the reliability of published medical
research, which has been increasing for some time, has been prompted
by concern over the influence of drugs-company funding.
A recent study at the Yale School of Medicine showed
that 80 per cent of clinical trials backed by drug manufacturers
reported positive findings - compared to 50 per cent of those
carried out by independent academics.
Other studies have shown evidence of a bias against
unclear trial results being published in academic journals, and of
positive results being repeatedly published - giving the impression
that a drug is far more effective than it really is.
The Oxford team's findings, which are published in
the latest edition of the Journal of the American Medical
Association, are based on an assessment of the original paperwork
from more than 100 trials of medical techniques ranging from drug to
surgical methods.
It compared the supposed aims of the trials with what
was finally published and found that in half of the studies, results
that would have given a more accurate picture of the effectiveness
of the treatment being studied were not fully reported.
In almost two-thirds of these cases, the results
omitted concerns over potential harmful effects. Independent
researchers were just as prone to bias as those funded by industry.
Crucial information, from the intensity of pain to
survival rates, was either downgraded in importance or omitted from
the published report.
In more than half of the trials examined,
discrepancies were found between the original aims of the study and
those finally reported. Such changes are in direct contravention of
official guidelines on trials. Despite this, not one trial report
made clear that the original aims of the research had been
altered.
When contacted by the Oxford team, almost 90 per cent
of the research teams denied that they had failed to report
everything, despite evidence to the contrary. Dr Doug Altman, a
professor of statistics in medicine at the Institute of Health
Sciences, Oxford, and a member of the research team, said: "All
trials should be published honestly and transparently, and this
study shows neither is happening.
The most worrying aspect is that over 50 per cent of
the outcomes found by the trials weren't reported - and so can't be
included in the reviews used to assess different treatments. This
has serious implications for the reliability of the recommendations
made to the National Health Service."
Dr Altman said that there was no evidence that the
cherry-picking of results was because of fraudulent intent, and
claimed instead that it was because of pressure from journals to
present findings on medical advances.
"There is a lot of pressure from medical journals to
publish positive findings and to keep the length of papers down,
which can lead to negative results being omitted," he said.
Dr Altman added, however, that this did not explain
why the original design aims of trials were altered so often. He
called for the original design aims and findings of all clinical
trials to be made publicly available, to prevent changes going
undetected.
Other leading authorities on medical research
expressed dismay at the findings of the study. Sir Iain Chalmers,
the founder of the Cochrane Collaboration, which publishes reviews
of clinical evidence for doctors worldwide said: "In the days when
no one paid any attention to scientific evidence, this would not
have been a worry, but now people are investing huge amounts in
research, these skeletons are coming out of the cupboard."
The Association of the British Pharmaceutical
Industry said that it agreed that researchers had a duty to reveal
any changes made to the original trial design, and supported full
disclosure - but only when the trials had been completed.
Richard Ley, a spokesman for the association, said:
"There can be commercially confidential information in trial
protocols, and with a new drug costing £500 million and 10 years to
produce, that is an important issue."
A spokesman for the National Institute for Clinical
Excellence, which provides the NHS with recommendations on best
practice, said that it was already taking steps to avoid the
problems of biased reporting of results.
"Our appraisal committee considers all of the
evidence - including an independent systematic review of the
manufacturer's submission - and other literature, which is designed
to take into account possible bias in the reporting of trial
outcomes."
|
.gif)