Clinical drug trials often are hailed as the standard for relevant and reliable information on potential new products.
But a recent editorial in the British Medical Journal suggests that researchers often fail to report relevant trial data (BMJ 2012;344:d8158).
“We are not dealing here with trial design, hidden bias or problems of data analysis—we are talking simply about the absence of the data,” the authors wrote. This behavior, in turn, biases research, wastes health care resources and may harm patients, the editorialists argued.
“Moreover, researchers or others who deliberately conceal trial results have breached their ethical duty to trial participants,” they wrote. In fact, a slew of studies accompanying the editorial in the journal “confirm the fact that a large proportion of evidence from human trials is unreported, and much of what is reported is done so inadequately,” the editorial argued. The studies explore the extent and consequences of leaving out data from clinical trials.
In one study, investigators showed that the addition of unpublished data to published meta-analyses of drug trials often changed the results (BMJ 2012;344:d7202). The researchers integrated previously unpublished data into existing meta-analyses of nine FDA-approved drugs and showed that the recalibrated trial data produced identical estimates of drug efficacy in three of 41 cases (7%), but 46% greater and 46% lower drug efficacy in the remaining 38 cases (19 for each).
The editorialists acknowledged that “it is sometimes assumed that incorporation of missing data will reduce estimates of drug benefits, but this study shows that ‘publication bias’ can cut both ways. Each increment of data can change the overall picture, but in most cases with no certainty that the picture is complete.”
Not only is evidence frequently missing from trials, but the requirements for mandatory trial registration and appropriately timed sharing of results often are not followed properly. Another study found that fewer than half of trials funded by the National Institutes of Health (NIH) are published in a peer-reviewed journal within 30 months of trial completion, and even at 51 months, one-third of results remained unpublished (BMJ 2012;344:d7292). Furthermore, in the United States, in 2009, only 22% of drug trials subject to mandatory reporting requirements disclosed their results within the required one year after the trial ended (BMJ 2012;344:d7373).
“So it seems that most trials haven’t reported results, which we think is serious,” said lead author Andrew P. Prayle, BMedSci, clinical research fellow at the University of Nottingham, in the United Kingdom.
A recent Cochrane review provided an example of allegedly unreported data (Cochrane Database Syst Rev 2012;1:CD008965). Attempting to study the anti-influenza antiviral drugs zanamivir (Relenza, GlaxoSmithKline) and oseltamivir (Tamiflu, Genentech USA, Inc., member of the Roche Group), the researchers received cooperation from GlaxoSmithKline, but reported that they were “unable to obtain the full set of clinical study reports or obtain verification of data” from Roche despite five requests between June 2010 and February 2011.
Tom Jefferson, MD, an independent epidemiologist in Rome, leader of the study, said his group had reached “tentative conclusions which are at odds with the manufacturers’ statements,” but “full testing of all these findings, which may have a profound public health impact, cannot be done in the absence of the complete data set.”
Roche maintained that it does make “detailed clinical trial reports” available and that it “stands behind the robustness and integrity of our data supporting the efficacy and safety of Tamiflu.”
The lack of reporting appears to violate the FDA Amendments Act of 2007. That prompted Rep. Henry A. Waxman (D-Calif.) and congressional colleagues to write to the heads of the NIH and the FDA, for an explanation, and an answer to why the penalty of $10,000 per day for violating the law had apparently not been enforced.
The NIH declined to comment and said it would respond to the representatives.
Pat El-Hinnawy, an FDA spokeswoman, said a delay in publishing data may result if the sponsor of a clinical trial is seeking approval of a new use of a drug or device. “FDA has identified a number of factors that skew the data and impact the percentage of results Prayle [et al] reported,” Ms. El-Hinnawy said.
As for enforcement, Ms. El-Hinnawy said that the agency has focused on providing information and assistance “to encourage compliance and to ensure an understanding of the responsibilities.”
Frederick Stearns, JD, partner at Keller and Heckman LLP, in Washington, D.C., said he suspected that policing the clinical trial reporting requirements “is a lower-priority issue, given all of the other matters the agency is responsible for.” In addition, he said, the $10,000 per day monetary penalty may seem “unduly harsh for a data submission requirement.”
Elizabeth Loder, MD, MPH, a BMJ editor and co-author of the editorial, noted that most clinical interventions in use today are based on trials carried out before the era of mandatory registration. “And here the task of data retrieval by systematic reviewers and national advisory bodies becomes impossible,” the editorialists wrote. “Our patients will have to live with the consequences of these failures for many years to come.”
Potential solutions to the problem of underreporting of data have been proposed. Dr. Prayle suggested that “a greater awareness of the reporting requirement will go a long way.” Dr. Jefferson added, “Reform and transparency are needed across the board.”
Dr. Loder proposed a harsher solution: “Penalties for not reporting trial data need to be enforced,” adding that academic institutions and professional organizations need to be involved, and researchers doing meta-analyses need to look beyond published trials.