The U.S., England and others have spent hundreds of millions of dollars stockpiling the drug following its approval by the Food and Drug Administration in 1997. But some have questioned the drug’s effectiveness because of research that was hidden from the public eye, but available to FDA regulators. These hidden trials showed the drug to be higher-risk and less effective at treating the flu.
Timothy Platts-Mills, an assistant professor of emergency medicine at UNC’s medical school, said not having the complete picture affects doctors’ decision-making.
“We take care of a lot of patients who come in with flu symptoms, and we have to decide whether to give them things like Tamiflu or not,” he said. “It is completely insane and unethical and unacceptable that we don’t have all the data available because a private company hid it.”
Since 2000, clinical trials have started being registered on clinicaltrials.gov to keep track of the them and prevent researchers from hiding them. Clinical trials are highly regulated tests to determine medical treatments’ risks and effectiveness prior to being approved for public use.
“Now we can at least see how many studies were done,” Platts-Mills said. “That doesn’t mean we can see the results necessarily; basically, at least we know that these studies were done.”
Weeding out the noise
Now nearly 200,000 trials are registered on clinicaltrials.gov. Including the Lineberger Comprehensive Cancer Center, UNC has 737 trials registered, according to UNC’s Office of Clinical Trials.
But even the largest research universities are dwarfed by the private sector. Pharmaceutical giant GlaxoSmithKline alone is listed as a sponsor or collaborator on 3,896 trials. In 2004, New York’s attorney general sued GSK for failing to publish negative trial results for its antidepressant drug Paxil. The federal trial registry was set up to make it harder to hide trials and make unpublished trials easier to track down.
GSK eventually settled the lawsuit for $105 million and made a public promise to make its trial data more accessible to researchers and regulators.
Trial registration might have the ancillary benefit of weeding out statistical noise that clouds real results. A study published in PLOS One in August showed that the percentage of studies sponsored by the National Heart, Lung and Blood Institute that showed a statistically significant positive result dropped from 57 percent prior to trial registration in 2000 to just 8 percent after.
Newer drugs might be more incremental improvements than the breakthrough advances of earlier drugs, but the difference in significant results could also be due to “p-hacking,” where researchers look at enough possible results until they are able to find a statistically valid outcome.
Looking for a solution
Platts-Mills said registration curbs this practice because one can compare what outcome the published result studied versus what the researchers said they were going to study when they registered.
“Sometimes, they’ll say they’ll look at whether people died on this drug, but then we publish that we looked at whether they got a heart attack or not while on this drug,” he said. “That’s different.”
William Powers, a neurologist at the UNC medical school, agreed that registration can curb cherry-picking favorable data. He also said there are valid reasons to switch measurements midtrial so long as it is done transparently.
Platts-Mills said it is up to journal editors to hold researchers accountable for deviations from their registration goals.
“They can publish it, but if they were going to study death but ended up on cardiovascular events, they need to report that it changed and why,” he said. “If they just report the other outcome, then it leaves ambiguity.”
Over the past 15 years, the types of trials that are required to register on the website have expanded greatly. The FDA began requiring that clinical trials register if their results were to be used in getting a drug or treatment approved. In 2005, the International Committee of Medical Journal Editors — which oversees many highly regarded medical journals — began requiring trials be registered on clinicaltrials.gov or a handful of other registries as a condition of publication.
Kirstin Borgerson, a professor at Canada’s Dalhousie University who studies bioethics and medical philosophy, said trial registration is a key to protecting patients.
“If you run 38 trials but only two are positive, but those are the ones that get published, that is very misleading,” Borgerson said.
She said she used the example of the heart medication Aprotinin in a recent paper because it was studied 64 times.
“It was clear that after a few times that it worked, but because of researchers not knowing about the other trials and other constraints, these trials kept going and going and going over many years,” she said. “In these redundant trials, there’s usually a placebo group, and that means there is a group of people that are not getting a drug that has been proven to work. That’s completely unethical.”
Platts-Mills said that clinical trials can cost up to $10,000 per person enrolled and that unnecessary trials or burdensome regulations add to the development costs that get passed on to patients.
“We don’t want to obstruct people from doing trials,” he said. “I don’t think the registries are obstructive at all.”
Registration is effective in preventing clinical trials from being completely hidden, but it is only a first step, according to many in the field. Platts-Mills, who has co-authored a number of papers analyzing the effectiveness of the registration system, said there is still a problem with unpublished trial results.
A lack of results
Roughly 20 percent of trials are not published six years after the trial’s completion, according to a 2013 study in the British Medical Journal.
One published estimate stated that only 46 percent of clinical trials ever publish results, but those registered to clinicaltrials.gov were significantly more likely to make them available.
In 2008, the website added a section that allows researchers to add study results to a completed trial. Fewer than 20,000 trials have posted results, as only certain trials are obligated to report their results.
Powers has five trials listed on the site and said the lack of study results is partly due to medical journals’ rules.
“Most medical journals require that you don’t give out the results before publication,” Powers said. “Maybe at a conference, but they really don’t want them available prior to publication.”
He also said the initial registration is fairly straightforward, but reporting results is trickier because of the wide range of ways results can be presented, depending on the trial.
“Shoehorning your data in can get very difficult if it’s not exactly lining up with the way they’ve categorized it,” Powers said.
Borgerson said that in an ideal world, all trial data would be available, both to ensure transparency and minimize redundant trials, but trial registration and results reporting are important steps.
“I would like researchers to know what has been done and bring that knowledge into their research design,” she said. “There (are) a lot of really crappy studies being designed all the time, and part of that is ignorance of what is out there.”