Some 30,000 oncologists, medical researchers and marketers meet every year to present the latest advancements in cancer treatment at the annual meeting of the American Society of Clinical Oncology. This year, however, the meeting oriented itself around a oncology study that was anything but a breakthrough.
Dr. Mark R. Gilbert, a professor of neuro-oncology at the University of Texas M. D. Anderson Cancer Center in Houston, presented the results of a clinical trial testing the drug Avastin in patients newly diagnosed with glioblastoma multiforme, an aggressive brain cancer. Previously, two other trials using this drug reported that tumors shrank and the disease seemed to stall for several months. However, Gilbert’s study didn’t find a difference in survival between patients who were given Avastin and those who had been given a placebo. Gilbert’s trial was considered the gold standard. Although the outcome was disappointing, the study was a victory for science over guesswork of real data versus hunches.
Previous studies were referred to as “single-arm” in nature. Single-arm means there was no comparison group. Gilbert’s trial had more than 600 brain cancer patients, and they were randomly assigned to two evenly balanced groups. These groups were an intervention arm (standard treatment plus Avastin) and a control group (standard treatment plus placebo). Furthermore, Gilbert’s trial was double-blind. Double-blind refers to a situation where neither patients nor doctors know who was in which group. This randomized, controlled trial had worked. However, doctors had no more understanding about how to treat brain cancer than they did before.
Gilbert’s trial demonstrated that some patients did better on Avastin but the trial was unable to discover these “responders” nor was it able to determine what would have accounted for the differences. Trying to figure out the differences is next on the agenda.
According to a spokesperson for Genetech, a division of the Swiss pharmaceutical giant Roche, which makes the drug, “Despite looking at hundreds of potential predictive biomarkers, we do not currently have a way to predict who is most likely to respond to Avastin and who is not.” Some 400 completed clinical trails in various cancer have still not made it clear why Avastin works or doesn’t in any single patient. Roche sells $6 billion globally of Avastin, and there have been 16 years of clinical trials involving tens of thousands of patients. Nevertheless, they are confused about other long-tested therapies. These therapies include neuroprotective drugs for stroke, erythropoiesis-stimulating agents for anemia, antiviral drugs such as Tamiflu and rosiglitazone (Avandia) for diabetes.
This begs the question: do clinical trials work?
Anatomy of a Clinical Trial
Researchers and clinicians are beginning to realize how individualized human physiology and pathology really are. Consider the genetic level were tumors in one person with pancreatic cancer almost surely won’t be identical to those of another. Not to mention how any two patients would react to the same drug. Even with rigorous monitoring of clinical trials, 16 medicines were withdrawn from the market between 2000 to 2010. Each individual has a unique pharmacogenomics.
Drug trials are generally divided into three phases. The first phase evaluates the safety of a new compound in a small population determining the best way to deliver it and what an optimal dose would be. Phase 2 looks at a much larger population, continuing to monitor safety and assess whether the drug works. Essentially this means, does the drug have any positive effect as seen through some blood marker associated with the disease. If so, then phase 3 commences. Apparently, most experimental drugs fail before they get to Phase 3 trials.
If a drug goes to Phase 3, safety and efficacy is monitored in hundreds or thousands of patients. Phase 3 outcomes are compared head-to-head for those getting a placebo or standard-of-care therapy. The FDA requires that two “adequate and well-controlled” trials confirm that a drug is safe and effective before it can go to market.
The results are run through rigorous statistical tests to make sure the outcomes are not the result of chance. When the measured effects are small (most clinical trial results) it is difficult to determine whether a drug is working.
Dr. John P.A. Ioannidis, a professor of medicine at Stanford and authority on statistical analysis, published what has become a famous article back in 2005 in The Journal of the American Medical Association. Ioannidis examined 48 high-profile trials that found a specific medical intervention to be effective. Twenty-six randomized, controlled studies were followed up by larger trials (larger sample size) in which 3 cases (12 percent) were wholly contradicted. In another 6 cases (23 percent), the later trials found the benefits to be less than half of what was published. In these trials, the therapy wasn’t changed, just the sample size. Ioannidis believes that if more rigorous studies were done, the refutation rate would be far higher.
Donald A. Berry, a professor of biostatistics at M. D. Anderson, agrees with Ioannidis. Berry has actually made a sport of predicting this evaporation effect. Berry points out that the failures of the last 20 or so Phase 3 trials testing drugs for Alzheimer’s disease could have been predicted based on the lackluster results from Phase 2 results. However, the payoff for a successful Phase 3 trial can be great, such that drug makers will often roll the dice. Actually, not on the prospect that the therapy will work, but on the chance that the trial will suggest it does.
Considering that the pharmaceutical companies sponsor and run most of the drug trials themselves, it brings what Ioannidis calls a “constellation of biases” to the process. Ioannidis goes on further to say that, trials are held against “a straw-man comparator” like a placebo rather than a competing drug. So, the studies don’t really help us understand which treatments for a disease work best.
According to Hal Barron, the chief medical officer and head of global development at Roche and Genentech, “When you do any kind of trial, you’re really trying to answer a question about truth in the universe. And, of course, we can’t know that. So we try to design an experiment on a subpopulation of the world that we think is generalizable to the overall universe” — that is, to the patients who will use the drug.
These very rules that govern clinical trial enrollment end up creating trial populations that are much younger, have fewer health issues, and have been exposed to few medical therapies than the individuals who are likely to use the drug.
According to Barron, “Listen, it’s not lost on anybody that about 95 percent of drugs that enter clinical testing fail to ever get approved. It’s not hard to imagine that at least some of those might have failed because they work very, very well in a small group. We can’t continue to have failures due to a lack of appreciation of this heterogeneity in diseases.”
To get at a solution for these difficulties, Genentech decided to attack subtypes of diseases that are already known and design small trials and enroll individuals who have the appropriate molecular marker or genetic signature. That’s exactly what they did in developing the breast cancer drug Herceptin, which homes in on tumor cells that have an abundance of the protein HER2. Sixty percent of the new drugs currently being developed at Genentech/Roche are being developed with a companion diagnostic test so they can identify the patients who would most likely benefit from it.
Even so, this piecemeal approach will be a slow process. Rather than trying to fit patients to a drug a handful at a time, it would be better to change the clinical trials themselves. They suggest that, a breast cancer trial (I-SPY 2), sponsored by Biomarkers Consortium (a partnership that includes the Foundation for the National Institutes of Health, the FDA and others), would be a good model to follow. The goal here is to figure out whether neoadjuvant therapy for breast cancer – administering drugs before a tumor is surgically removed – decreases recurrence of the disease, and if so, which drugs work best. They are testing up to a dozen drugs from multiple companies, phasing out those the don’t appear to work and subbing in others, without stopping the study.
Physicians will be using a statistical technique called Bayesian analysis which gives them the ability to glean information about which therapies are working. This way, doctors can learn during the process and then incorporate that information into the ongoing trial.