Scientific Evidence and Regulatory Review Involving Efficacy, Safety, and Futility
On November 6, 2020, a US Food and Drug Administration (FDA) advisory committee reviewed issues related to the efficacy and safety of aducanumab, a human IgG1 anti-Aβ monoclonal antibody specific for β-amyloid oligomers and fibrils implicated in the pathogenesis of Alzheimer disease.1 Given the importance of drug innovation for this common and often devastating disease, the abandonment of prior monoclonal antibodies targeting β-amyloid, and the clinical, regulatory, and market effects that approval of aducanumab could have, there has been significant interest in the development and regulatory review of aducanumab.
Determination of Futility. The primary evidence of efficacy for aducanumab was intended to be 2 nearly identically designed, phase 3, double-blind, placebo-controlled randomized clinical trials (RCTs) of high- and low-dose aducanumab (“study 301,” ENGAGE [NCT02477800] and “study 302,” EMERGE [NCT02484547]). The studies were initiated after a phase 1b safety and dose-finding study indicated suitable drug safety (NCT01677572).2 Approximately halfway through the phase 3 studies, a planned interim analysis met prespecified futility criteria and, in March 2019, the sponsor announced termination of the trials.
However, following this decision, and augmenting the data set with additional trial information that had been gathered after the futility determination, conflicting evidence of efficacy was identified in the 2 studies.1(p59-61) Study 301 (n = 1647 randomized patients) did not meet its primary end point of a reduction relative to placebo in the Clinical Dementia Rating–Sum of Boxes (CDR-SB) score. According to prespecified plans to protect against erroneous conclusions when performing multiple analyses, no statistically valid conclusions could therefore be made for any of the secondary end points in study 301. By contrast, study 302 (n = 1638 patients) reached statistical significance on its primary end point, estimating a high dose treatment effect corresponding to a 22% relative reduction in the CDR-SB outcome compared with placebo (P = .01). In the low-dose aducanumab group in study 302, the effect was not statistically significant compared with placebo, and based on the prespecified analytic plan, this precluded the ability to assess efficacy with respect to secondary outcomes in both the high- and low-dose groups.
Substantial Evidence From a Single Trial? While the FDA has usually preferred 2 adequate and well-controlled trials to demonstrate substantial evidence of efficacy for a new drug, the Federal Food, Drug, and Cosmetic Act was amended in 1997 to allow the FDA to approve a new drug based on evidence from a single study.3 Although no explicit rules govern exactly when a single pivotal trial might be sufficient, regulatory guidance notes the importance of characteristics that “support the persuasiveness of a single trial in supporting the conclusion that there is substantial evidence of effectiveness.”4 In the case of aducanumab, the sponsor worked with the FDA to further analyze the pivotal trials as well as its earlier phase 1b study to determine the importance of the statistically significant results of the high-dose group compared with the placebo group in study 302. This undertaking reflected an unusual degree of collaboration between the FDA and manufacturer of aducanumab, and the arrangement has been criticized as having potentially compromised the FDA’s objectivity in reviewing the New Drug Application.5
From 2012-2016, product approvals based on a single pivotal trial typically have been associated with statistically significant results with a P < .01.6 In addition, the minimum clinically important difference of the primary end point used in the aducanumab trials, CDR-SB, is generally considered to be 1 to 2 on a scale from 0 to 18,7 while the 22% reduction in the CDR-SB outcome observed in the high-dose group in study 302 reflected an absolute difference of 0.39. The FDA endorsed any statistically significant effect on the CDR-SB as a clinically meaningful outcome in studies 301 and 302, but a “responder analysis,” while prespecified, was not presented to the advisory committee to allow for an understanding of the proportion of individuals who achieved a predefined level of improvement at a given point.
Can Post Hoc Analyses Help Explain Why the Findings From These Trials Differ? Post hoc analysis of trials that change the populations of interest, end points, or methods of analysis introduce what may be regarded as unacceptable threats to statistical validity and scientific rigor, and they are usually performed as hypothesis-generating exercises. As pointed out by a statistical reviewer at the FDA,8 analyses based on a post hoc selection of the better of 2 RCTs—the one reaching statistical significance—without methods that acknowledge this purposeful choice increase the risks of inadvertently selecting data precisely because those data were consistent with the outcomes that were hoped for. For example, when used post hoc as a single pivotal study, a true P value for study 302 would be higher than .021, rather than .01 as computed when reporting both studies; the adjustment must be greater than a simple Bonferroni adjustment to account for both the selection of the best of 2 independent studies, as well as the data-driven departure from evaluating both studies according to the prespecified analysis plan.
More than 25 negative RCTs have tested the “amyloid cascade hypothesis,” and thus the observed discordance between study 302 and study 301 is consistent with a type I error. Nevertheless, a wide variety of post hoc analyses were presented during the FDA advisory committee meeting in an attempt to explain the null findings of study 301 under the presumption that study 302 was a true-positive result.1(p62-64) Much of the focus was on the potential effect of a protocol amendment increasing the dose of aducanumab provided to participants who were carriers of the apo ε4 allele. Due to variation in accrual patterns, slightly fewer participants in study 301 had the opportunity to receive the high dose, and it was speculated that this might explain the absence of a treatment effect.1(p73-83) However, the high-dose group in study 301 had less evidence of treatment benefit than did the low-dose group in that same study, despite both studies 301 and 302 showing statistically significant dose response trends as expected for measures of brain amyloid.1(p73-83)
Several other approaches were attempted to understand the discordance between the trials. There was no evidence of important differences in baseline characteristics among participants who enrolled in the 2 trials, nor evidence of a failure in randomization. Another analysis identified a greater number of individuals with accelerated deterioration, or “rapid progressors,” in the high-dose group of study 301 compared with study 302.1(p69-72) Although their exclusion resulted in modest improvement in the estimated efficacy of aducanumab, the sponsor was unable to determine criteria that would prospectively identify such patients to inform future trials or clinical practice, the theory of rapid progressors was introduced post hoc, and statistical analyses to account for them did not change the overall null results of study 301.1(p69-72) Any treatment will appear to be more effective if individuals in whom it works least are removed from the analysis. In short, while post hoc analyses are useful for generating interesting hypotheses to be tested in future trials, the post hoc analyses regarding aducanumab provided limited information useful in deciding the benefit of this new drug and these post hoc analyses should not be the basis for FDA approval.
Considerations of the Safety of Aducanumab. Ultimately, any determination of readiness for market must also consider a drug’s overall benefit-risk balance. The pivotal trials of aducanumab were carefully designed to minimize the potential harms from amyloid-related imaging abnormalities (ARIA). While ARIA, including vasogenic edema (ARIA-E), occurs early in treatment and is typically asymptomatic, rates of ARIA-E varied markedly between those who received placebo vs drug in studies 301 and 302 (placebo, 2.7% vs high-dose aducanumab, 35.2%). In addition, as many as 0.9% of participants with ARIA experienced severe symptoms, including confusion, disorientation, gait disturbance, ataxia, visual disturbance, headache, nausea, falls, and blurred vision. The FDA’s statistical review suggested evidence of potentially greater falls among individuals treated with high-dose aducanumab,8(p68) which as with many of the other symptoms of ARIA, are especially complicated clinically because of their potential overlap with underlying disease progression. While briefing materials suggested risk of ARIA can be mitigated by monitoring via imaging and dosing management,1(p118) it is unclear how consistently and comprehensively this could be performed in clinical practice.
Looking Forward. As compelling public testimony during the FDA’s advisory committee meeting made clear, Alzheimer disease poses a major burden on millions of people and their families, and there is an overwhelming demand for safe and effective new treatments. In light of this, the sponsor of aducanumab deserves recognition for a development program that included the design and conduct of 2 well-controlled, randomized, potentially pivotal trials that should be published in the peer-reviewed literature. However, considering that these efficacy trials were stopped for futility, there is no reason to favor the trial with the positive signal in 1 of 2 treatment groups over the trial with the negative outcome in both treatment groups, and there is no persuasive evidence to support approval of aducanumab at this time. Randomized trials should remain the primary means that regulators use to assess product efficacy, and that patients, physicians, and policy makers rely on, to have confidence in the safety and effectiveness of new therapeutics.
Accordingly, based on the extensive evidence presented, the advisory committee voted on the question of whether study 302, independent of study 301, provides “strong evidence that supports the effectiveness of aducanumab for the treatment of Alzheimer’s disease” as follows: 1 yes, 8 no, and 2 uncertain.9 The FDA will consider this recommendation against approval in its evaluation of aducanumab, and a decision on the drug application is expected by June.
Read more here : https://jamanetwork.com/journals/jama/fullarticle/2778191?guestAccessKey=c30a531f-ac49-4e21-ad37-315c7acdcaeb&utm_content=weekly_highlights&utm_term=040421&utm_source=silverchair&utm_campaign=jama_network&cmp=1&utm_medium=email
Comentários