A Synthesis of Adaptive Designs in Clinical Trials
Publication date: May 2016 / Download PDF
Author: Roy Nitulescu, Agnihotram V Ramanakumar, and Vatché Bartekian
Abstract
Since 2006, when the adaptive design working group presented their work in Washington D.C., the literature on adaptive designs has greatly expanded. Some designs have emerged as inherently better suited to exploratory-phase trials, while others are more appropriate for confirmatory phases of drug development. This article provides a high-level synthesis of available adaptive design methods and discusses their respective roles, advantages, and disadvantages. General challenges associated with the practice of adaptive trials such as concerns about operations, project management, regulatory authorities, necessary technologies, and ethics are also discussed.
Keywords. adaptive designs, flexible designs, clinical trials, exploratory, confirmatory
Introduction
In 2004, the United States Food and Drug Administration (FDA) published a report as a part of its Critical Path Initiative [5] outlining their views on issues with the medical product pipeline in the United States. According to the report, the rate at which new discoveries made in laboratories are being translated into safe and effective therapies available to patients (i.e., the product pipeline) has slowed down considerably over the past few decades despite aggregate spending on product development greatly increasing over the same period. Shortly after the FDA published its Critical Path report, the concept of adaptive designs started gaining popularity—initially, through a series of influential white papers published by a working group of statisticians practicing in the biopharmaceutical industry [7]. Adaptive designs in clinical trials have the potential to improve the efficiency with which new drugs are evaluated and subsequently brought to market. Over the past decade, the literature on adaptive designs and associated methods has greatly expanded, and regulatory agencies in some developed countries (e.g., United States and Europe) have begun issuing their own guidance on the topic as well [4, 6]. Recently, in 2013, the Tufts Center for the Study of Drug Development hosted a roundtable discussion featuring key opinion leaders, executives in the fields of clinical research, biostatistics, project management and operations, and representatives from the FDA and the European Medicines Agency. According to the report that followed the roundtable, approximately one in five clinical trials in the United States are using simple adaptive designs [19]. Moreover, the roundtable sponsors have stressed that the use of adaptive designs in exploratory-phase trials is expected to increase significantly in the near future.
According to the adaptive design working group, adaptive design clinical trials draw on accumulating data in order to decide how to alter the conduct of an ongoing study while maintaining its statistical and interpretational validity [7]. For example, some designs allow for periodic reallocation of dose or treatment regimens among enrolled subjects if initial regimens are found to be unsafe or ineffective. Other designs improve the speed and efficiency with which the highest effective and tolerable doses of a new drug can be identified. Such early-phase designs ultimately help in the planning of later-phase trials as they yield rich information regarding patient response and tolerance to wider ranges of doses and treatment courses than is possible with more traditional study designs. Furthermore, early interim data collected during the course of a later-phase trial can be used to address limitations present at the time of the trial’s design that no longer hold true once enough data is available. Modifications implemented on the basis of such interim data can potentially reduce the incidence of inconclusive or failed studies. With richer information from exploratory trials and better planned confirmatory studies, it is possible to bring safe, effective, and innovative new therapies to patients more quickly than with traditional clinical trials. Nevertheless, with all their benefits, adaptive designs also come with increased operational and methodological complexities which require rigorous multi-disciplinary planning.
Given the great and multifaceted burden of complexities that come with adaptive designs, there is a need for a high-level synthesis of all the well-established designs and methods recently developed in the literature to better inform decision-makers. In this article, the important aspects of adaptive designs are presented with the aim of helping practitioners and decision-makers in the biopharmaceutical field to come to grips with the benefits and critical challenges that adaptive designs pose. In particular, this article will outline the various types of adaptive designs, their associated adaptations, and their advantages and disadvantages. Furthermore, the article will also discuss the practical, regulatory, technological, and ethical concerns associated with the use of adaptive designs, and highlight important aspects to keep in mind when planning adaptive design clinical trials.
Choosing the right type of adaptive design
Adaptive designs are very attractive to individual researchers and industry sponsors due to their flexibility and resource-saving potential. However, there are no clear guidelines on the regulatory review requirements for these designs. This puts clinical trial designers in a difficult position in which they must select the appropriate designs and methods that will achieve their goals and satisfy regulatory authorities at the same time despite having relatively little clear-cut guidance on the matter. Although adaptive designs come with attractive benefits, researchers need to weigh these against the level of complexity and potential bias these designs will introduce into the trial. The biases due to these adaptations could be substantial depending on the types of adaptive designs that are chosen or the types of adaptations that are eventually employed within the scope of the design.
In general, the main methodological concern of regulatory agencies with regards to adaptations in clinical trials is the loss of control over Type I error rates (e.g., concluding that a drug has a certain effect when it, in fact, does not) [6]. In particular, the standard two-sided 5% error rate used in most trials cannot be guaranteed in the presence of complex scenarios of changing adaptations. Another related and important worry that has received much attention is the operational bias introduced into a study following adaptations [2]. For example, making alterations to the choice of study endpoints, patient populations, or treatment allocations based on unblinded interim results may lead to biases toward favorable study outcomes [6]. Analytical methods must be adjusted in order to yield valid estimates and inferences.
One way to avoid the difficulties in controlling Type I error rates is to consider using a Bayesian approach rather than a frequentist one. Indeed, Bayesian methodology provides a mechanism for the analysis of accumulating results during the course of a trial. This approach can be used for calculating the predicted probability distribution (i.e., posterior distribution) of future results on the basis of current results. However, it remains challenging to accurately determine the adaptive study’s operating characteristics such as sample size, randomization, or when to drop treatment arms based on interim results. Moreover, Bayesian posterior distributions are subject to other potential biases depending on the selection of available evidence (i.e., choice of prior distribution) and the choice of the likelihood model. In practice, the application of such methods, whether frequentist or Bayesian, requires the use of computationally intensive procedures and simulations based on a multifaceted set of scenarios.
Exploratory phase
Among all the available adaptive designs, some are more appropriate for early-phase trials (i.e., Phases I and IIa) and others are more appropriate for later phases of drug development (i.e., Phases IIb and III). Designs that allow for the reallocation of current subjects across treatment arms or the dropping of study arms are appropriate for exploratory trials, but they are not appropriate for confirmatory phases of drug development as they cannot offer adequate and well-controlled comparisons of treatment arms. However, such designs can yield very rich information that can help in the planning of later-phase trials. For example, by periodically reallocating subjects across treatment arms (e.g., adaptive treatment response designs) or dropping ineffective ones (e.g., “drop-the-loser” designs), it is possible to quickly identify the most promising treatments or doses among a pre-determined set. Similarly, by periodically reallocating subjects to other drug doses (e.g., adaptive dose-finding designs), it is possible to efficiently find the highest effective and tolerable dose of a drug. Table 1 depicts the pertinence of various types of adaptive designs along with their advantages and disadvantages concomitant with them in early phases of drug development.
Confirmatory phase
The role of adaptive designs in later-phase (i.e., confirmatory) trials is more complex. For example, investigators can use accumulating data to increase the likelihood of a trial being conclusive by either re-powering the study based on the analysis of interim data (e.g., sample size re-estimation) or adapting the trial from a superiority study to a non-inferiority one, if necessary (e.g., hypothesis-adaptive designs). Other study types can be used to identify subgroups of patients who may benefit more than others from the treatment under study (e.g., biomarker-adaptive designs)—particularly when the overall study sample does not show strong evidence of treatment response. Finally, some types of designs may be useful to mitigate ethical concerns associated with either continuing trials despite strong evidence regarding safety or efficacy (e.g., adaptive group sequential designs) or allocating new subjects to potentially inferior treatment arms (e.g., response-adaptive randomization)
When choosing between different appropriate adaptive designs for confirmatory trials, methodological and interpretational concerns should also be considered. Moreover, regulatory authorities are generally more cautious about permitting a large number of adaptations in the later phases of drug development. It is thus advisable to limit the number of planned adaptations in such cases. Table 2 describes various types of adaptive designs appropriate for confirmatory trials along with their advantages and disadvantages.
Challenges of adaptive designs
Adaptive designs are subject to all the challenges that are inherent in traditional group sequential studies used to terminate trials early either for superiority, safety, or futility. However, they also introduce new risks due to their ability to affect the actual conduct of a trial.
Practical and project management challenges
Similar to how many studies are terminated too early for benefit with superiority demonstrated by sequential analysis of interim data [13, 15], there is the potential for adaptive trials to make modifications too early on in the trial and thus distract the trial from reaching a valid conclusion upon completion. For example, since early results often reflect wide variations due to the play of chance rather than actual treatment effects, early interim analyses often report exaggerated effects [1, 10]. There is thus a risk of adapting a trial’s design too early.
The confidentiality of interim data is another practical concern—particularly for commercial drug development, where sponsors need to have access to some interim data to make important business decisions, but without risking to disqualify the trial from regulatory approval [7]. On the other hand, observers who do not have access to interim data can often make an inference about the general trend in the data based on the knowledge contained in the protocol and observed adaptations [7].
Another practical challenge of adaptive design clinical trials is the proper management of drug supplies. Since investigational drugs are not usually readily available for mass distribution, periodic changes in treatment randomization ratios during the course of a study can lead to unexpectedly high demand for a particular treatment. If the manufacturing process is not ready to meet these demands, the proper functioning of the study can be jeopardized.
Regulatory concerns
Health Canada’s regulatory framework does not explicitly deal with drug marketing submissions based on adaptive design clinical studies, although they have published a document in 2008 summarizing their future plans to modernize their regulatory framework [9]. In the United States, the FDA’s draft guidance has clarified their position on the matter to a certain extent, but it remains to be settled whether or not the FDA ultimately approves of the use of adaptive designs using unblinded interim analyses (other than group sequential designs) in confirmatory trials [8]. The FDA also highlighted the importance of modeling and trial simulations in understanding the operating characteristics of adaptive trials under alternate scenarios. During the trial planning phase, comprehensive trial simulation data should be submitted to the FDA along with all other relevant trial documentation. The European Medicines Agency (EMA) has also released a reflection paper on adaptive designs [4]. In general, the tone of the FDA’s guidance document is more positive than that of the EMA [8].
Since adaptive designs are still relatively novel and regulatory guidance is still lacking in many countries, few clinical trials have successfully used adaptive designs. As of the writing of this article, there are no published systematic reviews comparing the success rates of adaptive design clinical trials to fixed-design ones. Still, a report published in 2006 by the Pharmaceutical Research and Manufacturers of America (PhRMA) stated that, as of 2002, adaptive designs had been applied in 46 studies reviewed by the FDA for approval purposes [18]. A more recent article by the Tufts Center for the Study of Drug Development reported that about one in five clinical trials in the United States are using simple adaptive designs [19]. The same report also suggested that the use of adaptive designs in exploratory-phase trials is expected to increase significantly in the near future.
Technological challenges
Adaptive studies require the effective use of digital technology such as electronic data capture (EDC) systems with real-time data access. Such systems need to be designed in a way that data can be reliably collected, cleaned, monitored, and analysed in a timely fashion as to not impede the proper conduct of the adaptive trial. Beyond the data management aspects of running a study, advanced Interactive Web Response Systems (IWRS) are also needed to manage the operational complexity of adaptive designs. These systems are designed to manage and integrate the operational aspects of clinical trials such as randomization and drug supply management. Since adaptive trials are designed in a way that allows for periodic modifications to trial conduct, each stage has the potential to be considerably different from previous ones in terms of the randomization features. In particular, the recruitment, blinding process, and drug supply (i.e., packaging, distribution, storage, usage) all have to be planned before the trial starts and then modified and revalidated as per protocol. Hence, a well-designed IWRS system is needed to foresee, plan for, and then manage all possible tasks, including possible adaptations.
From a statistical stand-point, the increasingly sophisticated methods applied to plan adaptive studies and subsequently analyse the data require equally sophisticated analytical software, particularly when it comes to the simulation of multiple adaptation scenarios.
Ethical concerns
When a trial design is adapted following interim analysis, it is understood that this will have an important impact on present and future patients. For example, when early evidence appears to support one treatment regimen over another, current subjects on sub-optimal regimens may be put on more optimal ones and newly recruited subjects may be initiated on similar regimens as well. However, if the premature conclusions were false, these patients may suffer in the long-term. Therefore, there is an ever-present dilemma as to whether it is more ethical to immediately alter study conduct based on early evidence to avoid exposing patients to ineffective or toxic regimens, or to wait for more evidence to avoid reassigning subjects to worse treatment courses.
Concluding remarks
Adaptive designs offer a diverse set of research tools that can help drug developers gain insights about their investigational products (e.g., promising drugs, highest effective and tolerable dose) with high efficiency during the early phases of clinical development. Such insights can be very useful in planning later-phase trials. Furthermore, other designs can be employed to address limitations that existed at the time of trial design that are no longer present once enough interim data is available. Thus, later-phase trials can be altered in course to reduce the incidence of inconclusive or failed studies.
Methodologies for planning adaptive design clinical studies and subsequently analyzing interim and final data are continuously under development. In fact, the literature is rich with statistical methods to deal with the added statistical uncertainties arising from adaptations. However, it will take some time before these methods are extensively put to use by researchers since there is still no global agreement on what the standard tools should be.
One of the most important challenges facing academia and industry today is to train a new cohort of biostatisticians to meet the latest requirements necessary for the practice of adaptive design clinical trials and to deal with the complex analytical procedures necessary in such studies. Although most of the clinical trial operations and statistical services are conducted by a contract research organization (CRO) with substantial expertise in statistical methods, these organizations need to quickly acquire the expertise and experience in dealing with the specific statistical methods, simulation techniques, and advanced outcome analysis approaches used in adaptive design clinical studies. In order to succeed in bringing new products to patients using adaptive designs, pharmaceutical companies need to maintain strong professional relationships with their CRO partners, and in some cases may need to find new CRO partners with the most up-to-date expertise. Indeed, it is important that it be emphasized that there is a great deal of multi-disciplinary collaboration needed to successfully plan and conduct adaptive trials. Sponsors must coordinate with a team of scientists, regulatory professionals, and statisticians with the requisite expertise who understand the increased operational demands of adaptive designs. The team should also have experience in the designing and execution of all phases of clinical trials as well as a proven track record of selecting and validating available methods and synchronizing them with regulatory and scientific needs.
In conclusion, there is evidence of an increase in the implementation of adaptive designs in clinical trials. Since regulatory agencies appear to be more comfortable with adaptive designs in the exploratory phases of drug development than in the confirmatory phases (provided that suitable methods were applied and were supplemented by rigorous simulation data at the planning phase), sponsors are encouraged to adopt these designs early on in their drug development process.
References
[1] Abraham, E., Reinhart, K., Opal, S., Demeyer, I., Doig, C., Rodriguez, A. L., Beale, R., Svoboda, P., Laterre, P. F., Simon, S., et al. Efficacy and safety of tifacogin (recombinant tissue factor pathway inhibitor) in severe sepsis: a randomized controlled trial. JAMA 290, 2 (2003), 238-247.
[2] Chow, S.-C., and Chang, M. Adaptive design methods in clinical trials-a review. Orphanet J Rare Dis 3, 11 (2008), 169-90.
[3] Chow, S.-C., and Chang, M. Adaptive design methods in clinical trials. CRC press, 2011.
[4] EMEA/CHMP. Reflection paper on methodological issues in confirmatory trials with flexible design and analysis plan (draft chmp/ewp/2459/02, 23-mar-2006), 2007. Retrieved from: http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2009/09/WC500003616.pdf.
[5] FDA. Innovation or stagnation: challenge and opportunity on the critical path to new medical products, 2004. Retrieved from: http://www.fda.gov/downloads/ScienceResearch/SpecialTopics/CriticalPathInitiative/CriticalPathOpportunitiesReports/ucm113411.pdf.
[6] FDA. Adaptive design clinical trials for drug and biologics draft guidance, 2010. Retrieved from: http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/ucm201790.pdf.
[7] Gallo, P., Chuang-Stein, C., Dragalin, V., Gaydos, B., Krams, M., and Pinheiro, J. Adaptive designs in clinical drug development—an executive summary of the phrma working group. Journal of biopharmaceutical statistics 16, 3 (2006), 275-283.
[8] He, W., Pinheiro, J., and Kuznetsova, O. M. Practical Considerations for Adaptive Trial Design and Implementation. Springer, 2014.
[9] Health-Canada. Clinical trials regulatory review: targeted measures for a strengthened framework, March 2008. Retrieved from: http://www.hc-sc.gc.ca/dhp-mps/alt_formats/hpfb-dgpsa/pdf/prodpharma/ct_regrev_ce_exareg-eng.pdf.
[10] Ioannidis, J. P. Contradicted and initially stronger effects in highly cited clinical research. JAMA 294, 2 (2005), 218-228.
[11] Kang, X., Bai, Y., and Li, W. Adaptive design. Transl Perioper & Pain Med 1, 2 (2016), 1-3.
[12] Lang, E. Adaptive trial designs: improving the efficiency of drug development. Covance, 2015. Retrieved from: http://www.covance.com/content/dam/covance/assetLibrary/whitepapers/Applied%20Clinical%20Trials-WPCDS005.pdf.
[13] Montori, V. M., Devereaux, P., Adhikari, N. K., Burns, K. E., Eggert, C. H., Briel, M., Lacchetti, C., Leung, T. W., Darling, E., Bryant, D. M., et al. Randomized trials stopped early for benefit: a systematic review. JAMA 294, 17 (2005), 2203-2209.
[14] Pan, H., Xie, F., Liu, P., Xia, J., and Ji, Y. A phase i/ii seamless dose escalation/expansion with adaptive randomization scheme (sears). Clinical Trials 11, 1 (2014), 49-59.
[15] Pocock, S. J. When to stop a clinical trial. BMJ 305, 6847 (1992), 235-240.
[16] Pocock, S. J. When (not) to stop a clinical trial for benefit. JAMA 294, 17 (2005), 2228-2230.
[17] Pocock, S. J., Clayton, T. C., and Stone, G. W. Challenging issues in clinical trial design: part 4 of a 4-part series on statistics for clinical trials. Journal of the American College of Cardiology 66, 25 (2015), 2886-2898.
[18] PPD. Implementing adaptive trial design: operational considerations and the role of the cro, 2012. Retrieved from: http://www.ppdi.com/-/media/Files/PPDI-Files/Expert-Community/Whitepapers/ImplementingAdaptiveTrialDesign.pdf.
[19] Tufts-CSDD. The adoption and impact of adaptive trial designs, May 2013. Retrieved from: http://csdd.tufts.edu/files/uploads/tuftscsddbrief1final_new.pdf.
About Vantage BioTrials
Vantage BioTrials is a leading Canadian Contract Research Organization (CRO) that uses innovative clinical trial management strategies for the life science industry with a focus on patient safety & advancing new therapies to market. We offer pharmaceutical, biotech & medical device companies a complete and integrated set of full-service clinical trial management solutions.
Contact us today at [email protected] and discover how we can add value to your next clinical trial program.