Multi-Armed Bandit Problems for Assignment of Patients in Clinical Trials

Mentors:
Lead: Andrew Papanicolaou (Mathematics, NCSU)

Intellectual merit and significance:
Multi-armed bandits (MABs) are a class of problems that have proven effective for attaining optimal exploration-exploitation tradeoffs in online learning. In clinical trials, MABs can be used for assigning subjects to specific treatments, with each treatment serving as an arm [2]. The goal of the trial is to determine the best treatment by observing successes and failures among the subjects. However, the subjects in clinical trials are also patients, and hence there are ethical concerns if a newly entering subject is assigned to a low-performing treatment primarily to increase exploration. Thus, MABs for clinical trials should be designed with a criterion that is suitable to the needs of patients as well as to developers of treatments.

Objectives:
(i) Study the implementation of bandit algorithms, such as Upper Confidence Bound (UCB) [1], in simulated clinical trials. (ii) Investigate the effective size of studies when conducting a trial with an ethical MAB algorithm, e.g., finding the minimum number of recruits needed for statistical accuracy. (iii) Explore new MABs utilizing survival-function estimation [3] in long-term clinical trials with censored subjects.

Outcomes:
Novel contribution toward the improvement of statistical accuracy in clinical trials, with prioritization of patient well-being. Innovation of an MAB algorithm specifically tailored to studies where long-term patient survival is the goal.

References:
1. Lai T, Robbins H, “Asymptotically efficient adaptive allocation rules”, Advances in Applied Mathematics, Volume 6, Issue 1, 1985, Pages 4-22, ISSN 0196-8858, https://doi.org/10.1016/0196-8858(85)90002-8.

2. Villar S, Bowden J, Wason J, “Multi-armed Bandit Models for the Optimal Design of Clinical Trials: Benefits and Challenges.” Statist. Sci. 30 (2) 199 – 215, May 2015. https://doi.org/10.1214/14-STS504

3. Goel MK, Khanna P, Kishore J. “Understanding survival analysis: Kaplan-Meier estimate”. Int J Ayurveda Res. 2010 Oct;1(4):274-8. doi: 10.4103/0974-7788.76794. PMID: 21455458; PMCID: PMC3059453.