Adaptive Therapy Optimization via Surrogate Tumor Digital Twins and Deep Reinforcement Learning
PI: Soundar Kumara (Industrial Engineering)
Overview: Tumor Digital Twins (TDTs) are dynamic, virtual counterparts of a patient’s tumor, designed to simulate its response under various conditions and treatment scenarios. They offer a platform to test therapeutic hypotheses in silico before applying them in vivo, thereby enabling personalized medicine and potentially improving patients’ outcomes. However, the optimization of such adaptive cancer therapies faces significant hurdles due to tumor heterogeneity and complex evolutionary dynamics, which require running simulation models with millions of parameters. Such “curse of dimensionality” obstructs efficient exploration of the optimal treatment solution through conventional computational approaches, thus limiting the practical application of personalized oncology. This proposed project aims to address the aforementioned computational bottleneck by developing physically and biologically informed surrogate Tumor Digital Twins (TDTs) integrated with Deep Reinforcement Learning (DRL) for adaptive therapy optimization. Specifically, a large-scale TDT dataset will be curated by running high-fidelity Agent-Based Models (ABMs) for simulating the tumor microenvironment, which will be utilized to train surrogate models as fast approximations of the TDTs. These surrogate models enable efficient representation of tumor dynamics across diverse states, thereby facilitating the deployment of Deep Reinforcement Learning for Adaptive Therapy Optimization.
Methodology:
(1) Foundational Layer Establishment: Develop robust, high-fidelity AgentBased Models (ABMs) for Tumor Digital Twins (TDTs) that capture cellular heterogeneity and the spatio-temporal dynamics of the tumor microenvironment. The results of these sophisticated ABMs will serve as the mechanistic core of developing the following surrogate models and Deep Reinforcement Learning agent.
(2) AI-Accelerated Surrogate Modeling for Tumor Digital Twins: Create computationally efficient, biologically and physically informed surrogate models by utilizing the results from high-fidelity AMBs as the training data.
(3) Deep Reinforcement Learning (DRL) for Adaptive Therapy Optimization: Train DRL agents to discover optimal treatment strategies by interacting with the surrogate-enhanced TDT environment.
Computational Resources and Workflow: We will leverage the HPE Frontier supercomputer at Oak Ridge National Laboratory and our GPU-accelerated platform to run large-scale ABMs to collect massive datasets for robust surrogate model training. Simulations are expected to run concurrently on over 1,000 GPUs. Surrogate model training and deep reinforcement learning (DRL) will leverage Penn State’s Roar supercomputing infrastructure (expected NVIDIA A100/H100 GPUs, 80GB memory), with distributed training applied as needed. The surrogate will operate on GPU to efficiently handle large 3D tensors and enable fast transitions in the simulated environment, while the lightweights policy network will also reside on GPU to support highthroughput parallel simulations.We will implement the system in Python, using PyTorch for surrogate training and RL libraries such as Stable-Baselines3 or RLlib to run PPO or DDPG with a custom environment.
Alignment with ICDS Mission: This project directly supports the ICDS mission by uniting deep learning, high-performance computing, and biological sciences to address critical challenges in precision oncology. It exemplifies interdisciplinary collaboration, leveraging computational methods and domain expertise to drive transformative research in computational health.