A Digital Twin–Reinforcement Learning Framework for Adaptive, Patient-Specific Antineoplastic Drug Dosing
Abstract
Background. Conventional fixed-dose chemotherapy protocols fail to account for inter-patient variability in drug pharmacokinetics, tumour biology, and immune response. This results in subtherapeutic exposures or dose-limiting toxicities that compromise both efficacy and quality of life.
Methods. We present a Digital Twin–Reinforcement Learning (DT-RL) framework that couples a physiologically-based pharmacokinetic (PBPK) model with a Gompertz tumour growth ordinary differential equation and a Soft Actor-Critic (SAC) reinforcement learning agent. The system continuously updates a patient-specific virtual model and learns adaptive dosing policies that maximise tumour control while enforcing haematological safety constraints.
Results. In a 180-day in silico trial across three virtual patient profiles, the RL-optimised regimen achieved ≥99.2% tumour volume reduction in all patients, reduced cumulative tumour burden (AUC) by 42–44% relative to standard fixed-dose protocols, and maintained absolute neutrophil count above the safety threshold throughout therapy. Time-in-range compliance reached 98.1% and zero constraint violations were recorded.
Conclusions. The DT-RL framework demonstrates that real-time, model-guided adaptive dosing can substantially outperform population-average protocols in silico. These findings motivate prospective clinical validation and establish a computational blueprint for precision oncology decision support.
Introduction
Cancer remains one of the leading causes of mortality worldwide, with over 19 million new cases annually. Systemic chemotherapy, despite decades of refinement, continues to rely on population-averaged dosing protocols that fail to capture the profound heterogeneity in patient pharmacokinetics, tumour biology, and immune reserve. The consequences are clinically significant: subtherapeutic dosing permits tumour regrowth and resistance, while excessive dosing precipitates haematological toxicity, organ damage, and treatment discontinuation.
The concept of personalised medicine has long been aspirational in oncology, yet its implementation in routine clinical practice remains constrained by the absence of real-time, patient-specific computational tools capable of integrating multimodal physiological data and translating them into actionable dosing decisions. Physiologically-based pharmacokinetic (PBPK) modelling offers a mechanistic foundation for such tools, but static PBPK models cannot adapt to the dynamic, non-linear evolution of tumour response and patient physiology over a treatment course.
Digital Twins — continuously updated virtual replicas of individual patients — have emerged as a transformative paradigm in precision medicine. When coupled with machine learning, particularly reinforcement learning (RL), Digital Twins can serve not merely as predictive simulators but as active decision-support agents that learn optimal treatment strategies through trial-and-error interaction with the virtual patient model.
Reinforcement learning has demonstrated compelling results in sequential decision-making problems across medicine, including sepsis management, radiotherapy planning, and HIV treatment scheduling. However, its application to adaptive chemotherapy dosing — where the action space is continuous, the reward signal is delayed, and safety constraints are non-negotiable — demands a carefully designed framework that balances exploration with constraint satisfaction.
This work presents a complete, integrated DT-RL framework for antineoplastic drug dosing. We describe the mathematical architecture of the Digital Twin, the formulation of the RL problem as a constrained Markov Decision Process, the three-tier safety architecture, and the results of a 180-day in silico trial. We further contextualise our findings within the broader landscape of RL-guided dosing literature and discuss the translational pathway toward clinical deployment.
Methods
3.1 Digital Twin Architecture
The Digital Twin (DT) is a patient-specific computational model comprising three tightly coupled subsystems: (i) a multi-compartment PBPK model governing drug distribution and elimination; (ii) a Gompertz ordinary differential equation (ODE) representing tumour growth dynamics modulated by drug effect; and (iii) a haematological toxicity model tracking absolute neutrophil count (ANC) as a proxy for bone marrow suppression.
3.2 Physiologically-Based Pharmacokinetic (PBPK) Model
Drug disposition is described by a five-compartment PBPK model: plasma, tumour, liver, kidney, and peripheral tissue. Mass-balance ODEs govern drug transport between compartments according to blood flow rates, tissue volumes, and partition coefficients calibrated to literature-reported values for representative antineoplastic agents.
The tumour compartment drug effect is modelled as:
E(C) = Emax·C / (EC50 + C)
Gompertz growth ODE with Emax pharmacodynamic kill term. V = tumour volume (cm³); r = intrinsic growth rate; K = carrying capacity; C = drug concentration.
| Parameter | Symbol | Value | Source |
|---|---|---|---|
| Plasma volume | Vp | 3.0 L | PBPK literature |
| Tumour volume | VT | 0.5 L | Estimated |
| Elimination rate constant | kel | 0.18 h⁻¹ | Atezolizumab PK |
| Gompertz growth rate | r | 0.012 day⁻¹ | Fitted |
| Carrying capacity | K | 100 cm³ | Clinical data |
| Emax | Emax | 0.045–0.050 day⁻¹ | Patient-specific |
| EC50 | EC50 | 1.2 µg/mL | Literature |
3.3 Reinforcement Learning Formulation
The dosing optimisation problem is formulated as a constrained Markov Decision Process (CMDP) with the following components:
- State space S: Tumour volume, plasma drug concentration, ANC, day of cycle, cumulative dose, and rate of change of tumour volume — a 6-dimensional continuous state vector updated daily.
- Action space A: Continuous scalar dose in [0, Dmax], where Dmax = 150 mg. The SAC agent outputs a Gaussian-distributed action clipped to this range.
- Reward function R: Composite reward balancing tumour kill (positive), ANC maintenance (positive), dose efficiency (negative penalty for excessive dosing), and hard safety violations (large negative penalty).
- Constraint: ANC ≥ 1,500 cells/µL at all times; violations trigger a −100 reward penalty and dose suspension.
Composite reward function. Weights: w₁=1.0 (tumour kill), w₂=0.5 (ANC penalty), w₃=0.1 (dose efficiency), w₄=100 (hard constraint violation).
3.4 Soft Actor-Critic (SAC) Algorithm
SAC was selected as the primary RL algorithm on the basis of its maximum-entropy objective, which promotes robust exploration, and its demonstrated superiority in continuous-action domains. SAC optimises a policy π that maximises the expected cumulative reward augmented by an entropy term:
SAC maximum-entropy objective. α = temperature parameter controlling exploration-exploitation balance; H = policy entropy.
3.5 Three-Tier Safety Architecture
A hierarchical safety system operates at three levels to prevent unsafe dose recommendations from reaching the virtual patient:
- Pharmacological layer: Hard dose bounds enforced by action clipping (0–150 mg).
- Haematological layer: ANC-based dose hold — if predicted ANC falls below 1,500 cells/µL, dose is set to zero regardless of RL output.
- Protocol layer: Minimum inter-dose interval (48 h) and maximum cycle dose cap enforced by rule-based override.
3.6 Virtual Patient Cohort
Three virtual patient profiles were defined to represent the spectrum of clinical phenotypes encountered in practice:
| Patient | Initial Tumour Vol. | Drug Sensitivity (Emax) | Clearance | Clinical Phenotype |
|---|---|---|---|---|
| P1 | 20 cm³ | 0.045 day⁻¹ | Standard | High tumour burden, average responder |
| P2 | 12 cm³ | 0.042 day⁻¹ | High | Low burden, favourable responder |
| P3 | 15 cm³ | 0.044 day⁻¹ | Slow | Intermediate burden, partial responder |
Results
All simulations were run for 180 days. The RL agent was trained for 500 episodes prior to evaluation. Results are reported as mean ± range across the three virtual patient profiles unless otherwise stated.
4.1 Primary Outcome: Tumour Volume Dynamics
The animated comparison below shows tumour volume trajectories over 180 days for RL-optimised dosing (left panel) versus fixed-dose protocol (right panel) across all three virtual patients. The RL agent consistently achieves earlier and deeper tumour suppression, particularly in P1 (highest initial burden).
4.2 Cumulative Tumour Burden (AUC)
Cumulative tumour burden, quantified as the area under the tumour volume–time curve (AUC, cm³·day), was reduced by 43.0% (P1), 42.2% (P2), and 43.8% (P3) in the RL arm relative to fixed-dose controls. This metric captures integrated disease burden over the entire treatment course and is a stronger predictor of long-term outcomes than endpoint tumour volume alone.
4.3 Dosing Efficiency and Safety Metrics
4.4 Primary Results Summary
| Metric | P1 RL | P1 Fixed | P2 RL | P2 Fixed | P3 RL | P3 Fixed |
|---|---|---|---|---|---|---|
| Tumour reduction (%) | 99.5 | 99.5 | 99.2 | 99.2 | 99.3 | 99.3 |
| Cumulative dose (mg) | 95.6 | 55.7 | 117.7 | 68.6 | 105.9 | 61.7 |
| AUC (cm³·day) | 916 | 1,608 | 565 | 977 | 692 | 1,231 |
| AUC reduction (%) | 43.0% | — | 42.2% | — | 43.8% | — |
| ANC violations | 0 | 3 | 0 | 2 | 0 | 4 |
| TIR compliance (%) | 98.3 | 87.2 | 98.1 | 89.4 | 97.9 | 86.1 |
Discussion
The DT-RL framework presented here demonstrates that a computationally tractable, patient-specific dosing system can achieve substantial improvements in tumour control and safety outcomes relative to fixed-dose protocols in a simulated environment. The 43% mean reduction in cumulative tumour burden is particularly noteworthy, as it reflects not merely endpoint tumour size but the integrated disease burden that drives metastatic risk and treatment-related morbidity.
5.1 Algorithm Selection and Performance
The selection of SAC as the primary RL algorithm was motivated by its maximum-entropy formulation, which prevents premature policy convergence and maintains sufficient exploration to discover dose schedules that balance efficacy with toxicity avoidance. Among the six algorithms evaluated (DQN, DDPG, TD3, SAC, PPO, TRPO), SAC exhibited the best combination of convergence stability, sample efficiency, and constraint satisfaction rate. Proximal Policy Optimisation (PPO) performed comparably in terms of tumour control but showed higher variance in ANC compliance during early training episodes.
5.2 Nanoparticle-Mediated Delivery and the Digital Twin
A clinically relevant extension of the DT-RL framework is its integration with nanoparticle-mediated drug delivery systems. Nanoparticle formulations — including liposomes, polymeric nanoparticles, and antibody-drug conjugates — offer tunable pharmacokinetic profiles that can be parameterised within the PBPK model. The Digital Twin can incorporate nanoparticle-specific parameters (release kinetics, EPR effect magnitude, surface ligand binding affinity) to predict tumour-site drug concentrations with greater fidelity than conventional PK models.
The RL agent, informed by nanoparticle-specific PK predictions, can optimise not only the dose magnitude but also the timing of nanoparticle infusion relative to the tumour vascular permeability cycle — a dimension of personalisation inaccessible to conventional dosing protocols. This positions the DT-RL framework as a natural computational substrate for next-generation nanomedicine clinical trials.
5.3 Comparison with Prior Work
Our results are consistent with, and extend, prior computational studies of RL-guided dosing. Komorowski et al. (2018) demonstrated that RL could outperform physicians in ICU sepsis management; our work applies analogous principles to the oncology domain with the additional complexity of tumour dynamics and haematological constraints. Wu et al. (2022) reported a DT-based pCR prediction AUC of 0.88 for triple-negative breast cancer — our framework extends this predictive capability to a closed-loop adaptive dosing system. The federated learning literature (Haas et al., 2023; McMahan et al., 2017) provides a roadmap for multi-institutional deployment of the DT-RL system while preserving patient privacy.
5.4 Clinical Translation Pathway
Translation of the DT-RL framework to clinical practice requires a structured validation pathway. We propose a three-phase approach: (1) retrospective validation against electronic health record data from completed chemotherapy trials; (2) prospective observational deployment as a decision-support tool with clinician override capability; and (3) adaptive platform trial with pre-specified RL arm and Bayesian outcome monitoring. Regulatory engagement with the FDA's Software as a Medical Device (SaMD) framework and the EU's MDR/IVDR classification will be required from the outset.
Nanoparticle-Based Drug Delivery
How engineered nanocarriers interface with the Digital Twin PBPK model to enable targeted, stimulus-responsive antineoplastic drug delivery — reducing systemic toxicity while maximising tumour drug exposure.
Enhanced Permeability & Retention (EPR) Effect
Passive TargetingSolid tumours exhibit abnormal vasculature with large endothelial gaps (200–1200 nm) and impaired lymphatic drainage. Nanoparticles in the 10–200 nm size range selectively extravasate through these gaps and accumulate within the tumour interstitium — a phenomenon known as the EPR effect. Unlike free drug molecules that distribute non-specifically throughout the body, nanocarriers exploit EPR for passive tumour targeting, achieving 4–6× higher tumour drug concentrations while reducing systemic exposure.
Tumour neovasculature has 200–1200 nm fenestrations vs 5–10 nm in normal vessels — enabling selective nanoparticle extravasation.
Impaired lymphatic clearance in tumours prolongs nanoparticle retention — extending drug exposure at the tumour site for 48–72 hours.
EPR-mediated accumulation is modelled as an enhanced tumour-compartment uptake rate (k_epr) in the PBPK framework, calibrated per patient.
Nanocarrier Platforms for Antineoplastic Delivery
Platform TypesPhospholipid bilayer vesicles (80–200 nm) that encapsulate hydrophilic drugs in the aqueous core and hydrophobic drugs in the lipid bilayer. PEGylation extends circulation half-life to 48–72h. The most clinically advanced nanocarrier platform with FDA-approved formulations.
Biodegradable polymer matrices (50–300 nm) enabling controlled drug release via hydrolytic degradation. Release kinetics are tuneable by adjusting polymer molecular weight and composition — directly parameterisable in the PBPK model as a time-varying release function.
10–100 nm gold cores with surface-conjugated drug molecules. Near-infrared laser irradiation triggers photothermal drug release specifically at tumour sites — enabling spatially controlled drug delivery that can be synchronised with RL dosing cycles.
Highly branched, tree-like polymers (1–10 nm) with precise molecular weight and multivalent surface functionality. Ideal for surface conjugation of targeting ligands (folate, transferrin) and imaging agents — enabling theranostic applications within the DT framework.
Superparamagnetic iron oxide nanoparticles (5–50 nm) guided to tumour sites by external magnetic fields. Enable MRI-based Digital Twin state estimation — providing real-time tumour volume and drug distribution data to update PBPK model parameters.
Ionisable lipid-based carriers (60–100 nm) for nucleic acid delivery — enabling gene silencing (siRNA) or therapeutic protein expression (mRNA) in tumour cells. Validated by COVID-19 vaccine success; now being adapted for oncology applications.
Active Targeting Ligands
PrecisionSurface-functionalised nanoparticles bearing tumour-specific ligands achieve receptor-mediated endocytosis — concentrating drug payload specifically within cancer cells overexpressing the target receptor.
FR-α overexpressed in 40% of human cancers (ovarian, breast, lung). Folate-conjugated NPs achieve 10–20× higher tumour uptake vs untargeted NPs.
Trastuzumab-coated NPs for HER2+ breast cancer — combine receptor blockade with intracellular drug delivery for synergistic anti-tumour effects.
RGD peptide-functionalised NPs target αvβ3 integrin overexpressed on tumour neovasculature — enabling anti-angiogenic drug delivery.
Stimulus-Responsive Release
Smart NPsSmart nanocarriers release their drug payload only when triggered by tumour-specific microenvironmental cues — minimising premature drug release in healthy tissues and maximising tumour-site drug concentrations.
Tumour microenvironment pH 6.5–6.8 vs systemic pH 7.4. Acid-labile linkers (hydrazone, acetal) trigger drug release selectively at tumour site.
LCST-polymer coatings (e.g. PNIPAM) collapse above 41°C, releasing payload during mild hyperthermia — synchronisable with RL dosing cycles.
Matrix metalloproteinase-cleavable peptide linkers release drug in response to MMP-2/9 overexpressed in tumour stroma — achieving highly selective intratumoral release.
Integration with the Digital Twin-RL Framework
DT-NP InterfaceNanoparticle-based delivery fundamentally alters the pharmacokinetic profile of antineoplastic agents — introducing new parameters that must be captured within the PBPK model and communicated to the RL agent. In the DT-RL framework, nanoparticle characteristics are encoded as additional state variables, enabling the SAC agent to account for carrier-mediated pharmacokinetics when making dosing decisions.
Nanoparticle-specific PBPK parameters added to the model: EPR uptake rate (k_epr), carrier degradation rate (k_deg), ligand binding affinity (K_d), and stimulus-release threshold (pH / T). These extend the 5-compartment model to a 7-compartment NP-PBPK architecture.
The SAC state vector is augmented with NP-specific observables: tumour pH estimate, circulating NP concentration, EPR accumulation index, and MMP-2/9 activity biomarker — enabling the agent to make NP-aware dosing decisions in real time.
The RL reward function is modified to include a nanoparticle efficiency term: rewarding high tumour-to-plasma drug ratios (selectivity index) and penalising premature NP degradation in systemic circulation — steering the agent towards NP-optimal dosing strategies.
The convergence of nanoparticle engineering and Digital Twin-enabled RL represents a next-generation theranostic paradigm — where the carrier, the drug, and the dosing strategy are co-optimised in silico before any clinical administration, dramatically reducing the risk of treatment failure and off-target toxicity.