Séance Séminaire

Séminaire de Probabilités et Statistique

Deep Reinforcement Learning for Bayes-Adaptive Impulse Control of PDMPs for Cancer Patient Follow-Up

Cancer management requires long-term monitoring and adaptive treatment, as the disease typically alternates between remission and relapse. Clinical decisions often rely on the evolution of biological markers, which are only partially observable, while the underlying disease dynamics remain uncertain. Designing effective treatment policies under such conditions poses a significant challenge. To address this, we propose a Bayes-adaptive control framework based on Piecewise-Deterministic Markov Processes (PDMPs), a class of continuous-time hybrid models that that captures both the deterministic and stochastic elements of disease evolution. PDMPs provide a compact yet flexible representation of disease evolution using a limited number of interpretable parameters. We formulate the optimal follow-up and treatment problem as a hybrid-state Bayes-Adaptive Partially Observable Markov Decision Process (BAPOMDP). This formulation explicitly incorporates model and state uncertainty by augmenting the latent state space with beliefs over both disease states and unknown parameters. The resulting model enables offline policies computation, without requiring prior interaction with the patient population, by accounting for uncertainty in disease dynamics and observation processes. Since computing exact optimal policies in such high-dimensional hybrid models is intractable, we employ simulation-based deep reinforcement learning (DRL) algorithms to approximate effective control strategies. The learned policies aim to maximize patient survival while minimizing treatment costs and side effects, balancing long-term outcomes and treatment burden. Numerical experiments in a patient follow-up scenario demonstrate the feasibility and robustness of the proposed approach. The results highlight the potential of Bayes-adaptive deep reinforcement learning for personalized and uncertainty-aware decision-making in medical domains characterized by partial observability and complex disease dynamics.

Séminaire en salle 109, également retransmis sur zoom : https://umontpellier-fr.zoom.us/j/7156708132