Optimization Online


An Analysis of Structured Optimal Policies for Hypertension Treatment Planning: The Tradeoff Between Optimality and Interpretability

Gian-Gabriel Garcia (giangarcia***at***gatech.edu)
Lauren Steimle (steimle***at***gatech.edu)
Wesley Marrero (wmarrerocolon***at***mgh.harvard.edu)
Jeremy Sussman (jeremysu***at***med.umich.edu)

Abstract: Problem definition: In medical decision-making, Markov Decision Processes (MDPs) are useful for deriving optimal treatment policies when a patient's health evolves stochastically over time. Yet, optimal policies may lack structure that is interpretable to human decision-makers. When interpretability is valued by practitioners, suboptimal yet interpretable policies may be preferred over uninterpretable optimal policies. Interpretability is especially critical in hypertension treatment, where complicated clinical guidelines have drawn substantial controversy from practitioners. In this research, we design and analyze a class of interpretable policies for MDPs which leverage the natural interpretability of monotonicity, i.e., the intensity of the prescribed action increases with the severity of the state. Methodology/results: We present mixed integer programming formulations to obtain the optimal monotone policy and class-ordered monotone policy (CMP). The novel CMP generalizes monotone policies by imposing monotonicity over classes of states and actions rather than states and actions themselves. We show that the optimal monotone policy can be found in polynomial time, but the degree is non-trivial. We then analyze the performance gap of monotone policies and CMPs relative to the optimal policy using the price of interpretability (PI). Under mild conditions, we prove that the CMP achieves a PI no greater than the optimal monotone policy. Finally, we demonstrate the practicality of these methods for hypertension treatment and derive patient-level and population-level insights. Overall, the CMP’s flexibility allows it to outperform the monotone policy, achieving the greatest benefit for patients with stage 2 hypertension. Nevertheless, both interpretable policies retain low PIs and save over 3,200 quality-adjusted life years and prevent nearly 300 cardiovascular events over clinical guidelines while retaining clinically intuitive structure. Managerial implications: CMPs retain the interpretability of monotone policies while achieving superior performance. In practical applications, CMPs can provide more instinctive strategies than the optimal policy with minimal losses in performance.

Keywords: Markov decision processes, healthcare applications, medical decision-making, interpretability, cardiovascular disease, personalized treatment planning

Category 1: Applications -- OR and Management Sciences

Category 2: Other Topics (Dynamic Programming )

Category 3: Applications -- Science and Engineering

Citation: Institution Address: H. Milton Stewart School of Industrial and Systems Engineering, 765 Ferst Dr NW, Atlanta, GA 30332 Month/Year: August/2021

Download: [PDF]

Entry Submitted: 08/09/2021
Entry Accepted: 08/09/2021
Entry Last Modified: 08/09/2021

Modify/Update this entry

  Visitors Authors More about us Links
  Subscribe, Unsubscribe
Digest Archive
Search, Browse the Repository


Coordinator's Board
Classification Scheme
Give us feedback
Optimization Journals, Sites, Societies
Mathematical Optimization Society