Optimization Online


Robust Markov Decision Processes for Medical Treatment Decisions

Yuanhui Zhang(yuanhui.zhang***at***gmail.com)
Brian T. Denton(btdenton***at***umich.edu)

Abstract: Medical treatment decisions involve complex tradeoffs between the risks and benefits of various treatment options. The diversity of treatment options that patients can choose over time, and uncertainties in future health outcomes result in a difficult sequential decision making problem. Markov decision processes (MDPs) are commonly used to study medical treatment decisions; however, optimal policies obtained by solving MDPs may be affected by the uncertain nature of the model parameter estimates. In this article, we present a robust Markov decision process treatment model (RMDP-TM) with a controllable uncertainty set formulation for the transition probability matrices (TPMs) of the underlying Markov chain. We show that the RMDP-TM can overcome the common problem of over-conservativeness of the worst-case optimal policy obtained from the RMDP model with fixed uncertainty set formulations for TPMs. We present theoretical analysis to establish computationally efficient methods to solve the RMDP-TM and present its application to a medical treatment decision problem of optimizing the sequence and the start time to initiate medications for glycemic control for patients with type 2 diabetes.

Keywords: robust optimization; robust Markov decision process; medical decision making; type 2 diabetes; glycemic control

Category 1: Applications -- Science and Engineering

Category 2: Applications -- Science and Engineering (Biomedical Applications )

Category 3: Robust Optimization

Citation: Yuanhui Zhang, Graduate Program of Operations Research, North Carolina State University, Raleigh, NC 27695, yuanhui.zhang@gmail.com, Brian T. Denton, Department of Industrial and Operations Engineering, University of Michigan, Ann Arbor, MI 48109, btdenton@umich.edu, October 1, 2015

Download: [PDF]

Entry Submitted: 10/01/2015
Entry Accepted: 10/01/2015
Entry Last Modified: 10/01/2015

Modify/Update this entry

  Visitors Authors More about us Links
  Subscribe, Unsubscribe
Digest Archive
Search, Browse the Repository


Coordinator's Board
Classification Scheme
Give us feedback
Optimization Journals, Sites, Societies
Mathematical Optimization Society