Optimization Online


An algorithm for solving infinite horizon Markov dynamic programmes

Regan Baucke(r.baucke***at***auckland.ac.nz)

Abstract: We consider a general class of infinite horizon dynamic programmes where state and control sets are convex and compact subsets of Euclidean spaces and (convex) costs are discounted geometrically. The aim of this work is to provide a convergence result for these problems under as few restrictions as possible. Under certain assumptions on the cost functions, infinite horizon cost-to-go functions can be bounded by a pair of convex, Lipschitz-continuous bounding functions; we seek to refine these bounding functions until an epsilon convergence criteria is met. We prove a convergence result for a simplified version of our problem, and then apply this result for the stochastic version problem where uncertainty is governed by a discrete Markov process. Further, our algorithm is deterministic and requires no Monte-carlo simulation to estimate an upper bound on the cost of a given policy.

Keywords: dynamic programming, decomposition, multistage, stochastic programming, infinite horizon

Category 1: Stochastic Programming

Category 2: Other Topics (Dynamic Programming )

Category 3: Convex and Nonsmooth Optimization (Convex Optimization )

Citation: University of Auckland, 9th of April 2018.

Download: [PDF]

Entry Submitted: 04/08/2018
Entry Accepted: 04/09/2018
Entry Last Modified: 04/08/2018

Modify/Update this entry

  Visitors Authors More about us Links
  Subscribe, Unsubscribe
Digest Archive
Search, Browse the Repository


Coordinator's Board
Classification Scheme
Give us feedback
Optimization Journals, Sites, Societies
Mathematical Optimization Society