An algorithm for solving infinite horizon Markov dynamic programmes
Abstract: We consider a general class of infinite horizon dynamic programmes where state and control sets are convex and compact subsets of Euclidean spaces and (convex) costs are discounted geometrically. The aim of this work is to provide a convergence result for these problems under as few restrictions as possible. Under certain assumptions on the cost functions, infinite horizon cost-to-go functions can be bounded by a pair of convex, Lipschitz-continuous bounding functions; we seek to refine these bounding functions until an epsilon convergence criteria is met. We prove a convergence result for a simplified version of our problem, and then apply this result for the stochastic version problem where uncertainty is governed by a discrete Markov process. Further, our algorithm is deterministic and requires no Monte-carlo simulation to estimate an upper bound on the cost of a given policy.
Keywords: dynamic programming, decomposition, multistage, stochastic programming, infinite horizon
Category 1: Stochastic Programming
Category 2: Other Topics (Dynamic Programming )
Category 3: Convex and Nonsmooth Optimization (Convex Optimization )
Citation: University of Auckland, 9th of April 2018.
Entry Submitted: 04/08/2018
Modify/Update this entry
|Visitors||Authors||More about us||Links|
Search, Browse the Repository
Give us feedback
|Optimization Journals, Sites, Societies|