Optimization Online


Random-Sampling Multipath Hypothesis Propagation for Cost Approximation in Long-Horizon Optimal Control

Shankarachary Ragi (shankarachary.ragi***at***sdsmt.edu)
Hans D. Mittelmann (mittelmann***at***asu.edu)

Abstract: In this paper, we develop a Monte-Carlo based heuristic approach to approximate the objective function in long horizon optimal control problems. In this approach, we evolve the system state over multiple trajectories into the future while sampling the noise disturbances at each time-step, and find the weighted average of the costs along all the trajectories. We call these methods random sampling - multipath hypothesis propagation or RS-MHP. These methods (or variants) exist in the literature; however, the literature lacks - a) approaches on how to contain the exponential computational growth typically observed in these methods; b) performance guarantees. This paper fills these knowledge gaps to an extent. We derive convergence results for the cost approximation error from the MHP methods and discuss their convergence (in probability) as the sample size increases. As a case study, we apply MHP to approximate the cost function in a linear quadratic control problem and demonstrate the benefits of MHP approximation against an existing and closely related approximation approach called nominal beliefstate optimization.

Keywords: Long horizon optimal control, cost approximation, approximate dynamic programming, multipath hypothesis propagation.

Category 1: Applications -- Science and Engineering

Category 2: Applications -- Science and Engineering (Control Applications )


Download: [PDF]

Entry Submitted: 02/09/2020
Entry Accepted: 02/09/2020
Entry Last Modified: 07/03/2020

Modify/Update this entry

  Visitors Authors More about us Links
  Subscribe, Unsubscribe
Digest Archive
Search, Browse the Repository


Coordinator's Board
Classification Scheme
Give us feedback
Optimization Journals, Sites, Societies
Mathematical Optimization Society