The purpose of this article is to establish epigraphical convergence of the sample averages of a random lower semi-continuous functional associated with a Harris recurrent Markov chain with stationary distribution $\pi$. Sample averages associated with an ergodic Markov chain with stationary probability distribution will epigraphically converge from $\pi$-almost all starting points. The property of Harris recurrence allows us to replace ``almost all" by ``all", which is potentially important when running Markov chain Monte Carlo algorithms. That result on epi-convergence is then applied to establish the consistency of the optimal solutions and optimal value of a stochastic optimization problem involving expectation functional of the form $ E_{\pi}[f(x,\xi)].$ Moreover, we develop asymptotic normality of the statistical estimator of the optimal value using a Markov chain central limit theorem.
Citation
MOR-2020-092, The University of Chicago, March/2020