ON THE CONVEXITY OF VALUE FUNCTIONS FOR A CERTAIN CLASS OF STOCHASTIC DYNAMIC PROGRAMMING PROBLEM

It is a common practice in stochastic dynamic programming problems to assume a priori that the value function is either concave or convex and later verify this assumption after the value function has been identified. It is often a difficult task to establish the concavity or convexity of the value function directly. In this paper, we prove that the value function of a certain type of infinite horizon stochastic dynamic programming problem is convex. This type of value function arises frequently in economic modeling.