This paper explores sufficient conditions for a continuous stationary Markov optimal policy and a concave value function
in stochastic dynamic programming problems. Also, the paper addresses conditions needed for the differentiability of the value
function. The paper uses conditions such as first order stochastic dominance, second order stochastic dominance and concave
stochastic dominance that are widely applied in economics.
Keywords and Phrases:Dynamic programming, Stochastic dominance, Concave value function, Differentiable value function.
JEL Classification Numbers:C61, O41, D80, D90.
Received: February 23, 2001; revised version: May 19, 2002
RID="*"
ID="*" I am deeply indebted to Prajit Dutta for patient assistance and advice. This paper has benefited from discussions with
Tsz Cheong Lai, Tackseung Jun, Werner Stanzl and Satyajit Bose as well as feedback from an anonymous referee.