Neuro-dynamic programming is a methodology for sequential decision making under uncertainty, which is based on dynamic programming. The key idea is to use a scoring function to select decisions in complex dynamic systems, arising in a broad variety of applications from engineering design, operations research, resource allocation, finance, etc. This is much like what is done in computer chess, where positions are evaluated by means of a scoring function and the move that leads to the position with the best score is chosen. Neuro-dynamic programming provides a class of systematic methods for computing appropriate scoring functions using approximation schemes and simulation/evaluation of the system’s performance.