While many real-world problems can be formulated as Markov Decision Processes, they have historically suffered in practice due to Bellman’s curse of dimensionality making computation intractable. However, over the last decade or so, researchers have been making significant strides in the area of approximate dynamic programming. A wide range of practical applications have now been successfully explored, in areas ranging from supply chain and distribution management, dynamic pricing and revenue management, marketing/advertising, industrial organization, finance, and others.
In this course, students will learn the state-of-the-art theory and methods for approximate dynamic programming. We will cover the
full range of approaches available, including simulation-based methods and exact methods in the following broad categories: neurodynamic programming/reinforcement learning, gradient estimation, discretization/interpolation, fixed point projection, and math programming. As such, the course is ideal preparation for students in any field of study who are interested in solving dynamic programs that capture real-world features, but are intractable using standard methods. Students will implement the various numerical methods learned on a problem of their choosing, ideally coming from their own research.