Course Description 

The digital world has a wealth of data, such as internet of things (IoT) data, business data, health data, mobile data, urban data, security data, and many more, in the era of “Big Data” and the Fourth Industrial Revolution (Industry 4.0 or 4IR). Extracting knowledge or useful insights from these data can be used for intelligent and automated decision-making in various applications domains. Advanced analytics methods including machine learning modeling can provide actionable insights or deeper knowledge about data, which makes the computing process automatic and smart. Data- driven discovery is revolutionizing the modeling, prediction, and control of dynamic complex systems. 

This graduate course introduces the foundations of sequential decision-making models and algorithms (Markov decision processes, dynamic programming, Q-learning, TD learning, SARSA, actor-critic, policy gradient and bandits). We will illustrate the theory and algorithms via numerous application examples, drawn from the areas of finance, logistics, supply chain management, pricing and revenue management, and robotics etc. By the end of the course, the student will build solid understanding to conduct research on sequential decision-making problems. They also should be able to apply the theories and analysis skills in modelling dynamic engineering problems and designing algorithms to solving sequential decision-making problems in manufacturing and service applications.

The course includes 3 hours of lectures per week. Homework includes both written exercises and programming exercises. Depending on the instructor or the need, the course can be offered with a final quiz and a course project (including presentation and report).

Prerequisites 

Real Analysis, Linear Algebra, Operational Research, Statics and Probability, Optimization (Linear and Convex). In general, the course will be very much self-contained.