DATA8007 - Foundations of Sequential Decision-Making
Course Instructor
Dr Wenjie HUANG
Research Assistant Professor
HKU Musketeers Foundation Institute of Data Science and
Department of Data and Systems Engineering, HKU
Dr. Wenjie Huang is Research Assistant Professor in Department of Data and Systems Engineering, The University of Hong Kong. He received Ph.D. degree from the Department of Industrial Systems Engineering and Management, National University of Singapore (NUS) in 2019 and B.S. degree in the Department of Industrial Engineering from Shanghai Jiao Tong University, China in 2014. Prior to joining HKU, he held joint postdoc positions at School of Data Science, The Chinese University of Hong Kong, Shenzhen and Group for Research in Decision Analysis (GERAD), Quebec, Canada. His research projects have been supported by NSFC research funds, NRF Singapore and NUS Young Investigator Award.
Course Description
The digital world has a wealth of data, such as internet of things (IoT) data, business data, health data, mobile data, urban data, security data, and many more, in the era of “Big Data” and the Fourth Industrial Revolution (Industry 4.0 or 4IR). Extracting knowledge or useful insights from these data can be used for intelligent and automated decision-making in various applications domains. Advanced analytics methods including machine learning modeling can provide actionable insights or deeper knowledge about data, which makes the computing process automatic and smart. Data- driven discovery is revolutionizing the modeling, prediction, and control of dynamic complex systems.
This graduate course introduces the foundations of sequential decision-making models and algorithms (Markov decision processes, dynamic programming, Q-learning, TD learning, SARSA, actor-critic, policy gradient and bandits). We will illustrate the theory and algorithms via numerous application examples, drawn from the areas of finance, logistics, supply chain management, pricing and revenue management, and robotics etc. By the end of the course, the student will build solid understanding to conduct research on sequential decision-making problems. They also should be able to apply the theories and analysis skills in modelling dynamic engineering problems and designing algorithms to solving sequential decision-making problems in manufacturing and service applications.
The course includes 3 hours of lectures per week. Homework includes both written exercises and programming exercises. Depending on the instructor or the need, the course can be offered with a final quiz and a course project (including presentation and report).
Prerequisites
Real Analysis, Linear Algebra, Operational Research, Statics and Probability, Optimization (Linear and Convex). In general, the course will be very much self-contained.
HKU IDS
Research Postgraduate Programme
DATA8007 - Foundations of Sequential Decision-Making (Foundation)
Course Instructor
Dr Wenjie HUANG
Course Description
The digital world has a wealth of data, such as internet of things (IoT) data, business data, health data, mobile data, urban data, security data, and many more, in the era of “Big Data” and the Fourth Industrial Revolution (Industry 4.0 or 4IR). Extracting knowledge or useful insights from these data can be used for intelligent and automated decision-making in various applications domains. Advanced analytics methods including machine learning modeling can provide actionable insights or deeper knowledge about data, which makes the computing process automatic and smart. Data- driven discovery is revolutionizing the modeling, prediction, and control of dynamic complex systems.
This graduate course introduces the foundations of sequential decision-making models and algorithms (Markov decision processes, dynamic programming, Q-learning, TD learning, SARSA, actor-critic, policy gradient and bandits). We will illustrate the theory and algorithms via numerous application examples, drawn from the areas of finance, logistics, supply chain management, pricing and revenue management, and robotics etc. By the end of the course, the student will build solid understanding to conduct research on sequential decision-making problems. They also should be able to apply the theories and analysis skills in modelling dynamic engineering problems and designing algorithms to solving sequential decision-making problems in manufacturing and service applications.
The course includes 3 hours of lectures per week. Homework includes both written exercises and programming exercises. Depending on the instructor or the need, the course can be offered with a final quiz and a course project (including presentation and report).
Prerequisites
Real Analysis, Linear Algebra, Operational Research, Statics and Probability, Optimization (Linear and Convex). In general, the course will be very much self-contained.