Multi-agent Reinforcement Learning: Statistical and Optimization Perspectives

Multi-agent Reinforcement Learning: Statistical and Optimization Perspectives (via Zoom)

Abstract: Multi-agent Reinforcement learning (MARL), which studies how a group of interacting agents make decisions autonomously in a shared dynamic environment, is garnering significant interest in recent years. However, theoretical understandings on the non-asymptotic sample and computational efficiencies of MARL algorithms remain elusive, even for zero-sum Markov games, especially when it comes to overcoming the curse of multi-agents in the presence of large state-action spaces. This talk aims to address learning in zero-sum Markov games through both statistical and optimization perspectives. From a statistical angle, we provide the first minimax-optimal algorithm for learning zero-sum Markov games in the generative model, highlighting a delicate design of bonus terms that leverage the optimism principle in adversarial learning with a tighter regret analysis. From an optimization angle, we describe a single-loop policy optimization method, with two-timescale policy and value updates, that admits fast last-iterate convergence by leveraging entropy regularization and optimistic multiplicative weighted updates.
Bio: Dr. Yuejie Chi is a Professor in the department of Electrical and Computer Engineering, and a faculty affiliate with the Machine Learning department and CyLab at Carnegie Mellon University. She received her Ph.D. and M.A. from Princeton University, and B. Eng. (Hon.) from Tsinghua University, all in Electrical Engineering. Her research interests lie in the theoretical and algorithmic foundations of data science, signal processing, machine learning and inverse problems, with applications in sensing, imaging, and societal systems, broadly defined. Among others, Dr. Chi received the Presidential Early Career Award for Scientists and Engineers (PECASE), the inaugural IEEE Signal Processing Society Early Career Technical Achievement Award for contributions to high-dimensional structured signal processing, and held the inaugural Robert E. Doherty Early Career Development Professorship. She was named a Goldsmith Lecturer by IEEE Information Theory Society and a Distinguished Lecturer by IEEE Signal Processing Society.