Some Recent Advances in Optimization for Machine Learning

罗珞

复旦大学

2025年12月30日 星期二

下午 14:00-15:00

This talk considers optimization theory and algorithms for machine learning, addressing challenges in distributed optimization, second-order optimization, minimax problem, and nonconvex nonsmooth problem. We first focus on distributed finite-sum optimization, proposing a sampling strategy that allows different nodes to use different batch sizes at each iteration. This results the near-optimal communication complexity and computation complexity with respect to the global smoothness. We then consider second-order methods for minimization and minimax optimization, introducing block Broyden updates, the squared technique, and partial quasi-Newton methods with the fast superlinear local convergence rates. We further study the constrained nonconvex nonsmooth optimization by introducing the notion of generalized Goldstein stationary point, which characterizes the convergence of the stochastic algorithms for minimizing the Lipschitz continuous function. We also applied this idea to address the general nonsmooth nonconvex-concave minimax problem.