High-Dimensional Non-Convex Landscapes and Gradient Descent Dynamics

Author:

Tony Bonnaire, Davide Ghio, Kamesh Krishnamurthy, Francesca Mignacco, Atsushi Yamamura, Giulio Biroli

Keyword:

Condensed Matter, Disordered Systems and Neural Networks, Disordered Systems and Neural Networks (cond-mat.dis-nn), Statistical Mechanics (cond-mat.stat-mech)

journal:

date:

2023-08-06 16:00:00

Abstract

In these lecture notes we present different methods and concepts developed in statistical physics to analyze gradient descent dynamics in high-dimensional non-convex landscapes. Our aim is to show how approaches developed in physics, mainly statistical physics of disordered systems, can be used to tackle open questions on high-dimensional dynamics in Machine Learning.

PDF: High-Dimensional Non-Convex Landscapes and Gradient Descent Dynamics.pdf