The Physical Effects of Learning

Author:

Menachem Stern, Andrea J. Liu, Vijay Balasubramanian

Keyword:

Condensed Matter, Disordered Systems and Neural Networks, Disordered Systems and Neural Networks (cond-mat.dis-nn), Soft Condensed Matter (cond-mat.soft), Statistical Mechanics (cond-mat.stat-mech)

journal:

date:

2023-06-21 16:00:00

Abstract

Interacting many-body physical systems ranging from neural networks in the brain to folding proteins to self-modifying electrical circuits can learn to perform specific tasks. This learning, both in nature and in engineered systems, can occur through evolutionary selection or through dynamical rules that drive active learning from experience. Here, we show that learning leaves architectural imprints on the Hessian of a physical system. Compared to a generic organization of the system components, (a) the effective physical dimension of the response to inputs (the participation ratio of low-eigenvalue modes) decreases, (b) the response of physical degrees of freedom to random perturbations (or system ``susceptibility'') increases, and (c) the low-eigenvalue eigenvectors of the Hessian align with the task. Overall, these effects suggest a method for discovering the task that a physical network may have been trained for.

PDF: The Physical Effects of Learning.pdf