Enrico Ventura

Projects

• Glassy Physics of Diffusion Models

  Diffusion models represent the state of the art in image and video generation. These models learn the distribution of the data through a diffusive trajectory in the data-space, the same type of dynamical processes studied by stochastic thermodynamics. Statistical mechanics has recently helped at providing for crucial insights about the complex dynamics with which such models learn the data distribution. We are now studying the learning dynamics in case of stractured training data, i.e. data-points that live on a low dimensional manifold. We are exploiting Random Matrix Theory and the physics of Random Energy Models to answer the following questions:

  • Does the structure of the data help reducing overfitting and improving generalization?
  • Can we infer the way the model is learning the data-structure by looking at the geometry of the diffusive trajectory?

• Learning & Unlearning in Recurrent Networks

  The human brain is a complex system that is fascinating scientists since a long time. Its remarkable capabilities include classification of concepts, retrieval of memories and creative generation of new examples. At the same time, modern artificial neural networks are trained on large amounts of data to accomplish the same type of tasks with a considerable degree of precision. By contrast with biological systems, machines appear to be either significantly slow and energetically expensive to train, suggesting the need for a paradigmatic change in statistical learning. We evaluate a known training procedure for recurrent neural networks that can be split into a prior Hebbian learning phase (Learning) and a subsequent anti-Hebbian one (Unlearning). We are progressively proving that this unsupervised prescription is capable of performing classification, memorization and generation of examples with a high degree of efficacy while aligning with some modern biological theories of learning.

Publications (scholar)

• E. Ventura, S. Cocco, R. Monasson, F. Zamponi (2024). Unlearning regularization for Boltzmann machines. Machine Learning: Science and Technology, 5(2), 025078.
• E. Ventura, M. Benedetti. (2023). Training neural networks with structured noise improves classification and generalization. arXiv:2302.13417, 6.
• E. Ventura, M. Benedetti, E. Marinari, G. Ruocco, F. Zamponi (2022). Supervised perceptron learning vs unsupervised Hebbian unlearning: Approaching optimal memory retrieval in Hopfield-like networks. The Journal of Chemical Physics, 156(10), 21.

Gender (in)equality in Academia

I'm active in studying and divulgating issues relative to gender discrimination and inequalities in academia, especially in the STEM environment.

Here is a series of posters assembled with the Gender Balance Working Group of La Sapienza Physics Department, obtained from comic strips by Did This Really Happen?! and displayed at La Sapienza in Autumn 2023.