Researchers from the Basque Center for Applied Mathematics, Hokkaido University and Kyoto University have deciphered the thermodynamic “arrow of time” in large-scale complex networks
This breakthrough is anchored in the Ising model, a cornerstone in statistical physics and machine learning
The result sheds light on recent findings in biological systems, like the emergence of an arrow of time in signals from the brain
The paper has been published in the journal Nature Communications
New research from the Basque Center for Applied Mathematics – BCAM, Hokkaido University and Kyoto University has found an innovative mathematical solution for the non-equilibrium thermodynamics of large-scale neuron networks based on their temporal asymmetry. This breakthrough helps to better understand the behavior of biological systems like networks of neurons, or the behavior of machine learning and AI tools. This research has been carried out by Dr. Miguel Aguilera Ikerbasque Research Fellow at BCAM and Professor Hideaki Shimazaki, associate professor at Kyoto University’s Graduate School of Informatics, and has been published in the research journal Nature Communications.
Life, from the perspective of thermodynamics, is a system out of equilibrium: it goes on maintaining an internal harmonic structure while continuously exchanging energy and matter with the environment. According to the second law of thermodynamics, over time systems tend to move from order to disorder. This increase in the level of disorder is known as entropy. If this increase is larger than zero, the system is out of equilibrium, and its dynamics have the characteristic of being irreversible over time. This link between entropy and irreversibility is expressed as “the arrow of time” by the English physicist Arthur Eddington in 1927.
The idea of the arrow of time is sometimes exemplified by the idea of watching a movie of a cup of glass being shattered. As the cup breaks, entropy increases as the system changes to a more disordered state, which feels like a natural sequence of events. Conversely, if we were to watch the film in reverse, it would seem obvious that it is not possible for the pieces to just become a whole cup again, moving from disorder back to order. This seems so unintuitive, as such events require the arrow of time to be reversed, and a decrease of the entropy of the system.
An emerging field of physics, stochastic thermodynamics, has rapidly advanced our understanding of the microscopic non-equilibrium processes that occur, for example, in colloidal particles and molecular motors. These particles and motors are very small systems that play a role in biological systems (e.g., in cell membranes or transporting substances within a cell). However, the nonequilibrium thermodynamics and the arrow of time of large-scale complex networks, from neural circuits to flocks of birds, remained an enigma – until now.
As a prototypical model of large-scale complex networks, the researchers studied the Ising model, a model of recurrently connected neurons, which is also studied in statistical physics and machine learning. When connections between neurons are symmetric, the Ising model is in equilibrium state and presents complex disordered states called spin glasses. The mathematical solution of this state led to the award of the 2021 Nobel Prize in Physics to Dr. Giorgio Parisi. However, unlike living systems, spin crystals are in equilibrium and their dynamics are time-reversible. Instead, the researchers worked on the time-irreversible Ising dynamics caused by asymmetric connections between neurons. The study offers, for the first time, an exact mathematical solution of the temporal asymmetry –also known as entropy production– of nonequilibrium disordered Ising networks.
This research shows that there are two distinct regimes of behavior that give rise to temporal asymmetry in complex networks. The first is related to the mysterious “edge of chaos”, a critical regime on the border between order and disorder. The second is a phase of disorder with quasi-deterministic temporal dynamics. The result bridges the gap between the temporal asymmetry found in neuron network activity in the brain and its underlying mechanisms. “Neuroscientists report that the temporal asymmetry in brain dynamics increases in states of heightened awareness or cognitive effort, but we do not know how it is generated. Our research provides two possible solutions: one based on highly flexible stochastic states that combine order and disorder, and another based on combinations of chaotic but repetitive patterns,” said Dr. Miguel Aguilera, Ikerbasque Researcher at the Basque Center for Applied Mathematics.
(From left to right) Miguel Aguilera (Basque Center for Applied Mathematics – BCAM) and Hideaki Shimazaki (Kyoto University), researchers involved in the new study.
The exact solutions obtained serve as benchmarks for developing approximate methods for learning artificial neural networks. The development of learning methods that work robustly in multiple phases discovered in this study will advance machine learning studies. “The Ising model is a prototype of the artificial neural networks that underpin recent advances in deep learning and generative artificial neural networks. Understanding its behavior offers critical insights into both biological and artificial intelligence in general,” added Professor Hideaki Shimazaki, associate professor at Kyoto University’s Graduate School of Informatics
Dr. Miguel Aguilera also stressed the multidisciplinary nature of their study: “Our findings are the result of an exciting interdisciplinary collaboration involving insights from physics, neuroscience and mathematical modelling. This unique combination of disciplines opened the door to novel methods for understanding the organization of large-scale complex networks”.
Link to the paper: https://www.nature.com/articles/s41467-023-39107-y