Condensed Matter Theory

Abstract:

Deep learning models can exhibit what appears to be a sudden ability to solve a new problem as training time, training data, or model size increases, a phenomenon known as emergence. In this paper, we present a framework where each new ability (a skill) is represented as a basis function. We solve a simple multi-linear model in this skill-basis, finding analytic expressions for the emergence of new skills, as well as for scaling laws of the loss with training time, data size, model size, and optimal compute. We compare our detailed calculations to direct simulations of a two-layer neural network trained on multitask sparse parity, where the tasks in the dataset are distributed according to a power-law. Our simple model captures, using a single fit parameter, the sigmoidal emergence of multiple new skills as training time, data size or model size increases in the neural network.

Abstract:

We review the network model for the integer quantum Hall effect. The model provides a simplified description of Anderson localization in this context. It represents non-interacting electrons moving in two dimensions under the combined influence of a strong magnetic field and a smooth disordered potential. In this setting, electron eigenstates form disorder-broadened Landau levels and their character varies with energy across the Landau level. States in both the low-energy and the high-energy tails of the Landau level are localized, with a spatial extent characterized by the localization length. At the center of the Landau level there is a transition between phases with different quantized values of the Hall conductance and the localization length is divergent. The network model captures universal features of this transition.

Abstract:

A classical wave-particle entity (WPE) can materialize as a millimeter-sized droplet walking horizontally on the free surface of a vertically vibrating liquid bath. This WPE comprises a particle (droplet) that shapes its environment by locally exciting decaying standing waves, which, in turn, guides the particle motion. At high amplitude of bath vibrations, the particle-generated waves decay very slowly in time and the particle motion is influenced by the history of waves along its trajectory. In this high-memory regime, WPEs exhibit hydrodynamic quantum analogs where quantum-like statistics arise from underlying chaotic dynamics. Exploration of WPE dynamics in the very high-memory regime requires solving an integrodifferential equation of motion. By using an idealized one-dimensional WPE model where the particle generates sinusoidal waves, we show that in the limit of infinite memory, the system dynamics reduce to a 3D nonlinear system of ordinary differential equations (ODEs) known as the diffusionless Lorenz equations (DLEs). We use our algebraically simple ODE system to explore in detail, theoretically and numerically, the rich set of periodic and chaotic dynamical behaviors exhibited by the WPE in the parameter space. Specifically, we link the geometry and dynamics in the phase-space of the DLE system to the dynamical and statistical features of WPE motion, paving a way to understand hydrodynamic quantum analogs using phase-space attractors. Our system also provides an alternate interpretation of an attractor-driven particle, i.e., an active particle driven by internal state-space variables of the DLE system. Hence, our results might also provide new insights into modeling active particle locomotion.