Magnetic Control of Tokamak Plasmas Through Deep Reinforcement Learning

Nuclear fusion using magnetic confinement, in particular in the tokamak configuration, is a promising path towards sustainable energy. A key challenge is to shape and maintain a high-temperature plasma within the tokamak vessel. This requires regulating the plasma position and shape via magnetic fields. In this work, we introduce a new architecture for designing a tokamak magnetic controller based on deep reinforcement learning. The controller is entirely trained on a physics-based simulator and then deployed on the tokamak hardware. We successfully produced and controlled a diverse set of plasma configurations on the Tokamak à Configuration Variable (TCV) device. The control architecture replaces separate controllers used in traditional architectures with a single control policy and allows focus on ‘what’ to control rather than ‘how’. This represents a notable advance for tokamak feedback control, showing the potential of reinforcement learning to accelerate research in the fusion domain, and is one of the most challenging real-world systems to which reinforcement learning has been applied.

Jonas Buchli is a Senior Research Scientist with Deepmind, London. He has been working at the intersection of Machine Learning and Control for most of his career. He has been a contributor to a variety of interdisciplinary research projects in Disaster Assistance, Architecture, Biomedical Technology and Paleo-anthropology among others.

Federico Felici is a Research Scientist at the Swiss Plasma Center (SPC) at EPFL, Lausanne. He currently leads the research activities in the area of advanced plasma control. His current research interests include all aspects of tokamak plasma control, with a strong focus on model-based approaches for practical implementation of control across various current and future fusion research devices.