Breakthrough Listen

Next-generation human-computer interaction (HCI) is moving towards more seamless, intuitive,
and personal modes of communication, redefining how we interact with technology and one another.
Within this landscape, silent speech recognition (SSR) offers a powerful new interaction paradigm,
enabling hands-free, private interaction while supporting individuals with speech impairments and
enabling communication in noisy or sensitive environments. Recent advances in miniaturized sensors and artificial intelligence (AI) have accelerated the development of more sophisticated wearable
SSR systems, driven by growing demand for effortless and accessible communication. Although
electrophysiological (ExG) modalities, particularly electromyography (EMG), have dominated early
efforts in developing wearable SSR, critical challenges remain. Limited generalizability across
users, sensor-skin interface issues, and difficulties with the comfort of use are all current roadblocks
to reliable, high-fidelity signals in a wearable form factor. We propose that magnetometers offer
a promising alternative to ExG and have the potential to unlock more robust, generalizable, and
user-friendly SSR systems. We demonstrate that magnetometers embedded in a headphone form
factor achieve a per-user SSR accuracy of 86%, significantly outperforming previously reported
state-of-the-art wearable headphones combining ExG and inertial measurement units (IMUs). In
addition, we show that wearable magnetometry enables generalization across individuals for SSR.
Extending beyond headphones, we also introduce a necklace form factor with magnetometers that
is capable of decoding both silent and overt speech in ambient conditions, further showcasing the
versatility of magnetometers across different wearable designs in real-world conditions.