Posted by Xiaoran “Van” Fan, Experimental Scientist, and Trausti Thormundsson, Director, Google
The market for true wireless stereo (TWS) active noise canceling (ANC) hearables (headphones and earbuds) has been rapidly growing in recent years. In fact, by 2023, the global shipment volume of TWS ANC hearables is expected to nearly double that of smart wristbands and watches. This growth can be attributed to advancements in ANC, transparency mode, and artificial intelligence, which have significantly extended the on-head time for hearables.
Hearables are no longer just used for music listening; users now wear them for various purposes such as exercising, focusing, and mood adjustment. However, the health aspects of hearables are still relatively unexplored in the consumer market.
In our paper “APG: Audioplethysmography for Cardiac Monitoring in Hearables,” presented at MobiCom 2023, we introduce a new active in-ear health sensing modality. This modality, called Audioplethysmography (APG), allows ANC hearables to monitor a user’s physiological signals, such as heart rate and heart rate variability, without the need for additional sensors or compromising battery life.
APG has several advantages: it is highly resilient to motion artifacts, adheres to safety regulations, remains unaffected by seal conditions, and is inclusive of all skin tones. It works by transmitting a low intensity ultrasound wave through the ANC headphone’s speakers and collecting the receiving wave via the on-board feedback microphones. The APG signal is a pulse-like waveform that synchronizes with the heartbeat and provides rich cardiac information.
The ear canal is an ideal location for health sensing because it receives its blood supply from the deep ear artery, which forms a network of smaller vessels that permeate the auditory canal. Slight variations in blood vessel shape caused by the heartbeat can lead to changes in the volume and pressure of the ear canals, making it an ideal location for health monitoring.
Previous research has explored using hearables for health sensing by incorporating multiple sensors and a microcontroller. However, this approach adds cost, weight, power consumption, acoustic design complexity, and form factor challenges to hearables, making widespread adoption difficult.
ANC hearables already have feedback and feedforward microphones that can detect or record bio-signals inside and outside the ear canal. This passive sensing paradigm has prompted various mobile applications, but consumer-grade ANC headphones come with high-pass filters that limit the quality of the signals. It is challenging to embed health features that rely on low-frequency signals in commercial ANC headphones.
APG overcomes these challenges by using a low intensity ultrasound probing signal. This signal triggers echoes that are modulated by the tiny ear canal skin displacement and heartbeat vibrations. Coherent detection is used to retrieve the micro physiological modulation, resulting in an APG waveform that closely resembles a photoplethysmogram (PPG) waveform but provides more detailed cardiac information.
During our experiments, we found that APG works robustly even with poor earbud seals and music playing. However, it can be noisy and susceptible to body motion interference. To address this, we introduced channel diversity by transmitting multiple frequencies simultaneously. This allows us to capture both cardiac activities and head movements, and select the best frequency for high-quality pulse waveform measurement.
Our field studies with 153 participants showed that APG consistently achieved accurate heart rate and heart rate variability measurements across various activity scenarios. Unlike PPG, APG was resilient to variations in skin tone, seal conditions, and ear canal size.
Overall, APG presents a promising new modality for in-ear health sensing in ANC hearables. It eliminates the need for additional sensors, provides accurate measurements, and overcomes the limitations of existing passive sensing methods.
Source link
