Movement provides insight into how the brain controls the body. From traditional observation methods to modern artificial intelligence techniques, tracking movement has evolved significantly. The latest cutting-edge approach involves using artificial intelligence to automatically track body parts. However, training these models is still time-consuming and requires manual marking of each body part hundreds to thousands of times.
Associate Professor Eiman Azim and his team have developed GlowTrack, a non-invasive movement tracking method that utilizes fluorescent dye markers to train artificial intelligence. GlowTrack is efficient, high definition, and capable of tracking a single digit on a mouse’s paw or numerous landmarks on a human hand.
Published in Nature Communications on September 26, 2023, this technique has wide-ranging applications in biology, robotics, medicine, and beyond.
“In recent years, there has been a revolution in behavior tracking due to powerful artificial intelligence tools in the lab,” says Azim, the senior author and holder of the William Scandling Developmental Chair. “Our approach enhances the versatility of these tools, allowing us to capture diverse movements in the lab. Improved movement quantification provides better understanding of how the brain controls behavior and could aid in studying movement disorders like ALS and Parkinson’s disease.”
Current methods for capturing animal movement typically involve manually marking body parts on a computer screen, which is time-consuming and prone to human error. Human annotation limits the use of these methods to specific testing environments, as artificial intelligence models specialize in the limited training data they receive. If any factors, such as lighting, body orientation, or camera angle, change, the model may fail to recognize the tracked body part.
To overcome these limitations, the researchers used fluorescent dye markers to label animal or human body parts. These “invisible” markers allow for the creation of visually diverse data, which can be quickly fed into artificial intelligence models without the need for human annotation. With this robust data, the models can track movements in a wider range of environments and at a resolution that would be challenging to achieve through manual labeling.
This enables easier comparison of movement data between studies, as different labs can use the same models to track body movement across various situations. According to Azim, comparison and reproducibility of experiments are crucial in the scientific discovery process.
“Fluorescent dye markers were the perfect solution,” says first author Daniel Butler, a Salk bioinformatics analyst. “Similar to invisible ink on a dollar bill that only lights up when desired, our fluorescent dye markers can be activated and deactivated quickly, allowing us to generate a vast amount of training data.”
In the future, the team aims to combine GlowTrack with other tracking tools that reconstruct movements in three dimensions and analysis approaches to uncover patterns in these extensive movement datasets.
“Our approach can benefit various fields that require sensitive, reliable, and comprehensive movement capturing tools,” says Azim. “I am eager to see how scientists and non-scientists adopt these methods and the unique and unexpected applications that may arise.”
Other authors involved in this work include Alexander Keim and Shantanu Ray of Salk.
This work received support from the UC San Diego CMG Training Program, a Jesse and Caryl Philips Foundation Award, the National Institutes of Health, the Searle Scholars Program, the Pew Charitable Trusts, and the McKnight Foundation.
