Brain surgery training from an avatar | MIT News

Renowned neurosurgeon Benjamin Warf from Boston Children’s Hospital is seen in the MIT.nano Immersion Lab. Meanwhile, thousands of miles away, his virtual avatar is assisting Matheus Vasconcelos in Brazil as the resident practices a delicate surgery on a baby’s brain model.

Equipped with virtual-reality goggles, Vasconcelos observes Warf’s avatar demonstrating a brain surgery procedure before replicating it himself and engaging in conversation with Warf’s digital twin.

Warf describes the experience of watching his avatar interact with the residents as almost out-of-body, comparing it to having an identical twin.

The goal was to allow Warf to be present in two places simultaneously through his digital twin. Vasconcelos, a neurosurgery resident at Santa Casa de São Paulo School of Medical Sciences in Brazil, expressed confidence and comfort in applying the technique in real patients after training with the avatar.

The avatar project was initiated by EDUCSIM, a medical simulator and augmented reality company, as part of the 2023 cohort of START.nano, MIT.nano’s deep-tech accelerator. The project aimed to provide early-stage startups with discounted access to MIT.nano’s laboratories.

Giselle Coelho, EDUCSIM’s scientific director and a pediatric neurosurgeon, collaborated with the MIT.nano Immersion Lab technical staff to create Warf’s avatar, which began training future surgeons like Vasconcelos by November.

Coelho had the idea to create Warf’s avatar to enable surgeons worldwide to benefit from his expertise. The avatar was developed using high-fidelity motion-capture technology and various VR/AR technologies at MIT.nano’s Immersion Lab.

Warf’s avatar operates in both synchronous and asynchronous modes. The training Vasconcelos received was in the asynchronous mode, where residents can observe demonstrations and ask questions, with AI algorithms providing answers based on previous research and Warf’s input.

The avatar project aims to revolutionize medical training, particularly for remote and underserved areas like the Amazon region of Brazil, where access to expert surgeons is limited. Coelho believes that training surgeons with the avatar can have a significant impact on improving healthcare in these regions.