i3 | August 14, 2017

Virtual Reality is the Latest Tool for Surgeons

Jeremy Snow

How Minnesotan doctors used virtual reality to “walk into the heart” of conjoined twins to save their lives

How can doctors better prepare for complex surgeries? At the University of Minnesota Masonic Children’s Hospital the answer lies in virtual reality (VR). 

In February, twins Paisleigh and Paislyn Martinez were born connected from their chests to their belly-buttons and had their two hearts joined together. While conjoined twins are already extremely rare, doctors saw the sisters as even a bigger medical anomaly: one of the sisters was essentially acting as a heart, lung and dialysis machine for the other, explained one doctor. For both to survive, their best chance was separation. 

But before the nine-hour operation came more than two months of preparation, where doctors performed multiple mock surgeries and studied every part of the procedure, down to the exact positioning of the babies. And to study the twin’s anatomy, they turned to VR to truly understand what was going on inside. More and more, VR is being incorporated into hospitals to help doctors prepare for highly difficult procedures. With a 3D model right in front of them, doctors can see a patient’s organs in new ways to discover information they may have never found. 

For the Martinez babies, the VR experience allowed the doctors to “walk into [their] heart,” said Dr. Arthur Erdman, the director of the Earl E. Bakken Medical Devices Center at the University of Minnesota. The center is one of the leading locations for innovative medical tools, such as 3D printing, virtual reality and other cutting edge techniques.

“When you get into complex situations, even the most skilled radiologists and surgeons can benefit from being able to scan through the anatomy in whatever direction they want in real time,” he told i3. “They can enlarge it, turn it around and try to understand the complexities.”

To do this, the team at the Medical Devices Center enlisted Ph.D. students and colleagues to manually combine CT scans to craft a 3D model of the connected hearts. By late April, surgeons used VR glasses to view the model as if it was right in front of them. 

While wearing them, each doctor in the room could see the twin’s connected heart, at the same time. As they looked into the detailed digitization, they explored and studied the model, planning each step of the surgery as they examined it, Erdman said. 

But they found something shocking. The scans pinpointed a 1 millimeter vessel connecting the twins’ hearts that would need to be cut. 

“If they hadn’t realized that was there, it might have been devastating during the separation,” he said.

Thanks to these cutting edge techniques and world class doctors, the surgery was successful – Today, the twins are separated and recovering. 

VR and Beyond

Erdman first envisioned using an interactive VR system after a 2006 conference for the American Society of Mechanical Engineers and the Food and Drug Administration. By combining 3D models with interactive displays and a user interface, Erdman realized it could help engineers and physicians optimize medical devices and better understand patient-specific anatomy. To Erdman, VR belongs in the same toolbox as a medical textbook or an X-ray machine. Preparing surgeries with 3D objects, not just 2D renderings or images, has an obvious advantage. 

Also in that toolbox: 3D printing, which doctors also used to help the Martinez babies. The University of Minnesota’s Visible Heart Laboratory and the Medical Devices Center 3D printed a number of physical versions of the twin’s combined hearts to see exactly how the girl’s hearts connected. The MDC will soon upgrade to a printer that can print six different materials, Erdman said – the current one can print two. 

Now, the industry, is even starting to take notice – VR is also being used to educate patients what to do during procedures. Through immersive learning, medical students can see firsthand what steps need to be taken. One platform even livestreamed an operation in a 360° viewpoint of the doctor. 

As time goes forward, Erdman hopes VR technology becomes more commonplace to help doctors prepare for complex surgeries. The Medical Devices Center, for example, already is developing a second generation system that can project images “like a hologram.” 

“In future years, this [technology] will end up in all surgery suites,” Erdman said. “You’ll have the ability to pull up images with voice command and be able to pull up patient info just as easily.”

 July/August 2017 i3 Cover Issue

Subscribe to i3 Magazine

i3, the flagship magazine from the Consumer Technology Association (CTA)®, focuses on innovation in technology, policy and business as well as the entrepreneurs, industry leaders and startups that grow the consumer technology industry. Subscriptions to i3 are available free to qualified participants in the consumer electronics industry.