We’ve talked previously about the technological pioneers who are implementing Augmented Reality into medicine, from the Operating Room to the building of 3D models from CT Scans or MRI Reconstructions. However, there’s a dilemma. The AR user needs a continuous feed of the tissues during surgery in real time for the duration of the operation. Of course, ultrasound can do this, but there are limitations with it and many of the other real-time tissue monitoring systems such as fluoroscopy: are not precise. This is a problem, and all good advancements come from addressing problems.
One solution comes in the form of biomechanical deformable volumetric registration by the researchers at Shenzhen Institutes of Advanced Technology in Shenzhen, China. This approach uses a deformable virtual model built from high-resolution hexahedral mesh constructed based on MRI images, in this case, from a 3D abdominal phantom (Model 057A, Computerized Imaging Reference Systems, Inc. – Norfolk, Virginia). The researchers monitored the behavior of the model in response to controlled external forces such as compression and torsion. Looking at the liver as their reference organ, their virtual model was built from six image sets. One was taken under physiological conditions, without compression, and the other five were taken under varying levels of compression. Each set consisted of 111 images and 25 landmarks for the liver.
The process.
Without getting too technical, the researchers took these six image sets and manually segmented the 3D MRI images of the liver, classifying them as either parenchyma or vessels and tumors. They then cut each image into voxels and constructed the hexahedral mesh by connecting the voxels in each image. This allowed them to build a hexahedral mesh that is a more accurate and efficient model than the tetrahedral mesh currently used in other volumetric registration methods, and without the same structural problems as the tetrahedral. Basically, they used an MRI image to create a more accurate and physically realistic virtual model.
While this sounds like a lot to understand (and it is a complex model), the ultimate goal was simply to create a virtual model of the desired organ which is able to accurately respond to external stimuli such as pushing, pulling, and cutting. By doing this, researchers are a step closer to being able to apply this technique to real life surgery. Laparoscopic surgery could be monitored this way instead of with standard cameras and ultrasound, and with the right development and attention, the AR method could become more convenient and accurate than the ultrasound and camera imaging. We want to see augmented reality improving surgeries all over the world, giving each individual patient the best chance at recovery and minimizing complications.
Of course, this technology has its limitations–the controlled conditions were not always applicable to real patients. However, this is still a work in progress, and researchers say they are working to integrate MRI images and ultrasound in vivo. Ultrasound has its limitations, but it is a very useful starting point for other imaging technologies. Augmented reality, just like Isaac Newton, must stand on the shoulders of giants in order to see far. While this technology is still very new and there is not a whole lot of research available on it, we are excited to watch this project develop. AR and MR could be useful in both surgical planning and during the actual surgery, monitoring the behavior of the tissues in real time and helping surgeons access much more information than they have now. It could even be useful in recuperation, allowing the physicians to check the vitality of organs after surgery. We at ARinMED will be following this project closely! What do you think about it? We would love to hear your input in the comments section.
Source: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5319060/pdf/12938_2017_Article_321.pdf
