Every day at ARinMed, we see new attempts to integrate Augmented/Mixed Reality into medical scenarios. Surgery seems to embrace this new technology more than others, which in most cases works great for patients and researchers alike. However, most applied technologies are divided into two types: Preoperative data overlaid into the operative field, or intraoperative data overlaid into the operative field. All of this data is obtained from X-ray, SPECT, Ultrasound, etc., but sometimes this leads all of this Augmented data blocking the view for the surgeon, or also the user’s hands or surgical instruments can occlude or interfere with the visibility in the operative field, diminishing the benefits of the system, which is also bad for the surgery.
This brings up a question: Is there any other way to take advantage of Mixed Reality for surgery? A solid yes was answered by researchers from the Technische Universitat Munchen in Germany by bringing up a new system, unlike anything we have seen before. This system consists of a Mixed Reality multi-layer visualization during surgery using a stereo-RGBD augmented C-arm fluoroscope along with 2 Microsoft Kinect for optical tracking. The information obtained through this dispositive is portrayed through a head mounted device, which allows multiple layers to be synthesized into one single view, this video output is totally customizable since the blending values can be altered in every step of the process.
Basically, the premise of the dispositive is to give the user an X-ray adjustable vision, which goes from erasing the surgeon hand or instruments from his point of view, to giving him a full fluoroscopy vision and everything in between. This system was tested using an experimental protocol that included a fictitious operative field where several random occlusions with a hand using a latex glove and surgical instruments happened. Triggering the blending mechanism, the system successfully ‘erased’ the occluding hand and instruments, allowing to keep a free operative field, making the experiment a success.
Of course, there are some concerns, like the fact that not every background is 100% recoverable, but as the researchers stated in their study, this depends on the RGBD camera and by bringing more cameras to the device this could be solved. It does not take a clinical trial to know how visual occlusions can affect a surgery, just common sense. This is especially true when talking about trauma and orthopedic surgeries, where surgeons have to implant surgical materials such as wires, intramedullary nails, and even prosthesis (knee cap, hip), and they have to maintain a biomechanical standard for every structure they are repairing, which usually takes a lot of experience, but we believe that with new technologies like this one make the learning curve easier for new surgeons in this medical field.
What do you think about this new modality of Mixed Reality? Let us know in the comments section!
Source: https://arxiv.org/abs/1709.08962
