AI + XR: convergence of artificial intelligence and extended reality in healthcare

“At that time, we were not yet in a position to implement a comprehensive digital twin. I had been in the IT industry for over 30 years at the time and was firmly convinced that computers would be powerful enough someday to bring my ideas to life.”

-Michael Grieves, American digital pioneer and progenitor of the digital twin concept

Extended reality (XR) includes its myriad of modalities: augmented (AR), virtual (VR), and mixed (MR) reality. Artificial intelligence and extended reality together can create a special synergy to help manage complicated operations and complex systems.

A special manifestation of AI + XR technology is the digital twin, the virtual representation that simulates real-life object(s), process, and/or system. Digital twins, therefore, can be utilized to improve design and production of complicated processes as well as complex systems. Digital twins, made intelligent by extensive and continuous monitoring via internet of things (IoT) sensors and coupled with simulations, is essentially a synergy of human and machine intelligence in the creation of a representation of a complex system. Although NASA was among the first to use this technology for space exploration missions, other sectors such as manufacturing, construction, and Formula I racing have also taken advantage of this technology. There is even a digital twin of the entire country of Singapore.

While the concept of digital twins is reasonably known in the manufacturing domain, it is relatively underleveraged in healthcare. One aspect of digital twins is the capability to synthesize data, an aspect that will be immensely helpful in certain situations in healthcare. An example would be to use a digital twin in the planning of scenarios in an operating room to maximize efficiency and safety. In addition, digital twins can accommodate a new feature in a complex system to predict its safety profile and cost structure. Lastly, digital twins can include the addition of machine learning and other types of artificial intelligence to a process to enhance its intelligence. The advent of deep reinforcement learning has particularly high potential in such a digital twin environment as many iterations of an agent action can occur to find the most optimal reward. The possibilities in utilizing this resource of AI + XR and digital twins are limitless, from workflow optimization in various healthcare venues to clinical in silico trials of various therapies (such as mechanical ventilation strategies and interventional catheterization) as well as chronic disease management (as a strategy for precision medicine).

Recommended Posts