“Virtual reality allows you to be in a situation, a simulation, whatever it may be, and it can have you learn by doing. And so, through this medium, you can actually get smarter people. ”
Yao Huang, CEO of The Hatchery and Human Interact
I had an opportunity to give a short talk at an event (International Society of Pediatric Innovation, or iSPI) on the use of artificial intelligence in virtual reality.
Here is a summary of a few of my comments:
Virtual reality, if enabled even more with artificial intelligence, can become a more “intelligent” virtual experience. This capability can be of immense value in healthcare for the following reasons:
(1) More real
The virtual experience can be multimodal (visual, auditory, and haptics) so that this experience in a clinical setting can be even more realistic than what exists now. In addition, real world data (imaging, time series, etc.) can be used in an augmented format so that even 3-dimensional medical images can be used in a very realistic setting. This more advanced form of virtual reality will be particularly useful in procedure-oriented training such as operations and interventions and provides a more realistic operator experience.
(2) More natural
The interactive feel and capabilities of extended reality can be expanded with artificial intelligence. The resultant experiences will feel more natural and real world so that the boundary between the real and virtual worlds can start to blur. In addition, this natural extended reality can be extremely helpful in patient-clinician type interactions such as those in mental health and chronic diseases as well as in challenging situations such as death and dying discussions. The recent surge in natural language processing capabilities can add to the realistic feel.
(3) More precise
The virtual experiences can become individualized and precise so that the visual experience becomes personalized with memory. This aspect is ideal for medical education and clinical training, as there will be a permanent mentor who can not only educate and train on a one-on-one basis but will also remember the prior lessons. A clinician can also have repeated sessions with resuscitation, for instance, with different actors and situations with the experience that can also cover a range of patients of varying sizes and situations.
Extended reality and artificial intelligence can be an increasingly important part of citizen medicine or research to solve issues and enigmas in biomedicine. This part-gamification of biomedicine such as drug design and even procedures and diagnostic enigmas can attract some of the younger generation to become more interested in clinical medicine and healthcare. In addition, this combined VR + AI = IR domain can have some convergence possibilities with digital/virtual twins for a new paradigm of clinical research with real world evidence generation and a concomitant individualized dimension.
