Weill Cornell Medicine explores XR technology for training

“Everyone in healthcare is overworked,” says Dr. Rohan Jotwani, MD, MBA, an anesthesiologist and pain specialist at Weill Cornell Medicine. 

“Most hospitals are operating at 95-100 percent of clinical staff utilization,” he continued. “So, finding time to do training is probably one of the hardest things that we do.”

Weill Cornell is a teaching hospital and Dr. Jotwani sees this problem daily in his additional duties as an assistant professor of Anesthesiology – and as the assistant director of digital information. 

“We’re always looking for new ways to improve our educational and clinical workflows by introducing new technology,” he says. Extended reality, or XR, which is an all-encompassing term for immersive learning technologies, including virtual reality (VR) and augmented reality (AR), is a special area of interest for him.   

While training is a high priority in medicine, delivering healthcare is even higher. The hospital simply can’t ask its 80 anesthesiology resident doctors to take an hour off from work to complete a training module. If it did, the hospital would effectively shut down.

Higher fidelity training at the right moments 

Dr. Jotwani believes XR can help solve that problem and other training challenges in medicine as well. Simulating the complexity medical teams encounter from patient to patient in a multidisciplinary environment is one example. “Every clinical pathway is fairly unique,” he says. Patients may have similar issues, but there’s nuance and every individual case is different.

Knowledge retention is another challenge. He points to Advanced Cardiac Life Support (ACLS) certification as an illustration. This training is required every two years, but learning assessments show that people forget what they learned. As time passes, knowledge retention fades to the extent it’s “like you haven’t really done” the training.

“The problem we’re trying to solve is creating training with a great deal of fidelity,” but also “make training ever present,” he notes. “We want to give people the training that they need at the moment that they need it.”

XR makes it easy. Staff can simply put on a headset and start training for a very specific medical procedure. “You could do it on your own time, and it can involve the point of view of other people, without actually having to have those other people in the room.” 

Training on soft skills and patent interaction

Soft skills in medicine are the “real power skills,” says Dr. Jotwani. Soft skills are effectively “how you make people better.” 

It’s also crucial to medical team dynamics – and success in high-stress situations. 

“If there's a code in the operating room, the teams that communicate well are the teams that save a life,” he said. “Those that don't communicate well are the ones where the mistakes are made, and people die.”

How can XR tech help? Weill Cornell is experimenting with avatars in the Virti system that are enhanced with generative artificial intelligence (AI). The avatars can look any way needed, perform in an almost unlimited number of scenarios, and speak in 18 different languages. 

The traditional training status quo is to hire actors, which is logistically challenging, and the clinician scheduling problems remain. When the session is over – it’s over – you can’t take the training again and again as you can in XR.   

10 out of 10 pain level

The team is currently working on developing an AI-avatar model of a patient that lives in chronic pain for soft skills training. This is a “classic scenario” that happens every day, where medical professionals are sometimes dismissive of patient-reported pain levels.

For example, he says, a provider will ask a patient on a scale of one-to-10, “How much pain are you in?” The patient invariably answers, “10 out of 10.” However, to the provider, the patient doesn’t look like they are experiencing such a pain level and so they dismiss it as an exaggeration. 

“That's a really interesting model to create because we want people to come to terms with their bias,” says Dr. Jotwani. “Evidence suggests that women’s pain is often dismissed. We know that people don't take pain from underserved populations seriously. But we don't really have any good ways to actually train people to get better at this without these kinds of characters.”

Results to date

The effort to incorporate XR at Weill Cornell Medicine is still in the early stages. However, the team has noticed that it’s easy to use and highly customizable, which makes a big difference.  

XR seems intimidating because people think there’s coding involved – but there isn’t.  Dr. Jotwani notes it’s easy for someone without a technical background to build customer experience and training. 

“One of the frustrating parts about XR is that people will design games or design content that is close but doesn't quite match what we need,” he says. “One of the great things about Virti is that the technology puts you in the driver's seat.”

“Virti is decreasing that barrier to entry in XR,” says Dr. Jotwani. “It's been a good experience so far and we look forward to learning and doing more.”