Transcript:

Host: Jon Brouchoud, CEO of Arch Virtual Guest: Dr. Liz O., Director of Institutional Technology, Dr. Kiran C. Patel College of Osteopathic Medicine (KPCOM), Nova Southeastern University (NSU)

Jon Brouchoud: Alright, today we have a very special guest on the Simulation Pulse: Dr. Liz O. is joining us from NSU. I met Dr. Liz many, many years ago, I think it was 2018 at IMSH, the International Medical Simulation and Healthcare conference. We were there with a very early VR application called the Airway Lab project that we did with Adtalem Global Education. I was demonstrating that, and Dr. Liz came up and immediately could see the value of what we were doing. She pulled up a chair and spent some time getting to know me and getting to know the work that we were doing. Ever since then, we’ve had a tremendous collaboration. We’ve worked together on some really ambitious and exciting projects.

Dr. Liz is a visionary. I think of all the people we work with, many are very visionary, but Dr. Liz started, I think, all the way back in 2006-2007 using Virtual Worlds and has been continuously innovating and developing. I’m very, very excited to have this conversation today. So welcome to the Simulation Pulse, Dr. Liz!

Dr. Liz O.: Thank you so much.

Jon Brouchoud: So tell us, your title, the organization you’re with, in your own words, to make sure that I didn’t miss anything.

Dr. Liz O.: My name is Dr. Liz O. I work at Nova Southeastern University with the Dr. Kiran C. Patel College of Osteopathic Medicine (KPCOM). I’m in charge of everything that has to do with KPCOM technology, directing the management and operation, and also running a department, the Division of Institutional Technology. Under that umbrella, we bring in innovative solutions to our curriculum. Our vision is to transform our curriculum by leveraging innovative technology to increase or enhance our student academic success, even to the extent of our faculty and staff professional development.

Jon Brouchoud: Excellent, excellent. And what got you into technology to begin with? You were using these 3D real-time virtual environments for many, many years now. What was your original interest in that? How did you find your way into the virtual world?

Dr. Liz O.: Actually, my background started with physics education. I delved into doing my master’s in computer science and went into management information systems as well, and delved into biomedical informatics. I worked in the area of Information Technology Administration in every aspect of technology for many years. My goal was to leverage technology in medical education. That was what actually brought me into doing my PhD in Computing Technology in Education. That desire to leverage technology and use it in an innovative way to address complexity when it comes to medical education actually propelled me into trying to figure out the best way to do this.

In 2007, I think we got our first high-fidelity mannequin, and it was so exciting to get it because it was like in human form and had all these robotic things we could do with it – the voice and everything. I was in charge of configuring it, so I had to configure it from scratch, working with the vendor, getting it ready for our students. Then, a couple of months afterwards, I was thinking to myself, this is really cool, all the students have to come in and use this mannequin. But what if we have this in virtual environments? Then our students could actually do this from anywhere. That was what was a driving force: okay, how can I leverage that? The mannequin is one thing, but what else can we do to enhance our students, to help them reinforce their understanding? Mannequins are so expensive.

And that was why I delved into Second Life. I started looking at Second Life virtual world environments to see what we can do, and that kind of brought me into the 3D environment. From that time onward, I’ve developed so many things within Second Life, trying to leverage that environment in the education of our students.

Jon Brouchoud: Outstanding. And even as I started, even to this day, when I do research and look up precedent for some of the work that we’re doing, I’ll find your work. It’s one of the first things to pop up. I’ll think I have a great idea for something that could be done in a virtual world, I’m looking at different precedent, how has anybody else tried this? Oh, sure enough, Dr. O. did it already in Second Life many years ago! So you’ve been really innovating in this space for a long time and really have done a lot of the groundwork that has now sort of become something that’s increasingly… I don’t want to say that it’s mainstream yet, but it’s definitely growing. So it’s very exciting, I would imagine, to be at a point now where you’re seeing these virtual technologies being adopted in schools all over the world. You were there from the beginning. What is your perspective on that now that it’s become something integrated into the curriculum at this scale?

Dr. Liz O.: I’m very excited because I remember when I was talking about immersive technology, nobody understood what I was saying. I remember one of my Deans said, “You know, you’re the only one that understands what you’re saying, right? We don’t know what immersive technology is all about.” But you know what, when COVID hit, and everybody had to stay at home, and there was no way we could do anything else… I guess that actually opened the door to what we’ve been working on. People are now realizing that, okay, there’s a platform out there that you can actually use, that you can leverage, that students can actually use from anywhere. And that kind of frog-leaped us into the limelight when everybody now understands what it means to be immersed in your education using virtual reality environments, which have always been there, but people were not really paying close attention to it up until after or during COVID, and subsequently after COVID.

So I’m really excited that it’s happening; it’s like a dream come true for me. And I’m very excited that even at KPCOM, we have fully integrated virtual reality experiences into almost all our courses. We’re working closely with our Deans, our faculty, and our students, and we are also doing research on what we’re doing as well. Because, you know, there’s one thing to use technology, there’s another thing to see how effective it is, whether it’s working or not. My goal is not just to use technology for the sake of technology; my goal is to use technology to solve that problem that nothing else can solve.

Medical education or medical practice is a very complex endeavor. It’s a matter of life and death when it comes to that. So you want to teach your students, and you want your faculty to have the skill to be able to teach those students. And you want the student to be able to enhance their understanding of the concept being taught by going through it, repeating the skill over and over again until they master it, in an environment that not only engages them but also allows them to use all their learning skills, you know, the sensory, visualizing it, involving themselves in the concept or the skills that they have to master.

And that was one of the things that we did with medical procedures. Because a medical procedure is a skill of ‘how-to’, and it’s steps that have to be carried out in a particular order. If you don’t follow that step in that particular order, you put the life of the patient at risk. So, we realized that virtual reality is a platform that actually allows us to do that, allows our students to come in and experience that and do it all over again until they’re able to master the skill. In addition to even using the mannequin, we’re able to combine virtual reality with the mannequin, with the classroom, everything blended together to enhance our students’ skills when it comes to particular procedures.

So, in answering your question, all the schools are doing it now, I’m very excited. It’s like, okay, this is where we are supposed to be. We’re getting there. And for KPCOM, actually, we’re also partnering with industries outside, like yourself and even some other vendors, even the manufacturers of the devices that we are using, to make sure that we are getting the best of what we need to help our students and also to evaluate what we’re doing, reaching out back to those vendors saying, okay, this is what is working, this is what is not working, how can we improve on this?

Jon Brouchoud: That’s amazing. That is absolutely amazing. And I think we became fast friends because we shared a lot of that interest in Second Life. We saw what was possible there, but I think you and I both were using that platform in a very non-traditional way compared to what most of the other users of that platform were using it for. It was a very sort of multi-purpose platform. There were people in there using it as a kind of a virtual chat room. But the interesting and innovative thing about Second Life was that they gave the users the tools to be able to create the content and design the content. That was one of the first places that was possible. Before that, people had to either write their own game engines, or Unity really wasn’t the thing at that point that far back.

So that’s I think where the innovation came from. You took those tools and you were able to use them to achieve very specific goals that you were setting out to achieve, rather than hiring a team of programmers or having to do all the programming yourself from the ground up. This was a real-time multi-user environment that gave you the tools to be able to create that content and innovate and iterate and build that. And there wasn’t really anything else like that at the time. Eventually, there were other platforms that sort of spun out of that, but back in 2007, that was the only place you could do that.

Dr. Liz O.: Yeah, exactly. That was the only place, and it was so fascinating to see that you can actually build up something. You create your own avatar, you have your students flying into the clinic that I created, and then they actually blend, using art observation to actually go into the patient room and be able to use the knowledge that they gained from observing the artwork to diagnosing the patient’s situation, cases, and things like that. So it was really interesting to see that we were able to do it far back then, and now it’s a lot easier, coming all the way and everything coming to play, and people are now using it.

Jon Brouchoud: Absolutely. Well, and what was really interesting is that when you eventually got to the point where you were working with these high-fidelity mannequins, I think it’s similar to the genius of mannequins insofar as it wasn’t like the mannequin came with a pre-programmed set of things that only you can do. It was a platform, right? The mannequins are essentially a platform that allows you, as the innovator, as an instructor, to be able to say, here’s what we need to do with this mannequin, here are the problems that we’re seeing that students need to address, or opportunities that we’re seeing. So, again, building on that narrative of a platform that gives the tools to the educators to create the content, mannequins were like that too.

And I think we saw that potential, and that’s what we tried to do with Acadicus – build on that platform concept. So instead of going and building a library of pre-programmed experiences, we decided to build tools, I think growing from a lot of the lessons and opportunities that we saw in Second Life, and sort of bring that into the VR environment where you’re actually putting on a headset and being immersed. And so, that’s where I think you immediately saw the opportunity there because you were like, this program isn’t going to tell me how to do it; this program is going to give me tools and an opportunity so that I can have a voice in shaping and creating these experiences that we’re going to build together.

Dr. Liz O.: And that was the beauty of your product, actually. Because I was looking for a product when we were moving away from Second Life. I wanted something else that could allow me to build stuff, in addition to having the tailor-made, but also I wanted to be able to do something on my own easily, customize what we need to do to what will benefit us more. And Acadicus seemed to be the first one I saw as a platform that could help us do that. We could easily use the assets that you already have to create a whole room with an avatar patient, with the virtual patient in a virtual world, equivalent to the mannequins, and have students use that or have our faculty also go in and record their own instructions.

So, that was very useful. It was a very useful skill and very useful features that your product has that have actually helped us so far. And we have used it in so many ways, and we are making progress. I always give you guys feedback, I always stretch you guys, okay, we want this done, how can we get there? Trying to get the best of what we can get out of the product. And you guys have always been very forthcoming, taking our feedback and being able to give us what we need, no matter even how hard it is, challenging your team. And that has been very helpful in the product that we have so far that we have been using for our students.

Jon Brouchoud: Yeah, and that was a tremendous collaboration too, starting almost five years ago. You really identified several procedures that would be very valuable in medical education and brought that to us. And I think together, we sort of grappled with this spectrum between wanting it to be a sandbox where it’s truly flexible and these are building blocks, like Lego pieces that you can reuse and recombine in different ways, but the skills are very much stepwise procedures – there’s only really one way that you want to do these things. And in order to have, from a programmatic perspective, things like, we socketed this in, we connected this, therefore this happens… we have to get into some pre-programmed elements to it, which is sort of the antithesis of the open sandbox model.

So we were kind of always going back and forth on how can we keep this as open and flexible as possible, but also make it so that it’s a standalone experience that students can do in a self-guided way that doesn’t always require a facilitator to be present. That’s kind of the balance that we walked with that. And I think it was really interesting, the way you brought this idea about building the content but then having an instructor do a 3D recording of the demonstration. I think that the 3D recording feature in Acadicus really allowed for flexibility where you still have a voice in creating it, you can still change that script, your instructors can create their own recordings of those demonstrations rather than sort of depending on what was given to them. So I think that was a great sort of collaboration where we were able to kind of give and take a little bit and work together. And now we’ve got these procedures, and students are using them. So you’ve been measuring, like you say, doing surveys and studies to measure the efficacy. How have the results been so far?

Dr. Liz O.: Well, with the medical procedures, the last cohort that actually finished the entire thing, they gave positive feedback on that. They were very impressed about being able to use the virtual reality, the virtual environment, to learn the skill. Some of them were like, okay, how can we do this more? I said, oh well, we have open labs, keep coming in, and then you can come and use this tool for as many times as you want.

And part of those students actually were interested in doing research with me. So I had a whole cohort of students that actually focused on immersive technology research and development with me. We actually go into different topics that they narrow their research based on. Like one of them was trying to measure psychomotor skill – how can virtual reality help to do that? And we used one of the medical procedures as the scenario, which was the EKG. The student measured it and had a survey that other students actually completed after the experience they had with the EKG. The feedback was really good because we were able to see that the way you move, the way you turn, actually helps the students, their position, the way they stand to actually put in those lead placements, the stickers and the lead placement on the patient. And it also teaches them balance as well. She was able to measure all that in the research that she did.

Then we had another student that did research on learning styles. Out of all the learning styles that we have, which one fared better using virtual reality? And we’re able to gauge that kinesthetic and visualizing – being able to visualize what you’re doing – and also some auditory learning skills actually gained some experience in that as well. So we figured out that okay, it was not just only the kinesthetic part of it, it actually has benefit for every aspect of the learning styles: being able to visualize, being able to hear (that was the 3D recording), being able to visualize what you’re doing, and being able to use, to do it with your own hands as well, the kinesthetic part of it. The research came back that every aspect of learning style benefits from it. By default, you think kinesthetic and auditory learning styles would be higher up, which was true, but all the other ones too came up. Being able to visualize was even higher. And that helps us to actually understand the value in the use of immersive technology in medical education.

And I also have another student that measured confidence. That was what she measured. The confidence of doing medical procedures. How does immersive technology help you to get to that point? And what we found out from our own research was being able to do this over and over again – you made a mistake and you try it again. Because when they come to my class, they are not leaving until they get that green checkmark that everything was done correctly. So they have to keep doing it. Doing that over and over again until they get it increases their confidence in the fact that, okay, when I get to the clinic, or when I see a real patient, or even when I go to the mannequin and I go to try it, I’m going to be a lot better. And that was also measured, and we found positive feedback in that area.

Jon Brouchoud: That’s fascinating. Well, and even like you say, preparing to use the mannequin, I think that’s an important distinction. A lot of people think of VR as an attempt to replace mannequins, as if we’re trying to sort of move away from that. But I really don’t see it like that. I think it’s a way to support and augment what you’re already doing with mannequins. And in your case, preparing students to make really efficient, effective use of their time when they do get that time in the Sim lab. Because the Sim labs are very expensive facilities, you can’t just wander in there anytime you want. So you want to be ready, you want to be prepared. Maybe this opportunity to go into a VR lab and to sort of preview and practice and go over and over, like you say, until you get that green checkbox, then you’re ready to go into the lab. And I would like to think that when you go to work with the mannequins and/or with a real patient, you have déjà vu. You’ve already done this before many times. You’ve watched instructors, you’ve watched them do it from multiple angles, you’ve done it yourself, and you’re ready. Hopefully, it sounds like that’s playing out in the research that you’re doing as well.

Dr. Liz O.: Yeah, that is actually one of the things that we wanted to see – how we can transfer the knowledge, the skill that you gain using the virtual reality environment, into all these other aspects, the physical aspect, whether being mannequin or even in real life. So we are open to actually following our students, those that are doing research, to follow them from being in school using virtual reality and using mannequin, and go to the real-life use during their rotation, and see what they’re doing and being able to leverage virtual reality within that environment as well when they’re doing their rotation.

And that was what brought up our grant that we have now, the USDA grant for the rural area that we’re using. We developed case scenarios whereby our preceptors at that site can actually use virtual reality to connect with the students from wherever they are and still continue to teach them in preparation for them going to see their real patient. So by the time they go to see the real patient, they know all those skills because they’ve already done it with a preceptor, and then they are able to do it better. That will be a different discussion that once the grant is completely done, myself and Dr. Florence Carney will come and then we can talk more about the details of that, about what we’re doing when it comes to the immersive learning virtual medical education.

Jon Brouchoud: That’s fantastic. I’m really looking forward to that content as well. In addition to the procedures, that’s sort of another project that we worked on that was separate from the batch of procedures that we created. But the environments are so realistic, just going in there and walking around and seeing the way the check-in desk is, and then going back around to the exam room. We didn’t really have an environment quite like that in Acadicus, so that was a really exciting space to be able to sort of model that out and have such a realistic environment. And then you just go around into the exam room and there the patient is sitting in there. It’s just really exciting to see how that plays out and to have another interview and talk more about that one. So that’s really cool.

Dr. Liz O.: Definitely. And we also worked together on the GWEP project, you know, the Geriatric Workforce Enhancement Program. We had Dr. Sweta Tiwari on as a guest. So if you are watching this now and want to learn more about that project, you could watch that episode. But from your perspective, you sort of guided them through this implementation of VR. How was your experience working on that project?

Dr. Liz O.: It was very interesting and fascinating. I was just so excited that I had all these faculty that were interested in doing this. It was hard work because we had to do it several times, we had to write the script, we had to be sure everything was in order by which they wanted it to work. But the beauty of it is, once everything was done, we were able to incorporate that into our curriculum because the GWEP cases in Acadicus are being used in the geriatric classroom. The faculty themselves put on the headset and actually teach within that environment, going through the case scenario, using it to teach.

And we do it across campus. We have two campuses, one in Tampa/Clearwater and one in Davie. During our lectures, we connect through Acadicus to teach the content. It’s always very fascinating. They go in interacting, discussing, and they ask the students questions. The faculty in Davie will begin, and the faculty in Clearwater will take it up, and they will all interact in this environment to teach the students. Then afterwards, the students will now come to my lab to reinforce their understanding and now use the VR themselves to go into those scenarios with the patient and then go into the scavenger hunt to see all those things that they have to be able to know in order to prevent falls. So it was really, really interesting, and it’s been a great success. Usually when we run it, the students are always excited about it.

Jon Brouchoud: That’s fascinating. I’ve got an older family member, and I was in their apartment the other day looking around, and I identified several fall hazards. It was like I had déjà vu from being in that scavenger hunt, trying to find… poor Millie is trying to live at home, and she’s got all these different trip hazards and fall hazards. So yeah, you sort of really get a sense of having already been in that environment and identifying those risks. Hopefully, as we scale that up, that’ll continue to provide really effective training for geriatricians.

Dr. Liz O.: And you know what we are also doing with Acadicus, actually, you guys helped us develop an opioid case together, which is like a standalone. What I’ve done in the past semester, two semesters ago, was to actually introduce that into the clinic as well, like a soft skill training. Whereby I have medical assistants coming into my lab to learn how to take care of patients that have opioid overdose and to learn how to use Narcan, because that’s a tool that they have, and most of them don’t know how to use it. So we were able to use this product, the virtual reality, to train them how to apply Narcan and what you do to be able to resuscitate the patient. The medical assistants, they were very excited about it. They were like, “Oh my, this is good.” And now I’m going to follow them and see whether the knowledge that they acquired coming to the lab, they were able to use that in real time to apply Narcan to patients.

That is how we are actually leveraging this skill. And the same thing is going to… we are also hoping to do the same thing with the GWEP product as well, so that we can expose that application of the VR experience to our medical assistants for them to actually understand what it means to have the 4Ms, and also what it means to be able to guard against falls and all those things that they have to do. So that’s the next step that we’re going to be taking with GWEP, to introduce that into the clinic environment and use it as a soft-skill learning tool for our medical assistants.

Jon Brouchoud: Excellent. Yeah, that’s very, very exciting. So you had mentioned there’s two different campus locations. Does each one of them have their own VR lab?

Dr. Liz O.: Yes, yes, we do. And actually, I’m sitting in one of them right now. My background, if I take off my background, you might be able to see it. Let me do that.

Jon Brouchoud: Yeah, I’d love to see it.

Dr. Liz O.: Oh, there it is! There’s Simon EKG patient.

Jon Brouchoud: Yeah, I have the EKG patient…

Dr. Liz O.: And I’m going to move the camera around so you can see the lab.

Jon Brouchoud: Oh wow! Look at that. So each one of the stations has several seats, and then there’s a TV, a large screen TV there so you can see. So the instructor can walk around, or the facilitator can walk around and see what each of the students are doing. But it’s not a facilitated experience? It doesn’t require a facilitator to be in there live? It’s an experience that you go through, and you place the EKGs, you go through the entire procedure, and then you’re able to confirm that you did the steps right?

Dr. Liz O.: Yes. But the facilitator is able to still watch what the students are doing, answer any questions that they have, and to be able to verify that they’ve done it correctly.

Jon Brouchoud: Yes.

Dr. Liz O.: And we are here because it’s all about technology. So I basically teach this class being here with the students, to make sure that they actually do what they need to do after watching the lesson and understanding what they have to do. Each time I have to remind them about the different controllers, what it means to use the controllers, what you have to do, the three buttons you have to press to be able to do this. All those things have to be done each time because if they don’t do it all the time, when they come in, they have to have a refresher each time.

But the good thing about that, once they master it, we’re there to provide the support for them when it comes to technical issues, if they have any technical problem with the VR. But most of the time, they’re good, and we’re able to guide them through the steps. It’s been very successful. We’ve been able to do this many, many, many times. They come in groups because we have large numbers of students. We have 160-something plus students in Clearwater and we have 250 students in Davie. So we have them rotate the lab depending on the case. Some cases will be like 40 minutes, and some cases will be like 45 minutes, and we put some cushion time for us to switch the students as they rotate through the lab.

Jon Brouchoud: So there’s over 400 students per class year, is that right? Across the campuses? [Recalculating based on 160+ and 250] Okay, over 400 students. And they’re across two different campuses. And how far apart are those campuses?

Dr. Liz O.: Oh, well, if you’re driving, it’s like almost four hours’ drive.

Jon Brouchoud: Four hours’ drive apart. So they’re pretty far apart. Yeah. And when you facilitate these, you’re facilitating across both campuses at the same time, is that correct?

Dr. Liz O.: Well, so in some cases, for the medical procedures, we run them… we have three days, and that three days includes mannequin experience, immersive experience, and they also have another like low-fidelity experience as well. So they have so many things that they have to do within those three days. Medical procedures will be two days, so immersive experience is one of such.

Sometimes we run the two campuses at the same time, but sometimes we run it at different times over those three days. So I might run Davie Monday and Wednesday, and I might run Clearwater on Tuesday. More so because of the capacity of the staff that we have. So we are able to run those sessions, but we don’t run them simultaneously for medical procedures all the time.

It depends. The courses that we have introduced immersive experience into, like the Geriatric (GWEP) course, the faculty will teach the course using the virtual reality in the classroom, right? Then the students will come to the lab. In that kind of situation, sometimes… we made open lab for the students, but this coming semester, we’re going to make it mandatory for the student to come to the lab. And when we do that, we actually do that connecting the two labs together. So we do that. My team will be at each campus, and then we connect together, and we do it at the same time. And that’s what we’ve been doing for those courses that we are gradually introducing virtual reality into, that we are tying the lab into it.

But medical procedure is a very skillful set of steps that they have to take, so we usually spend more time with them, and we want to give them all the attention that they need and immediate solution to any problem that they might have. So we make sure that at least we give them that day in person.

Jon Brouchoud: Yeah. Well, and similar to working with mannequins, I would imagine you have technicians that are setting up the computers and making sure the VR headsets are configured and ready to go, so that when the students get there, all they have to do is put on the headset and away they go.

Dr. Liz O.: Yeah, and that’s what we’ve been doing. So I have my team here in Clearwater and my team in Davie. They’re always there to make sure that everything… we don’t leave the students alone by themselves. They just come in, they use the headset, they have to do what they have to do, and they have to make sure that they do it well because they are being graded on it. So it’s not a matter of you coming in and, you know, I mean, it’s fun, you come to play, but you’re learning while having fun. And sometimes we just have to remind them, okay, if you want to come back, you can come back and play another day.

Jon Brouchoud: Yeah, yeah. I think that’s what happens as students get in the headsets and they’re pretty excited. They want to be able to do… and that’s the idea, is to get them engaged. And I think when they’re excited and they’re leaning in like that, they’re more likely to retain what they’re learning for longer, they’re more likely to remember what they’re learning. So that engagement, that excitement and enthusiasm, I think is part of what makes VR so effective.

Dr. Liz O.: Yeah. And now we are making sure that our faculty are also trained and comfortable. What I do with my faculty, any one of them that actually were able to use VR in the classroom, they’re my champions. So I celebrate them because it’s not easy. And for them to take the step that, okay, I’m going to try this technology, I’m going to take advantage of the best of it and use it in my lecture, it’s very exciting to me. I’m always very excited when I have faculty that have done that, and I celebrate them all the time.

Based on that, we did our Medical Physiology. Physiology is an abstract course. So I met… usually I do my one-on-one with faculty that are interested. So I met with Dr. B. Masuga. She immediately saw Acadicus, she said, “Oh, I like it. I think I would like to use it.” So I trained her, and she picked it up easily. She was very into it. And then we looked at her course and saw how we could leverage Acadicus. The case that she had was shortness of breath, and the topic was autonomic nervous system that she was trying to illustrate. It’s always been difficult, students don’t do well when you ask them that question.

So this time around, we used this innovation, we used this application to teach the students. She was able, after so many trainings, she got it, and we were able to go live with her in lecture. She had a VR headset on, she went into Acadicus, she did what she needed to do there. We pulled in the assets, built a room with a patient, with a five-year-old patient. And then she was able to go in there. She had three rooms actually, three sides to the room. Room one was all about telling the objective of the case. And then room two was actually doing it with the students, and we had a couple of students in there with us during the first lecture. And then she had a debriefing room. She had all the students again, they went into the debriefing room, and then they talked about the case.

We did that, and the exam came, and they did well. Twenty… I think she had like a 27% or 21% increase in the performance compared to the last time that the students did it. And that shows that using this technology was able to improve the student performance, and they were able to understand the concept, and they were able to answer the question correctly. And that improves the performance of the students. Actually, we won an award for this in the AACOM, the American Association of Colleges of Osteopathic Medicine. They gave us an award for this.

Jon Brouchoud: Congratulations! Well deserved, very cool.

Dr. Liz O.: Yeah. So we got an award from AACOM for introducing virtual reality in the teaching of medical physiology. So Dr. Masuga and I actually won an award for doing that, and Acadicus was one of the products that we used in getting to that.

Jon Brouchoud: Excellent. Bringing VR to the classroom, I think that’s a very exciting new frontier, and people are just starting to figure out that that’s possible. And it sounds like you’re also just, in keeping with the theme of your innovation, you’re one of the first to be really in there doing that. And I think that’s another opportunity to really… again, simulation centers are vitally important places, but again, they can’t just be open all the time. These are very expensive mannequins, very expensive equipment. But if we can sort of increase access beyond the simulation lab and get it into the classroom like you’re describing, it’s like a whole new world. What students wouldn’t give to see an interactive VR lecture like that rather than a PowerPoint, right? Like, what an incredible opportunity to really get them engaged. And I think that’s again, you’re seeing that engagement, so you see the increased efficacy in terms of how well they do in the follow-up study. So that’s fantastic.

Dr. Liz O.: Yes, yes. And well, we still have more work to do. So we are making progress. We always are. We’re getting there. Now that AI is the buzzword now for everybody, we’re getting to that front as well too.

Jon Brouchoud: Yeah, a lot of exciting opportunities there as well, for sure. I wanted to ask a little bit about the process of creating some of the procedures because it reminds me of… I think it’s lumbar puncture, male and female Foley catheter, EKG, a wound treatment experience, endotracheal intubation, and we’re just adding the CVL procedure. So I think that’s a total of seven simulations, those are what we call the procedures. And then we did the geriatrics project, and then we did the rural project, and of course the opioid project was another. So we’ve worked on a lot of stuff together, didn’t we?

Dr. Liz O.: Yes! That’s a lot of VR content!

Jon Brouchoud: Yeah, that’s a lot! So can you talk a little bit about the process of creating those? Because I know, particularly with the GWEP project, there was a script that was involved where we followed Millie from being at home, then she’s in the hospital, she’s in the ICU. But, and so we created 3D recordings from our team, but that wasn’t as interesting until Dr. Guida put on a headset, right? And Dr. Guida put on the headset, and he recorded this entire lecture. And now when students go in there, they’re not just seeing some generic person teaching a lecture, it’s Dr. Guida, it’s their teacher. [Voiceover of Dr. Guida’s recording: “You’re about to embark on a self-guided healthcare journey of an elderly woman named Millie. The scenario will take place at various stages of her journey, which we hope will illustrate the 4M framework for you.”] So that was a big part of this. But can you talk a little bit about the process of creating those procedures in terms of how our teams worked together to be able to make sure that we represented these procedures accurately?

Dr. Liz O.: Yes. So, the beginning of the procedure begins with a lot of dialogue with the subject matter experts, which is Dr. Alonso, Dr. Michelle Johnson, and Dr. Eric Gomez. These subject matter experts, they are the experts when it comes to the content of medical procedures. So we begin with them. They have to evaluate the scripts, and it has to actually match what it means to have EKG placement, okay? What it means to have lumbar puncture. What are those skills, what are those steps that have to be taken?

So we go through the process of identifying, working together with Dr. Alonso, with Dr. Michelle Johnson, and your team. We have several meetings together, right? First of all, we identify, okay, what video do we want to use for you guys to actually see what we’re talking about? Alright? So sometimes we provide video for you guys to see what are the steps that we want to be able to have our students master when they use virtual reality. And then Dr. Alonso and Dr. Michelle Johnson will come in and review that to make sure that the scripts actually align with what it means to be doing this in real life, in a hospital or in the clinic.

Once that process has taken place and we have established the content, then your team would take that up. And then we work together to make sure that technically, everything that’s supposed to follow actually happens. And I work closely with your team to make sure that we have all these ins and outs in place, the things that are supposed to be there, the things that don’t look the way we want them to look, and to make sure that we are not deviating from the true picture of what the student is supposed to learn. So that when they go out, when they actually go to a real patient or when they go to the clinic or the hospital, the skill that they have learned using virtual environments is extended, they can see it and they can make use of it.

Even to the extent that sometimes, like for the lumbar puncture and for the CVL, those are kind of complicated scenarios. And the last time that we did those tools in the medical procedure immersive experience, the students actually, they got excited about it because they don’t see it that easily to be able to do, but the immersive experience allowed them to be able to understand, to see it better and understand the steps that they have to take. And that is Dr. Alonso and Dr. Michelle Johnson and Dr. Eric Gomez providing, making sure that the content aligns with what the students will see when they get to the clinic. So that was very important, that was actually a very, very important aspect of the steps that we take in the process.

And then once that is established, we meet with your team couple of times to make sure that everything looks the way we want it to look and feels the way we want it to feel. And also, I usually bring in my research students – we’ve done that a couple of times – because I make them test it out first. Because when we test it out first, we do find some issues or some things that are not being done well. We want to get feedback from the student perspective as well. So I will have my research students kind of play around with it, and then we come back to you guys, and then you guys will make all the adjustments. At the end of the day, we now come back, and then we all look at it again. You do some recordings for us, and then we look at the recordings, and then we say, “Oh yeah, this is perfect, we can use it now, we’re ready.” And that’s how the process has been from the beginning to the end.

Jon Brouchoud: I think it speaks to how memorable being in a VR headset is, but I remember those review sessions like they were yesterday. Like we were all together, you’ve got all the team members there, and our developers are all there, and we’re standing around this patient, and we’re going through the procedure. Maybe we could do it this way, maybe we should try it that way, and kind of going through that iterative cycle to get it just right. But those review sessions were a lot of fun. I always enjoyed that.

Dr. Liz O.: Yes, yes, yes. And that’s very important actually, when it comes to development. There’s one thing for you to develop, there’s another thing for you to review and provide feedback to enhance the development. When it comes to virtual reality, you want to create a scenario or a case that is almost as realistic as possible, alright? And that is what makes it innovative. Because you want to create it in such a way that, I want my students, my faculty to make use of this tool to enhance their understanding and the skill that they want to acquire.

And you’re trying to do this in a way that is memorable, that they can remember once they leave that… that’s what the immersion is all about. Because they’re making use of everything that has to do with their sensory organs to actually learn this. So you want them to leave and have the feeling that, okay, I’m ready. I’m ready to do this in real life. I’m ready to do this with the mannequins. And that’s the beauty of it because once you are in that VR headset, you’re just in your own bubble, and then you just want to give it everything.

Jon Brouchoud: Absolutely, absolutely. I’m wondering from your perspective, we talk to clinicians and educators all over the country, and we’re seeing this picture emerging of really a pretty devastating shortage of clinicians and a shortage of educators in the medical and healthcare education space. Can you speak to your experience with that? Is there a shortage of instructors? Is there a shortage of clinicians that you’re seeing in Florida? And do you think it’s going to get worse? Are there any solutions on the horizon, and is VR potentially part of that solution?

Dr. Liz O.: Talking about shortage of physicians and educators, I think what we have identified is that in the rural area, we don’t have as many doctors, physicians, or healthcare [providers]. And that was one of the things that propelled us in the writing of the proposal for the USDA grant – to actually take virtual reality, to see how virtual reality can help train our preceptors or our students in that aspect, across [distances]. So that even if they don’t have the educators or the physicians always present in the rural environment, the fact that virtual reality, being able to connect from anywhere, can actually leverage that technology to enhance the provision of service.

I know one of the things that we actually requested for the USDA grant was holographic experience. We wanted to hologram into the clinic because we’re looking at, during COVID, nobody could go into the room with the patient. So one of the things that we requested for was the holographic experience, which we are still doing within our classroom here. But imagine doing that, holographically being able to assess your patient, bringing the physician to the patient where they don’t have a physician in person, but you have a physician that can holograph into the room and be able to see the patient.

And that brings us to telemedicine, to telehealth. And that’s another thing, that’s the next phase of what we are doing when it comes to using immersive technology. My hope is to bridge the border, to bridge the gap, and go across this border to be able to connect us in the virtual world and be able to bring service from wherever to whether being the rural or even in the urban place. Just connect educators, physicians, students. And I believe the immersive experience, the virtual world, can actually help us a lot. We can leverage these environments to enhance, to bridge the gap between shortage of physicians or instructors, or even medical student training.

Sometimes the way you train in certain parts of the state is different from the way you train in other parts of the state. So if you all train together using the virtual environment platform, the education, what you get from both sides or from wherever you are, will be the same because you’ll be able to structurally structure your teaching in such a way that you can communicate to everybody at the same time, and they will obtain the same skill. So, in answering your question, I think there’s future in leveraging virtual environments when it comes to being able to bridge the gap and help us in enhancing or supplementing the shortage of physicians or instructors across our borders or across our states or even our local rural environments as well.

Jon Brouchoud: That’s fantastic. That’s a great vision and kind of dovetails into my last question for you, and you maybe have already answered this. But what do you see as the future of VR? What’s your… if all obstacles are removed and you really get your way and everything is implemented flawlessly over the next five years, what does the future of VR look like?

Dr. Liz O.: It’s going to be better. It has come to stay; it’s not going anywhere. It’s going to become even more immersive in the sense that virtual reality, mixed reality, augmented reality are going to become one, tied to AI. That’s where we are going. Now, there are so many other things that are going to come up with being able to combine those things together, but that’s the future, especially when it comes to healthcare education.

And what I want to tell everybody is: don’t use technology just for the fun of it. Use technology to achieve those things that nothing else can achieve. So if we have that goal, then we’re going to always think outside the box and be able to stretch our boundaries to actually bring out the best in what we are doing. Being able to use this environment to the extent by which it’s going to give us the best and make us achieve the best when it comes to medical education, whether being practice or training and delivery as well.

Because even in the hospital environment, not only in the education environment, in the clinic environment, in the hospital environment, you bring in virtual reality or mixed reality or augmented reality combined with AI into preparing your preceptors, your faculty, your doctors, your physicians – continuing medical education, right? You want them to be able to use that, and virtual reality or immersive technology can help that. Because then the doctors can actually use that medium to practice what they have to go and do before they go do it, or use it to kind of reinforce their understanding, or prepare for their board exams because they have to renew their license, right?

So those are the ways by which we can use this. And they’re always very busy, but if you bring it to the environment, they’re already there. They can use that from wherever they are to practice, to continually be in there revisiting what they have to do, and then taking that information and showing it to the patients before any surgery occurs or any procedure occurs. You tell the patient, “This is what you’ll be expecting to see.” We have tools that we can leverage to do that.

So, I think the future is… it’s not going anywhere. It’s only going to be better. And we don’t have a choice; we have to keep following it and making it the best that we can.

Jon Brouchoud: Absolutely. Very well said, very well said. Well, I sincerely appreciate your time meeting with me today and sharing the insights that you’ve shared. I look forward to the next interviews. I think, you know, we’ve been working together for quite a few years now, but I feel like we’re just getting started. The best is yet to come. There’s a lot of exciting new places that we can go. So, very, very honored to be working with you and appreciative of all of the work that we’ve done together and the projects that you’ve brought to us and the innovation that you’ve brought to us. It’s just very, very much appreciated by myself and everyone on our team.