VR Clinical Skills App

Project Overview

Client: Institute of Medical Science, University of Toronto
Content Advisors: Nicole Harnett
Supervisors: Dr. Jodie Jenkinson, Michael Corrin
Industry: clinical research, education
Audience: MSc and PhD graduate research students
Type: virtual reality interactive experience
Tools: Figma, Unity, Insta360 Studio, Autodesk Maya, Adobe Creative Cloud
Timeline: 3 years (2022-2025)
Role: Lead Product Designer, VR Developer

Clinical work in the healthcare field is becoming increasingly common in graduate research-based programs. This is especially prevalent in interdisciplinary academic settings such as the Institute of Medical Science (IMS) at the University of Toronto, where graduate supervisors often manage research laboratories alongside clinical work. Subsequently, many graduate students now extend into patient-facing clinical roles. Navigating the professional setting is extremely difficult, with students often thrown into fast-paced clinical environments with minimal training or understanding of their role.

To address these concerns, the University of Toronto IMS looked to develop a new clinical skills course that combined traditional teaching methods with an immersive virtual reality (VR) component. In recent years, health program curriculums have been shifting towards problem-based learning and immersive active learning techniques. New literature also supports the use of VR as a clinical education tool. This project specifically aimed to enrich the IMS curriculum by developing an interactive VR-based learning tool to be used in the graduate level clinical skills course.

Design Solution

Project Workflow
This project was a departmental collaboration within the University of Toronto IMS Biomedical Communications and Doctoral programs. Working with the research faculty, stakeholders, and graduate student end users, I employed an iterative design process that focused on leveraging emerging technologies for interactive education, developing a robust and technically sound final application to train incoming new post-graduate research students in practicing sensitive clinical interactions in a safe, repeatable, virtual environment.

Research
To begin, the project started with research on existing literature and education media, identifying gaps in clinical skills training for our target learners of graduate research students. I conducted a review and audit of readily available learning resources, including educational training videos, online eLearning modules, and interactive simulations, assessing the effectiveness in how these products addressed real-world interpersonal challenges faced by graduate student researchers. This review process revealed a limited number of immersive interactive tools for interdisciplinary student researchers to actively practice clinical decision-making and communication skills in realistic scenarios, particularly around the topics of consent and patient confidentiality.

Ideation
Following the discovery of these educational gaps, I collaborated with the client to define the scope of the project, including establishing the VR application goals and objectives, budget for product development, and timelines around implementation of the product.

The content for the VR module was established through interviews in consultation with the IMS Curriculum Committee Working Group, an advisory panel comprised of subject matter experts, stakeholders, and student graduate researchers. In addition to these interviews, to ensure a human-centered learner experience, I developed journey maps and user personas to better understand the target user needs, motivations, and challenges. These tools helped align the education outcomes with both institutional objectives and learner goals.

Script
Based on the identified learning objectives, I created a shortlist of key clinical topics that would benefit most from a VR environment. These topics focused on scenarios where communication, judgement, and ethical decision-making were critical in the interaction between patient and researcher. Detailed scripts were then developed for the scenarios, ensuring that dialogue, pacing, and branching moments reflected realistic clinical interactions that provided meaningful opportunities for learner reflection and skill development.

Storyboard
Each VR scenario was then visualized into a storyboard to map out narrative flow, visual framing, and user interaction points. Two primary scenarios around patient consent were developed, one centered around a standard patient consent process and a second involving a more complex interaction involving ethical and cultural dilemmas. The storyboarding process was critical in planning camera placement, learner perspective, and scene transitions, ensuring that each scenario supported immersion while maintaining clarity and instructional intent.

Wireframes
To establish the user interface and interactive flow of the application, I created wireframes in Figma that mapped the learner’s experience from the home page, through module explorations, and to scenario completion. These wireframes were used to test interactions, evaluate usability, and refine navigation patterns before the development of the prototype application. I applied design thinking principles to ensure the interface was intuitive, minimizing cognitive load while maintaining immersion. Design requirements were designed for both core functionality, such as module selection and progression, and general functionality, including environment exploration and instructional overlays.

Film and Video Editing
The core visual content for the immersive VR experience was created using 360-degree panoramic filmed clinical scenarios. During production, I addressed technical challenges including tripod visibility and unwanted external artifacts through framing and post-production editing. The video editing process focused on maintaining visual realism while ensuring clarity. The audio editing process focused on noise removal and voice dubbing of dialogue to ensure sound quality and audible comprehension.

Unity Prototype Development
The prototype was developed using Unity, a 3D software development platform. The application combined fully immersive VR with a mixed reality approach, integrating the real-world 360-degree video footage with 2D and 3D interface assets. This combination approach allowed learns to observe realistic interactions while also engaging with elements designed to reinforce learning goals and objectives.

Reflection
This experience in end-to-end product development strengthened my ability to connect qualitative insights with design strategy for the execution of an effective user experience across the project. It demonstrated my ability to lead complex, interdisciplinary design work that spans research, instructional design, UI, UX, visual production, and technical development. Through the combination of evidence-based learning strategies with technology, this VR application showcases how thoughtful design can create meaningful educational experiences and tools that support skill-building in an immersive clinical environment.

Video Demo
The video demo walkthrough of The Institute of Medical Science Virtual Reality Clinical Skills Initiative for graduate research students to learn about the clinical environment. Watch the prototype demo video!

Results
Two VR modules were developed and implemented as part of the University of Toronto MSC1121H Clinical Research Skills course. Read about how graduate students engaged with these VR simulations to practice patient‑facing communication scenarios in this peer-reviewed article: Virtual Reality for Developing Patient-facing Communication Skills in a Medical Science Graduate Education Course: A Mixed-Methods Pre-Post Study.