Year of Publication: 2022
Effectiveness and Utility of Virtual Reality Simulation as an Educational Tool for Safe Performance of COVID-19 Diagnostics: Prospective, Randomized Pilot Trial.
Tanja Birrenbach, Josua Zbinden , George Papagiannakis, Aristomenis K Exadaktylos, Martin Müller, Wolf E Hautz, Thomas Christian Sauter
(JMIR Serious Games 2021;9(4):e29586)
Background: Although the proper use of hygiene and personal protective equipment (PPE) is paramount for preventing the spread of diseases such as COVID-19, health care personnel have been shown to use incorrect techniques for donning/doffing of PPE and hand hygiene, leading to a large number of infections among health professionals. Education and training are difficult owing to the social distancing restrictions in place, shortages of PPE and testing material, and lack of evidence on optimal training. Virtual reality (VR) simulation can offer a multisensory, 3-D, fully immersive, and safe training opportunity that addresses these obstacles.
Objective: The aim of this study is to explore the short- and long-term effectiveness of a fully immersive VR simulation versus a traditional learning method regarding a COVID-19–related skill set and media-specific variables influencing training outcomes.
Methods: This was a prospective, randomized controlled pilot study on medical students (N=29; intervention VR training, n=15, vs control video-based instruction, n=14) to compare the performance of hand disinfection, nasopharyngeal swab taking, and donning/doffing of PPE before and after training and 1 month later as well as variables of media use.
Results: Both groups performed significantly better after training, with the effect sustained over one month. After training, the VR group performed significantly better in taking a nasopharyngeal swab, scoring a median of 14 out of 17 points (IQR 13-15) versus 12 out of 17 points (IQR 11-14) in the control group, P=.03. With good immersion and tolerability of the VR simulation, satisfaction was significantly higher in the VR group compared to the control group (median score of User Satisfaction Evaluation Questionnaire 27/30, IQR 23-28, vs 22/30, IQR 20-24, in the control group; P=.01).
Conclusions: VR simulation was at least as effective as traditional learning methods in training medical students while providing benefits regarding user satisfaction. These results add to the growing body of evidence that VR is a useful tool for acquiring simple and complex clinical skills.
Assessing unconstrained surgical cuttings in VR using CNNs
Ilias Chrysovergis, Manos Kamarianakis, Mike Kentros, Dimitris Angelis, Antonis Protopsaltis, George Papagiannakis
In the recent years, industry and academia have massively adopted Virtual Reality (VR) applications to train students and personnel. Despite the effort, only limited systems involve procedures for assessing user progress inside the immersive environment, that either evaluate only trivial tasks or require a huge amount of time by the reviewers. On the other hand, the need for real-time automated evaluation of user’s actions is constantly increasing. State-of-the-art methods for similar tasks either require the development of complicated task specific computer vision algorithms or support very simple tasks.
This work proposes a deep learning based system, that is able to assess, in real-time, user actions within a VR training scenario. The method enables the rapid development of trained assessment functions, since it utilizes data augmentation to minimize the amount of labelled data that need to be collected. Furthermore, by using transfer learning, these assessment functions can be reconfigured to support similar tasks, thus reducing even more the amount of training data. In this paper, we present the results of our method for the task of tearing a deformable 3D model.
Different machine and deep learning algorithms were considered and compared. Ultimately, our proposed model is a Convolutional Neural Network (CNN), trained on a dataset created with a data augmentation technique.
Recording and replaying psychomotor user actions in VR
Manos Kamarianakis, Ilias Chrysovergis, Mike Kentros, George Papagiannakis
Nowadays, session recording and playback of a single or multiuser VR session has become an increasingly market-required asset. The need for effective VR recording and replaying (VRRR) is especially highlighted in virtual training applications, as replaying user actions can serve as an additional and powerful educational tool. Despite the effort, achieving VRRR is a task not natively undertaken by modern game engines and therefore most VR applications do not include such a feature by default.
Current bibliography contains numerous studies of how the VR record and replay features can enhance the learning impact that VR educational-oriented applications provide, by mainly measuring the performance of users. Usually, the data are captured in video format and a post process of this high-dimensional data is required to obtain any further analysis. Since video data is not sufficient for reasons we explain in Section 2, our approach is close to Kloiber et al., who proposed an analysis of user’s motion by recording their hands and head trajectories. Current ongoing research also explores the proper methods and data structures that must be employed to achieve real-time logging while keeping the required data storage manageable and allowing effective replay.
Realistic soft-body tearing under 10ms in VR
Manos Kamarianakis, Antonis Protopsaltis, Michail Tamiolakis, George Papagiannakis
Rigged animated models are one of the most researched areas in computer graphics and have been vastly adopted in Virtual Reality (VR) applications. VR experts experiment with various animation and deformation techniques that can yield realistic real-time outputs. To cover the needs that arise from a variety of use cases, our research revolves around the ability to perform realistic tears, i.e., small cuts, on the surface of a model. Current bibliography describes diverse ways on how to cut a 3D model, but most of these methods are not suitable for VR, since the specific calculations must be performed in a real-time manner within a few ms to preserve user immersion. Furthermore, to avoid the uncanny valley in VR, we emulate realistic effects while performing cuts on certain materials, such as a sponge or human tissues, that one would expect to occur in real life.
Latest developments allow for basic operations, such as cutting, tearing or drilling on a rigged mesh model, to be run in near real-time. Furthermore, the replication of the physical behaviour of soft bodies, when applying external forces to them in VR, would greatly increase the overall realism of the simulation. The ongoing research for increased realism in virtual environments heavily impacts educational-oriented applications, especially the ones regarding VR medical training.
Less Is More: Efficient Networked VR Transformation Handling Using Geometric Algebra
Manos Kamarianakis, Ilias Chrysovergis, Nick Lydatakis, Mike Kentros, George Papagiannakis
As shared, collaborative, networked, virtual environments become increasingly popular, various challenges arise regarding the efficient transmission of model and scene transformation data over the network. As user immersion and real-time interactions heavily depend on VR stream synchronization, transmitting the entire data sat does not seem a suitable approach, especially for sessions involving a large number of users. Session recording is another momentum-gaining feature of VR applications that also faces the same challenge. The selection of a suitable data format can reduce the occupied volume, while it may also allow effective replication of the VR session and optimized post-processing for analytics and deep-learning algorithms. In this work, we propose two algorithms that can be applied in the context of a networked multiplayer VR session, to efficiently transmit the displacement and orientation data from the users’ hand-based VR HMDs. Moreover, we present a novel method describing effective VR recording of the data exchanged in such a session. Our algorithms, based on the use of dual-quaternions and multivectors, impact the network consumption rate and are highly effective in scenarios involving multiple users. By sending less data over the network and interpolating the in-between frames locally, we manage to obtain better visual results than current state-of-the-art methods. Lastly, we prove that, for recording purposes, storing less data and interpolating them on-demand yields a data set quantitatively close to the original one.
Virtual Reality Medical Training for COVID-19 Swab Testing and Proper Handling of Personal Protective Equipment: Development and Usability
Paul Zikas, Steve Kateros, Nick Lydatakis, Mike Kentros, Efstratios Geronikolakis, Giannis Evangelou, Manos Kamarianakis, Ioanna Kartsonaki, Achilles Apostolou, Tanja Birrenbach, Aristomenis K. Exadaktylos, Thomas C. Sauter, George Papapagiannakis
Efficient and riskless training of healthcare professionals is imperative as the battle against the Covid-19 pandemic still rages. Recent advances in the field of Virtual Reality (VR), both in software and hardware level, unlocked the true potential of VR medical education (Hooper et al., The Journal of Arthroplasty, 2019, 34 (10), 2,278–2,283; Almarzooq et al., Virtual learning during the COVID-19 pandemic: a disruptive technology in graduate medical education, 2020; Wayne et al., Medical education in the time of COVID-19, 2020; Birrenbach et al., JMIR Serious Games, 2021, 9 (4), e29586). The main objective of this work is to describe the algorithms, models and architecture of a medical virtual reality simulation aiming to train medical personnel and volunteers in properly performing Covid-19 swab testing and using protective measures, based on a world-standard hygiene protocol. The learning procedure is carried out in a novel and gamified way that facilitates skill transfer from virtual to real world, with performance that matches and even exceeds traditional methods, as shown in detail in (Birrenbach et al., JMIR Serious Games, 2021, 9 (4), e29586). In this work we are providing all computational science methods, models together with the necessary algorithms and architecture to realize this ambitions and complex task verified via an in-depth usability study with year 3–6 medical school students.
Cloud-based XR Services: A Survey on Relevant Challenges and Enabling Technologies
Theodoros Theodoropoulos, Antonios Makris, Abderrahmane Boudi, Tarik Taleb, Uwe Herzog, Luis Rosa, Luis Cordeiro, Konstantinos Tserpes, Elena Spatafora, Alessandro Romussi, Enrico Zschau, Manos Kamarianakis, Antonis Protopsaltis, George Papagiannakis and Patrizio Dazzi10
In recent years, the emergence of XR (eXtended Reality) applications, including Holography, Augmented, Virtual and Mixed Reality, has resulted in the creation of rather demanding requirements for Quality of Experience (QoE) and Quality of Service (QoS). In order to cope with requirements such as ultra-low latency and increased bandwidth, it is of paramount importance to leverage certain technological paradigms. The purpose of this paper is to identify these QoE and QoS requirements and then to provide an extensive survey on technologies that are able to facilitate the rather demanding requirements of Cloud-based XR Services. To that end, a wide range of enabling technologies are explored. These technologies include e.g. the ETSI (European Telecommunications Standards Institute) Multi-Access Edge Computing (MEC), Edge Storage, the ETSI Management and Orchestration (MANO), the ETSI Zero touch network & Service Management (ZSM), Deterministic Networking, the 3GPP (3rd Generation Partnership Project) Media Streaming, MPEG’s (Moving Picture Experts Group) Mixed and Augmented Reality standard, the Omnidirectional MediA Format (OMAF), ETSI’s Augmented Reality Framework etc.
Year of Publication: 2021
Effectiveness and Utility of Virtual Reality Simulation as an Educational Tool for Safe Performance of COVID-19 Diagnostics: Prospective, Randomized Pilot Trial.
Tanja Birrenbach, Josua Zbinden , George Papagiannakis, Aristomenis K Exadaktylos, Martin Müller, Wolf E Hautz, Thomas Christian Sauter
(JMIR Serious Games 2021;9(4):e29586)
Background: Although the proper use of hygiene and personal protective equipment (PPE) is paramount for preventing the spread of diseases such as COVID-19, health care personnel have been shown to use incorrect techniques for donning/doffing of PPE and hand hygiene, leading to a large number of infections among health professionals. Education and training are difficult owing to the social distancing restrictions in place, shortages of PPE and testing material, and lack of evidence on optimal training. Virtual reality (VR) simulation can offer a multisensory, 3-D, fully immersive, and safe training opportunity that addresses these obstacles.
Objective: The aim of this study is to explore the short- and long-term effectiveness of a fully immersive VR simulation versus a traditional learning method regarding a COVID-19–related skill set and media-specific variables influencing training outcomes.
Methods: This was a prospective, randomized controlled pilot study on medical students (N=29; intervention VR training, n=15, vs control video-based instruction, n=14) to compare the performance of hand disinfection, nasopharyngeal swab taking, and donning/doffing of PPE before and after training and 1 month later as well as variables of media use.
Results: Both groups performed significantly better after training, with the effect sustained over one month. After training, the VR group performed significantly better in taking a nasopharyngeal swab, scoring a median of 14 out of 17 points (IQR 13-15) versus 12 out of 17 points (IQR 11-14) in the control group, P=.03. With good immersion and tolerability of the VR simulation, satisfaction was significantly higher in the VR group compared to the control group (median score of User Satisfaction Evaluation Questionnaire 27/30, IQR 23-28, vs 22/30, IQR 20-24, in the control group; P=.01).
Conclusions: VR simulation was at least as effective as traditional learning methods in training medical students while providing benefits regarding user satisfaction. These results add to the growing body of evidence that VR is a useful tool for acquiring simple and complex clinical skills.
Effectiveness and Utility of Virtual Reality Simulation as an Educational Tool for Safe Performance of COVID-19 Diagnostics: Prospective, Randomized Pilot Trial.
Tanja Birrenbach, Josua Zbinden , George Papagiannakis, Aristomenis K Exadaktylos, Martin Müller, Wolf E Hautz, Thomas Christian Sauter
(JMIR Serious Games 2021;9(4):e29586)
Background: Although the proper use of hygiene and personal protective equipment (PPE) is paramount for preventing the spread of diseases such as COVID-19, health care personnel have been shown to use incorrect techniques for donning/doffing of PPE and hand hygiene, leading to a large number of infections among health professionals. Education and training are difficult owing to the social distancing restrictions in place, shortages of PPE and testing material, and lack of evidence on optimal training. Virtual reality (VR) simulation can offer a multisensory, 3-D, fully immersive, and safe training opportunity that addresses these obstacles.
Objective: The aim of this study is to explore the short- and long-term effectiveness of a fully immersive VR simulation versus a traditional learning method regarding a COVID-19–related skill set and media-specific variables influencing training outcomes.
Methods: This was a prospective, randomized controlled pilot study on medical students (N=29; intervention VR training, n=15, vs control video-based instruction, n=14) to compare the performance of hand disinfection, nasopharyngeal swab taking, and donning/doffing of PPE before and after training and 1 month later as well as variables of media use.
Results: Both groups performed significantly better after training, with the effect sustained over one month. After training, the VR group performed significantly better in taking a nasopharyngeal swab, scoring a median of 14 out of 17 points (IQR 13-15) versus 12 out of 17 points (IQR 11-14) in the control group, P=.03. With good immersion and tolerability of the VR simulation, satisfaction was significantly higher in the VR group compared to the control group (median score of User Satisfaction Evaluation Questionnaire 27/30, IQR 23-28, vs 22/30, IQR 20-24, in the control group; P=.01).
Conclusions: VR simulation was at least as effective as traditional learning methods in training medical students while providing benefits regarding user satisfaction. These results add to the growing body of evidence that VR is a useful tool for acquiring simple and complex clinical skills.
Virtual Reality surgical medical training in the post-Covid-19 pandemic era
G. Papagiannakis, E. Kenanidis, N. Milonakis, M. Potoupnis, E. Tsiridis
(Surgical Life: The Journal of the Association of Surgeons of Great Britain and Ireland, Number 60)
Surgical training has progressed extensively from the historical apprenticeship of the early surgeons to a modern complicated structured training programme with numerous duties and evaluations. Expertise must be gained in a shorter period whilst requiring a greater set of skills than previously. Numerous novel surgical techniques have been developed, demanding the current trainees to learn a broader array of specialist skills, despite having less time to do so. As the number of trainees increases, the chances to acquire procedural and technical skills become gradually limited. Furthermore, the introduction of working hour restrictions and a drive towards senior-led care have reduced the available training hours during the des-ignated training period.
ORamaVR Streamlines Medical Training for Healthcare Professionals
According to the World Health Organization, our planet will need more than 40 million new doctors, nurses, frontline healthcare workers, and other healthcare professionals by the year 2030, which is double the current medical workforce. This demand will be fueled by several different factors – most notably, a growing geriatric population. People are living much longer, thanks in part to improved healthcare. But that brings a rise in chronic conditions such as cardiovascular disease, diabetes, and cancer.
In light of the COVID-19 pandemic, it’s sobering to realize that if we don’t act now to implement new medical training solutions, an additional deficit of 18 million healthcare workers will compound the current existing shortage. However, the 150-year-old training model (a master teaching an apprentice over the course of several years) is unable to meet the level of healthcare professionals needed.
Empirical evidence from other industries clearly demonstrates virtual reality (VR) technology is an effective and efficient way of improving training. However, VR implementation in the medical industry has been slow because of the cost to develop and customize software, limiting accessibility where it is needed the most – in the hands of medical instructors and learners.
A computational medical XR discipline
Architectures for SLAM and Augmented Reality Computing
Never ‘Drop the Ball’ in the Operating Room: An efficient hand-based VR HMD controller interpolation algorithm, for collaborative, networked virtual environments
Inter-operability and Orchestration in Heterogeneous Cloud/Edge Resources: The ACCORDION Vision
Covid-19 – VR Strikes Back: innovative medical VR training
An All-In-One Geometric Algorithm for Cutting, Tearing, and Drilling Deformable Models
Α Virtual Reality App for Physical and Cognitive Training of Older People With Mild Cognitive Impairment: Mixed Methods Feasibility Study
Year of Publication: 2020
Covid-19 – VR Strikes Back: innovative medical VR training
MAGES 3.0: Tying the knot of medical VR
In this work, we present MAGES 3.0, a novel Virtual Reality (VR)-based authoring SDK platform for accelerated surgical training and assessment. The MAGES Software Development Kit (SDK) allows code-free prototyping of any VR psychomotor simulation of medical operations by medical professionals, who urgently need a tool to solve the issue of outdated medical training. Our platform encapsulates the following novel algorithmic techniques: a) collaborative networking layer with Geometric Algebra (GA) interpolation engine b) supervised machine learning analytics module for real-time recommendations and user profiling c) GA deformable cutting and tearing algorithm d) on-the-go configurable soft body simulation for deformable surfaces.
Scenior: An Immersive Visual Scripting system based on VR Software Design Patterns for Experiential Training
Year of Publication: 2019
Effectiveness and Utility of Virtual Reality Simulation as an Educational Tool for Safe Performance of COVID-19 Diagnostics: Prospective, Randomized Pilot Trial.
Tanja Birrenbach, Josua Zbinden , George Papagiannakis, Aristomenis K Exadaktylos, Martin Müller, Wolf E Hautz, Thomas Christian Sauter
(JMIR Serious Games 2021;9(4):e29586)
Background: Although the proper use of hygiene and personal protective equipment (PPE) is paramount for preventing the spread of diseases such as COVID-19, health care personnel have been shown to use incorrect techniques for donning/doffing of PPE and hand hygiene, leading to a large number of infections among health professionals. Education and training are difficult owing to the social distancing restrictions in place, shortages of PPE and testing material, and lack of evidence on optimal training. Virtual reality (VR) simulation can offer a multisensory, 3-D, fully immersive, and safe training opportunity that addresses these obstacles.
Objective: The aim of this study is to explore the short- and long-term effectiveness of a fully immersive VR simulation versus a traditional learning method regarding a COVID-19–related skill set and media-specific variables influencing training outcomes.
Methods: This was a prospective, randomized controlled pilot study on medical students (N=29; intervention VR training, n=15, vs control video-based instruction, n=14) to compare the performance of hand disinfection, nasopharyngeal swab taking, and donning/doffing of PPE before and after training and 1 month later as well as variables of media use.
Results: Both groups performed significantly better after training, with the effect sustained over one month. After training, the VR group performed significantly better in taking a nasopharyngeal swab, scoring a median of 14 out of 17 points (IQR 13-15) versus 12 out of 17 points (IQR 11-14) in the control group, P=.03. With good immersion and tolerability of the VR simulation, satisfaction was significantly higher in the VR group compared to the control group (median score of User Satisfaction Evaluation Questionnaire 27/30, IQR 23-28, vs 22/30, IQR 20-24, in the control group; P=.01).
Conclusions: VR simulation was at least as effective as traditional learning methods in training medical students while providing benefits regarding user satisfaction. These results add to the growing body of evidence that VR is a useful tool for acquiring simple and complex clinical skills.
iSupport: Building a Resilience Support Tool for Improving the Health Condition of the Patient During the Care Path
From Readership to Usership and Education, Entertainment, Consumption to Valuation: Embodiment and Aesthetic Experience in Literature-based MR Presence
Virtual Reality Simulation Facilitates Resident Training in Total Hip Arthroplasty: A Randomized Controlled Trial
Digital Health Tools for Perioperative Stress Reduction in Integrated Care
Year of Publication: 2018
Year of Publication: 2017
Digital Health Tools for Perioperative Stress Reduction in Integrated Care
Real-time rendering under distant illumination with Conformal Geometric Algebra
Gamification and Serious Games
Augmented Cognition via Brainwave Entrainment in Virtual Reality: An Open, Integrated Brain Augmentation in a Neuroscience System Approach
Year of Publication: 2016-before