Year of Publication: 2024
To keep the first publication closed
Paul Zikas, Antonis Protopsaltis, Manos Kamarianakis, Mike Kentros, Nick Lydatakis, Dimitris Angelis, Michalis Tamiolakis, Michael Dodis, George Kokiadis, Ilias Chrysovergis, Flora Fyka, Maria Pateraki, George Papagiannakis
In this work, we propose MAGES 4.0, a novel Software Development Kit (SDK) to accelerate the creation of collaborative medical training scenarios in VR/AR. Our solution offers a versatile authoring platform for developers to create medical simulations in a future-proof, low-code environment. MAGES breaks the boundaries between realities since students can collaborate using virtual and augmented reality devices at the same medical scene. With MAGES we provide a solution to the 150-year-old training model which is unable to meet the level of healthcare professionals needed. Our platform incorporates, among others, the following novel advancements: a) 5G edge-cloud remote rendering and physics dissection, b) realistic real-time simulation of organic tissues as soft-bodies, c) a highly realistic cutting and tearing algorithm, d) neural network assessment for user profiling and, e) a VR recorder to record and replay or resume the training simulation from any perspective.
64. Non-linear, Team-based VR Training for Cardiac Arrest Care with enhanced CRM Toolkit
Kentros Mike, Kamarianakis Manos, Cole Michael, Popov Vitaliy, Protopsaltis Antonis,George Papagiannakis
This paper introduces iREACT, a novel VR simulation addressing key limitations in traditional cardiac arrest (CA) training. Conventional methods struggle to replicate the dynamic nature of real CA events, hindering Crew Resource Management (CRM) skill development. iREACT provides a non-linear, collaborative environment where teams respond to changing patient states, mirroring real CA complexities. By capturing multi-modal data (user actions, cognitive load, visual gaze) and offering real-time and post-session feedback, iREACT enhances CRM assessment beyond traditional methods. A formative evaluation with medical experts underscores its usability and educational value, with potential applications in other high-stakes training scenarios to improve teamwork, communication, and decision-making.
63. Geometric Algebra Meets Large Language Models: Instruction-Based Transformations of Separate Meshes in 3D, Interactive and Controllable Scenes
Dimitris Angelis, Prodromos Kolyvakis, Manos Kamarianakis, George Papagiannakis
This paper introduces a novel integration of Large Language Models (LLMs) with Conformal Geometric Algebra (CGA) to revolutionize controllable 3D scene editing, particularly for object repositioning tasks, which traditionally requires intricate manual processes and specialized expertise. These conventional methods typically suffer from reliance on large training datasets or lack a formalized language for precise edits. Utilizing CGA as a robust formal language, our system, shenlong, precisely models spatial transformations necessary for accurate object repositioning. Leveraging the zero-shot learning capabilities of pre-trained LLMs, shenlong translates natural language instructions into CGA operations which are then applied to the scene, facilitating exact spatial transformations within 3D scenes without the need for specialized pre-training. Implemented in a realistic simulation environment, shenlong ensures compatibility with existing graphics pipelines. To accurately assess the impact of CGA, we benchmark against robust Euclidean Space baselines, evaluating both latency and accuracy. Comparative performance evaluations indicate that shenlong significantly reduces LLM response times by 16% and boosts success rates by 9.6% on average compared to the traditional methods. Notably, shenlong achieves a 100% perfect success rate in common practical queries, a benchmark where other systems fall short. These advancements underscore shenlong’s potential to democratize 3D scene editing, enhancing accessibility and fostering innovation across sectors such as education, digital entertainment, and virtual reality.
62. Computational Medical XR (CMXR): Pioneering the Future of Healthcare Through Computational Medicine and Extended Reality
George Papagiannakis, Oliver Kanape
Computational Medical Extended Reality (CMXR), brings together life sciences and neuroscience with mathematics, engineering, and computer science. It unifies computational science (scientific computing) with intelligent extended reality and spatial computing for the medical field. It significantly differs from previous “Clinical XR” and “Medical XR” terms, as it is focusing on how to integrate computational methods from neural simulation to computational geometry, computational vision and computer graphics with deep learning models to solve hard problems in medicine and neuroscience: from low-code/no-code/genAI authoring platforms to deep learning XR systems for training, planning, real-time operative navigation, therapeutics, and rehabilitation.
61. Egocentric skilled human activity psychomotor training: A case study involving mechanical simulated part-based process
Dimitrios Sideris, Minas Dasygenis, George Papagiannakis, Antonis Protopsaltis
Virtual reality (VR) training shows promise for developing skilled psychomotor movements compared to traditional methods. This paper presents a case study of using VR simulation to train prospective drivers on proper snow chain installation technique. An anti-skid snow-chain was 3D modeled, rigged and animated with physical properties, providing realistic high-fidelity behavior for VR users. A virtual world and an immersive egocentric VR training scenario, supporting simulated VR user actions, tailored to specific user interactions, were further developed using Unreal Engine. A usability evaluation experiment with twenty-five drivers provided auto-logged quantitative metrics, and qualitative feedback, gathered during the training process. Despite the limited sample size, initial findings and the demonstration of high proficiency by trainees during the training scenario, suggest that VR enables meaningful advantages for the development of complex psychomotor skills, for the trainees involved in the specific training scenario. These initial conclusions, regarding the impact of VR training in such mechanical simulated part-based process, will be greatly useful for studies dealing with the learning impact and knowledge retention of VR training.
60. Edge-driven Docker registry: facilitating XR application deployment
Antonios Makris, Evangelos Psomakelis, Ioannis Korontanis, Theodoros Theodoropoulos, Ioannis Kontopoulos, Maria Pateraki, Christos Diou, Konstantinos Tserpes
In recent years, containerization is becoming more and more popular for deploying applications and services and it has significantly contributed to the expansion of edge computing. The demand for effective and scalable container image management, however, increases as the number of containers deployed grows. One solution is to use a localized Docker registry at the edge, where the images are stored closer to the deployment site. This approach can considerably reduce the latency and bandwidth required to download images from a central registry. In addition, it acts as a proactive caching mechanism by optimizing the download delays and the network traffic. In this paper, we introduce an edge-enabled storage framework that incorporates a localized Docker registry. This framework aims to streamline the storage and distribution of container images, providing improved control, scalability, and optimized capabilities for edge deployment. Four demanding XR applications are employed as use cases to experiment with the proposed solution.
59. GA-Unity: A Production-Ready Unity Package for Seamless Integration of Geometric Algebra in Networked Collaborative Applications
Manos Kamarianakis, Nick Lydatakis, George Papagiannakis
This paper introduces GA-Unity, the first Unity package specifically designed for seamless integration of Geometric Algebra (GA) into collaborative networked applications. Indeed, in such contexts, it has been demonstrated that using multivectors for interpolation between transmitted poses reduces runtime by 16% and bandwidth usage by an average of 50% compared to traditional representation forms (vectors and quaternions); we demonstrate that GA-Unity further enhances runtime performance. Tailored for 3D Conformal Geometric Algebra, GA-Unity also offers an intuitive interface within the Unity game engine, simplifying GA integration for researchers and programmers. By eliminating the need for users to develop GA functionalities from scratch, GA-Unity expedites GA experimentation and implementation processes. Its seamless integration enables easy representation of transformation properties using multivectors, facilitating deformations and interpolations without necessitating modifications to the rendering pipeline. Furthermore, its graphical interface establishes a GA playground for developers within the familiar confines of a modern game engine. In summary, GA-Unity represents a significant advancement in GA accessibility and usability, particularly in collaborative networked environments, empowering innovation and facilitating widespread adoption across various research and programming domains while upholding high-performance standards.
58. Decoupled Edge Physics algorithms for collaborative XR simulations
George Kokiadis, Antonis Protopsaltis, Michalis Morfiadakis, Nick Lydatakis, George Papagiannakis
This work proposes a novel approach to transform any modern game engine pipeline, for optimized per- formance and enhanced user experiences in Extended Reality (XR) environmentsDecoupling the physics engine from the game engine pipeline and using a client-server N-1 architecture creates a scalable solution, efficiently serving multiple graphics clients on Head-Mounted Displays (HMDs) with a single physics engine on edge-cloud infrastructure. This approach ensures better synchronization in multiplayer scenarios without introducing overhead in single-player experiences, maintaining session continuity despite changes in user participation. Relocating the Physics Engine to an edge or cloud node reduces strain on local hardware, dedicating more resources to high-quality rendering and unlocking the full potential of untethered HMDs. We present four algorithms that decouple the physics engine, increasing frame rates and Quality of Experi- ence (QoE) in VR simulations, supporting advanced interactions, numerous physics objects, and multi-user sessions with over 100 concurrent users. Incorporating a Geometric Algebra interpolator reduces inter-calls between dissected parts, maintaining QoE and easing network stress. Experimental validation, with more than 100 concurrent users, 10,000 physics objects, and softbody simulations, confirms the technical viability of the proposed architecture, showcasing transformative capabilities for more immersive and collaborative XR applications without compromising performance.
57. A Computational Medical XR Discipline
56. VR Isle Academy: A VR Digital Twin Approach for Robotic Surgical Skill Development
Achilleas Filippidis, Nikolaos Marmaras, Michael Maravgakis, Alexandra Plexousaki, Manos Kamarianakis, George Papagiannakis
Contemporary progress in the field of robotics, marked by improved efficiency and stability, has paved the way for the global adoption of surgical robotic systems (SRS). While these systems enhance surgeons’ skills by offering a more accurate and less invasive approach to operations, they come at a considerable cost. Moreover, SRS components often involve heavy machinery, making the training process challenging due to limited access to such equipment. In this paper we introduce a cost-effective way to facilitate training for a simulator of a SRS via a portable, device-agnostic, ultra realistic simulation with hand tracking and feet tracking support. Error assessment is accessible in both real-time and offline, which enables the monitoring and tracking of users’ performance. The VR application has been objectively evaluated by several untrained testers showcasing significant reduction in error metrics as the number of training sessions increases. This indicates that the proposed VR application denoted as VR Isle Academy operates efficiently, improving the robot – controlling skills of the testers in an intuitive and immersive way towards reducing the learning curve at minimal cost.
55. Effectiveness of virtual reality compared to video training on acetabular cup and femoral stem implantation accuracy in total hip arthroplasty among medical students: a randomised controlled trial
Eustathios Kenanidis, Panagiotis Boutos, Grigorios Voulgaris, Aikaterini Zgouridou, Eleni Gkoura, Zakareya Gamie, George Papagiannakis, Eleftherios Tsiridis
Purpose Virtual reality (VR) training efectiveness in improving hip arthroplasty surgical skills requires further evaluation.
We hypothesised VR training could improve accuracy and the time taken by medical students compared to a control group
with only video teaching.
Methods This single-centre randomized controlled clinical trial collected data from March to June 2023. Surgically naïve
volunteer undergraduate medical students performed three sessions on a VR training platform, either cup (VR-Cup=ControlStem) or stem (VR-Stem=Control-Cup) implantation. The primary outcome was the mean diference between predefned cup
inclination (60°) and stem anteversion (20°) compared to the actual implanted values in sawbones between VR and control
groups. Secondary outcomes were task completion time and mistake number between the groups.
Results A total of 101 students participated (VR-Cup 47, VR-Stem 54). Groups did not signifcantly difer concerning age
(p = 0.879), gender (p = 0.408), study year (p = 0.938), previous VR use (p = 0.269) and baseline medical and procedural
knowledge. The VR-Cup implanted the cup closer to the intended target (p < 0.001) and faster than the Control-Cup group
(p = 0.113). The VR-Stem implanted the stem closer to the intended target (p = 0.008) but not faster than the Control-Cup
group (p = 0.661). Stem retroversion was commoner in the Control-Stem than in the VR-Stem group (p = 0.016).
Conclusions VR training resulted in higher rates of accurate procedure completion, reduced time and fewer errors compared
to video teaching. VR training is an efective method for improving skill acquisition in THA.
54. A network application approach towards 5G and beyond critical communications use cases
Konstantinos C. Apostolakis, Barbara Valera-Muros, Nicola di Pietro, Pablo Garrido, Daniel del Teso, Manos Kamarianakis, Pedro R. Tomas, Hamzeh Khalili, Laura Panizo, Almudena Díaz Zayas, Antonis Protopsaltis, George Margetis, Josep Mangues-Bafalluy, Manuel Requena-Esteso, André S. Gomes, Luís Cordeiro, George Papagiannakis and Constantine Stephanidis
Abstract Low latency and high bandwidth heralded with 5G networks will allow transmission of large amounts of Mission-Critical data over a short time period. 5G hence unlocks several capabilities for novel Public Protection and Disaster Relief (PPDR) applications, developed to support first responders in making faster and more accurate decisions during times of crisis. As various research initiatives are giving shape to the Network Application ecosystem as an interaction layer between vertical applications and the network control plane, in this article we explore how this concept can unlock finer network service management capabilities that can be leveraged by PPDR solution developers. In particular, we elaborate on the role of Network Applications as means for developers to assure prioritization of specific emergency flows of data, such as high-definition video transmission from PPDR field users to remote operators. To demonstrate this potential in future PPDR-over-5G services, we delve into the transfer of network-intensive PPDR solutions to the Network Application model. We then explore novelties in Network Application experimentation platforms, aiming to streamline development and deployment of such integrated systems across existing 5G infrastructures, by providing the reliability and multi-cluster environments they require.
Year of Publication: 2023
To keep the first publication closed
Paul Zikas, Antonis Protopsaltis, Manos Kamarianakis, Mike Kentros, Nick Lydatakis, Dimitris Angelis, Michalis Tamiolakis, Michael Dodis, George Kokiadis, Ilias Chrysovergis, Flora Fyka, Maria Pateraki, George Papagiannakis
In this work, we propose MAGES 4.0, a novel Software Development Kit (SDK) to accelerate the creation of collaborative medical training scenarios in VR/AR. Our solution offers a versatile authoring platform for developers to create medical simulations in a future-proof, low-code environment. MAGES breaks the boundaries between realities since students can collaborate using virtual and augmented reality devices at the same medical scene. With MAGES we provide a solution to the 150-year-old training model which is unable to meet the level of healthcare professionals needed. Our platform incorporates, among others, the following novel advancements: a) 5G edge-cloud remote rendering and physics dissection, b) realistic real-time simulation of organic tissues as soft-bodies, c) a highly realistic cutting and tearing algorithm, d) neural network assessment for user profiling and, e) a VR recorder to record and replay or resume the training simulation from any perspective.
51. AR-Assisted Surgical Care via 5G networks for First Aid Responders
Manos Kamarianakis, Antonis Protopsaltis, George Papagiannakis
Abstract Surgeons should play a central role in disaster planning and management due to the overwhelming number of bodily injuries that are typically involved during most forms of disaster. In fact, various types of surgical procedures are performed by emergency medical teams after sudden-onset disasters, such as soft tissue wounds, orthopaedic traumas, abdominal surgeries, etc. HMD-based Augmented Reality (AR), using state-of-the-art hardware such as the Magic Leap or the Microsoft HoloLens, have long been foreseen as a key enabler for clinicians in surgical use cases, especially for procedures performed outside of the operating room.
This paper describes the Use Case (UC) “AR-assisted emergency surgical care”, identified in the context of the 5G-EPICENTRE EU-funded project. Specifically, the UC will experiment with holographic AR technology for emergency medical surgery teams, by overlaying deformable medical models directly on top of the patient body parts, effectively enabling surgeons to see inside (visualizing bones, blood vessels, etc.) and perform surgical actions following step-by-step instructions. The goal is to combine the computational and data-intensive nature of AR and Computer Vision algorithms with upcoming 5G network architectures deployed for edge computing so as to satisfy real-time interaction requirements and provide an efficient and powerful platform for the pervasive promotion of such applications. By developing the necessary Virtual Network Functions (VNFs) to manage data-intensive services (e.g., prerendering, caching, compression) and by exploiting available network resources and Multi-access Edge Computing (MEC) support, provided by the 5G-EPICENTRE infrastructure, this UC aims to provide powerful AR-based tools, usable on site, to first-aid responders.
50. A Multivariate Unimodality Test Harnessing the Dip Statistic of Mahalanobis Distances Over Random Projections
Prodromos Kolyvakis, Aristidis Likas
Abstract Unimodality, pivotal in statistical analysis, offers insights into dataset structures and drives sophisticated analytical procedures. While unimodality’s confirmation is straightforward for one-dimensional data using methods like Silverman’s approach and Hartigans’ dip statistic, its generalization to higher dimensions remains challenging. By extrapolating one-dimensional unimodality principles to multi-dimensional spaces through linear random projections and leveraging point-to-point distancing, our method, rooted in α-unimodality assumptions, presents a novel multivariate unimodality test named mud-pod. Both theoretical and empirical studies confirm the efficacy of our method in unimodality assessment of multidimensional datasets as well as in estimating the number of clusters.
49. Development and usability testing of a fully immersive VR simulation for REBOA training
T. Birrenbach, R. Wespi, W. E. Hautz, J. Berger, P. R. Schwab, G. Papagiannakis, A. K. Exadaktylos and T. C. Sauter
REBOA training in VR was found to be feasible without relevant VR-specific side-effects. Usability (SUS median 77.5, IQR 71.3–85) and sense of presence and immersion (Slater-Usoh-Steed median 4.8, IQR 3.8–5.5) were good,
the workload without under-nor overstraining (NASA-TLX median 39, IQR 32.8–50.2) and user satisfaction high (USEQ median 26, IQR 23–29). Confidence of trainees in conducting REBOA increased significantly after training (p < 0.001).
Procedural training of the REBOA procedure in immersive virtual reality is possible with a good accept- ance and high usability. REBOA VR training can be an important part of a training curriculum, with the virtual reality- specific advantages of a time- and instructor-independent learning.
48. Survey of New Applications of Geometric Algebra
Eckhard Hitzer, Manos Kamarianakis, George Papagiannakis,Petr Vašík.
This survey introduces 101 new publications on applications of Clifford’s geometric algebras (GA) newly published during 2022 (until mid-January 2023). The selection of papers is based on a comprehensive search with Dimensions.ai, followed by detailed screening and clustering. Readers will learn about the use of GA for mathematics, computation, surface representations, geometry, image- and signal processing, computing and software, quantum computing, data processing, neural networks, medical science, physics, electric engineering, control and robotics.
47. A lightweight storage framework for edge computing infrastructures/EdgePersist
Evangelos Psomakelis, Antonios Makris, Konstantinos Tserpes, Maria Pateraki.
The rise of Internet of Things (IoT) and the ever increasing volumes of data they are gathering and generating are putting a heavy strain on network infrastructures that are struggling to maintain the Quality of Service (QoS) requirements of applications. This work presents a novel edge storage solution that is moving most of the data storage, processing and exchange operation to the edge, near the devices that generate and/or use all of these data. Since this solution is meant for IoT edge networks it is designed to be decentralized, lightweight and inherently scalable, based on Kubernetes, MinIO and Dataset Lifecycle Framework.
46. Design, Technology, and Measurement Considerations in Virtual-Reality Assessment
Sean P. Gyll (Western Governors University, USA), Karen K. Shader (Western Governors University, USA), Paul Zikas (ORamaVR, Switzerland), and George Papagiannakis (University of Crete, Greece)
Virtual reality (VR) simulations as an assessment tool represent a much-needed effort to move beyond the shortcomings of today’s forms-based measures. Within VR, we assess for competency and problem-solving skills versus the content memorization typically supported by multiple-choice assessments. This chapter reviews the development process for the behavioral healthcare coordination VR assessment deployed at Western Governors University. It follows three patients undergoing behavioral health care treatment and highlights essential design, technology, and measurement considerations in developing a VR assessment. For any assessment program, construct validity is the chief validity component. This means that standards-based principles must be maintained to support the inferences drawn from test scores. However, without a framework for developing and maintaining those standards, assessment developers are left to their own devices to determine which practices are most likely to be effective. This chapter provides practical examples to aid assessment professionals in maintaining those standards.
45. Training for Better Transfer in an Online Competency-Based Higher Education Program: Technology's Role in Preparing the Next Generation Workforce
Sean P. Gyll (Western Governors University, USA), Karen K. Shader (Western Governors University, USA), Paul Zikas (ORamaVR, Switzerland), and George Papagiannakis (University of Crete, Greece)
Virtual reality simulations represent a much-needed effort to move beyond the shortcomings of traditional form-based assessments. Within VR, we assess competency and problem-solving skills versus the content memorization typically supported by conventional measures. This chapter explores an innovative VR simulation recently deployed at Western Governors University. The authors explored the utility of a VR simulation as an assessment tool when students engaged in more inclusive, immersive, and interactive experiences compared to conventional methods. The authors investigated students’ summative assessment scores across a 2D (desktop) and 3D VR (headset) version and how additional factors like motion sickness, cognitive workload, and system usability impacted their scores. The results showed that students in the Desktop version outperformed those in the VR version on the summative assessment while feeling equally immersed in the simulation. Implications for future research are discussed, especially for optimizing learning experiences in an online competency-based higher education program.
44. UniSG^GA: A 3D scenegraph powered by Geometric Algebra unifying geometry, behavior and GNNs towards generative AI
Manos Kamarianakis, Antonis Protopsaltis, Dimitris Angelis, Paul Zikas, Mike Kentros, George Papagiannakis
This work presents the introduction of UniSG^GA, a novel integrated scenegraph structure, that to incorporates behavior and geometry data on a 3D scene. It is specifically designed to seamlessly integrate Graph Neural Networks (GNNs) and address the challenges associated with transforming a 3D scenegraph (3D-SG) during generative tasks. To effectively capture and preserve the topological relationships between objects in a simplified way, within the graph representation, we propose UniSG^GA, that seamlessly integrates Geometric Algebra (GA) forms. This novel approach enhances the overall performance and capability of GNNs in handling generative and predictive tasks, opening up new possibilities and aiming to lay the foundation for further exploration and development of graph-based generative AI models that can effectively incorporate behavior data for enhanced scene generation and synthesis.
43. MAGES 4.0: Accelerating the world’s transition to VR training and democratizing the authoring of the medical metaverse
Paul Zikas, Antonis Protopsaltis, Nick Lydatakis, Mike Kentros, Stratos Geronikolakis, Steve Kateros, Manos Kamarianakis, Giannis Evangelou, Achilleas Filippidis, Eleni Grigoriou, Dimitris Angelis, Michail Tamiolakis, Michael Dodis, George Kokiadis, John Petropoulos, Maria Pateraki, George Papagiannakis
In this work, we propose MAGES 4.0, a novel Software Development Kit (SDK) to accelerate the creation of collaborative medical training applications in VR/AR. Our solution is essentially a low-code metaverse authoring platform for developers to rapidly prototype high-fidelity and high-complexity medical simulations. MAGES breaks the authoring boundaries across extended reality, since networked participants can also collaborate using different virtual/augmented reality as well as mobile and desktop devices, in the same metaverse world. With MAGES we propose an upgrade to the outdated 150-year-old master-apprentice medical training model. Our platform incorporates, in a nutsell, the following novelties: a) 5G edge-cloud remote rendering and physics dissection layer, b) realistic real-time simulation of organic tissues as soft-bodies under 10ms, c) a highly realistic cutting and tearing algorithm, d) neural network assessment for user profiling and, e) a VR recorder to record and replay or debrief the training simulation from any perspective.
Year of Publication: 2022
Effectiveness and Utility of Virtual Reality Simulation as an Educational Tool for Safe Performance of COVID-19 Diagnostics: Prospective, Randomized Pilot Trial.
Tanja Birrenbach, Josua Zbinden , George Papagiannakis, Aristomenis K Exadaktylos, Martin Müller, Wolf E Hautz, Thomas Christian Sauter
(JMIR Serious Games 2021;9(4):e29586)
Background: Although the proper use of hygiene and personal protective equipment (PPE) is paramount for preventing the spread of diseases such as COVID-19, health care personnel have been shown to use incorrect techniques for donning/doffing of PPE and hand hygiene, leading to a large number of infections among health professionals. Education and training are difficult owing to the social distancing restrictions in place, shortages of PPE and testing material, and lack of evidence on optimal training. Virtual reality (VR) simulation can offer a multisensory, 3-D, fully immersive, and safe training opportunity that addresses these obstacles.
Objective: The aim of this study is to explore the short- and long-term effectiveness of a fully immersive VR simulation versus a traditional learning method regarding a COVID-19–related skill set and media-specific variables influencing training outcomes.
Methods: This was a prospective, randomized controlled pilot study on medical students (N=29; intervention VR training, n=15, vs control video-based instruction, n=14) to compare the performance of hand disinfection, nasopharyngeal swab taking, and donning/doffing of PPE before and after training and 1 month later as well as variables of media use.
Results: Both groups performed significantly better after training, with the effect sustained over one month. After training, the VR group performed significantly better in taking a nasopharyngeal swab, scoring a median of 14 out of 17 points (IQR 13-15) versus 12 out of 17 points (IQR 11-14) in the control group, P=.03. With good immersion and tolerability of the VR simulation, satisfaction was significantly higher in the VR group compared to the control group (median score of User Satisfaction Evaluation Questionnaire 27/30, IQR 23-28, vs 22/30, IQR 20-24, in the control group; P=.01).
Conclusions: VR simulation was at least as effective as traditional learning methods in training medical students while providing benefits regarding user satisfaction. These results add to the growing body of evidence that VR is a useful tool for acquiring simple and complex clinical skills.
42. Towards Supporting XR Services: Architecture and Enablers
Tarik Taleb, Abderrahmane Boudi, Luis Rosa, Luis Cordeiro, Theodoros Theodoropoulos, Konstantinos Tserpes, Patrizio Dazzi, Antonis Protopsaltis, Richard Li
Emerging XR applications, including Holography, Augmented, Virtual and Mixed Reality, are characterized by unprecedented requirements for Quality of experience (QoE), largely exceeding those currently attainable. To cope with these requirements, noticeable efforts and a number of initiatives are ongoing to enhance the current communications technologies, especially in the direction of supporting ultra-low latency and increased bandwidth. This work proposes an architecture that puts together the key enablers to support future XR applications, highlighting the shortcomings of existing technologies and leveraging the ongoing innovations. It demonstrates the feasibility of the proposed architecture by describing the processes driving the platform with relevant use case scenarios, and mapping the envisioned functionality to existing tools.
41. A Digital Health Intervention for Stress and Anxiety Relief in Perioperative Care: Protocol for a Feasibility Randomized Controlled Trial
Haridimos Kondylakis, Irene Alice Chicchi Giglioli, Dimitrios G Katehakis, Hatice Aldemir, Paul Zikas, George Papagiannakis, Santiago Hors-Fraile, Pedro L González-Sanz, Konstantinos C Apostolakis, Constantine Stephanidis, Francisco J Núñez-Benjumea, Rosa M Baños-Rivera, Luis Fernandez-Luque, Angelina Kouroubali
Stress and anxiety are psychophysiological responses commonly experienced by patients during the perioperative process that can increase presurgical and postsurgical complications to a comprehensive and positive recovery. Preventing and intervening in stress and anxiety can help patients achieve positive health and well-being outcomes. Similarly, the provision of education about surgery can be a crucial component and is inversely correlated with preoperative anxiety levels. However, few patients receive stress and anxiety relief support before surgery, and resource constraints make face-to-face education sessions untenable. Digital health interventions can be helpful in empowering patients and enhancing a more positive experience. Digital health interventions have been shown to help patients feel informed about the possible benefits and risks of available treatment options. However, they currently focus only on providing informative content, neglecting the importance of personalization and patient empowerment.
40. Progressive tearing and cutting of soft-bodies in high-performance virtual reality
Manos Kamarianakis, Antonis Protopsaltis, Dimitris Angelis, Michail Tamiolakis, George Papagiannakis
We present an algorithm that allows a user within a virtual environment to perform real-time unconstrained cuts or consecutive tears, i.e., progressive, continuous fractures on a deformable rigged and soft-body mesh model in high-performance 10ms. In order to recreate realistic results for different physically-principled materials such as sponges, hard or soft tissues, we incorporate a novel soft-body deformation, via a particle system layered on-top of a linear-blend skinning model. Our framework allows the simulation of realistic, surgical-grade cuts and continuous tears, especially valuable in the context of medical VR training. In order to achieve high performance in VR, our algorithms are based on Euclidean geometric predicates on the rigged mesh, without requiring any specific model pre-processing. The contribution of this work lies on the fact that current frameworks supporting similar kinds of model tearing, either do not operate in high-performance real-time or only apply to predefined tears. The framework presented allows the user to freely cut or tear a 3D mesh model in a consecutive way, under 10ms, while preserving its soft-body behaviour and/or allowing further animation.
39. My two cents on the metaverse: why it is important and how to build it using latest computational science-based tools
George Papagiannakis
When the World Economic Forum brings together leading voices from the private sector, civil society, academia and policy to define the parameters of an economically viable, interoperable, safe and inclusive metaverse, it means that its forthcoming economic and social value is imminent, unavoidable and even already been implemented. Many might have objections with how the metaverse will be deployed, but I don’t think any organisations can afford to bypass or ignore its tremendous underlying potential and transformation on the way we work, play, learn and communicate.
In this short article, I will outline the latest science-based evidence of the importance of the metaverse and my proposal on how we should go about building it using science-based tools, in the most human-centric way and for the widest possible social benefits.
38. Relay-based network architectures for Collaborative Virtual Reality Applications
Fadia Hasnaoui, Lamia Zohra Mihoubi, Maria Pateraki, Miloud Bagaa
Currently deployed NAT devices are designed primarily around the client/server paradigm, in which relatively anonymous client machines inside a private network initiate connections to public servers with stable IP addresses and DNS names. Thus, the asymmetric addressing and connectivity regimes established by NAT devices have created unique problems for Peer-to-Peer (P2P) applications and protocols. Multiple NAT-traversal techniques have been developed to overcome these shortcomings, each offering a different set of pros and cons. In the context of a P2P collaborative virtual reality (CVR) system, the difficulty of selecting a convenient and effective NAT-traversal technique increases exponentially because of the added constraints related to CVR. In this view, this article discusses the trade-offs of different NAT-traversal techniques and the CVR challenges that need to be taken into account when choosing a NAT-traversal technique. Finally, it presents a relay-based approach that leverages container migration to mitigate the drawbacks that come with this solution and accentuate its advantages.
37. Assessing unconstrained surgical cuttings in VR using CNNs
Ilias Chrysovergis, Manos Kamarianakis, Mike Kentros, Dimitris Angelis, Antonis Protopsaltis, George Papagiannakis
In the recent years, industry and academia have massively adopted Virtual Reality (VR) applications to train students and personnel. Despite the effort, only limited systems involve procedures for assessing user progress inside the immersive environment, that either evaluate only trivial tasks or require a huge amount of time by the reviewers. On the other hand, the need for real-time automated evaluation of user’s actions is constantly increasing. State-of-the-art methods for similar tasks either require the development of complicated task specific computer vision algorithms or support very simple tasks.
This work proposes a deep learning based system, that is able to assess, in real-time, user actions within a VR training scenario. The method enables the rapid development of trained assessment functions, since it utilizes data augmentation to minimize the amount of labelled data that need to be collected. Furthermore, by using transfer learning, these assessment functions can be reconfigured to support similar tasks, thus reducing even more the amount of training data. In this paper, we present the results of our method for the task of tearing a deformable 3D model.
Different machine and deep learning algorithms were considered and compared. Ultimately, our proposed model is a Convolutional Neural Network (CNN), trained on a dataset created with a data augmentation technique.
36. Recording and replaying psychomotor user actions in VR
Manos Kamarianakis, Ilias Chrysovergis, Mike Kentros, George Papagiannakis
Nowadays, session recording and playback of a single or multiuser VR session has become an increasingly market-required asset. The need for effective VR recording and replaying (VRRR) is especially highlighted in virtual training applications, as replaying user actions can serve as an additional and powerful educational tool. Despite the effort, achieving VRRR is a task not natively undertaken by modern game engines and therefore most VR applications do not include such a feature by default.
Current bibliography contains numerous studies of how the VR record and replay features can enhance the learning impact that VR educational-oriented applications provide, by mainly measuring the performance of users. Usually, the data are captured in video format and a post process of this high-dimensional data is required to obtain any further analysis. Since video data is not sufficient for reasons we explain in Section 2, our approach is close to Kloiber et al., who proposed an analysis of user’s motion by recording their hands and head trajectories. Current ongoing research also explores the proper methods and data structures that must be employed to achieve real-time logging while keeping the required data storage manageable and allowing effective replay.
35. Realistic soft-body tearing under 10ms in VR
Manos Kamarianakis, Antonis Protopsaltis, Michail Tamiolakis, George Papagiannakis
Rigged animated models are one of the most researched areas in computer graphics and have been vastly adopted in Virtual Reality (VR) applications. VR experts experiment with various animation and deformation techniques that can yield realistic real-time outputs. To cover the needs that arise from a variety of use cases, our research revolves around the ability to perform realistic tears, i.e., small cuts, on the surface of a model. Current bibliography describes diverse ways on how to cut a 3D model, but most of these methods are not suitable for VR, since the specific calculations must be performed in a real-time manner within a few ms to preserve user immersion. Furthermore, to avoid the uncanny valley in VR, we emulate realistic effects while performing cuts on certain materials, such as a sponge or human tissues, that one would expect to occur in real life.
Latest developments allow for basic operations, such as cutting, tearing or drilling on a rigged mesh model, to be run in near real-time. Furthermore, the replication of the physical behaviour of soft bodies, when applying external forces to them in VR, would greatly increase the overall realism of the simulation. The ongoing research for increased realism in virtual environments heavily impacts educational-oriented applications, especially the ones regarding VR medical training.
34. Less Is More: Efficient Networked VR Transformation Handling Using Geometric Algebra
Manos Kamarianakis, Ilias Chrysovergis, Nick Lydatakis, Mike Kentros, George Papagiannakis
As shared, collaborative, networked, virtual environments become increasingly popular, various challenges arise regarding the efficient transmission of model and scene transformation data over the network. As user immersion and real-time interactions heavily depend on VR stream synchronization, transmitting the entire data sat does not seem a suitable approach, especially for sessions involving a large number of users. Session recording is another momentum-gaining feature of VR applications that also faces the same challenge. The selection of a suitable data format can reduce the occupied volume, while it may also allow effective replication of the VR session and optimized post-processing for analytics and deep-learning algorithms. In this work, we propose two algorithms that can be applied in the context of a networked multiplayer VR session, to efficiently transmit the displacement and orientation data from the users’ hand-based VR HMDs. Moreover, we present a novel method describing effective VR recording of the data exchanged in such a session. Our algorithms, based on the use of dual-quaternions and multivectors, impact the network consumption rate and are highly effective in scenarios involving multiple users. By sending less data over the network and interpolating the in-between frames locally, we manage to obtain better visual results than current state-of-the-art methods. Lastly, we prove that, for recording purposes, storing less data and interpolating them on-demand yields a data set quantitatively close to the original one.
33. Virtual Reality Medical Training for COVID-19 Swab Testing and Proper Handling of Personal Protective Equipment: Development and Usability
Paul Zikas, Steve Kateros, Nick Lydatakis, Mike Kentros, Efstratios Geronikolakis, Giannis Evangelou, Manos Kamarianakis, Ioanna Kartsonaki, Achilles Apostolou, Tanja Birrenbach, Aristomenis K. Exadaktylos, Thomas C. Sauter, George Papapagiannakis
Efficient and riskless training of healthcare professionals is imperative as the battle against the Covid-19 pandemic still rages. Recent advances in the field of Virtual Reality (VR), both in software and hardware level, unlocked the true potential of VR medical education (Hooper et al., The Journal of Arthroplasty, 2019, 34 (10), 2,278–2,283; Almarzooq et al., Virtual learning during the COVID-19 pandemic: a disruptive technology in graduate medical education, 2020; Wayne et al., Medical education in the time of COVID-19, 2020; Birrenbach et al., JMIR Serious Games, 2021, 9 (4), e29586). The main objective of this work is to describe the algorithms, models and architecture of a medical virtual reality simulation aiming to train medical personnel and volunteers in properly performing Covid-19 swab testing and using protective measures, based on a world-standard hygiene protocol. The learning procedure is carried out in a novel and gamified way that facilitates skill transfer from virtual to real world, with performance that matches and even exceeds traditional methods, as shown in detail in (Birrenbach et al., JMIR Serious Games, 2021, 9 (4), e29586). In this work we are providing all computational science methods, models together with the necessary algorithms and architecture to realize this ambitions and complex task verified via an in-depth usability study with year 3–6 medical school students.
32. Virtual Reality Systems, Tools, and Frameworks
Antonis Protopsaltis, George Papapagiannakis
A virtual reality system is a software system that generates the illusion of immersion and presence within a virtual environment. A virtual reality framework is a customizable application that consists of design patterns and components, which assist virtual reality developers with its modularity, reusability, and extensibility. Virtual reality systems and frameworks may be developed using special purpose virtual reality development tools.
31. Virtual Reality: A Model for Understanding Immersive Computing
Antonis Protopsaltis, George Papapagiannakis
Virtual reality model allows the user to subjectively be involved and become immersed within a computer-generated environment. As the physical world is hidden, the user realizes the virtual environment as real while the acquired sense of presence enhances the feeling of immersion.
30. Cloud-based XR Services: A Survey on Relevant Challenges and Enabling Technologies
Theodoros Theodoropoulos, Antonios Makris, Abderrahmane Boudi, Tarik Taleb, Uwe Herzog, Luis Rosa, Luis Cordeiro, Konstantinos Tserpes, Elena Spatafora, Alessandro Romussi, Enrico Zschau, Manos Kamarianakis, Antonis Protopsaltis, George Papagiannakis and Patrizio Dazzi10
In recent years, the emergence of XR (eXtended Reality) applications, including Holography, Augmented, Virtual and Mixed Reality, has resulted in the creation of rather demanding requirements for Quality of Experience (QoE) and Quality of Service (QoS). In order to cope with requirements such as ultra-low latency and increased bandwidth, it is of paramount importance to leverage certain technological paradigms. The purpose of this paper is to identify these QoE and QoS requirements and then to provide an extensive survey on technologies that are able to facilitate the rather demanding requirements of Cloud-based XR Services. To that end, a wide range of enabling technologies are explored. These technologies include e.g. the ETSI (European Telecommunications Standards Institute) Multi-Access Edge Computing (MEC), Edge Storage, the ETSI Management and Orchestration (MANO), the ETSI Zero touch network & Service Management (ZSM), Deterministic Networking, the 3GPP (3rd Generation Partnership Project) Media Streaming, MPEG’s (Moving Picture Experts Group) Mixed and Augmented Reality standard, the Omnidirectional MediA Format (OMAF), ETSI’s Augmented Reality Framework etc.
Year of Publication: 2021
30. Effectiveness and Utility of Virtual Reality Simulation as an Educational Tool for Safe Performance of COVID-19 Diagnostics: Prospective, Randomized Pilot Trial.
Tanja Birrenbach, Josua Zbinden , George Papagiannakis, Aristomenis K Exadaktylos, Martin Müller, Wolf E Hautz, Thomas Christian Sauter
(JMIR Serious Games 2021;9(4):e29586)
Background: Although the proper use of hygiene and personal protective equipment (PPE) is paramount for preventing the spread of diseases such as COVID-19, health care personnel have been shown to use incorrect techniques for donning/doffing of PPE and hand hygiene, leading to a large number of infections among health professionals. Education and training are difficult owing to the social distancing restrictions in place, shortages of PPE and testing material, and lack of evidence on optimal training. Virtual reality (VR) simulation can offer a multisensory, 3-D, fully immersive, and safe training opportunity that addresses these obstacles.
Objective: The aim of this study is to explore the short- and long-term effectiveness of a fully immersive VR simulation versus a traditional learning method regarding a COVID-19–related skill set and media-specific variables influencing training outcomes.
Methods: This was a prospective, randomized controlled pilot study on medical students (N=29; intervention VR training, n=15, vs control video-based instruction, n=14) to compare the performance of hand disinfection, nasopharyngeal swab taking, and donning/doffing of PPE before and after training and 1 month later as well as variables of media use.
Results: Both groups performed significantly better after training, with the effect sustained over one month. After training, the VR group performed significantly better in taking a nasopharyngeal swab, scoring a median of 14 out of 17 points (IQR 13-15) versus 12 out of 17 points (IQR 11-14) in the control group, P=.03. With good immersion and tolerability of the VR simulation, satisfaction was significantly higher in the VR group compared to the control group (median score of User Satisfaction Evaluation Questionnaire 27/30, IQR 23-28, vs 22/30, IQR 20-24, in the control group; P=.01).
Conclusions: VR simulation was at least as effective as traditional learning methods in training medical students while providing benefits regarding user satisfaction. These results add to the growing body of evidence that VR is a useful tool for acquiring simple and complex clinical skills.
29. Effectiveness and Utility of Virtual Reality Simulation as an Educational Tool for Safe Performance of COVID-19 Diagnostics: Prospective, Randomized Pilot Trial.
Tanja Birrenbach, Josua Zbinden , George Papagiannakis, Aristomenis K Exadaktylos, Martin Müller, Wolf E Hautz, Thomas Christian Sauter
(JMIR Serious Games 2021;9(4):e29586)
Background: Although the proper use of hygiene and personal protective equipment (PPE) is paramount for preventing the spread of diseases such as COVID-19, health care personnel have been shown to use incorrect techniques for donning/doffing of PPE and hand hygiene, leading to a large number of infections among health professionals. Education and training are difficult owing to the social distancing restrictions in place, shortages of PPE and testing material, and lack of evidence on optimal training. Virtual reality (VR) simulation can offer a multisensory, 3-D, fully immersive, and safe training opportunity that addresses these obstacles.
Objective: The aim of this study is to explore the short- and long-term effectiveness of a fully immersive VR simulation versus a traditional learning method regarding a COVID-19–related skill set and media-specific variables influencing training outcomes.
Methods: This was a prospective, randomized controlled pilot study on medical students (N=29; intervention VR training, n=15, vs control video-based instruction, n=14) to compare the performance of hand disinfection, nasopharyngeal swab taking, and donning/doffing of PPE before and after training and 1 month later as well as variables of media use.
Results: Both groups performed significantly better after training, with the effect sustained over one month. After training, the VR group performed significantly better in taking a nasopharyngeal swab, scoring a median of 14 out of 17 points (IQR 13-15) versus 12 out of 17 points (IQR 11-14) in the control group, P=.03. With good immersion and tolerability of the VR simulation, satisfaction was significantly higher in the VR group compared to the control group (median score of User Satisfaction Evaluation Questionnaire 27/30, IQR 23-28, vs 22/30, IQR 20-24, in the control group; P=.01).
Conclusions: VR simulation was at least as effective as traditional learning methods in training medical students while providing benefits regarding user satisfaction. These results add to the growing body of evidence that VR is a useful tool for acquiring simple and complex clinical skills.
28. Virtual Reality surgical medical training in the post-Covid-19 pandemic era
G. Papagiannakis, E. Kenanidis, N. Milonakis, M. Potoupnis, E. Tsiridis
(Surgical Life: The Journal of the Association of Surgeons of Great Britain and Ireland, Number 60)
Surgical training has progressed extensively from the historical apprenticeship of the early surgeons to a modern complicated structured training programme with numerous duties and evaluations. Expertise must be gained in a shorter period whilst requiring a greater set of skills than previously. Numerous novel surgical techniques have been developed, demanding the current trainees to learn a broader array of specialist skills, despite having less time to do so. As the number of trainees increases, the chances to acquire procedural and technical skills become gradually limited. Furthermore, the introduction of working hour restrictions and a drive towards senior-led care have reduced the available training hours during the des-ignated training period.
27. ORamaVR Streamlines Medical Training for Healthcare Professionals
According to the World Health Organization, our planet will need more than 40 million new doctors, nurses, frontline healthcare workers, and other healthcare professionals by the year 2030, which is double the current medical workforce. This demand will be fueled by several different factors – most notably, a growing geriatric population. People are living much longer, thanks in part to improved healthcare. But that brings a rise in chronic conditions such as cardiovascular disease, diabetes, and cancer.
In light of the COVID-19 pandemic, it’s sobering to realize that if we don’t act now to implement new medical training solutions, an additional deficit of 18 million healthcare workers will compound the current existing shortage. However, the 150-year-old training model (a master teaching an apprentice over the course of several years) is unable to meet the level of healthcare professionals needed.
Empirical evidence from other industries clearly demonstrates virtual reality (VR) technology is an effective and efficient way of improving training. However, VR implementation in the medical industry has been slow because of the cost to develop and customize software, limiting accessibility where it is needed the most – in the hands of medical instructors and learners.
26. Architectures for SLAM and Augmented Reality Computing
25. Never ‘Drop the Ball’ in the Operating Room: An efficient hand-based VR HMD controller interpolation algorithm, for collaborative, networked virtual environments
24. Inter-operability and Orchestration in Heterogeneous Cloud/Edge Resources: The ACCORDION Vision
23. Covid-19 – VR Strikes Back: innovative medical VR training
22. An All-In-One Geometric Algorithm for Cutting, Tearing, and Drilling Deformable Models
21. Α Virtual Reality App for Physical and Cognitive Training of Older People With Mild Cognitive Impairment: Mixed Methods Feasibility Study
Year of Publication: 2020
Covid-19 – VR Strikes Back: innovative medical VR training
20. MAGES 3.0: Tying the knot of medical VR
In this work, we present MAGES 3.0, a novel Virtual Reality (VR)-based authoring SDK platform for accelerated surgical training and assessment. The MAGES Software Development Kit (SDK) allows code-free prototyping of any VR psychomotor simulation of medical operations by medical professionals, who urgently need a tool to solve the issue of outdated medical training. Our platform encapsulates the following novel algorithmic techniques: a) collaborative networking layer with Geometric Algebra (GA) interpolation engine b) supervised machine learning analytics module for real-time recommendations and user profiling c) GA deformable cutting and tearing algorithm d) on-the-go configurable soft body simulation for deformable surfaces.
19. Scenior: An Immersive Visual Scripting system based on VR Software Design Patterns for Experiential Training
Year of Publication: 2019
Effectiveness and Utility of Virtual Reality Simulation as an Educational Tool for Safe Performance of COVID-19 Diagnostics: Prospective, Randomized Pilot Trial.
Tanja Birrenbach, Josua Zbinden , George Papagiannakis, Aristomenis K Exadaktylos, Martin Müller, Wolf E Hautz, Thomas Christian Sauter
(JMIR Serious Games 2021;9(4):e29586)
Background: Although the proper use of hygiene and personal protective equipment (PPE) is paramount for preventing the spread of diseases such as COVID-19, health care personnel have been shown to use incorrect techniques for donning/doffing of PPE and hand hygiene, leading to a large number of infections among health professionals. Education and training are difficult owing to the social distancing restrictions in place, shortages of PPE and testing material, and lack of evidence on optimal training. Virtual reality (VR) simulation can offer a multisensory, 3-D, fully immersive, and safe training opportunity that addresses these obstacles.
Objective: The aim of this study is to explore the short- and long-term effectiveness of a fully immersive VR simulation versus a traditional learning method regarding a COVID-19–related skill set and media-specific variables influencing training outcomes.
Methods: This was a prospective, randomized controlled pilot study on medical students (N=29; intervention VR training, n=15, vs control video-based instruction, n=14) to compare the performance of hand disinfection, nasopharyngeal swab taking, and donning/doffing of PPE before and after training and 1 month later as well as variables of media use.
Results: Both groups performed significantly better after training, with the effect sustained over one month. After training, the VR group performed significantly better in taking a nasopharyngeal swab, scoring a median of 14 out of 17 points (IQR 13-15) versus 12 out of 17 points (IQR 11-14) in the control group, P=.03. With good immersion and tolerability of the VR simulation, satisfaction was significantly higher in the VR group compared to the control group (median score of User Satisfaction Evaluation Questionnaire 27/30, IQR 23-28, vs 22/30, IQR 20-24, in the control group; P=.01).
Conclusions: VR simulation was at least as effective as traditional learning methods in training medical students while providing benefits regarding user satisfaction. These results add to the growing body of evidence that VR is a useful tool for acquiring simple and complex clinical skills.
17. iSupport: Building a Resilience Support Tool for Improving the Health Condition of the Patient During the Care Path
15. From Readership to Usership and Education, Entertainment, Consumption to Valuation: Embodiment and Aesthetic Experience in Literature-based MR Presence
14. Virtual Reality Simulation Facilitates Resident Training in Total Hip Arthroplasty: A Randomized Controlled Trial
13. Digital Health Tools for Perioperative Stress Reduction in Integrated Care
Year of Publication: 2018
Year of Publication: 2017
Digital Health Tools for Perioperative Stress Reduction in Integrated Care
10. Real-time rendering under distant illumination with Conformal Geometric Algebra
9. Gamification and Serious Games
8. Augmented Cognition via Brainwave Entrainment in Virtual Reality: An Open, Integrated Brain Augmentation in a Neuroscience System Approach
Year of Publication: 2016-before
Digital Health Tools for Perioperative Stress Reduction in Integrated Care
7. A Mobile, AR Inside-Out Positional Tracking Algorithm, (MARIOPOT), Suitable for Modern, Affordable Cardboard-Style VR HMDs
6. glGA: an OpenGL Geometric Application framework for a modern, shader-based computer graphics curriculum
5. A survey of mobile and wireless technologies for augmented reality systems
