Virtual and augmented reality

In today's advanced era of medical science and techniques, acquiring comprehensive knowledge for conducting medical examinations has become increasingly challenging. While theoretical learning remains essential in the medical field, such as when performing ultrasound scans, the demand for practical learning opportunities prior to working with real patients has grown significantly. By leveraging on VR technology, we propose a solution that provide learners with an immersive and realistic environment where they can undergo training in ultrasound scans. This approach aims to simulate the experience of conducting actual scans, enabling learners to develop their skills effectively. In this paper, we describe our solution focusing on the technique we developed to transform any set of DICOM images into visually realistic ultrasound images to be used inside our real-time VR-based training simulation. The use of VR can overcome the financial obstacles associated with procuring expensive equipment for training purposes, while the flexibility of our approach enhances the learning experience by adapting to various scenarios and accommodating different skill levels. Learners can navigate through a range of simulated cases, reinforcing their knowledge and competence in ultrasound techniques.

Keywords: Practical Learning , Ultrasound Simulation , Virtual Reality

Manufacturing industries today are undergoing a formidable process of transition to completely new and sometimes unfamiliar modes of operation. This process is also known as the 5th Industrial Revolution and involves the implementation of a set of technologies including, among others, digital technologies such as XR-Extended Reality, AI, Big Data and Digital Twins. In particular, Extended Reality, which encompasses the full spectrum of advanced 3D models and techniques for visualizing and interacting environments, is already a standard de facto method at some stages of the product life cycle. In the design phase of complex systems, for example, it supports designers, other business functions and end users in the collaborative analysis of complex assemblies. While these technologies are supported as enablers for the aforementioned transition in industries, their use at the academic level remains focused on the goal of creating more interactive educational experiences and enhancing understanding of complex concepts. In contrast, this paper analyzes the use of XR in mechanical engineering curricula with the goal of identifying the main challenges for creating advanced digital skills for engineers and empowering them to integrate such knowledge into industrial process optimization.

Keywords: Engineering Education , Extended Reality , Focus Group , Mechanical Engineering

Recently, most research efforts and resourced in Aeronautics are devoted to the development of Advanced Air Mobility (AAM) technologies. This term refers to all air mobility services based on autonomy and electrification of air systems and their infrastructure and when conducted in urban contexts is known as Urban Air Mobility (UAM). However, the integration of these systems and what is their environmental, social and economic impact is still unknown and such ignorance may decelerate its effective implementation and thus enjoyment of the benefits. In this context, visualization and interaction technologies such as Extended Reality Technologies (XR) can be leveraged as enabling technologies aiding in the deployment of AAM and UAM. In this paper, we present an overview of methodologies for designing human-machine interfaces in aeronautics and the results of a study of tools and methods for prototyping virtual urban environments for XR-UAM applications.

Keywords: Human Machine Interfaces , UAM - Urban Air Mobility , XR – Extended Reality

Design review is a tool used to identify errors in products before their physical prototype is realized. Currently, 2-dimensional (2D) Computer Aided Design (CAD) digital or paper drawings are used to conduct design review. There is the risk that users who are not practical with CAD software may be excluded from this process. In addition, the dimensions and scales of CAD models may be lost due to the complexity of displaying them on a PC screen. VR technologies overcome these limitations. The idea of “Virtual Design Review” (VDR) allows users to live an immersive experience and see, manipulate, test, examine, and identify errors of virtual products in real-time, realistically and in total safety already in the early design stages, without undertaking the time and costs of building them. The objective of this work is to propose a proper workflow for conducting VDR sessions on complex assemblies, whose design process is developed over a Product Lifecycle Management (PLM) platform like 3D Experience. VR modules offered by this platform to execute VDRs are explored and tested.

Keywords: 3D Experience platform , Cloud-Based Design and Manufacturing , Fusion Engineering , PLM , Virtual Design Review and Digital Twin

Despite the European Union's efforts to decrease workplace accidents, statistics show an increase in accidents in the last years. Therefore, training workers is crucial to prevent injuries. The thriving of new technologies, such as Immersive Virtual Reality, can offer innovative training methods in the industrial field. However, introducing innovative technologies to a user base with an increasing average age necessitates evaluating the impact on workers across various age groups. In this contribution, we developed an IVR system to train workers to learn procedures in industrial environments. We recruited 50 participants and divided them into three age groups: young (19–33 years), middle-aged (34–48 years), and older (49–63 years). Objective (execution errors and times) and subjective metrics (User Experience Questionnaire (UEQ), System Usability Scale (SUS), and Virtual Reality Sickness Questionnaire (VRSQ)) were collected. The results showed that the young group made more mistakes than the other two groups, but there were no statistically significant differences in execution times. The SUS score was statistically significantly higher for the young group than the others, the VRSQ did not show any significant increase.

Keywords: Age-related Performance , Immersive Virtual Reality (IVR) , Industrial Safety Training , User Experience Metrics , Workplace Accident Prevention

The industrial revolutions driven by Industry 4.0 and 5.0 are transforming work practices with new tools like Augmented Reality (AR), which promise to boost productivity in industrial settings. This study introduces an AR application for the HoloLens 2 to evaluate three interaction metaphors - hand-ray, near, and gaze interaction - for moving virtual objects, focusing on accuracy and ergonomics. ErgoSentinel software provided real-time ergonomic analysis through RULA scores, and the NASA-TLX questionnaire assessed perceived workload. Results indicate that ray interaction is the most ergonomically favorable for long-distance tasks, reducing physical strain and enhancing comfort. Near interaction is precise for short-distance tasks but increases physical strain over long distances. Gaze interaction balances physical strain reduction with potential precision loss in long-distance tasks, imposing a higher cognitive load.

Keywords: Augmented Reality , Ergonomics , Industry 5.0 , RULA

Modern design products are increasingly complex and emotionally significant, demanding versatile and collaborative customization. However, a literature and commercial review reveals a limited availability of flexible, multi-user, photorealistic Virtual Reality (VR) systems for product configuration. We introduce MAGIC (Multi-user Advanced Graphic Immersive Configurator), a collaborative platform combining realistic graphics with ergonomic validation using digital avatars, addressing the limitations of 2D visualization and existing tools. MAGIC is evaluated in a yacht design case study involving 30 participants in an immersive, co-located configuration of a sailing yacht to assess the system's usability and the potential of VR for customizing complex products. Results show MAGIC's feasibility in supporting multi-user configuration (100% success rate) and achieving a strong usability score (SUS = 80.83). User feedback highlights that high-quality graphics and additional content significantly enhance immersion and user engagement. However, encountered challenges with navigation methods and spatial perception indicate areas for improvement. MAGIC's collaborative and immersive capabilities can be extended to other industries demanding proactive customer engagement in the customization of large, heavy products and ergonomic design.

Keywords: collaboration , marine industry , multi-user , product configuration , virtual reality , yacht design

Virtual reality (VR) technologies have shown significant potential in enhancing the automotive design process by providing immersive, interactive, and collaborative environments for reviewing complex vehicle architectures. Traditional CAD models and physical prototypes often struggle to offer the flexibility needed for rapid iteration and spatial understanding, especially for systems such as hydrogen-powered commercial vehicles. This study aims to develop a VR platform to improve the collaborative review of the internal architecture of a hydrogen-powered commercial vehicle, specifically focusing on the spatial arrangement and interaction of critical components such as hydrogen tanks, fuel cells, and batteries. The goal is to enhance design processes, reduce iteration costs, and improve spatial understanding and decision-making within automotive teams. The VR environment was built using Unity 3D and optimized for mobile VR platforms, incorporating features such as interactive component manipulation, dual locomotion modes (teleportation and free movement), and real-time collaboration. The platform was tested with a range of users, including automotive professionals and students, to evaluate its usability, performance, and impact on spatial understanding and user engagement. User feedback demonstrated high levels of satisfaction, with 85% of participants reporting improved spatial understanding, 80% preferring VR over traditional CAD tools, and 90% noting increased engagement. The VR platform enabled faster completion of assembly tasks (40% faster than traditional methods) and improved collaborative decision-making. However, challenges such as system performance issues, motion sickness, and a steep learning curve were reported by some users, particularly nontechnical users. The VR platform developed in this study significantly advanced the collaborative and immersive review of automotive designs, offering tangible benefits in terms of engagement, spatial comprehension, and design iteration efficiency. While there were limitations related to hardware requirements and user accessibility, the results highlight the transformative potential of VR in automotive design and education.

The rise of the Metaverse is anticipated to extend the duration of time individuals spend within virtual indoor environments, potentially affecting human well-being. The hype generated by the launch of Apple Vision Pro has increasingly spread the idea of using virtual platforms as a workspace. Our research explores the idea of applying biophilic environmental design within the Metaverse, hence our definition: Metagreenverse. The biophilia hypothesis suggests that humans have an innate tendency to seek connections with the natural world and there is increasing evidence that biophilia can yield health benefits. In this preliminary study, we designed a metaverse office enriched with biophilic elements and investigated how the plants can affect the user experience of 24 participants compared to the same office without biophilic elements. We evaluated the user experience by submitting the User Experience Questionnaire (UEQ), the user emotional response with State-Trait Anxiety Inventory (STAI) questionnaire, the realism of the virtual office with Slater-Usoh-Steed (SUS) test, and the perception time. We found that the integration of biophilic elements into a metaverse workspace influences positively the users' experience.

Keywords: Biophilic Design , Metaverse , Virtual Reality , Well-being , Workspace

This study aims to explore the use of Augmented Reality (AR) visual assets to convey procedural instructions, specifically for conveying information about component orientation. We focused on assembly scenarios where no affordance is provided for orientation while maintaining a consistently high affordance for how components are mounted. This information is recurrent in tasks where users are familiar with components that fit together without needing specialized tools but lack knowledge of the specific orientations required for the assembly. A typical example is placing rubber gaskets that fit smoothly into grooves but where no markings indicate the correct sealing side. We evaluated six different AR presentation modes for conveying component orientation: image, video, static side-by-side product model, animated side-by-side product model, static in-situ product model, and animated insitu product model. The literature provides no clear agreement on which is the most effective. To fill this gap, we conducted a user study with 36 participants, measuring completion time, accuracy, and cognitive load across the six AR presentation modes. We also analyzed how users interacted with each of them and collected user subjective feedback.

Keywords: assembly , Augmented Reality , authoring , instruction , orientation , visual asset

Extended Reality (XR) refers to technologies that allow linking virtual and real worlds and include, in a broad sense, Virtual (VR), Augmented (AR), or Mixed (MR) Reality. The interest in XR is increasing mainly in training operations; however, it is important to consider that XR techniques could sacrifice realism and could require high costs to reproduce real environments. The present study addresses this problem and aims to experimentally test how the level of realism can be implemented and increased in practice and how different levels of realism impact the user experience. A framework for the reconstruction of 3D scenes using photogrammetry is proposed and an experimental campaign is presented to understand the user experience of an XR application, with particular attention given to participants' feedback about the level of realism of the virtual scenario. The focus of the research is to outline the improvement of the virtual experience owed to the photogrammetric reconstruction. Participants were asked to give their opinions on the comparison between the reconstructed 3D environment and a conventional shaded virtual environment made up of 3D CAD models. This paper contributes to the still limited empirical debate about the implementation of different levels of realism in virtual applications and its impact on participants' experience. Results show that participants appreciated the photogrammetric reconstructed realistic scenario, proving that realistic scenes can improve the user experience.

Keywords: 3D modeling , experimental campaign , photogrammetry , questionnaire , realism , user experience , virtual reality

The Radial Neutron Camera (RNC) is an ITER diagnostic designed to measure the un-collided 14 MeV and 2.5 MeV neutrons from deuterium-tritium (DT) and deuterium-deuterium (DD) fusion reactions, through an array of detectors covering a poloidal plasma section along collimated Lines Of Sight (LOS). The Ex-Port RNC is located in the Port Cell (PC) Interspace zone, the first area outside the Vacuum Vessel, and includes several detector units, mounted inside detector boxes. On the base of reliability considerations, the Radial Neutron Camera Maintenance Plan (RNC-MP) has foreseen a hands-on maintenance task to replace detectors unit equipment in case of failure. Despite not precluding workers access, the radiation field at shutdown in the PC Interspace zone requires optimization of hands-on operations to enable the minimization of occupational radiation exposure under the ALARA (As Low As Reasonably Achievable) principle. The relevant complexity and dimensions of the Ex-Port RNC requires a maintenance evaluation method different from a physical mock-up. The Virtual Reality technology may help to create a digital mock-up for evaluating the detectors replacement operations. The paper aims to demonstrate the potentialities of such technologies, in a field where safety and worker ergonomics are primary priorities. Thanks to the virtual scenario built, the RNC-MP updated steps are evaluated, improved and finally validated.

Keywords: Ex-port radial neutron camera , ITER , Radial neutron camera , Verification and validation , Virtual reality , VR simulation

The present paper proposes a description of the Virtual Reality (VR) use within the System Engineering approaches. The VR is spread among different design context and for different purposes, especially during Validation and Verification phase (V&V). A case study is introduced, derived from the nuclear fusion project ITER. The scope of the work is to validate the maintainability of the Radial Neutron Camera (RNC) Ex-Port System. The RNC Ex-port system is a diagnostic tool for detection of un-collided neutrons and gain information about the plasma state. The context of this system needs precise and safe maintenance procedures due to the harsh and potentially dangerous environment. The VR method has been used for the validation of procedures. The CAD model has been adapted into the software Unity and needed scripts have been introduced. The substitution of a detector (the 4He scintillator) is taken into consideration, verifying the feasibility for one operator.

Keywords: ITER , Maintenance , Systems Engineering , Verification &Validation , Virtual Reality

Background: Mandibular reconstruction has evolved significantly since its inception in the early 1900s. Currently, the fibula free flap (FFF) is considered the gold standard for mandibular and maxillary reconstructions, particularly for extensive defects, and the introduction of Extended Reality (XR) and virtual surgical planning (VSP) is revolutionizing maxillofacial surgery. Methods: This study focuses on evaluating the accuracy of using in-house cutting guides for mandibular reconstruction with FFF supported by virtual surgical planning (VSP). Planned and intraoperative osteotomies obtained from postoperative CT scans were compared in 17 patients who met the inclusion criteria. The proposed analysis included measurements of deviation angles, thickness at the centre of gravity, and the maximum thickness of the deviation volume. Additionally, a mandibular resection coding including 12 configurations was defined to classify and analyze the precision of mandibular osteotomies and investigate systematic errors. Preoperative, planned, and postoperative models have been inserted in an interactive VR environment, VieweR, to enhance surgical planning and outcome analysis. Results: The results proved the efficiency of adopting customized cutting guides and highlighted the critical role of advanced technologies such as CAD/CAM and VR in modern maxillofacial surgery. A novel coding system including 12 possible configurations was developed to classify and analyze the precision of mandibular osteotomies. This system considers (1) the position of the cutting blade relative to the cutting plane of the mandibular guide; (2) the position of the intersection axis between the planned and intraoperative osteotomy relative to the mandible; (3) the direction of rotation of the intraoperative osteotomy plane around the intersection axis from the upper view of the model. Conclusions: This study demonstrates the accuracy and reliability of in-house cutting guides for mandibular reconstruction using fibula free flaps (FFF) supported by virtual surgical planning (VSP). The comparison between planned and intraoperative osteotomies confirmed the precision of this approach, with minimal deviations observed.

Keywords: fibula free flap , maxillofacial surgery , virtual reality , virtual surgical planning , VR , VSP

This paper presents an innovative approach designed to simplify the decision-making process in the installation of Projector-based Spatial Augmented Reality (SAR) systems through the utilization of the Design for Mass Personalization methodology. The developed solution addresses the inherent complexities of defining platform hardware components, streamlining setup procedures, and minimizing the necessity for extensive technical expertise and time investment. Implemented as a Blender add-on, this solution provides users with a 3D model of the platform setup alongside a comprehensive list of components accompanied by technical specifications. Validation results demonstrate the configurator's capability to generate valid configurations when compared to existing setups. Additionally, studies involving experts in AR/VR technologies confirm its satisfactory usability.

Keywords: Decision Support , Design for Mass personalization , Projector-based augmented reality

Maintenance operators usually face complex challenges due to the complexity of modern sophisticated machinery and systems. Only highly experienced operators may have sufficient skills to perform recovery procedures properly and quickly; frequently, not expert operators need for assistance from the manufacturer's after-sales. Therefore, very long recovery times, high maintenance costs, and temporary drop in production take place. In this context, augmented reality (AR) can offer valid support to non-expert operators involved in maintenance activities. In this work, a framework based on user experience (UX) design approach was developed for designing highly usable AR applications to guide operators in performing maintenance procedures safely and efficiently. The proposed framework has been applied to the H2020 European project titled “XMANAI – Explainable Manufacturing Artificial Intelligence”. Results showed that the application developed is suitable to assist not expert operators, preventing them from making mistakes, leading to the upskill and increased productivity for the company.

Keywords: augmented reality , Industry 5.0 , Operator 4.0 , transdisciplinary engineering , user experience design

The fourth industrial revolution (I4.0) is driving the development of automated and intelligent manufacturing systems, but managing these systems poses challenges, especially for maintenance and process control. The emerging Industry 5.0 (I5.0) concept advocates for a human-centered approach, integrating human skills with advanced technologies to address such challenges. This study introduces a novel holistic evaluation of ergonomics, moving beyond traditional physical assessments to include cognitive ergonomics, thus providing a more comprehensive analysis of operator interactions with industrial systems. The proposed approach leverages Virtual Reality (VR) simulations combined with wearable devices to collect data on users' movements and physiological parameters. This enables simultaneous evaluation of physical workload and cognitive demands, fostering early identification of ergonomic issues in the design process. Unlike traditional methods, our framework emphasizes a deeper integration of devices for enhanced data analysis, ensuring that both physical and cognitive factors are addressed. As a case study, the redesign of an automated packaging machine was undertaken. Results demonstrated significant ergonomic improvements, reducing the EAWS (European Assessment Work-Sheet) score (from 30% to 50% approx.) and incorporating cognitive ergonomics into the evaluation. These findings highlight the potential of this approach to enhance the operator's experience and optimize human-machine interaction in modern industrial environments.

Keywords: digital simulation , ergonomics and human factors , Human-centered design , industry 5.0 , virtual maintenance , virtual reality

The rapid evolution from Industry 4.0 to Industry 5.0 has heightened the need for integrating sustainability practices within complex system designs, particularly in the manufacturing sector. The research presents the development of a Sustainability Digital Twin (SDT) framework aimed at enhancing sustainable design practices for complex engineering systems. This framework integrates digital product models with sustainability indicators and uses Virtual Reality (VR) and Augmented Reality (AR) to promote Green Design principles. The approach begins with the definition of specific Sustainability Key Performance Indicators (S-KPIs) obtained by Life Cycle Assessment (LCA) tools. After that, using VR and AR allows to design and develop customized interactive dashboards to provide a comprehensive overview of sustainability indicators across the social, environmental, and economic dimensions, and to support decision-making along the product lifecycle. A core element of this transdisciplinary methodology is the adoption of a human-centered design philosophy, ensuring that user interfaces are intuitive and user-friendly, to Green Design tools, with the support of advanced digital technologies. The integration of human-machine interaction models aims at extending the current Green Design practices, mainly focused on environmental performances, towards social areas, including human-centric aspects such as usability, accessibility, comfort and pleasure in use. The research applies the proposed transdisciplinary approach to an industrial case study on automatic production lines, focusing on the ceramics industry, developed in collaboration with a world leading company in this sector.

Keywords: Augmented reality , Digitization , Green design , Human-Centered Design , Industry 5.0, Virtual reality , Sustainability digital twin

Despite the significant surge in innovation and the ongoing trend of increasing automation levels in manufacturing and logistics brought along by Industry 4.0, companies continue to heavily depend on human labor to meet market demands for product customization and reduced time-to-market. Consequently, there is a pressing need for companies to enhance workforce skills within a shorter timeframe, leading to an increased demand for new training methodologies aimed at accelerating knowledge acquisition at a high-quality level. As the Industry 5.0 paradigm emerges, there is a renewed emphasis on human-centricity, encouraging collaboration between humans and machines to address new challenges in industrial environments. Extended Reality (XR) technologies offer a promising solution to bridge the gap between operators' needs and industrial requirements, enabling operators in navigating complex indoor spaces efficiently and safely, optimizing operational workflows and enhancing overall productivity. This research explores the application of XR technologies in indoor navigation, a crucial aspect of modern industrial environments. Two distinct approaches are introduced and compared: AR-Tablet-based and MR-HoloLens-based. Through a real industrial case study focused on indoor navigation in a spare part warehouse, technical insights are gathered to inform effective implementation strategies in logistics settings.

Keywords: Extended Reality , Human factors , Indoor Navigation , Microsoft HoloLens 2 , Operator training

Purpose: Minimally invasive surgery (MIS) has emerged in clinical practice to minimize surgical trauma, providing patients with faster recovery, reduced pain and complications and enhanced aesthetic results compared to traditional surgery. However, this approach increase the risk of iatrogenic damage, i.e. the accidental injury to sensitive anatomical structures (eg. nerves and vascular strcuteres) not directly visible during a percutaneous access. Augmented reality (AR) can effectively mitigate these drawbacks by overlaying graphical information onto the surgical field and providing real-time feedback, offering support in training settings and clinical practice. Implementation challenges have limited the number of case studies in the scientific literature. This study presents a novel simulation paradigm for orthopedic surgery training, filling a gap in surgical skill development resources for trainees, and demonstrating the effectiveness of this approach. Methods: The proposed methodology provides a framework for building a cost-effective and easily reproducible surgical training simulation environment. To address the challenge of mental spatial navigation during MIS procedures, the framework's rationale is to address the challenge of mental spatial navigation during MIS procedures. A surgical gesture tracking system using a commercial depth camera for comfortable simulation was developed. The principles of the acquisition system, image processing, and spatial computation mechanics are detailed to illustrate the framework's applicability. Digital environments customization with game engines to simulate expensive medical instrumentation, such as the C-arm, is also demonstrated. The simulation platform comprises a Computer Vision (CV) module, an X-ray machine simulation module, and an AR module. Results: System validation involved analysis at different framework levels. From texture analysis of acquired images to application accuracy evaluation, the influence of various parameters on system performance is demonstrated. The simulation system is a valuable tool for learning minimally invasive procedures and for developers building AR systems for medical applications. The implementation is focused on the insertion surgical devices, including screws and K-wires. This is results in real application in minimizing the risk of iatrogenic injury to neural and vascular structures. To demonstrate the effectiveness of highly reproducible accuracy between real and virtual environment an analysis of errors and accuracies is illustrated at level of different subsystems. Measurement between comparative measurement between vernier caliper and simulation system methods shows a R>0.9 with a p<0.01. Application accuracy was evaluated using the following parameters. The relative point-to-point accuracy averaged 1.02mm with a standard deviation of 2.82mm.

Keywords: Augmented reality , Image guided surgery , Minimally invasive surgery , Orthopedic