Virtual and augmented reality

The challenges of sustainable development require a consistent transformation of decision-making practices in society and in the industry. In this regard, Virtual Reality (VR) is an effective tool, providing experiences that are not accessible in the real world. The overview effect is a feeling of interconnectedness and responsibility towards our planet and its inhabitants described by astronauts after seeing the Earth during spaceflights. We present a VR application merging the overview effect with data visualization. We illustrate the design process and perform a pilot test to assess the emotions raised by the VR experience. Furthermore, we report feedback from design engineering and sustainability experts discussing the applications' potential in decision-making contexts, including product development processes.

Keywords: Decision making , Emotional design , Human behaviour in design , Sustainability , Virtual reality

More than 60 years has passed since the installation of the first robot in an industrial context. Since then, industrial robotics has seen great advancements and, today, robots can collaborate with humans in executing a wide range of working activities. Nevertheless, the impact of robots on human operators has not been deeply investigated. To address this problem, we conducted an empirical study to measure the errors performed by two groups of people performing a working task through a virtual reality (VR) device. A sample of 78 engineering students participated in the experiments. The first group worked with a robot, sharing the same workplace, while the second group worked without the presence of a robot. The number of errors made by the participants was collected and analyzed. Although statistical results show that there are no significant differences between the two groups, qualitative analysis proves that the presence of the robot led to people paying more attention during the execution of the task, but to have a worse learning experience.

Keywords: empirical study , human error analysis , human–robot interaction , virtual reality

Virtual museum systems have been shown to play a key role in enhancing visitor's experience and increasing the accessibility of cultural artifacts. In this context, the use of haptic interfaces based on force feedback could increase the level of immersivity of these systems and the quality of the interaction between visitors and cultural artifacts, introducing tactile information that could enrich the experience of all people, also in the case of visitors with disability. However, HD present limits concerning the ease of use of the device. This paper provides the results of an exploratory research carried out within the Research center of Teaching and learning, Disability and Educational Technology of the University of Macerata (TIncTec), consisting in two studies. Study 1 aims to analyze the learning performance of people with and without disabilities concerning this device. Study 2 aims to assess whether HD can provide useful support for conceptualizing the shape of virtual objects. To this end, we considered the 6 DoF high fidelity force feedback Geomagic Touch X by 3D system. A total of 30 people has been involved, including both children and adults with and without disabilities. The considered VR applications were developed in Unity 3D, using the 3D Systems Openhaptics Unity Plugin. The results provide useful insights for the design of future HD-based applications, showing how haptic technology can properly support visitors with disability in virtual museum environments in an inclusive perspective.

Keywords: Accessibility , Cultural Heritage , Force Feedback , Haptic Interfaces , Inclusive Museum , Special Education , Virtual Museum , XR Technologies

Most cultural promotion and dissemination are nowadays performed through the digitization of heritage sites and museums, a necessary requirement to meet the new needs of the public. Augmented Reality (AR), Mixed Reality (MR), and Virtual Reality (VR) have the potential to improve the experience quality and educational effect of these sites by stimulating users' senses in a more natural and vivid way. In this respect, head-mounted display (HMD) devices allow visitors to enhance the experience of cultural sites by digitizing information and integrating additional virtual cues about cultural artifacts, resulting in a more immersive experience that engages the visitor both physically and emotionally. This study contributes to the development and incorporation of AR, MR, and VR applications in the cultural heritage domain by providing an overview of relevant studies utilizing fully immersive systems, such as headsets and CAVE systems, emphasizing the advantages that they bring when compared to handheld devices. We propose a framework study to identify the key features of headset-based Extended Reality (XR) technologies used in the cultural heritage domain that boost immersion, sense of presence, and agency. Furthermore, we highlight core characteristics that favor the adoption of these systems over more traditional solutions (e.g., handheld devices), as well as unsolved issues that must be addressed to improve the guests' experience and the appreciation of the cultural heritage. An extensive search of Google Scholar, Scopus, IEEE Xplore, ACM Digital Library, and Wiley Online Library databases was conducted, including papers published from January 2018 to September 2022. To improve review reporting, the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were used. Sixty-five papers met the inclusion criteria and were classified depending on the study's purpose: education, entertainment, edutainment, touristic guidance systems, accessibility, visitor profiling, and management. Immersive cultural heritage systems allow visitors to feel completely immersed and present in the virtual environment, providing a stimulating and educational cultural experience that can improve the quality and learning purposes of cultural visits. Nonetheless, the analyzed studies revealed some limitations that must be faced to give a further impulse to the adoption of these technologies in the cultural heritage domain.

Keywords: Augmented reality (AR) , Cultural heritage , Digital heritage , Head-mounted display (HMD) , Mixed reality (MR) , Virtual reality (VR)

In this paper, a human-centered design methodology is proposed to support the interiors design of a Maritime Patrol Aircraft (MPA). The combined use of Immersive Virtual Reality (VR) tools and Digital Human Models (DHMs) are exploited to include both objective and subjective measurements in the ergonomic evaluation process. The main issue consisted in defining the optimal ergonomic design configuration for Operator Console Area and Observer Window Area of MPA's digital mock-up, provided by Leonardo S.p.A. The methodology is based on a five-step iteration process: once having identified the requirements, required input data and the design variables, the ideated configuration is evaluated using DHMs in order to retrieve objective measurements (i.e., interferences, visibility, reachability); finally, a subjective assessment within immersive VR environment is conducted. A real-time RULA analysis is carried out on DHMs, calibrated on selected users representing specific percentages of target population, by means of a full-body tracking system. The subjective assessment in immersive VR allows to take into account also other human factors (i.e. human ability, dexterity and cognitive aspects) that have not been previously considered and may significatively influence the validation of the final design configuration of MPA interiors.

Keywords: Digital Human Modeling , Digital Mock-up , Ergonomic Validation , Maritime Aircraft , Virtual Reality

This paper investigates the use of Virtual Reality (VR) to develop virtual tour applications for marketing purposes. The aim is to explore how virtual technologies can support the creation of knowledge about a specific food product and the achievement of user engagement by a multi-sensory virtual tour of the real production site. The study provides design guidelines to create a valuable, multisensory experience by VR tours and demonstrate how the adoption of a user-driven approach, instead of a technology-driven approach, allows to achieve a positive intention to buy. The case study was represented by one of the excellences among Italian food products, the Parmigiano Reggiano (PR) cheese. The PR virtual tour was validated by a user testing campaign, involving more than 70 users: users' reactions and feedback were collected by human physiological data monitoring and questionnaires' administration. The research results demonstrated how virtual technologies could effectively help people to create a solid knowledge about a food product to support the marketing process and to form an intention to buy thanks to a better understanding of the quality of the local and traditional productions.

Keywords: User experience , User-centered design , Virtual reality , Virtual tours , Virtual travel , Virtualization

Featured Application: Future interactive devices. Additive Manufacturing (AM) facilitates product development due to the various native advantages of AM when compared to traditional manufacturing processes. Efficiency, customisation, innovation, and ease of product modifications are a few advantages of AM. This manufacturing process can therefore be applied to fabricate customisable devices, such as bespoke interactive devices for rehabilitation purposes. In this context, a two-day workshop titled Design for Additive Manufacturing: Future Interactive Devices (DEFINED) was held to discuss the design for AM issues encountered in the development of an innovative bespoke controller and supporting platform, in a Virtual Reality (VR)-based environment, intended for people with limited dexterity in their hands. The workshop sessions were transcribed, and a thematic analysis was carried out to identify the main topics discussed. The themes were Additive Manufacturing, Generative Design Algorithms, User-Centred Design, Measurement Devices for Data Acquisition, Virtual Reality, Augmented Reality, and Haptics. These themes were then discussed in relation to the available literature. The main conclusion of this workshop was that a coherent design for AM tools is needed by designers to take AM considerations throughout the design process, since they lack the AM knowledge required to develop bespoke interactive devices.

Keywords: design for AM knowledge , future interactive devices , user experiences

The "An Ocean of Science" project encourages effective interventions and innovative educational paths to support the growth of knowledge and skills in schools, helping competent institutions to guarantee more inclusive cohesion policies with a high cultural and scientific impact. The project also provides innovation in education and training by providing cultural and socio-educational development opportunities in the framework of cultural heritage located in the marine environment and sea protection. The purpose of this study was to evaluate the effects and involvement of secondary school students during several educational laboratory activities, focusing on the development of their practical skills and training. With a view to active learning, which makes the students actors, responsible and aware of the learning process, this study shows an experimental work designed to demonstrate the usefulness and the validity of the scientific approach as the main pillar of the teaching of the sciences related to the marine issues, with special focus to submerged cultural heritage and sea pollution.

Keywords: Educational innovation , serious games , teaching/learning experiences , underwater cultural heritage , virtual reality

In recent years, research reached a very high level of development and validation of augmented prototyping systems in support of collaborative design activities. However, there is still great scepticism in companies when it comes to integrating these new technologies within a consolidated working model. Among others, the main barrier to overcome concerns the lack of understanding of the impact of AR systems on the key objectives of a business, such as improving its efficiency and revenue. For this reason, this paper aims to quantify these indicators by observing the technological impact not on a single design session but on an entire product development process, during which the aspects related to its integration are also considered. Thanks to the collaboration with a design agency, it was possible to compare parameters such as the lead time, number of iterations, person-hours and costs between two similar and realistic projects in which only one was supported by projection-based AR technology.

Keywords: co-design process , industrial validation , key performance indicator , spatial augmented reality , technology acceptance

Spatial-Augmented Reality (SAR) technology has the potential to impact decision-making in various design fields, such as architecture, engineering, and product design. However, despite its benefits, SAR technology has not yet been fully embraced by professionals. This study aims to identify the essential features of a SAR platform that would meet the demands and expectations of different design activities, including product appearance, interface, and ergonomics. The research involved experts who used a SAR platform in real-life product development, and data was collected through semi-structured questionnaires and interviews. The analysis revealed a significant correlation between the features of the technology, considering both potential and already available ones, and the needs of specific design sectors during product development processes. The study's outcomes provide valuable insights for advancing SAR technology in the design field and promoting its adoption by professionals.

Keywords: Augmented Reality , Design activity , Technology assessment

This article reports a study evaluating the effect of continuous assistance via an active exoskeleton during orientation tasks of the human wrist in a serious game. The proposed platform features a complete system that may be used for robot-assisted rehabilitation in clinics, offering support for both the therapist and the patient. It gives the possibility to customize the rehabilitation session by regulating the assigned task difficulty, the assistance level offered by the exoskeleton, and the wrist Degrees of Freedoms (DoFs) to be supported, providing a versatile, easy-to-use, rehabilitation station. The setup consists of a 3-DoFs wrist exoskeleton activated by three electric motors, connected to a wheeled station containing the control unit and the host PC to actuate the exoskeleton. The exoskeleton allows to monitor the positions of the human wrist joints and the force applied to the device by the user while performing a movement. Advanced software has been implemented to create a stimulating environment during therapy through virtual reality. The Flight Simulator serious game has been used to investigate the user's wrist Range of Motion (RoM) covered while reaching twenty targets. Thirteen healthy subjects tested the system under two different conditions, considering the device assistance on/off. Results show a significantly higher performance under assistance, rather than without it, on the exploitation of the wrist RoM and on the intuitiveness of movements during the orientation task. A reduction of around 30% of the overall muscle stress has been measured, highlighting the assistance efficacy of the system.

Keywords: Human-Robot Interaction , Mechanism Design , Medical and Rehabilitation Robots , Motor Control , Wearable Device , Wrist Exoskeleton

As technology advances, we are surrounded by more complex products that can be challenging to use and troubleshoot. We often turn to online resources and the help of others to learn how to use a product's features or fix malfunctions. This is a common issue in both everyday life and industry. The key to being able to use a product or fix malfunctions is having access to accurate information and instructions and gaining the necessary skills to perform the tasks correctly. This paper offers an overview of how artificial intelligence, digital twins, and the metaverse-currently popular technologies-can enhance the process of acquiring knowledge, know-how, and skills, with a focus on industrial maintenance. However, the concepts discussed may also apply to the maintenance of consumer products.

Keywords: computer-aided design , cyberphysical system design and operation , eXtended reality , maintenance , metaverse

Nowadays, industrial training is gaining popularity, and in particular, Virtual Reality (VR) technologies are often adopted to recreate training experiences that can transform the learning process by exploiting the concept of learning by doing. Moreover, VR allows us to recreate realistic and immersive environments that provide trainees with hands-on experience in a safe and controlled setting. One of the advantages of using VR technologies to perform training simulations lies in the ability that these tools offer to mix different senses as if they were independent variables and to manipulate them one at a time. In this way, the effects of using one sense on the specific task under analysis can be discovered in a controlled scenario in which the others are controlled. One of the least used senses in VR is undoubtedly that of smell. In this paper, we intend to use a new olfactory device specifically designed to be adopted with VR technologies, integrate it with an immersive VR training application, and propose a series of uses allowed by this combination. In the paper, we report the details of the multisensory approach that integrates the different senses. Additionally, the paper presents the findings of a pilot experiment conducted to validate the adopted approach and assess the results. The study demonstrates that the developed Olfactory Display had no adverse impact on users' interaction with VR content.

Keywords: Olfactory Display , Virtual Reality , Virtual Training

This chapter provides final remarks on the book's content and explores the promising future of prototyping in product design, particularly with the advent of emerging technologies such as the metaverse and Artificial Intelligence, which offer significant possibilities and advancements for the prototyping process.

Immersive technologies are often used in museum visits due to their numerous advantages, including the possibility of enhancing Cultural Heritage dissemination, improving accessibility, and learning activities. Despite the advantages, museum professionals may be reluctant to integrate immersive technologies during the museum visit due to their potential intrusiveness. In addition, a technology-driven approach is often used, which sometimes leads to scattered results and does not exploit the technological potential to meet the museum's objectives. Moreover, the design process for the creation of immersive exhibition visits is based on a trial-and-error approach rather than on specific guidelines regarding the use of immersive technologies in the museum context. The paper presents a study that investigates the immersive technology-related factors that influence the visitors' experience in museum exhibitions and the immersive technology awareness, benefits, and hindering factors perceived by museum professionals. Specifically, the study focuses on the occurring design process for the integration of immersive technology in museum exhibitions involving multidisciplinary professionals. It presents mixed methods that include experimental case studies, online surveys, semi-structured interviews, and a participatory action research activity. The result consists of a conceptual framework for a new collaborative design process that aims to facilitate the design of immersive museum exhibitions and, consequently, to help museums achieve their objectives and improve visitors' experience.

Keywords: Design process , Extended Reality , Immersive technology , Museum exhibitions

The Metaverse is defined as an interconnected network of 3D environments that enable multisensory interactions with virtual environments, digital objects, and people. A central component of a Metaverse is the realism of human interaction with the environment and each other, which is achieved through emotional expression. However, there are limitations in realistically representing human emotions and facial expressions in real time through virtual avatars. In this paper, the authors present a research project called Meta-EmoVis aimed at improving the realism of the Metaverse by detecting human facial expressions and translating them into emotional feedback that influences aspects of the virtual avatar and environment. The Emoj tool is used to collect data and map facial expressions to basic emotions, as well as detect the human engagement level. The goal is to create a world of real social interaction and emotional sharing in virtual spaces that are sensitive and adaptive to human emotional and mental states. Preliminary user tests to evaluate the effectiveness of the application are presented.

Keywords: Emotion , Metaverse , Virtual Reality

The use of digital technologies in museums is becoming increasingly popular, with a growing trend towards using multisensory eXtended Reality (XR) technologies to enhance visitor engagement and involvement. XR technologies like Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR) have the potential to bring numerous advantages to museums, including increased accessibility and inclusivity. By allowing visitors to experience a sense of immersion and actively engage with cultural artifacts, XR technologies can improve emotional engagement and learning activities. However, the integration of these technologies is often related to temporary exhibitions, and there are concerns about the costs associated with implementation and the potential for technology to negatively impact the visitor experience. Despite these challenges, evaluating the impact of multisensory XR technologies on visitor experience is difficult. This paper outlines a laboratory-based research activity that compares the impact of traditional exhibitions and XR technologies on visitor experience and learning performance. The study simulates a museum exhibition and allows the user to interact with different paintings using different XR technologies in a controlled environment.

Keywords: Cultural Heritage , Multisensory XR technologies , Museum exhibitions

Virtual Reality (VR) technologies are increasingly being adopted in fashion marketing to create innovative new channels for sales, advertising, brand communication, and strengthening relationships with consumers. In addition, virtual technologies can be useful for personalizing the offer, gathering insights, and user engagement. However, there seem to be few occurrences where the potential of virtual technologies on User eXperience (UX) and engagement is explored, evaluating the advantages compared to real-life experiences. Analyzing these aspects for virtual fashion experiences can be challenging. For measuring the UX, quantitative methods such as sensors capable of recording and examining the body's responses and automatic systems used to recognize emotions by tracking facial movements and expressions have been increasingly employed. In the context of fashion marketing, they can help analyze the implications that virtual experiences have, also considering content personalization and company promotion. This paper presents the design and development of a VR application that includes emotion-tracking technology that can be used to evaluate users' emotional reactions to virtual experiences and compare them with real ones. The final aim of the study is to increase the degree of user engagement and the quality of the virtual fashion show UX of the Italian clothing brand Bacon. Preliminary test sessions were organized to evaluate users' emotional reactions to the virtual experience and verify their effectiveness in relation to the objectives of the study.

Keywords: Emotion tracking , User eXperience , Virtual Fashion Show

The paper presents the development of a Virtual Reality (VR) application to perform dives in the Christoforos Shipwreck. The huge wreck lies in the Panormos Bay of Skopelos island (Greece) at a depth of 45 meters. Based on a well-known methodology designed for ancient shipwrecks, the 3D reconstruction of the underwater site has been carried out by adjusting the workflow to survey modern shipwrecks. In particular, the methodology, based on photogrammetry, is capable to provide a highly detailed 3D reconstruction of the shipwreck in a few dives, considering issues like working depth, elevations, and thin elements modelling. Moreover, the paper describes the optimization of the 3D model and the software to be executed in low-performance Head Mounted Display (HMD) devices. The resulting VR application, realized for touristic purposes, recreates the exact ambient conditions inside and outside the water simulating the flora and fauna of the place, the coastline, allowing users to live a recreational and educational experience by virtual diving in the underwater site.

Keywords: computer graphics technologies , cultural heritage , modern shipwreck , underwater photogrammetry , virtual reality

Craniomaxillofacial surgeries are performed using custom-made physical cutting guides and resin dental splints that present several drawbacks (e.g. time and cost required for their design and production). The literature commonly provides augmented/mixed reality (AR/MR) solutions for assisting maxillofacial osteotomies and repositioning without any interactive guide. This work proposes a new MR application, useful for osteotomy and repositioning, providing interactive, fast, and intuitive feedback to the surgeon, who is then supported in performing the bone fragment resection and replacement frame by frame. The proposed application speeds up the surgery and reduces under/overshooting errors. Moreover, the idea of integrating osteotomy and repositioning assistance in the same MR application is rarely found in the literature. It is an entirely novel approach to craniomaxillofacial surgery. The MR application has been designed with a three-button menu. The “App Start” calibrates the app, the “Osteotomy Mode” visualises the holograms of the cutting lines and drilling points, and the “Repositioning Mode” visualises the step-by-step real-time feedback to precisely support the surgeon placing the osteotomised bone fragment towards the final pre-planned position. The MR app has been developed in Unity and deployed on Microsoft HoloLens V2. A laboratory test bench was realised to validate the accuracy of the proposed MR-based approach. The validation protocol consists of two tasks to test the osteotomy and repositioning modes using a 3D-printed skull phantom. For osteotomy, the accuracy is 0.89 mm (genioplasty), 1.24 mm (maxillary osteotomy), 1.33 mm (orthognathic surgery), and 2.89 mm (mandibular angle osteotomy). For repositioning, the accuracy is 0.6 mm (anteroposterior deviation), 0.7 mm (mediolateral deviation), and 0.6° (angular deviation).

Keywords: Augmented reality , Hololens , Maxillofacial surgery , Mixed reality , Surgical guide