Virtual and augmented reality

The design research and education landscapes are changing. The widespread development and use of technologies such as Mixed Reality (MR) and the diffusion of Head-Mounted Displays (HMDs) available at low cost are causing a shift in design education toward the Metaverse. In this ever-changing scenario, there is a need to rethink design and teaching methods. However, scientific literature lacks the ability to provide contributions that include MR technology education in the industrial design program. We, therefore, present an innovative laboratory with an integrated multidisciplinary approach that starts from the fundamentals of interaction design and aims to teach students how to design next-generation MR interfaces for the Metaverse. The lab combines theory and practice within three courses: Information Design, Information Systems, and Virtual Design and Simulation. Industrial design students follow a precise multidisciplinary method consisting of five steps, from state-of-the-art analysis to the presentation of a final group design of an MR user interface. Thus, we introduce a class case study by presenting the outcomes of a semester project in the field of household appliances. Evaluation of the teaching method is conducted through a semi-structured questionnaire. Preliminary results show positive outcomes from students in terms of acceptance, effectiveness, usefulness, efficiency, and satisfaction with the teaching method adopted for the laboratory.

Keywords: industrial design , interaction design , metaverse , mixed reality , teaching method , user interface design

Nowadays, manufacturing plants are required to be flexible to respond quickly to customer demands, adapting production and processes without affecting their efficiency. In this context, Industrial Robots (IRs) are a primary resource for modern factories due to their versatility which allows the execution of flexible, reconfigurable, and zero-defect manufacturing tasks. Even so, the control and programming of the commercially available IRs are limiting factors for their effective implementation, especially for dynamic production environments or when complex applications are required. These issues have stimulated the development of new technologies that support more efficient methods for robot control and programming. The goal of this research is to identify and evaluate the main approaches proposed in scientific papers and by the robotics industry in the last decades. After a critical review of the standard IR control schematic, the paper discusses the available control alternatives and summarizes their characteristics, range of applications, and remaining limitations.

Keywords: industrial robots , instruction streaming , open controller , robot control , robot programming , trajectory streaming

Reduction of global bee populations is one of the relevant topics in the list of environmental issues defined in the United Nations Sustainable Development Goals Agenda. Virtual Reality technologies can play a strategic role in raising awareness and education for sustainability by allowing users to visualize, demonstrate and emphasize information, making it more accessible. Virtual Reality can also elicit emotional involvement, which is important for influencing user behavior. The authors have designed and developed an educational Virtual Reality application called Colonies, which allows users to understand the issues related to the bee problem and empathize with bee colonies. The experience also offers suggestions for more sustainable behaviors.

Keywords: Behavioral research , Sustainable development

With the high growth and prosperity of e-commerce, the retail industry needs to explore new technologies that improve digital shopping experiences. In the current technological scenario, Virtual Reality (VR) emerges as a tool and an opportunity for enhancing shopping activities, especially for the fashion industry. This study explores whether using Immersive Virtual Reality (IVR) technologies enhances the shopping experience in the fashion industry compared to Desktop Virtual Reality (DVR). A within-subject experiment was carried out involving a sample of 60 participants who completed a simulated shopping experience. In the first mode (DVR), a desktop computer setup was used to test the shopping experience using a mouse and keyboard for navigation. The second mode (IVR) exploited a Head-Mounted Display (HMD), and controllers, that allowed navigation while seated on a workstation to avoid sickness. Participants had to find a bag in the virtual shop and explore its features until they were ready to purchase it. Post-hoc measures of time duration of the shopping experience, hedonic and utilitarian values, user experience, and cognitive load were compared. Results showed that participants experienced higher hedonism and utilitarianism in the IVR shop compared to DVR. The cognitive load was comparable in both modes, while user experience was higher in IVR. In addition, the time duration of the shopping experience was higher in IVR, where users stayed immersed and enjoyed it for longer. This study has implications for fashion industry research, as the use of IVR can potentially lead to novel shopping patterns by enhancing the shopping experience.

Keywords: Fashion industry , Retailing , Shopping experience , User study , Virtual reality

Design for Disassembly (DfD) and Augmented Reality (AR) have become promising approaches to improve sustainability, by providing efficient delivery and learning assets. This study combines DfD and AR to deliver a method that helps to streamline maintenance processes and operator training. It focuses on a common part in the process industry that requires frequent maintenance and repair. DfD was applied to the pump's design to ease disassembly and reduce material waste, energy consumption, and maintenance time. AR was used to provide an interactive guide to improve the operator understanding of its internal parts and assembly/disassembly procedures. The resulting DfD-AR led to a reduction in maintenance time and shows potential to deliver better training. This highlights the potential of DfD and AR to enhance sustainability, learning, and productivity. The resulting disassembly sequence was taken to an AR simulation, helping process designers to better understand the procedure and further optimize the solution with other constraints.

Keywords: Augmented reality (AR) , design for disassembly (DfD) , optimization , recycling

Port Cell (PC) rooms in the ITER tokamak building host the devices required for the Port Plugs operation and provide connection to services. During Plasma Operation State, the Vacuum Vessel ports sealing interface provides vacuum tightness and first confinement barrier as essential functions. When a Port Plug needs to be replaced or maintained, this significant sealing component must be removed and reinstalled. The general strategy for this operation has been reviewed resulting in modification proposals to address the requirements for static and dynamic confinement and management of radiation considering the ALARA (As Low As Reasonably Achievable) approach. As Dose Reduction Measures (DRM), additional provisions have been proposed to ensure a robust confinement control during the machine shutdown operation. They mostly consist in an airlock that allows the extension of the first confinement barrier equipped with specific handling equipment to perform human-assisted operations on the sealing flanges. In particular, this paper presents the study carried out in TBM dedicated PC during the conceptual design phase of the maintenance cabin airlock and the handling tools by means of Virtual Reality (VR). VR techniques allow working with virtual prototypes at full scale from the early design stage to ease the design process and the concepts validation with important results concerning the assessments of accessibility issues. The study describes also the outcomes of the VR studies and explains how VR can be an effective instrument to conduct integration studies with complex kinematics and human access operations in the execution of the engineering validation process.

Keywords: ALARA , Early design validation , eXtended reality , Integration , Test blanket modules , Virtual reality

User experience (UX) application in the practice of engineering and product design is still limited. The present paper provides insights into research on UX design and recommendations for design practitioners by pointing out common criticalities. These outcomes are achieved through a literature review on how UX relates to design. First, issues in benefitting from UX understanding in design are identified with a specific focus on theoretical contributions. Second, experimental papers investigating UX and design are analysed in relation to previously identified issues. Although issues are present to some extent in all the contributions, the empirical studies dealing with UX in design are overall valid. The results highlight UX's support in revealing design requirements, but its capability of steering design processes is arguable, as concrete guidelines for practitioners are not well described. Based on identified issues, the authors propose a checklist to make UX studies in design more reliable and their outcomes more comparable.

Keywords: best practices , design process , user experience , UX applications , UX issues

Interactive data visualization plays a crucial role in the interpretability of large datasets. Virtual reality offers unique advantages in exploring data, beyond traditional 2-D views. This article presents a set of interaction artifacts designed for analyzing and interpreting complex datasets through immersive 3-D graph visualization and interaction. Our system makes complex datasets easier to work with by offering a wide range of visual customization tools and intuitive methods for selection, manipulation, and filtering. It also provides a cross-platform, collaborative environment that can be accessed by remote users through traditional computers, drawing tablets, and touchscreen devices.

Keywords: Data visualization , Large dataset , Three dimensional computer graphics , Visualization

On October 21, 2022, the 21st IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2022) was successfully completed in Singapore. ISMAR is the leading international conference in the fields of augmented reality, mixed reality, and virtual reality. This was the first time that ISMAR was held in Southeast Asia and the first time in hybrid mode. ISMAR 2022 achieved a historically high number of papers and attendees, witnessing the steady growth of the community and the scientific contributions. In this article, we report the key outcomes, impressions, research trends, and lessons learned from the conference.

Keywords: Augmented reality

This chapter presents selected case studies showcasing the design and testing of user experiences through prototypes created with eXtended Reality (XR) technologies. These case studies were developed by students participating in the Virtual and Physical Prototyping course at the School of Design, Politecnico di Milano. The projects assigned to student teams aimed to create prototypes prioritizing user interaction and experience, such as apps connecting young people with nature or aiding language learning. The students followed a simplified product development process, starting with defining the target audience and requirements, followed by concept development and design execution, and appropriate XR technologies selection for the implementation. The chapter presents a set of guidelines that outline this simplified product development process for crafting XR application prototypes. Subsequent chapters showcase how these guidelines were applied by the students in their respective case studies.

Keywords: Product design , Product development , Software prototyping

This chapter presents three examples demonstrating the advantages of multisensory experiences in eXtended Reality applications. The first case study involves a training application designed to familiarize users with laboratory machinery, incorporating the senses of sight, hearing, and smell. The second case study concerns the utilization of multisensory experiences in Augmented Reality to facilitate language learning and enhance the exploration of a new city. The third one concerns a multisensory application including visual components, sounds, and odors for learning music.

Keywords: Augmented reality

This chapter presents three examples that demonstrate the effective integration of Reality and Virtuality, resulting in captivating applications that significantly enhance and improve the user experience through their seamless combination. The primary objective of the first case study is to explore and examine various methods of fostering connections among individuals through the utilization of Augmented Reality technology for collaborative art experiences. The second case study focuses on proposing a connection between humans and plants through Virtual Reality to enhance mental and physical well-being, as well as overall quality of life. The third case study focuses on leveraging Augmented Reality technology for maintenance operations.

Keywords: Arts computing , Augmented reality , User experience

This chapter provides an overview of eXtended Reality (XR), a term that encompasses technologies such as Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR). XR allows the real world to be enriched with virtual data, objects, and information, offering varying levels of immersion and interaction. The chapter explores the components of XR and discusses classification frameworks, including Milgram's Reality-Virtuality Continuum and the 3I model. It further delves into the developments and technologies of VR, highlighting stereoscopic vision, VR headsets and immersive walls. The chapter also explores AR, its accessibility through everyday devices, and its applications. The importance of considering all sensory modalities for creating immersive XR experiences is emphasized, and the role of multimodal interaction in enhancing user experiences is discussed. The chapter highlights the significance of a multisensory approach in achieving a sense of presence and explores examples of early immersive and multisensory technology. Finally, it discusses the impact of touch in human interaction and the incorporation of tactile sensations in VR applications, as well as of olfaction, which plays a significant role in human perception and memory, as certain smells can evoke strong emotional responses and trigger vivid recollections of past experiences. Incorporating odors into XR applications can enhance the overall sense of presence and realism by providing users with scent cues that align with the virtual environment. Olfactory displays are devices that release specific odors or scents to accompany virtual content.

Keywords: Behavioral research , Display devices , Mixed reality , Odors , Stereo image processing

In this chapter, three examples are given of how Augmented Reality (AR) technologies can be used to promote sustainability. The initial case study involves an application that illustrates how the environment affects biochemical processes. The second case study features sustainable handmade toys that educate children on electronic circuits. The third one supports children in understanding the entire lifespan of a plant and learning its scientific microscopic processes.

Keywords: Augmented reality , Toys

The industry's interest in Virtual and Augmented Reality (VR and AR) technologies started from the beginning of their appearance in the research world. Over the years, scholars observed ups and downs, to which various factors contributed. In recent years these technologies, now known as eXtended Reality (XR), have returned to fascinate the industrial world, mainly because most of the related enabling technologies have improved to the point of pushing companies to re-invest in them. The introduction of approaches such as the digital twin one and the recent hype on the metaverse is also a push in this direction. A few questions arise: what are the benefits of such technologies in the industry today, and what are the unexplored possibilities? Starting from a systematic literature review and exploring the practical implications of integrating technologies in the industrial field, the paper tries to answer these questions. The paper is not intended as a technological forecast but as a stimulus for future research.

Keywords: Industry 4.0 , Training , Virtual reality

Product development stages are typically characterized by different forms of representations and degrees of specification, which potentially affect user\'s perception and evaluation. These effects are worth investigating more closely also because of the growing relevance of new technologies such as Virtual Reality (VR) in the design field. The objective of this paper is to elucidate the mutual relations between forms of representation, visual behaviour, and people\'s evaluations. The focus is on differences between virtual and physical prototypes. In the illustrated experiment, participants visited a tiny house in an immersive VR (360° images acquisition). The results were compared with a past experiment where the physical prototype of the same product was similarly evaluated. The dwell times on Areas of Interest (AOIs) pertaining to the tiny house were compared and correlated to variables concerning subjective evaluations. The results show just a few similarities of visual exploration in terms of gazed AOIs. Substantial differences in terms of how the duration of gazing affects evaluations have been found too. The larger number of significant correlations between observations and evaluations in the virtual exploration emerged.

Keywords: Case study , Evaluation , Prototype , Representation form , Virtual reality

Among the main features of Industry 4.0, digitization and the evolution of the human-machine interaction occupy a central role. These concepts are transferring even in the health domain, moving toward Healthcare 4.0. The new concept of Industry 5.0 further promotes the human-centric perspective focusing on the consideration of human factors. In this context, training for workers, both in the industry and in the healthcare sectors, needs to be strongly human-centred to be efficient and effective. This paper refers to simulation-based training and aims to provide a transdisciplinary framework for the simulation assessment from the learners' perspective. The final scope is to outline a set of data-driven guidelines for the simulation optimization and redesign, throughout a human-centred approach, aiming to improve the workers' performance and the overall learning process, considering the physical, cognitive, and emotional conditions. The proposed method is suitable for each kind of training (both traditional and with the use of virtual reality/augmented reality systems) and relevant for every sector. Two different use cases are presented, respectively referring to the healthcare and industry fields, proposing a unique assessment protocol. The healthcare use case considered the low-fidelity simulation of lumbar puncture, while the industrial use case referred to the replacement of the engine oil filter on tractors. Although the great differences between the content of the use cases, the results obtained about performance as well as cognitive and emotional states are close enough to define a common set of guidelines to redesign and optimize the simulation-based training.

Keywords: Design optimization , Ergonomics , Healthcare 4.0 , Human Factors , Industry 4.0 , Simulation-based training

The realization of a specific REMote HAndling Test and Training Facility (REMHAT) is necessary for the development and optimization of the Remote Handling (RH) system, to demonstrate the fulfilment of all its design requirements, for testing and validation of the RH maintenance procedures, for the qualification and acceptance testing of RH equipment and tooling. Furthermore, it provides training and certification to the RH maintenance operators, by making available to operators the knowledge, instructions and experiencing of the Virtual Simulator platform, the control room devices, the robots and tooling to be used for RH tasks. In this work, the preliminary architecture of the DTT REMHAT facility is described, with a focus on the added value of the adoption of Virtual Reality (VR) technologies enabling real-time alignment of virtual robots' configuration with the real ones (Digital Twin). A first approach to the facility design is proposed, starting from the technical requirements: Environmental, Design, Physical, Functional, Operational, Human Factors and Product assurance and safety. The proposed logical architecture consists of four Sub-Systems (S/S): building and auxiliaries, mock-ups, robots and Control Room. In the present work, a particular attention is paid on the last one. The idea is to create a digital twin of the RH equipment mock-ups to be coupled with software of real-time structural simulation, in order to have at disposition and show in VR environment real-time additional information about the real environment, like robots' deflections and/or vibrations, clearances and possible collisions detection. These data could be crucial for the operator's decision-making process about a sudden intervention on the RH system, further to their employment for future studies and optimization. Therefore, for each subsystem of the facility, the components identification and a description of the software and hardware aspects are provided. In conclusion, an overview of the critical issues of such complex system is presented, in order to prepare the ground for future research works.

Keywords: Digital twin , DTT , Remote handling system , Test and training facility , irtual reality

System engineering (SE) methods and principles are nowadays widely adopted in the product development processes, especially in the industrial sector, where saving production time and costs are primary goals. This work describes an application of a particular SE methodology, the V-model-based design, in which the system development lifecycle is divided on the basis of a graphical V-shaped scheme, called V-model. Following this approach, a new concept of charging arm for Robotic Train Inspection Monorail (TIM) of Large Hadron Collider (LHC) at CERN (Conseil Européen pour la Recherche Nucléaire) has been developed. The current charging arm version is affected by several issues and limits that have led to the necessity of a new solution. Starting from the first stages of functional requirements (FRs) definition and decomposition (left side of the “V”), a new concept has been implemented, in order to be tested for its verification and validation (right side of the “V”). As part of the principles of SE, the process has been based on virtual models of the product and on virtual simulations of its operation, rather than on the realization of time-consuming and expensive physical models and tests, even if a final physical prototype has also been built and some physical operative tests have also been carried out on it. These tests have showed that the new product appears to fulfill each one of its FRs and overcome the limits imposed by the previous version. The future commissioning and operative tests in the real operating condition and location shall definitely validate the new product.

Keywords: Additive manufacturing , Design for manufacturing , Finite element method , System engineering , V-model-based design , Virtual reality

ugmented reality (AR) is increasingly considered to support scenarios of co-located and remote collaboration. Thus far, the core goal has been advancing the supporting technologies and assessing how they perform to inform design and development, thus providing support toward their maturity. Nevertheless, while understanding the performance and impact of supporting technology is indisputable groundwork, we argue that the field needs to adopt a framework that moves from answering questions about the proposed methods and technologies to a more holistic view, also encompassing collaboration. However, moving toward this goal challenges how evaluations are designed, adding complexity and raising several questions about what needs to be considered. In this article, we briefly examine the different dimensions entailed in collaborative AR and argue in favor of a distinctive evaluation framework that goes beyond current practice and sets its eyes on the elements that allow judging how collaboration unfolds while informing the role of the supporting technology.

Keywords: Software engineering