Virtual and augmented reality

The study of an automated system for intralogistics requires an important use of time and resources, starting from the input data analysis up to the definition of the technical solution. While many commercial tools are available for testing and optimizing the plant performance during the advanced design stages, little work has been done concerning the workflow to be followed at the pre-sales design phase. In this context, the present paper focuses on the definition of the best practices for the correct preliminary definition of a robotic cell for palletization. To simplify and speed up the pre-sales feasibility study and estimate the performance of the proposed robotic system, an engineering approach based on a simplified theoretical model is reported and integrated within a dynamic calculation table. As the main output, the proposed tool calculates the robot saturation which is a key index for the plant preliminary definition.

Keywords: Design Tool , Industry 4.0 , Palletizing Robotic System , Performance Definition , Pre-Sales Design

The concept of Industry 5.0 (I5.0) promotes the human-centricity as the core value behind the evolution of smart manufacturing systems (SMSs), based on a novel use of digital technologies in the design and management of modern industrial systems to take up the socio-technical challenges. In this context, the paper proposes a Smart Manufacturing Systems Design (SMSD) framework enabling I5.0, based on the human-automation symbiosis. Thanks to an ‘Augmented Digital Twin' (ADT) able to integrate and digitize all the entities of the factory (i.e. machines, robots, environments, interfaces, people), AI-driven applications can be built to support the user domain and make people and machines co-evolve thanks to a systematic data sharing between physical and digital assets (e.g. digital twin, virtual mock-ups, human-machine interfaces), optimizing factory productivity and workers wellbeing. In this framework, machines and humans can both generate knowledge and learn from each other, generating a virtuous co-evolution, supporting the understanding of the human-machine interplay and the creation of an effective collaboration between people and SMSs. The framework was conceived and validated involving four industrial companies, belonging to diverse sectors, interested in overcoming the current limits of I4.0 lines by including the human factors for future SMS management.

Keywords: augmented digital twin , human-automation symbiosis , Industry 5.0 , Operator 4.0 , Operator 5.0 , smart manufacturing systems

In recent years, virtual reality technology has grown more widely available. This creates new opportunities and methods in the field of education, particularly for STEM (science, technology, engineering, and mathematics) disciplines. This study describes the design and the development of an immersive virtual reality laboratory (VR Lab) that aims to introduce novel activities and practical experiments in the field of industrial engineering and material science that can be highly beneficial for students. The laboratory enables students to experience and learn the macro and micro behavior of various engineering materials. The VR Lab is designed as an open-space room that is digitally divided into multiple learning stations, each of which is dedicated to a specific aspect of the subject. A virtual mechanical tensile test machine may operate with various materials and display the results by combining finite element analysis simulations and stress-strain curve. In another station, students can be transported inside atomic and molecular structures of various materials and can investigate how dislocations and slipping planes influence the mechanical behavior of metals or how the alignment of molecular chains affects the strength of polymers. Immersive VR Lab showed great potential for education. The developed virtual learning stations can be used to complement learning activities and physical experiments that are generally too risky, too expensive, or simply too time-consuming to be carried out in a real classroom, particularly in the STEM area.

Keywords: Engineering Education , Material Science Teaching , STEM , Virtual Reality , VR Lab

[No abstract available]

Stroke affects approximately fifteen million people worldwide annually, with impaired hand function being one of its most common effects. Hemiparetic post-stroke patients suffer a mild loss of strength involving one side of their body: though not fully debilitating, it still impacts their everyday life activities. To prevent mobility deterioration, patients must perform well-focused and repetitive exercises during chronic rehabilitation. Virtual Reality (VR) emerges as an interesting tool in this framework, offering the possibility of training and measuring the patient's performances in ecologically valid, engaging, and challenging environments. In recent years, there has been an increasing diffusion of accessible head-mounted displays that enhance the sense of realism and immersion in a virtual scene. To explore the feasibility and efficacy of VR immersion and game mechanics in rehabilitation programs, a VR system that allows users to rehabilitate their motor skills in a home-based environment has been designed and tested considering standard measures related to usability, immersion, workload, and simulator sickness, and with the involvement of rehabilitation experts. The results demonstrate how users and experts have received the application positively, highlighting the potential of VR applications for the future development of home-based rehabilitation programs.

Keywords: Exergames , Post-Stroke Rehabilitation , Virtual Reality

In the manufacturing industry, Augmented Reality (AR) has shown significant potential in enhancing operators' capabilities while performing inspection and assembly activities. However, the augmented visualization of virtual models on physical components can present challenges and potential misunderstandings, as the visualization mode greatly influences the perception of components and the amount of information received. This study investigates the impact of rendering techniques on user performance during industrial assembly inspection tasks. In particular, a set of rendering techniques, suitable for industrial AR applications, have been selected and implemented through a dedicated AR tool. The selected AR rendering techniques have been compared, in terms of qualitative and quantitative metrics, in order to test their efficiency and effectiveness for industrial assembly inspection activities. 17 domain experts and 33 representative users have been involved in the experimental study which outcomes reveal that the rendering techniques play a significant role in assisting operators for identifying design discrepancies in industrial products. Furthermore, a correlation has been observed between the two AR rendering techniques best suited to support industrial inspection activities and the types of assembly errors detected.

Keywords: AR rendering techniques , Augmented Reality , Industrial assembly inspection , Industry 4.0 , Quality inspection

Augmented Reality (AR) has been widespread over the years in different areas, especially in the manufacturing field, to facilitate the digitization of production processes, and increase productivity and product quality. While AR has grown in popularity, significant barriers exist to its adoption in industry, including high development costs and ineffective interoperability, which limit high flexibility and rapid adaptability required for competitive production systems. Therefore, handling interoperability properly is a key issue in this scenario, especially between AR technology and existing IT systems, widely adopted throughout the product lifecycle. Many of those challenges are related to the compatibility of data representation across these different systems, resulting in data loss or sharing of unusable information. Based on these considerations, this paper first analyzes the main issues related to the interoperability between AR and CAD systems, including data exchange, AR content authoring, and virtual data specifications. To bridge the gap between AR and CAD interoperability, a CAD automation tool is then presented that can support unskilled operators in preparing AR content for industrial applications. Preliminary experimentation is carried out to evaluate the main potentialities due to its adoption, which provides the basis for further implementation and investigation in a real industrial setting.

Keywords: Augmented Reality , CAD systems , CAD-AR interoperability

Virtual reality (VR) applications for surgical planning and mentoring are currently becoming more widespread. Within these fields of applications, haptic systems are crucial to help the understanding of the feedback provided by the tissue during the interaction with the surgical tool. The accuracy of the VR scenario is related to many aspects, among them the ability of the haptic device to provide the force feedback and thus the adoption of proper characterization and modeling of the soft tissue behavior during deformation. This paper presents the current development concerning a VR environment to be equipped with a force feedback haptic device to simulate and experience clamping in colorectal surgery. The reaction at the clamp handle is provided starting from a FEA of the contact between the surgical tool and the tissue, coupled with the kinematic analysis of the mechanism. Connecting the virtual scenario to a microcontroller board, the reaction force is applied to the the haptic device, shaped as a clamping tool handle, so that the user may experience the tissue impedence during clamping. By simulations made through Simscape, a concept is evaluated according to some simulation paths that involve common scenarios of the virtual reality experience (such as collision, partial or total clamping operation). The novelty of the research regards the system design oriented to increase the accuracy of the soft tissue material behaviour through a preliminary off-line dedicated FEA investigation. Through Simulink simulations, the system design is verified so that the VR system set-up may be carried out. The results highlighted: (a) a good correspondence among the theoretical study of the reference kinematism that the haptic device must reproduce; (b) the implementation of a bilinear material model, derived off-line from FEA, is suitable to capture hyperelastic and geometric non-linearities of the soft tissue; (c) the feasibility of the open loop haptic control in accordance to the explored kinematism. The final goal of the research is to propose a system design approach able to tune the VR experince for teaching and mentoring through a in-house set-up easy to be tailored.

Keywords: FEA , Force feedback , Metamodeling , Surgical simulation , Virtual prototyping

As the Metaverse gains popularity due to its use in various industries, so does the desire to take advantage of all its potential. While visual and audio technologies already provide access to the Metaverse, there is increasing interest in haptic and olfactory technologies, which are less developed and have been studied for a shorter time. Currently, there are limited options for users to experience the olfactory aspect of the Metaverse. This paper introduces an open-source kit that makes it simple to add the sense of smell to the Metaverse. The solution is modular, allowing for the simultaneous use of multiple odors and compatibility with both desktop and wearable applications. The details of the solution, including its technical specifications, are outlined to enable potential users to utilize, test, and enhance the project and make it available to the scientific community.

Keywords: Metaverse , olfactory display , smell , virtual and augmented reality environments

The increasing interest in technologies such as Metaverse, Augmented Reality (AR), and Artificial Intelligence (AI) opens up numerous possibilities that have the potential to shape our daily experiences in the future. We investigated how integrating today's popular chatbots and multisensory AR technologies can create personalized museum visiting experiences. The paper outlines the concept of a digital assistant-based application for museum visits. The visitor, equipped with AR technology, can ask questions about the exhibits and receive precise and contextually relevant answers from the digital assistant who can virtually appear as an avatar with accompanying multimedia AR information. In the paper, we describe the implementation of a prototype of the application.

Keywords: Augmented Reality , Chatbots , Metaverse , Museums

Teaching technical contents related to product design and 3D modeling is a challenging task. Students enrolled in Master of Science courses often have different backgrounds and technical skills, due to their different educational paths at the bachelor's degree. From the student's perspective, one of the main difficulties concerns the definition of the 3D modeling procedure, understand technical drawings and figuring out the proper assembly sequence to get the final product. Over the years, some methods to fill these existing gaps have been experimented, such as tutoring and Massive Open Online Courses. Despite these passive approaches have been proven useful for students, digital technologies, such as Virtual Reality, can boost the implementation active learning sessions and hands-on experiences. In this paper, we present a Virtual Reality application for active learning developed to help students in the initial stage of the “Methods and tools for detailed design” course to acquire a set of lacking knowledge, which includes technical terminology related to mechanical parts and components, the ability to decompose mechanical systems into sub-parts, and to create a 3D model using a CAD tool given a 2D engineering drawing. To evaluate the effectiveness of the proposed approach, the application has been tested with a cohort of students with different backgrounds and knowledge about mechanical design and CAD.

Keywords: Active learning , hands-on experience , Virtual learning , Virtual Reality , Virtual Reality for education

With the rising concept of Industry 5.0, the worker's needs and well-being are placed at the center of the production process. The use of extended reality can be exploited to provide the operator with interactive training, allowing independent, step-by-step learning in the real environment, with instructional digital contents, superimposed over the machinery in the workplace. This kind of training promotes the operator's skills, and consequently empowers the industrial competitiveness. The aim of this paper is to assess and compare the effectiveness of two different marker-less mixed reality (MR) applications to train the operator in the wire harness activities over a tractor's drivetrain. An application based on Azure Spatial Anchors by Microsoft and another one based on the CAD model of the tractor's drivetrain have been developed and tested through the comparison with the traditional paper-based instructions. On-field trials have been accomplished with 10 end-users. User experience, usability, perceived workload, and training effectiveness have been assessed through specific questionnaires and by analyzing user performances. Even if the Anchors-based app showed better outcomes in terms of effectiveness and acceptability, both MR applications resulted in good usability. Operators' willingness of using this technology as a standard tool for training and support emerged.

Keywords: HoloLens , human centred manufacturing , Industry 5.0 , mixed reality , spatial anchors , training

The Fourth Industrial Revolution conceptualizes the rapid change of industries resulting from the convergence of technologies such as artificial intelligence, genetic editing, and advanced robotics that enable increasing interconnectivity and machines that can analyze and diagnosing problems without human intervention, through intelligent automation. In this scenario, the use of augmented reality technologies is of great interest. The paper aims to explore the use of augmented reality in support of traditional inspections for assisting textile experts in fabric defect detection. The contribution of this study consists of three main phases, necessary for the future development of the system: (1) the analysis of possible automatic defect detection techniques; (2) the analysis of hardware solutions for the realization of a system based on important criteria such as operator comfort, system footprint, and so on; (3) the proposal of a possible comprehensive solution. Considering these aspects this paper identifies and investigate the best scenario for the introduction of artificial intelligence and augmented reality technologies to help the operator in the detection of textile defects.

Keywords: Artificial intelligence , augmented reality , machine vision , textile defect detection

Implementing an alternative reality for Metaverse implies modeling optimized digital content to guarantee real-time interaction and high-quality rendering. Even if 3D reconstruction based on 3D scanning techniques provides a good replica of real objects, the output files are challenging to use in this application. On the other hand, manually developed optimized 3D models require much time and effort. This aspect becomes crucial in scenarios including thousands of 3D models with which humans should interact. This paper proposes a method to automate the 3D modeling process of items whose shapes can be classified according to predefined geometrical categories. The dataset for this study relates to products, which present a wide variety of shapes but are attributable to just a few formal archetypes. In the proposed pipeline, metric orthographic images of the object to be digitally reproduced are analyzed by Convolutional Neural Networks (CNN)s. Subsequently, the same images are analyzed with Computer-Vision (CV) algorithms to extrapolate the characteristic dimensions related to the assigned archetypes. The method has been tested on different items, and the results proved the effectiveness of the whole approach in terms of correct archetypes recognition, parameter extraction, and creation of the 3D model, which are comparable with digitized 3D models with high-quality scanning tools but much lighter in model size.

Keywords: 3D Object Database , Artificial Intelligence , Automatic 3D Modeling , Computer Vision , Convolutional Neural Network , Process Optimization

Point cloud 3D models are gaining increasing popularity due to the proliferation of scanning systems in various fields, including autonomous vehicles and robotics. When employed for rendering purposes, point clouds are typically depicted with their original colors acquired during the acquisition, often without taking into account the lighting conditions of the scene in which the model is situated. This can result in a lack of realism in numerous contexts, especially when dealing with animated point clouds used in eXtended reality applications, where it is desirable for the model to respond to incoming light and seamlessly blend with the surrounding environment. This paper proposes the application of physically based rendering (PBR), a rendering technique widely used in real-time computer graphics applications, to animated point cloud models for reproducing specular reflections, and achieving a photo-realistic and physically accurate look under any lighting condition. To achieve this, we first explore the extension of commonly used animated point cloud formats to incorporate normal vectors and PBR parameters, like roughness and metalness. Additionally, the encoding of the animated environment maps necessary for the PBR technique is investigated. Then, an animated point cloud model is rendered with a shader implementing the proposed PBR method. Finally, we compare the outcomes of this PBR pipeline with traditional renderings of the same point cloud produced using commonly used shaders, taking into account different lighting conditions and environments. Through these comparisons, we demonstrate how the proposed PBR method enhances the visual integration of the point cloud with its surroundings. Furthermore, it will be shown that using this rendering technique, it is possible to render different materials, by exploiting the features of PBR and the encoding of the surrounding environment.

Keywords: extended reality , point cloud , real-time rendering

The design of a product consists of several stages to be carried out in sequence until the result is achieved, some of them are performed trough the help of CAD software. This design step is very rigorous with as little margin for error as possible, and it often goes to make a clean break from other stages that precede it. On the other hand, virtual reality technology is certainly an element of interest within various design areas. One of the most promising uses of this type of technology is in the visualization and review of products in the design process. But in this paper we will see a next step compared to this just described, that is, the application of virtual reality within the design making itself. By delving into the design timeline, we will then go on to explore how this technology can provide support not limited to visualization alone, but extended to modeling, what possibilities can be offered by the realization of three-dimensional surfaces, modeled already at the correct scale, and how these aspects can be efficiently integrated within a methodology, assessing variations in timing and quality of work. We will also try to intercept any downsides to try to get a clear understanding of what possibilities could be offered soon by this technology and whether indeed these types of processes can replace current methods, established for years.

Keywords: Design Methods , Industrial Design , Virtual Reality

The Industry 5.0 paradigm emphasizes the importance of the operator's well-being by seeing human-centricity as one of its cardinal principles. This enables a twofold benefit, improving the sustainability of the production process and enhancing performance. Improving performance and increasing operators' safety is often related to cognitive load optimization. Confined spaces are working environments where elements of distraction, such as noise, can cause accidents with major, even fatal, consequences. Studies on the effects of noise on cognitive abilities present mixed results, and those concerning confined spaces are limited as they require an expensive experimental setup replicating the working scenario. Immersive Virtual Reality (IVR) technology enables overcoming this gap by replicating the experimental conditions in a synthetic environment. We exploited IVR to study noise's impact on cognitive performance in confined spaces. We compared the impact of a stationary continuous noise source with an intermittent non-periodic one by administering the Stroop Color and Word Test. We also assessed the perceived cognitive effort by administering the modified noise-induced task load index questionnaire. We also compared the effects on the operator's physiological activity through Heart Rate Variability (HRV) analysis. Results show that by keeping the equivalent noise level lower than 85 dB, noise has no statistically significant effects on cognitive performance and Heart Rate.

Keywords: Cognitive Performance , Heart Rate Variability , Noise Effects , Stroop Color and Word Test , Virtual Reality

The role played by emotions and attention is crucial for the development of advanced driver assistance systems that improve safety by flexibly adapting to the current state of the driver. In the present study, we used immersive virtual reality as a testing tool to investigate how different emotional states affect drivers' attention in a divided attention task. Two different emotional states, diversified by valence and arousal, were induced before performing a divided attention task in a driving simulation. The experimental task developed for this study allowed us to explore if and how two different emotional states can affect the way drivers divide their attention between a central driving-related task and a peripheral visual task. Our results showed that scared drivers presented lower reaction times at the central task compared to relaxed drivers. On the contrary, the emotional state did not affect the performance at the peripheral task, which revealed instead a significant effect of the eccentricity at which the visual stimuli were presented, influencing both the accuracy of targets' perception and participants' reaction times.

Keywords: ADAS , Attention , Emotions , Virtual reality

The wind has been a natural and renewable resource used for professional and recreational maritime transportation of small and large vessels since human history. Sailing is making a comeback due to the growing focus on sustainability, accelerated by the recent global energy crisis. Seafarers rely on wind and bearing angle visualization to navigate efficiently and safely, thanks to the use of sensors and compasses. This paper focuses on Augmented Reality in Head-Mounted Displays visualization of wind and bearing angle data. We analyzed the literature and generated a heatmap of the used areas in the user's field of view. Second, we designed and implemented two interfaces that use two different visualization techniques: Boat Stabilized (BS) and Head Stabilized (HS). We compared them in between the subject experiment (N = 44), using a simulated Virtual Reality simulator of the sailing scenario. The user's primary task is wind events recognition, while obstacles (buoys) detection is secondary. We measured both task errors and reaction time, and submit NASA RTLX, SUS, UEQ, and visive auditive and kinesthetic (VAK) questionnaires. We found that BS has a significantly lower reaction time and better usability in the primary and secondary tasks. Both visualization techniques have similar users perceived cognitive load and user experience evaluation. VAK test showed that BS is better for kinaesthetic types and HS is better for visual types.

Keywords: Augmented reality , Human–computer interaction , Nautical , Navigation , Sailing , User study

This work aims to validate the “minimal AR” authoring approach in a real industrial assembly scenario. It focuses on optimizing visual assets in Augmented Reality (AR) work instructions. The design of AR assembly documentation is influenced by three main variables: work instructions, affordance (dependent on equipment components and operator capabilities), and AR signifiers (combination of visual assets with their properties). In this study, we fixed the instruction complexity while exploring the relationship between affordance and AR signifiers. First, we set up a focus group of 10 experts in AR technical documentation to extract guidelines for the design of minimal AR signifiers for assembly instructions with a variable affordance. Then, we validated these guidelines through an industrial case study involving 34 participants in four assembly tasks. We verified if the candidate minimal AR signifier, obtained using the proposed guidelines, corresponded to the minimal AR signifier established by users. The results showed that in 33% of the cases, users exploited the candidate minimal AR signifier to accomplish the task successfully. Beyond the minimal AR signifier, an additional one conveying the notification about the task success must always be provided to ensure failure by those operators with reduced capabilities. We also found that, in 29% of the cases, users needed less information than the candidate minimal AR signifier due to their higher capabilities. However, as expected, this condition leads users to make more errors than with the candidate minimal AR signifier. Moreover, the study confirms that AR signifiers with redundant information or attractive appearance, such as animated product models, are unnecessary to improve task comprehension. Still, animations could be beneficial in reinforcing understanding when object properties are difficult to detect.

Keywords: Assembly , Authoring , Industrial augmented reality , Minimal information , Visual asset , Work instruction