Abstract: In recent years, Augmented Reality (AR) has been effectively proposed as a tool to support workers in manual procedural tasks in industry, such as assembly and maintenance. It is very common for workers to deal with complete equipment in maintenance. Then, instructions may refer to components located in blind areas, i.e., visually occluded by part of the equipment. Displaying in-situ AR instructions in such blind areas requires handling occlusions; otherwise, side-by-side instructions can be exploited. In the literature, it is still unclear which solution better leverage workers’ performance in maintenance tasks accomplished in blind areas. Thus, we designed three AR presentation modes to convey maintenance instructions in a real machine: 3D in-situ rendered with X-ray technique; 3D side-by-side on a CAD model replicating the blind area; 2D side-by-side on a virtual mirror. We conducted a user study comparing these three presentation modes with a 2D drawing extracted from the original maintenance documentation of the machine. The performance of 42 participants was evaluated in terms of completion time, accuracy, and cognitive load. The results revealed that both in-situ and 3D side-by-side presentation modes perform better than 2D drawing. Specifically, the in-situ presentation mode outperforms the 3D side-by-side mode in terms of completion time. The side-by-side virtual mirror does not improve performance with respect to 2D drawing, then it needs to be redesigned for effective use in AR maintenance interfaces.

Keywords: Augmented reality | Blind area | Industrial operator support | Maintenance | Occlusion | Work instructions

Abstract: Augmented Reality (AR) is one of the leading technologies of the Industry 4.0 revolution, offering innovative interfaces to promote the diffusion of digital contents into industrial processes, thanks to flexible and robust solutions and cost-effective devices. In this context, this paper explores the adoption of AR in industrial logistics where several open issues still discourage its effective use in everyday scenarios. After a review of objectives, approaches and technics of AR integration in logistics operations, the paper presents a framework to identify goods in a warehouse, retrieve data relative to the package, display info to the user to drive operations. The approach aims at easing and speeding up the activity of the warehouseman to identify goods, check the relative information and to put each good on the correct shelf. A prototypal application was developed within the Unity platform and integrated with the company ERP system to manage data on the products and retrieve images of the identification labels. A real use case involving a primary company producing agricultural tractors is proposed to test usability of the prototype. Results showed that the developed application allows relevant benefits in terms of process effectiveness, error prevention, aiming at reducing the operator mental workload.

Keywords: Augmented Reality | Industry 4.0 | Logistics | Warehouse operations

Abstract: Factory acceptance test is the inspection of equipment and components at the supplier’s premises before delivery or final inspection. However, this control can represent a considerable cost for the customer, especially when the manufacturer’s company is geographically far from the customer one and inspections must be frequent. In this paper, the authors present a framework to support the factory acceptance test based on augmented reality (AR) techniques and model-based definition aimed at dimensional checks that does not require the physical presence of the customer at the supplier's premises. The supplier must be previously equipped with an automatic measuring machine. Once the component under inspection is placed inside the machine, this reads the type and the position of the features to be measured along with the related specification limits directly from the annotated 3D model of the component. The results are automatically transmitted to the customer’s site. Through a tablet, the supplier, guided by the customer, reads the results of the measures directly on the measured object through augmented or mixed reality techniques. Any out-of-specification dimension can be remeasured in real time with the customer’s remote assistance using traditional measurement techniques. The proposed architecture, at an advanced stage of experimentation, is discussed with reference to an industrial case study proposed and using an entry level commercial 3D scanner.

Keywords: Augmented reality | Dimensional inspection | Factory acceptance test | Metrology

Abstract: This work is focused on the development of a “serious game”, i.e., a software with no playful purpose adopted to train and verify the skills of users to improve safety in the workplace. Particular attention has been given to the accidents that occur during typical operations done in factory sites, e.g., driving forklifts. Firstly, the different types of “serious games” already available on the market have been analyzed to identify the best way to carry on tests and simulations by means of virtual, mixed, and augmented reality. Afterward, once the best solution has been identified, the Unity development environment has been considered to define a standard that could also be used for future projects. So, the result is the development of a powerful editor of the scene, in which the user can program all the components adjusting the game to the tasks to be performed by the worker, the tools, the environment and the targets. So, an upgradeable “parametric path” has been created, which will be followed by the forklift, realizing a new game and test environment for the worker. Then, modular components have been conceived to contemplate the future developments of the project, including a graphic editor: these will lead to a software that can be easily modified according to the customer’s requests.

Keywords: Occupational safety | Serious games | Virtual reality

Abstract: Preface and Acknowledgements (Editorial)

Keywords: Editorial

Abstract: The evaluation of hip implantation success remains one of the most relevant problems in orthopaedics. There are several factors that can cause its failure, e.g.: aseptic loosening and dislocations of the prosthetic joint due to implant impingement. Following a total hip arthroplasty, it is fundamental that the orthopaedist can evaluate which may be the possible risk factors that would lead to dislocation, or in the worst cases, to implant failure. A procedure has been carried out with the aim of evaluating the Range of Movement (ROM) of the implanted prosthesis, to predict whether the inserted implant is correctly positioned or will be prone to dislocation or material wear due to the malposition of its components. Leveraging on a previous patented methodology that consists in the 3D reconstruction and movement simulation of the hip joint, this work aims to provide a more effective visualization of the simulation results through Mixed Reality (MR). The use of MR for the representation of hip kinematics and implant position can provide the orthopaedic surgeon with a deeper understanding of the orientation and position of implanted components, as well as the consequences of such placements while looking directly at the patient. To this end, an anchoring system based on a body-tracking recognition library was developed, so that both completely automatic and human-assisted options are available without additional markers or sensors. An Augmented Reality (AR) prototype has been developed in Unity 3D and used on HoloLens 2, integrating the implemented human-assisted anchoring system option.

Keywords: Computer-aided surgery | HoloLens 2 | Mixed reality | THA assessment | Total hip arthroplasty

Abstract: In recent days product design and review cycles are considerably held to many constraints and requirements. One of the main requirements regards the usability of solutions and the impact that design choices have on final users. Therefore, it is becoming crucial to anticipate human factors concerns in the preliminary phases of product development process. Also, since human factors affect the users subconsciously and influence the decision making in a significant way, they need to be enabled to observe the users while interacting with the product in real-time. In this framework, the platform that the human factors expert and the users use plays a vital role in influencing the human factor assessments and evaluations. Latest developments in Extended Reality opened the way toward the possibility to conceive new simulation platforms that allow experts to deeply explore the products in real-time through collaborative and interactive environments before the physical mockup of the product. The purpose of this paper is to outline a Mixed Reality (MR) tool in the field of aircraft interior design to demonstrate it as a potential co-creative platform for involving human factor experts in the loop while the task is ongoing. An MR multi-user, co-located, collaborative and interactive environment of an aircraft galley is developed where the HF specialist and flight crew member can co-exist to visualize the real scale model of the galley and perform an operational task. Such co-creative tool is foreseen to execute usability tests during design review phase and reduce time, costs of product development cycle while meeting the user requirements.

Keywords: Co-creation | Design review | Human factors and ergonomics | Mixed reality

Abstract: Collaborative robots (cobots) are designed to directly interact with human beings within a shared workspace. To minimize the risk of musculoskeletal disease for the workers, a physical ergonomic assessment of their interaction is needed. Virtual reality (VR) and motion capture (Mocap) systems can aid designers in building low-hazard collaborative environments. This work presents a framework based on VR and Mocap systems for the ergonomic evaluation of collaborative robotic workstations. Starting from the 3D models of the cobot and workstation components, a virtual environment is built in Unity and ROS is employed to manage the cobot behavior. The physical ergonomics is evaluated by means of RULA methodology, exploiting the body tracking capabilities of the device Kinect Azure, a low-cost markerless Mocap system. The framework has been tested by building a virtual environment for collaborative control of flanges with different diameters. The worker interacts with a six-axis Nyro One to move parts on the workstation. The ergonomic assessment is performed in real-time, and a report is generated for later uses and evaluations. The proposed framework fosters the design of collaborative robotics workstations based on an objective assessment of ergonomics. The results of this research work allow planning future development steps for the emulation of more complex workstations with cobots and the use of augmented reality to evaluate how to modify existing workstations to introduce a cobot.

Keywords: Collaborative robots | Ergonomics | Motion capture | Virtual reality

Abstract: The development of cutting-edge technologies in the industrial sector has led to the demand for increasingly specific tools in the optimization of efficiency problems. The automotive sector is the one that makes the most use of these technologies, such as, among all, Virtual Reality (VR) and Augmented Reality (AR). A virtual reality tool is inserted as a guide tool for the user in drawing as quickly as possible the key information for optimizing the process. This research work fits into this context, the goal of which was the implementation of an immersive platform for carrying out accurate measurements within an entirely virtual automotive environment. A VR system of this type allows to check the dimensional and shape tolerances of car components, performing measurement, with high precision and in real time, in a custom-made virtual environment, in which it is possible to simulate the presence of a myriad of components and test their mutual interaction. A comparative test was carried out, obtained by varying the graphical and geometrical model in the VR settings, in order to evaluate the level of usability and the degree of efficiency of a tool for measuring distances between objects in a virtual reality environment, depending on the system parameters. The validation of a measurement instrument in VR is part of an increasingly current technological context in which the need to optimize the time-cost curve embraces the need for increasingly accurate results.

Keywords: Assembly training | Tolerance measurements | Virtual reality

Abstract: Human-Robot Interaction (HRI) is an interdisciplinary research field aiming to study and develop platform in which robots (and in particular collaborative robots, also called cobots) can interact and collaborate with humans to execute dedicated tasks. Usually, cobots are intended as passive robotic devices for direct collaboration with a human operator within a shared workspace. They are designed to be used in open and uncontrolled environments; the robot shall be able to adapt its behaviors to the dynamic input of the surrounding environment. In this optic, Mixed Reality (MR) can play a crucial role supporting the flow of data between the actors (cobot and human) working in the shared environment, it can offer a simply, but remarkably advanced, communication interface between human and robot. Thanks to MR, tools to allow human operators, without particular experience or knowledge of robotics, to easily interact with the cobot can be developed. Our work is focused on development of a MR platform that integrates cutting-edge technologies, i.e. a Head Mounted Display (HMD), and a cobot in a shared environment. The experimental setup includes the Microsoft’s Mixed Reality HMD HoloLens 2 and the Franka Emika Robot System.

Keywords: Cobot | Head Mounted Display | Human Robot Interaction | Mixed Reality | Robotics

Abstract: Extended Reality (XR) technologies can be a valid tool for supporting operators with assembly instructions in real-time, directly superimposed on the product. A strategy to face dynamic effects, such as sudden environmental changes and objects in movement, still needs to be defined to implement effective XR applications in moving production lines. The physical environment must be tracked and recognized to determine the position and the orientation of digital content in space. This paper aims to address these open issues by proposing a Mixed Reality (MR) application to support workers in the wire harness process for the tractor’s drivelines production, which requires the product to be in motion. For this aim, the Microsoft Azure Spatial Anchors with Microsoft HoloLens 2 were adopted. The designed and developed MR application allows the operator to display, step-by-step, consecutive work instructions provided through textual indications, pictures, videos, and animations. Through the Azure Spatial Anchors, the digital content can be anchored over the driveline parts. By knowing the speed of the production line and the duration of each wiring task, a time-controlled application has been developed, to assure the visualization of the instruction in correspondence of the relative driveline elements, in the right time. The application has been tested on-field with expert operators. Despite the presence of some drawbacks related both to the driveline motion and technology, the operators’ feedback pointed out satisfactory and promising results since the application allowed them to reduce errors and forgetfulness.

Keywords: Azure Spatial Anchor | In-motion tracking | Industry 4.0 | Mixed Reality | Wire harness process

Abstract: In this work, we present a technique to simplify the authoring of Augmented Reality Technical Documentation, allowing technical writers with limited knowledge in Augmented Reality (AR) to produce this new type of documentation in their companies. Contrarily to Traditional Technical Documentation, AR offers the opportunity to provide the exact amount of information needed through a careful design of the AR interface. However, in the literature, there are no established techniques to break work instructions down into elemental pieces of information and define how to convey this information through visual assets in AR. In this work, we proposed identifying the information contained in work instructions of technical documentation, extending the method of “Therbligs,” already used in the literature to describe assembly tasks. First, we defined six classes of information types: identity, location, order, way-to, notification, and orientation. Then, we showed how to use these information types to break down the work instructions of an assembly manual used as a case study. We found that the six information types were enough to analyze the complete manual. The second contribution of this work is the proposal of the most suitable visual asset and its properties for each information type. This goal was accomplished through a technical discussion in a focus group with ten experts in the design of AR technical documentation.

Keywords: Augmented reality | Authoring | Information presentation | Technical documentation | Work instructions

Abstract: Augmented Reality (AR) has been proved to be effective in maintenance operations in the industrial field. In a concurrent engineering approach, the authoring of AR manuals, to convey instructions to operators exploiting AR, must be done during the design phase of the product lifecycle. A reliable solution is needed to speed up the AR manual development process when the product is not physically available. We compared three solutions for displaying a demo version of an AR manual when the real product is not available, opting to replace it with its CAD model. Based on the user study results, the main features for each demo version were collected. The Augmented Reality (AR) solution allows to show the product that is not physically available in a real scale with the drawback of needing a physical printed marker. The Desktop Virtual Reality (DVR) prototype overcomes the problem of managing different devices and real-life locations, but without a real scale and a natural interaction. The Augmented Desktop Virtual Reality (ADVR) prototype allows to distinguish more easily the virtual elements of the true AR from those simulating the real product, but with a less natural interaction due to the use of a secondary screen. As a case study, we chose a compressor that a local company is going to produce with its AR manual. Although users overall preferred the AR demo version, the company chose the ADVR solution due to a better perception of what would be the result of the true AR application.

Keywords: Augmented Reality | Prototyping | Task simulation | Work instructions

Abstract: For the sake of being competitive in an ever-changing market, industrial companies need a redefinition of traditional design and integration of parts, equipment, and services such a redefinition allows effectively addressing the interaction between machines and operators, particularly in the area of complex production lines. In this context, enhancing ergonomics is crucial to reduce fatigue and stress of workers and increase work-place efficiency and comfort. Moreover, identifying ergonomic flaws in three-dimensional human-machine design problems (e.g., body posture, reach, visibility) at an early stage of the engineering process allows to prevent these issues at a low cost. Virtual reality (VR) is emerging as a powerful tool to improve the ergonomic assessment in the design of complex production lines. However, VR is not yet a well-consolidated practice for industrial companies, and the state-of-the-art applications are limited to simplified, isolated, and customized experiments. This work proposes the use of a virtual golden zone (VGZ) as a standard and efficient VR method for the ergonomic analysis and optimization of operator activities in manual manufacturing stations. The resulting effectiveness and benefits are highlighted through the application of the approach to a real industrial case study. Finally, the outcomes of a usability questionnaire, compiled by the professionals involved in the VR reviews, are presented to evaluate the usability of the VGZ methodology in the design process of complex production lines.

Keywords: Complex production lines | Ergonomics | Human computer interaction | Virtual reality

Abstract: This work is part of a larger project aiming to develop a comprehensive Augmented Reality (AR) interface for recreative and professional nautical sailing navigation. Due to the complexity of the marine environment and the dynamism of the crew members on board, we propose to diversify the display of navigation information concerning the role, position and activity onboard. The interfaces are designed to be viewed by one or more crew members, solving the problem of everyone viewing the same data, and giving each one the information they need based on their role or position. The three novel proposals for sail-specific AR graphic interfaces are designed to be displayed on Head-Mounted Display HMD and each of them differs for the information stabilization: Screen-Stabilized, Body-Stabilized and Boat-Stabilized. Each interface is different from the others differentiated according to the type of navigation and the user who must use that information. These approaches have been prototyped and evaluated by a panel of experts in the field of sailing navigation and were able to showcase their potential for future evaluations in different scenarios.

Keywords: Augmented Reality | Interface | Nautical | Sailing | Spatiality | User-based

Abstract: In the ever-changing scenario of technology evolution, designers need to develop new interactive systems that respond to users’ needs. Augmented Reality (AR) could be a tool and an opportunity for designers to create novel interactive systems. AR has proven to be effective in several domains, showing potential for widespread deployment even in everyday life tasks such as the use of household appliances. Thus, leveraging the Human-Centred Design (HCD) approach, we integrate AR into the design and development process of an interactive system for household appliances. Based on our survey results with 463 participants, one of the appliances that could benefit most from AR is the kitchen machine. Starting from a case study, we develop a demo to prove the feasibility of designing interactive systems with the integration of AR technologies, following the HCD approach.

Keywords: Augmented Reality | Human-Centred Design | Industrial Design | Interactive System | User Interface

Abstract: Conventionally, visitors of exhibitions cannot directly interact with artworks but remain mainly passive. This work presents a new way to discover paintings through an extended reality application with the aim of enhancing visitors’ engagement. The workflow consists of digitally recreating a painting. All its constitutive elements are contoured and then removed by the artwork. These elements will be unveiled by the visitors with gesture interaction. The discovery of each element is followed by additional information on the subject. The application also proposes an alternative experience where the user paints the components of the artwork with gestures. The hands’ position was tracked by the Leap Motion Controller, an optical sensor by Ultraleap. The process has been applied to one of the Tintoretto’s pieces of art: “L’Annunciazione del Doge Grimani”. The extended reality application has been experienced by over twenty thousand visitors at the dedicated exhibition. Numerous positive comments received from both visitors and guides are encouraging.

Keywords: Cultural heritage | Exhibition | Extended reality | User interaction

Abstract: This paper proposes a methodology to exploit virtual prototypes for the development of both VR and AR-based Training systems, with the same Software-Hardware architecture. This architecture provides a unique collaborative environment to enable the interaction between several users even if geographically distant, allowing to switch easily and rapidly between VR, AR and mixed technologies. The proposed Training Systems concern a simple but structured workcycle about the assembly of some Lego bricks. The selected development platform is IC.IDO: a commercial software with an industrial footprint, produced by ESI Group. Despite IC.IDO is originally designed for VR and not compatible for AR applications, it resulted fully compliant with the requisites of the proposed methodology. The paper firstly describes the path outlined from the conversion of a Training session from VR to AR compatible, within IC.IDO. Thanks to the adoption of a colour-filtering software as OBS Studio, the IC.IDO session easily becomes AR-compatible, using a unique hardware architecture: the HTC Vive Pro HMD and a VIVE tracker. In this work, also the Oculus Rift S has been employed for the VR applications to demonstrate the system’s compatibility with other VR devices. For both VIVE and Oculus, the Virtual Desktop feature allowed to display the OBS Desktop window in the HMD. Furthermore, an innovative solution is proposed: thanks to IC.IDO’s Cooperative Environment, trainer and trainee will cooperate within the same scene using simultaneously VR and AR. Finally, the role of the trainer is discussed, to outline a process of progressive independence for the trainee.

Keywords: Augmented reality | Cooperative environment | Virtual prototype | Virtual reality | Virtual training system

Abstract: The importance of training for operators in industrial contexts is widely highlighted in literature. Virtual Reality (VR) is considered an efficient solution for training, since it provides immersive, realistic, and interactive simulations environments promoting a learn-by-doing approach, far from the risks of the real field. Its efficacy has been demonstrated by several studies, but a proper assessment of the operator’s cognitive response in terms of stress and cognitive load during the use of such technology is still lacking. This paper proposes an integrated methodology for the analysis of user’s cognitive states, suitable for each kind of training in the industrial sector and beyond, fostering the human-centred design and manufacturing perspective. The methodology has been assessed using an industrial case study where virtual training is used for assembly of agricultural vehicles. Experimental results highlighted that, with VR additional supportive information, while operators’ errors drastically decrease, the stress increases for complex tasks, due to the greater amount of information to manage. The proposed protocol allows understanding the operators’ cognitive conditions in order to optimize the VR training application, avoiding operators’ stress, mental overload, and improving performance.

Keywords: cognitive ergonomics | mental workload | stress | virtual assembly | Virtual reality | virtual training

Abstract: Airports are a fundamental node in the aviation system. The growth of this sector undergoes the evolution of the airports procedures and infrastructures. Air traffic control is considered to be one of the most important activity performed in an airport. This often involves numerous personnel, construction and maintenance costs etc. With the ongoing digitalisation process in various fields, the concept of remote and virtual control towers (RVT) has emerged owing to the innovation in this field. Technologies such as augmented reality (AR) have successfully paved their smooth way to bring in improvements to an RVT. The current work aims to enhance and improve the controller work in an RVT exploiting the adaption of virtual reality and AR systems. In particular, this concept is under investigation within the EU funded SESAR project: RETINA (Resilient Synthetic Vision for Advanced Control Tower Air Navigation Service Provision) and DTT (Digital Technologies for Tower). AR content generation by virtue of various display technologies are studied and, Optical Spatial See-through displays (O SSTDs) have been considered for its own advantages in developing this application. One of the necessary requirements of AR in this configuration is identified as the tracking of the operator’s point of view (eye-tracking), to provide virtual content consistent with its real position. Thus, this paper elaborates the design of specific eye-tracking system using Microsoft Kinect V2 for the virtual control tower application. The need to have binocular vision to use AR content is assessed and the interface has been tested with few subjects to evaluate the precision of the measurements detected through the proposed solution.

Keywords: Air traffic control | Augmented reality | Eye-tracking | Microsoft kinect | Remote control tower | Spatial see-through displays

Abstract: User experience (UX) application in the practice of engineering and product design is still limited. The present paper provides insights into research on UX design and recommendations for design practitioners by pointing out common criticalities. These outcomes are achieved through a literature review on how UX relates to design. First, issues in benefitting from UX understanding in design are identified with a specific focus on theoretical contributions. Second, experimental papers investigating UX and design are analysed in relation to previously identified issues. Although issues are present to some extent in all the contributions, the empirical studies dealing with UX in design are overall valid. The results highlight UX's support in revealing design requirements, but its capability of steering design processes is arguable, as concrete guidelines for practitioners are not well described. Based on identified issues, the authors propose a checklist to make UX studies in design more reliable and their outcomes more comparable.

Keywords: best practices | design process | user experience | UX applications | UX issues

Abstract: This article presents a software platform to design serious games for the rehabilitation of severe memory loss by means of virtual reality (VR). In particular, the focus is on retrograde amnesia, a condition affecting patient’s quality of life usually after brain stroke. Currently, the standard rehabilitation process includes showing pictures of familiar environments to help memory recover, while the proposed rehabilitation solution aims at developing patient-specific serious games for memory loss starting from 3D scanning of familiar environments. The Occipital Structure sensor and the Skanect application have been used for the virtualization of the real objects and the environment. Instead of following the traditional approach to design a video game during which the game logic is specifically developed for a virtual scene and the software code is not meant to be recombined, a modular procedure has been designed using Unity to interface the virtual objects of each acquired environment without modifying the game logic. In addition, the developed solution makes available a set of software modules for patient’s monitoring and data management to automatically generate medical reports, which can be easily connected to each new serious game. A test has been performed to assess the main features of the VR platform and its usability recruiting ten testers. Medical personnel evaluated positively the platform, and they highlighted the importance of objective data to improve the ecological validity of the cognitive rehabilitation for retrograde amnesia.

Keywords: Neurological disorders | Serious game | Virtual reality

Abstract: This work proposes a novel virtual reality system which makes use of wearable sensors for testing and validation of cooperative workplaces from the ergonomic point of view. The main objective is to show, in real time, the ergonomic evaluation based on a muscular activity analysis within the immersive virtual environment. The system comprises the following key elements: a robotic simulator for modeling the robot and the working environment; virtual reality devices for human immersion and interaction within the simulated environment; five surface electromyographic sensors; and one uniaxial accelerometer for measuring the human ergonomic status. The methodology comprises the following steps: firstly, the virtual environment is constructed with an associated immersive tutorial for the worker; secondly, an ergonomic toolbox is developed for muscular analysis. This analysis involves multiple ergonomic outputs: root mean square for each muscle, a global electromyographic score, and a synthetic index. They are all visualized in the immersive environment during the execution of the task. To test this methodology, experimental trials are conducted on a real use case in a human–robot cooperative workplace typical of the automotive industry. The results showed that the methodology can effectively be applied in the analysis of human–robot interaction, to endow the workers with self–awareness with respect to their physical conditions.

Keywords: Cooperative workplace | Ergonomic analysis | Human– robot physical interaction | Virtual reality | Wearable sensors

Abstract: As one of the most promising technologies for next-generation mobile platforms, Augmented Reality (AR) has the potential to radically change the way users interact with real environments enriched with various digital information. To achieve this potential, it is of fundamental importance to track and maintain accurate registration between real and computer-generated objects. Thus, it is crucially important to assess tracking capabilities. In this paper, we present a benchmark evaluation of the tracking performances of some of the most popular AR handheld devices, which can be regarded as a representative set of devices for sale in the global market. In particular, eight different next-gen devices including smartphones and tablets were considered. Experiments were conducted in a laboratory by adopting an external tracking system. The experimental methodology consisted of three main stages: calibration, data acquisition, and data evaluation. The results of the experimentation showed that the selected devices, in combination with the AR SDKs, have different tracking performances depending on the covered trajectory.

Keywords: Apple ARKit | benchmarking | Google ARCore | mobile augmented reality | simultaneous localization and mapping (SLAM) | tracking accuracy

Abstract: In the context of Industry 4.0, Operator 4.0 paradigm represents a key factor when dealing with the integration of new digital technologies into smart factories that are suited for workers with different skills, capabilities, and preferences. In this regard, to encourage the introduction of these new digital solutions and achieve high user acceptance, it is fundamental to consider human factors and put the worker at the center of the development process through the adoption of structured design strategies such as user-centered design (UCD) approaches. In this perspective, the paper proposes a novel Augmented Reality (AR) tool for supporting operators at the workplace, in real-time, while performing inspection activities on built products. The proposed tool has been developed according to a user-centered design approach by involving end-users in the various design and testing stages. Preliminary tests have been conducted with representative users on a real case study to assess the usability of the proposed solution. The outcomes are very encouraging and lead us towards further investigations for effective and valid implementation of this AR tool in an industrial scenario.

Keywords: Industrial augmented reality | Industry 4.0 | Operator 4.0 | Usability | User-centered design

Abstract: Numerical simulations and Finite Element Analysis (FEA) have currently increased their applications in medical field for making preoperative plans to simulate the response of tissues and organs. Soft tissue simulations, such as colorectal simulations, can be adopted to understand the interaction between colon tissues and surrounding tissues, as well as the effects of instruments used in this kind of surgical procedures. This paper analyses through FEA the interaction between a surgical device and a colon tissue when it is fully clamped. Sensitivity analysis in the respect of the material mechanical behaviour, geometric approximation and the effect of thickness variation are investigated with the aim of setting up a virtual prototype of the surgical operation to aid mentoring and preliminary evaluation via haptic solutions. Through this investigation, the force feedback estimation that is necessary in many virtual-reality applications, may be estimated without discharging nonlinear effects that occur during clamping and that usually cannot be simulated efficiently to guarantee real-time solutions. Results are aligned with experimental data, confirming the reliability and right the set-up of FEA. Through them, the preliminary set-up of a haptic force feedback has been described and simulated through Simulink 3D animation, confirming the feasibility of the concept.

Keywords: FEA | Force feedback | Haptic device | Metamodeling | Surgical simulation | Virtual prototyping

Abstract: In this work, we propose a Mixed Reality (MR) application to support laboratory lectures in STEM distance education. It was designed following a methodology extendable to diverse STEM laboratory lectures. We formulated this methodology considering the main issues found in the literature that limit MR’s use in education. Thus, the main design features of the resulting MR application are students’ and teachers’ involvement, use of not distracting graphics, integration of traditional didactic material, and easy scalability to new learning activities. In this work, we present how we applied the design methodology and used the framework for the case study of an engineering course to support students in understanding drawings of complex machines without being physically in the laboratory. We finally evaluated the usability and cognitive load of the implemented MR application through two user studies, involving, respectively, 48 and 36 students. The results reveal that the usability of our application is “excellent” (mean SUS score 84.7), and it is not influenced by familiarity with Mixed Reality and distance education tools. Furthermore, the cognitive load is medium (mean NASA TLX score below 29) for all four learning tasks that students can accomplish through the MR application.

Keywords: augmented and virtual reality | distance education and online learning | improving classroom teaching | mixed reality | mobile learning

Abstract: Although Virtual Reality Social Skills Training has proven its effectiveness in treating psychiatric disorders, this VR application field is still under-researched for two main reasons. The first one is the unavailability of low-cost VR technologies with sufficient computational capacity needed to render realistic Virtual Environments. The second one consists of the need for specialized VR application developers, usually far from the mental health research field. The recent diffusion of low-cost stereoscopic viewers and the introduction of easy and fast VR content authoring systems, such as Cinematic Virtual Reality (CVR), allow overcoming these limitations. CVR makes it possible to capture real scenes through 360 cameras, augment them with additional virtual objects, and finally immerse the user in these synthetic but highly immersive environments. We present the design and the features of the Entellect360 prototype -an innovative tool supporting the rehabilitation process of subjects affected by schizophrenia. It exploits CVR technology to create Virtual Environments aimed at the rehabilitation of psychiatric patients. The Entellect360 features allow for rehabilitation sessions and patient-performance data-collection even under conditions of social distancing. We also explain the experimental protocol and the validation procedure the prototype will undergo to assess its effectiveness.

Keywords: 360-degree virtual reality | Cognitive rehabilitation | Human-computer interaction | Mental health | Social skills training

Abstract: Mixed Reality (MR) could help students in the understanding of complex concepts as well as increase their motivation in the learning process. In this work, our aim is to propose a MR application for the support of engineering students in the understanding of assembly drawings of complex machines. We presented the application of our design methodology for this case study. Then, based on the results of a user study with a sample of students, we tried to improve the usability and the user experience of the MR application, proposing an updated version. The usability of the revised application was in the range “good-excellent” (mean SUS score 77.0). We also presented the lessons learned in this case study, that can be a starting point for a renewal of consolidated didactic processes aiming at future application of MR in other STEM courses.

Keywords: Augmented and virtual reality | Distance education and online learning | Improving classroom teaching | Mixed Reality | Mobile learning

Abstract: In the last years, interactive exhibitions based on digital technologies have become widely common, thanks to their flexibility and effectiveness in engaging visitors and creating memorable experiences. One of the topics in which digital technologies can be particularly effective is the communication of abstract concepts that are difficult for the human mind to imagine. An emblematic example is the astronomy discipline, which requires us to imagine and understand phenomena far away from our everyday life. In this paper, the authors present a research project, MARSS, in which digital technologies are used effectively to enhance the Users’ Experience of the Museo Astronomico di Brera located in Milan. Specifically, the MARSS project aims at designing and developing a new digital journey inside the museum to allow different categories of visitors to enjoy the exhibition in an engaging and interactive way. The paper presents the design and development phases of the experience and its evaluation with users. The results of the evaluation indicate that the digital interactive experience is appreciated by users and is successful in translating the content of high scientific value into more engaging and easily understandable elements.

Keywords: augmented reality | cultural heritage | extended reality | interactive exhibitions | science museums | user experience

Abstract: The development of Virtual Reality in a wide range of field, including engineering related applications, has pushed towards the investigation of novel solutions that are able to take advantage of such new possibilities, while possibly trying to seamlessly integrate them within currently established workflows. Regarding conceptual sketching, which commonly represents one of the first activities taking place across Product Design development work-flows, there are examples of applications that allow to shift from the 2D layout of traditional drawing to a fully immersive 3D environment where the user is able to produce strokes in space by means of a set of natural gestures. Despite sounding extremely intuitive, this kind of approach also comes with potential issues: the lack of a supportive surface onto which the user can rely on to produce strokes with a high degree of precision while not feeling tired after prolonged sessions can be problematic. Based on these premises, a new hybrid approach is proposed: the user is still immersed in the Virtual Environment, but is able to make use of a traditional tablet device which lays on a physical desk in order to produce visible strokes in Virtual Reality, while having the possibility to simultaneously manipulate the position and the orientation of the scene thanks to a hand tracking device to break into the third dimen-sion. As designed, the application supports the generation of simple line strokes and few basic commands, but a thorough testing session is still needed to validate the solution and investigate on the necessary improvements.

Keywords: Conceptual design | Product design | Virtual reality

Abstract: Design for Sustainability is a research area based on a multidisciplinary approach, which has become increasingly important in recent years. Among the several approaches to Design for Sustainability emerged in the past decades, great attention is paid to the “Design for sustainable behavior” approach, used to design products that can impact on users' behaviors, through embedded smart technologies, e.g., Internet of Things (IoT). In fact, IoT systems are able to "dialogue" with the users, supporting the identification of any misbehavior, and suggesting more sustainable ones. The authors identified the opportunity to design and develop AR interactive applications aiming to generate awareness about the impact of humans on Earth, make people reason about how they can directly contribute to limit the expansion of this problem and induce their behavioral change. The applications are meant to engage users in an active process of exploring and discovering informative contents and to foster them to elaborate a personal and critical vision and change their bad habits. Specifically, this paper presents two case studies about the design and development of Augmented Reality applications and IoT products to be used for supporting users towards more conscious food consumption in their daily life, in order to reduce food waste.

Keywords: Augmented reality | Design for behavioral change | Design for sustainable behavior | Food waste

Abstract: The European Commission defined the new concept of Industry 5.0 meaning a more human-centric, resilient, and sustainable approach for the design of industrial systems and operations. A deep understanding of the work environment and organization is important to start analysing the working conditions and the resulting User eXperience (UX) of the operators. Also, the knowledge about users’ needs and ergonomics is fundamental to optimize the workers’ wellbeing, working conditions, and industrial results. In this context, the paper presents a strategy to effectively assess the UX of workers to promote human-centric vision of manufacturing sites, enhancing the overall sustainability of the modern factories. A set of non-invasive wearable devices is used to monitor human activities and collect physiological parameters, as well as questionnaires to gather subjective self-assessment. This set-up was applied to virtual reality (VR) simulation, replicating heavy duty work sequence tasks that took place in an oil and gas pipes manufacturing site. This approach allowed the identification of possible stressful conditions for the operator, from physical and mental perspectives, which may compromise the performance. This research was funded by the European Community's HORIZON 2020 programme under grant agreement No. 958303 (PENELOPE).

Keywords: Cognitive ergonomics | Human-centred design | Industry 5.0 | User experience | Virtual reality

Abstract: The scientific literature highlights how Mixed Reality (MR) simulations allow obtaining several benefits in healthcare education. Simulation-based training, boosted by MR, offers an exciting and immersive learning experience that helps health professionals to acquire knowledge and skills, without exposing patients to unnecessary risks. High engagement, informational overload, and unfamiliarity with virtual elements could expose students to cognitive overload and acute stress. The implementation of effective simulation design strategies able to preserve the psychological safety of learners and the investigation of the impacts and effects of simulations are two open challenges to be faced. In this context, the present study proposes a method to design a medical simulation and evaluate its effectiveness, with the final aim to achieve the learning outcomes and do not compromise the students' psychological safety. The method has been applied in the design and development of an MR application to simulate the rachicentesis procedure for diagnostic purposes in adults. The MR application has been tested by involving twenty students of the 6th year of Medicine and Surgery of Università Politecnica delle Marche. Multiple measurement techniques such as self-report, physiological indices, and observer ratings of performance, cognitive and emotional states of learners have been implemented to improve the rigour of the study. Also, a user-experience analysis has been accomplished to discriminate between two different devices: Vox Gear Plus® and Microsoft Hololens®. To compare the results with a reference, students performed the simulation also without using the MR application. The use of MR resulted in increased stress measured by physiological parameters without a high increase in perceived workload. It satisfies the objective to enhance the realism of the simulation without generating cognitive overload, which favours productive learning. The user experience (UX) has found greater benefits in involvement, immersion, and realism; however, it has emphasized the technological limitations of devices such as obstruction, loss of depth (Vox Gear Plus), and narrow FOV (Microsoft Hololens).

Keywords: Augmented reality | Cognitive load | Medical education | Mixed reality | Simulation | Stress

Abstract: Since custom-made 3D printed surgical guides for maxillofacial surgery are usually expensive, Augmented Reality (AR) can be efficiently employed to overcome the high costs. The proposed work aims to develop and test an AR application for different maxillofacial surgeries. The application consists in overlaying the cutting lines on the patient’s mandible to guide the clinician during the procedure. It has been realized in Unity and preliminary tested with HoloLens 2 and a 3D printed mandible. Seven participants performed two consecutive trials. The mandible with the obtained surgical lines has been scanned after each test to digitally reconstruct the traced lines and compare them with the surgical lines previously designed. The results allowed the preliminary analysis of the developed AR system’s accuracy and precision. Mean distances from the designed surgical guides showed good accuracy for the genioplasty (deviation error around 1.03 mm) and orthognathic surgery (deviation error around 1.27 mm), suggesting the applicability of HoloLens 2 for these kinds of surgery. On the contrary, the application was not suitable for the mandibular angle osteotomy (deviation error over 2.50 mm).

Keywords: Augmented reality | HoloLens 2 | Maxillofacial surgery | Surgical guides

Abstract: In the industrial context, the wire harness represents a process with a high degree of manual work and significant customizability of final products. To reduce the workers’ mental demand, the augmented reality (AR) can be an effective tool. The existing industrial applications mainly focus on manual assembly assistance, but few solutions have been developed for the wire harness. This paper tries to enhance such a process by supporting operators with an AR application that shows the work instructions and allows reducing their cognitive workload. The main goals are to propose a user-friendly and versatile tool and carry out a structured and complete evaluation of the user experience. Two experimental sessions were conducted in the laboratory by simulating the wire harness assembly and quality inspection. The use of the AR application is efficient and effective especially when instructions far from the workstation are considered as a reference.

Keywords: Augmented reality | Hololens 2 | Human-centered manufacturing | Industry 4.0 | Wire harness assembly

Abstract: Human factors studies are becoming more and more crucial in the automotive sector due to the need to evaluate the driver.s reactions to the increasingly sophisticated driving-assistant technologies. Driving simulators allow performing this kind of study in a controlled and safe environment. However, the driving simulation.s Level of Detail (LOD) can affect the users. perception of driving scenarios and make an experimental campaign.s outcomes unreliable. This paper proposes a study investigating possible correlations between driver.s behaviors and emotions, and simulated driving scenarios. Four scenarios replicating the same real area were built with four LODs from LOD0 (only the road is drawn) to LOD3 (all buildings with real textures for facades and roofs are inserted together with items visible from the road). 32 participants drove in all the four scenarios on a fixed-base driving simulator; their performance relating to the vehicle control (i.e., speed, trajectory, brake and gas pedal use, and steering wheel), their physiological data (electrodermal activity, and eye movements), their subjective perceptions, opinions and emotional state were measured. The results showed that drivers. behavior changes in a very complex way. Geometrical features of the route and environmental elements constrain much more driving behavior than LOD does Emotions are not affected by LODs. Generally, different signals showed different correlations with the LOD level, suggesting that future studies should consider their measures while modeling the virtual scenario. It is hypothesized that scenario realism is more relevant during leisurely environmental interaction, whilst simulator fidelity is crucial in task-driven interactions.

Keywords: driver behavior | Driving simulator | environmental psychology | eye tracker | level of detail (LOD) | simulation reliability

Abstract: The paper presents the human-centered interdisciplinary methodology “SafeMob - Safe Mobility Experiential and Environmental Assessment” developed by the authors. The method combines Objective Safety (OS) with Perceived Safety (PS) to evaluate the performance of mobility solutions. The interrelation of data from the person (physio/psycho) and the environment (road, vehicle, and the surrounding or interacting context - including flows, buildings, and weather) makes the methodology holistic and interdisciplinary. The final goal is to provide a ‘Decision Support System’ for stakeholders in the mobility field, the automotive sector, and the urban planning area. The paper describes the overall theoretical approach and a specific case study application using a car simulator. Emotional reactions of users, driving through the same virtual scenario with different Level of Details, are assessed to gather information about the perceived safety of the environment.

Keywords: Drive simulator | Emotions | Level of Detail | Physiological measures | Virtual reality

Abstract: Projector-based Spatial Augmented Reality (P-SAR) is a technology that allows to alter the external appearance of a physical object by means of an almost infinite variety of computer-generated contents. Thanks to the adoption of colored lights, a projected image acquires coherent spatial properties with respect to the data it represents; this has been demonstrated to facilitate the users’ interpretation of complex information. The current paper presents the development of a P-SAR system aiming at supporting the real-time visualization and the inspection of engineering simulation results. Particular attention is paid on detailing the algorithms necessary for the generation of the color maps to be displayed on the prototype’s surface. These are interpolated starting from a discrete array of output data coming from a generic simulation to resemble the configuration of sensors commonly adopted in real experimental setups. An illustrative case study applied to CFD analysis is finally discussed to show the applicability of such immersive environments in engineering fields that require to perform testing activities with equipment like wind tunnels.

Keywords: Engineering analysis | Simulation | Spatial augmented reality

Abstract: Worldwide, stroke is the third cause of disability. The majority of people affected by this disease cannot perform activities of daily living. Bringing the therapy to the patients' home is complex, and in literature, there are still open challenges to face. Starting from therapists' and patients' needs, this paper describes a possible solution: HANDY, a rehabilitative active hand exoskeleton for post-stroke patients. With a desktop application, they perform three different types of exercises: passive, active and based on activities of daily living. They can also control the exoskeleton themselves in a serious-game approach with a leap motion controller. We evaluated our method with patients at the Villa Beretta rehabilitative center. Preliminary results from the session about comfort, usability and willingness to utilize the system are promising.

Keywords: Additive manufacturing | CAD modeling | Hand exoskeleton | Interactive applications | Stroke

Abstract: Successful aircraft cabin design depends on how the different stakeholders are involved since the first phases of product development. To predict passenger satisfaction prior to the manufacturing phase, human response was investigated in a Virtual Reality (VR) environment simulating a cabin aircraft. Subjective assessments of virtual designs have been collected via questionnaires, while the underlying neural mechanisms have been captured through electroencephalographic (EEG) data. In particular, we focused on the modulation of EEG alpha rhythm as a valuable marker of the brain’s internal state and investigated which changes in alpha power and connectivity can be related to a different visual comfort perception by comparing groups with higher and lower comfort rates. Results show that alpha-band power decreased in occipital regions during subjects’ immersion in the virtual cabin compared with the relaxation state, reflecting attention to the environment. Moreover, alpha-band power was modulated by comfort perception: lower comfort was associated with a lower alpha power compared to higher comfort. Further, alpha-band Granger connectivity shows top-down mechanisms in higher comfort participants, modulating attention and restoring partial relaxation. Present results contribute to understanding the role of alpha rhythm in visual comfort perception and demonstrate that VR and EEG represent promising tools to quantify human–environment interactions.

Abstract: In the last two decades a huge number of interactive and collaborative applications of Virtual Environments for designing products has been proposed. Such applications have been recommended as tools to implement Human Centered Design Approach in experiments where potential users are involved in participatory design sessions before going for production. In this kind of experiments, we observe that users are mainly involved in the validation of solutions previously elaborated by designers while in the last decade the most innovative approach in the creation of solutions seems to be moving from a user centered design to a co-design or co-creation process. Thus, it is essential to have a platform where the elements of co-creation can be fulfilled in building a successful project. The purpose of this paper is to outline the concept of co-creation and the significance of co-creation platforms alongside of proposing innovative tools for building the co-creative environments. The paper aims to layout a classification of the Extended Reality (XR) tools currently available and of their functionality as valuable means to actually embed co-design and co-creation concepts in Virtual Environments evaluating the advantages that this can bring to Industry through field studies. A review of innovative solutions like Virtual, Augmented and Mixed reality technologies is examined and drawn towards the requirements of the concept through a literature research. Additionally, a co-creative environment for designing aircraft cabin interiors is conceived and discussed a with company representative.

Keywords: Co-creation | Collaboration | Extended reality | Interaction platforms

Abstract: Many industrial technologies are developed to optimize products and bring innovation. In particular, the automotive sector is renewing itself according to the rules of green energy and consumption. This huge change requires a reinterpretation of the models on the market updating them to the present and the future needs of automotive industry. In this paper the best compromise between innovation and tradition is found for the Ford brand that has not yet presented electric cars in the sedan segment. Following the SDE method enriched with Quality Function Deployment (QFD), Benchmarking (BM) and Top Flop Analysis (TPA), it is possible to carry out an innovative project. All these technologies must, however, be ordered according to a specific product allowing the best result for the design process. It is therefore necessary identifying the most common stylistic trends in order to draw the external styling of the vehicle using virtual prototyping techniques. To achieve an innovative result, Augmented Reality (AR) is considered to complete the method substituting the static and expensive procedure of making maquettes.

Keywords: Additive manufacturing | Augmented reality | Benchmarking | Car design | Design engineering | QFD | Stylistic design engineering (SDE)

Abstract: Industrial Augmented Reality (iAR) has demonstrated its advantages to communicate technical information in the fields of maintenance, assembly, and training. However, literature is scattered among different visual assets (i.e., AR visual user interface elements associated with a real scene). In this work, we present a systematic literature review of visual assets used in these industrial fields. We searched five databases, initially finding 1757 papers. Then, we selected 122 iAR papers from 1997 to 2019 and extracted 348 visual assets. We propose a classification for visual assets according to (i) what is displayed, (ii) how it conveys information (frame of reference, color coding, animation), and, (iii) why it is used. Our review shows that product models, text and auxiliary models are, in order, the most common, with each most often used to support operating, checking and locating tasks respectively. Other visual assets are scarcely used. Product and auxiliary models are commonly rendered world-fixed, color coding is not used as often as expected, while animations are limited to product and auxiliary model. This survey provides a snapshot of over 20 years of literature in iAR, useful to understand established practices to orientate in iAR interface design and to present future research directions.

Keywords: Augmented reality | industry | reviews | user interfaces | visualization

Abstract: Industry 4.0 is characterized by great potential for innovation impacting the operator's role, increasingly engaged in smart activities of a decision-making nature. In such a working scenario, operators' working conditions can be effectively improved by applying a user-centered collaborative design approach. To this end, we developed a Virtual Reality-based multiplayer tool exploiting low-cost body tracking technology to evaluate ergonomic postural risk. The tool allows evaluating both in real-time and off-line the ergonomic postural risk according to the Rapid Upper Limb Assessment metrics. By applying this approach, a twofold advantage can be achieved. On the one hand, ergonomic experts can have an immersive three-dimensional visualization of postures even in off-line observations. On the other hand, it is possible to evaluate the ergonomics of workstations in the design phase by having the operator work on virtual mock-ups of workstations, thus allowing a sustainable approach to user-centered collaborative design.

Keywords: Collaborative design | Ergonomics | Kinect V2 | RULA | User-centered | Virtual Reality

Abstract: The introduction of the new generation of Head Mounted Displays (HMD) makes users’ experiences in Mixed Reality (MR) environments more engaging. However, these devices still have a limited field of view, which negatively affects the spatial localization process of virtual objects in the 3D environment. The literature presents several visualization techniques to address this issue, but they currently have several drawbacks, such as visual clutter, occlusion of the real scene, high user workload, and there is still no visualization technique that solves such issues definitively. Therefore, inspired by the gaming industry, we present CompassbAR a visualization technique for out-of-view objects. CompassbAR encodes the position of all out-of-view objects surrounding the user, in a 2D bar positioned at the top of the field of view. In addition, we propose a validation procedure and metrics that aim to evaluate the ability of the CompassbAR visualization technique to guide users towards the out-of-view objects.

Keywords: Head-mounted display | Mixed Reality | Visualization techniques

Abstract: In the era of ‘metaverse’, virtual environments are gaining popularity among new multimedia contents and are also recognized as a valuable means to deliver emotional content. This is favoured by cost reduction, availability and acceptance by end-users of virtual reality technology. Creating effective virtual environments can be achieved by exploiting several opportunities: creating artificial worlds able to generate different stories, mixing sensory cues, and making the whole interactive. The design space for creating emotional virtual environments is ample, and no clear idea of how to integrate the various components exists. This paper discusses how to combine multiple design elements to elicit five distinct emotions. We developed and tested two scenarios per emotion. We present the methodology, the development of the case studies, and the results of the testing.

Keywords: Affective Virtual Reality | Design Methodology | Emotions | Metaverse | Virtual Reality

Abstract: This work investigates the possibility of using a novel “minimal AR” authoring approach to optimize the visual assets used in augmented reality (AR) interfaces to convey work instructions in manufacturing. In the literature, there are no widely supported guidelines for the optimal choice of visual assets (e.g., CAD models, drawings, and videos). Therefore, to avoid the risk of having AR technical documentation based only on the author’s preference, our work proposes a novel authoring approach that enforces the minimal amount of information to accomplish a task. Minimal AR was tested through a simulated AR LEGO-based assembly task. The performance (completion time, mental workload, errors) of 40 users was evaluated with 4 combinations of visual assets in 4 tasks with an increasing amount of information needed. The main result is that visual assets with an excess of information do not significantly increase performance. Therefore, the location of a specified object should be “minimally” authored by an auxiliary model (e.g., a circle and an arrow). For identifying an object within a couple, color coding is preferred to using additional visual assets. If more than two objects must be identified, a drawing visual asset is also needed. Only when the orientation of a selected object must be conveyed, animated product models are required. These insights could be helpful for an optimal design of AR work instructions in a wide range of industrial fields.

Keywords: Authoring | Industrial augmented reality | Industrial metaverse | Minimal information | Visual asset | Work instruction

Abstract: In this paper, industrial design structure (IDeS) is applied for the development of two new full-electric sports sedan car proposals that go by the names Blitz Vision AS and Retro. With a deep analysis of the trends dominating the automotive industry, a series of product requirements was identified using quality function deployment (QFD). The results of such analysis led to the definition of the technical specifications of the product via benchmarking (BM) and top-flop analysis (TFA). The product architecture was then defined by making use of a modular platform chassis capable of housing a variety of vehicle bodyworks. The structured methodology of stylistic design engineering (SDE) was used. This can be divided in six phases: (1) stylistic trends analysis; (2) sketches; (3) 2D CAD drawings; (4) 3D CAD models; (5) virtual prototyping; (6) solid stylistic model. The chassis of the CAD model was verified structurally by means of FEM analysis, whereas the drag coefficients of the two vehicle proposals were compared with one of the main competitor’s vehicles via CFD simulations. The resulting car models are both aesthetically appealing and can be further developed, leading eventually to the production stage. This proves the effectiveness of IDeS and SDE in car design.

Keywords: additive manufacturing | augmented reality | car design | design engineering | industrial design | quality function deployment (QFD) | stylistic design engineering (SDE) | vehicle virtual design | virtual product development

Abstract: Laboratory experiences have proved to be a key moment of the educational path in most of the so-called Sciences, Technology, Engineering and Mathematics (STEM) subjects. Having the opportunity of practicing on actual experiments about the theoretical knowledge achieved during the classroom lectures is a fundamental step from a didactic point of view. However, lab activities could be forbidden in the presence of tests characterized by safety issues, thus limiting students' cultural growth; this is particularly true for physics experiments involving radioactive materials, sources of dangerous radiations. To face the considered problems, the authors propose hereinafter a mixed-reality solution involving augmented reality (AR) at students-side and actual instrumentation at laboratory-side. It is worth noting that the proposed solution can be applied for any type of experiment involving the remote control of measurement instruments and generic risk conditions (physical, chemical or biological). As for the considered case study on gamma radiation measurements, an ad-hoc AR application along with a microcontroller-based prototype allows students, located in a safe classroom, to (i) control distance and orientation of a remote actual detector with respect to different radioactive sources and (ii) retrieve and display on their smartphones the corresponding energy spectrum. The communication between classroom equipment and remote laboratory is carried out by means of enabling technologies typical of Internet of Things paradigm, thus making it possible a straightforward integration of the measurement results in cloud environment as dashboard, storage or processing.

Keywords: Augmented reality | Mixed-reality education | MQTT protocol | Physics experiments | Radiation measurements | Remote laboratory | Reverse engineering

Abstract: Background: Augmented Reality (AR) represents an innovative technology to improve data visualization and strengthen the human perception. Among Human–Machine Interaction (HMI), medicine can benefit most from the adoption of these digital technologies. In this perspective, the literature on orthopedic surgery techniques based on AR was evaluated, focusing on identifying the limitations and challenges of AR-based healthcare applications, to support the research and the development of further studies. Methods: Studies published from January 2018 to December 2021 were analyzed after a comprehensive search on PubMed, Google Scholar, Scopus, IEEE Xplore, Science Direct, and Wiley Online Library databases. In order to improve the review reporting, the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines were used. Results: Authors selected sixty-two articles meeting the inclusion criteria, which were categorized according to the purpose of the study (intraoperative, training, rehabilitation) and according to the surgical procedure used. Conclusions: AR has the potential to improve orthopedic training and practice by providing an increasingly human-centered clinical approach. Further research can be addressed by this review to cover problems related to hardware limitations, lack of accurate registration and tracking systems, and absence of security protocols.

Keywords: augmented reality | digital health | HoloLens | Human–Computer Interaction (HCI) | Human–Machine Interaction (HMI) | intraoperative | medical training | mixed reality | rehabilitation

Abstract: Virtual reality (VR) is among the main technologies revolutionizing numerous sectors, including tourism. In the latter context, virtual tours (VTs) are finding increasing application. Providing an immersive and realistic human–machine interaction, VR tours can bring visitors to virtually experience destination areas. The proposed research presents a theoretical and empirical investigation of the role played by some technical VR features (i.e., presence, immersion, ease-of-use) on VR visitors’ enjoyment, satisfaction, and, accordingly, on the physical visit intention of the production site and neighboring areas. After having experienced a 360-degree VR tour of a food production site, created specifically for this study, 140 visitors were surveyed online. Results—emerging from a PLS structural equation model—show that immersion and presence both directly impact the enjoyment and indirectly the user’s VR tour satisfaction and visit intention. Further, if the VR tour is perceived as easy to use, it influences visitors’ satisfaction and physical visit intention. This study contributes to the novel VR literature, applied in the tourism sector, evidencing how immersive and enjoyable scenarios, experienced via widespread devices such as smartphones, may impact tourists’ choices. In food tourism, VR technologies can be fundamental in attracting new visitors to the production sites and neighboring areas.

Keywords: Ease-of-use | Enjoyment | Human–machine interaction | Immersion | Intention to visit a food production site | Presence | Tourism | Virtual reality | Virtual tours | VR tour satisfaction

Abstract: Purpose: To evaluate the role of 3D models on positive surgical margin rate (PSM) rate in patients who underwent robot-assisted radical prostatectomy (RARP) compared to a no-3D control group. Secondarily, we evaluated the postoperative functional and oncological outcomes. Methods: Prospective study enrolling patients with localized prostate cancer (PCa) undergoing RARP with mp-MRI-based 3D model reconstruction, displayed in a cognitive or augmented-reality fashion, at our Centre from 01/2016 to 01/2020. A control no-3D group was extracted from the last two years of our Institutional RARP database. PSMr between the two groups was evaluated and multivariable linear regression (MLR) models were applied. Finally, Kaplan–Meier estimator was used to calculate biochemical recurrence at 12 months after the intervention. Results: 160 patients were enrolled in the 3D Group, while 640 were selected for the Control Group. A more conservative NS approach was registered in the 3D Group (full NS 20.6% vs 12.7%; intermediate NS 38.1% vs 38.0%; standard NS 41.2% vs 49.2%; p = 0.02). 3D Group patients had lower PSM rates (25 vs. 35.1%, p = 0.01). At MLR models, the availability of 3D technology (p = 0.005) and the absence of extracapsular extension (ECE, p = 0.004) at mp-MRI were independent predictors of lower PSMr. Moreover, 3D model represented a significant protective factor for PSM in patients with ECE or pT3 disease. Conclusion: The availability of 3D models during the intervention allows to modulate the NS approach, limiting the occurrence of PSM, especially in patients with ECE at mp-MRI or pT3 PCa.

Keywords: 3D modeling | Augmented reality | Prostate cancer | Robotic surgery | Surgical margins

Abstract: Games play an important role in the development of children. Especially when it comes to children with special needs like autism, it is important to have a different approach when it comes to learning. Shapes and colours are the most important fundamental skills that help in the recognition of objects around and help with letters and words. Balloon Pop is an application that is designed keeping in mind the requirements of children diagnosed with autism. The paper gives an idea of the user’s needs and requirements and shows how the project solves the problem with augmented reality game design. The goal of the project is to design a game for autistic children that will help them learn the abstract concept of shapes and colours. The Digital game-based learning methodology (DGBL) is used in developing the game. The paper consists of the design rules that have been followed in designing for better interaction along with hardware and software architecture describing the flow of the project. Also, at last, the test and evaluation and 8 users have been done by using the System Usability Scale (SUS) tool. The application has been proven to be helpful for children along with parents/therapists to achieve the goal to make children interact and communicate freely, increase focus, and learn the abstract concepts of shapes and colour easily.

Keywords: Augmented reality | Autism | Balloon pop | Children | Game

Abstract: The design of complex manufacturing systems (CMSs) is challenging, because of the requirements of efficiency, safety, and ergonomics, and the need of optimizing resources, i.e., space, machines, operators, and data. Virtual reality (VR) – one of the promising technologies at the base of Industry 4.0 – is able to address the design issues of CMSs, and even decrease costs and time when employed from the initial conception to the final validation of production lines, since it facilitates their virtual commissioning, i.e., it enables the full verification of systems and related components by virtual inspection and tests. Despite the above advantages, VR is still rarely used in the design of CMSs, and there is no standard VR approach in industry yet. In addition, the related scientific literature is scarce and often limited to small or simplified cases. To fill this gap, this work presents a novel VR-based approach for designing CMSs, composed of four phases: Three-dimensional CAD Export, Model Import, Scene Creation, and VR Review. The proposed approach is applied to a real industrial use case related to the virtual commissioning of an electric axles production line and it is evaluated through a questionnaire from industry professionals. The case study shows that using the VR technology enhanced the technical communication between experts in the teamwork, and it was particularly effective in finding ergonomics flaws like issues in visibility, reach, and posture using a virtual golden zone. In addition, all users found the VR interaction enjoyable and easy to learn, and beginner users perceived a comparable workload as advanced users.

Keywords: Complex manufacturing systems | Human computer interaction | Industry 4.0 | System design | Virtual commissioning | Virtual reality

Abstract: Following an oncological resection or trauma it may be necessary to reconstruct the normal anatomical and functional mandible structures to ensure the effective and complete social reintegration of patients. In most surgical procedures, reconstruction of the mandibular shape and its occlusal relationship is performed through the free fibula flap using a surgical guide which allows the surgeon to easily identify the location and orientation of the cutting plane. In the present work, we present a Mixed Reality (MR)-based solution to support professionals in surgical guide positioning. The proposed solution, through the use of a Head-Mounted Display (HMD) such as that of the HoloLens 2, visualizes a 3D virtual model of the surgical guide, positioned over the patient’s real fibula in the correct position as identified by the medical team before the procedure. The professional wearing the HMD is then assisted in positioning the real guide over the virtual one by our solution, which is capable of tracking the real guide during the whole process and computing its distance from the final position. The assessment results highlight that Mixed Reality is an eligible technology to support surgeons, combining the usability of the device with an improvement of the accuracy in fibula flap removal surgery.

Keywords: HoloLens 2 | mandibular reconstruction | maxillofacial surgery | mixed reality | surgical guide

Abstract: Numerous computer-based therapies have been designed for cognitive-behavioral interventions to support children with Autism Spectrum Disorder (ASD) in recent years. Among these technologies, Augmented Reality (AR) offers unique educational benefits because it provides children with direct guidance on their learning tasks. In this work, we propose “AR-brickhouse,” an AR application to support ASD children during LEGO therapy. It combines the benefits derived from AR technology and the LEGO tangible user interface with caregivers’ involvement. The novelty of our system concerns the improvement of ASD children’s basic skills such as positioning in space; focusing on tasks; acquisition of concepts of shape, color, and size. A preliminary user study involved eight ASD children and twelve therapists from a real medical center. Our results suggest that the proposed system is easy for therapists and allows children to improve the aforementioned basic skills. In fact, they were able to accomplish LEGO assembly tasks with better accuracy and in less time than traditional LEGO therapy.

Keywords: Augmented Reality | Autism Spectrum Disorder | LEGO assembly | Play therapy

Abstract: This paper presents the preliminary design of a teleoperation system for a bimanual bartending robot, with reference to the BRILLO (Bartending Robot for Interactive Long Lasting Operations) project. The aim is to simulate the remote control of the robotic bartender by the human operator in an intuitive manner, using Virtual Reality technologies. The proposed Virtual Reality architecture is based on the use of commercial Head Mounted Display with a pair of hand controllers and the virtual simulation of the remote environment of the robot, with the robotic simulator CoppeliaSim. Originally, virtual simulations of the robot environment have allowed to identify the possible scenarios and interactions between the customers and the different robotic systems inside the automatized bar: the totem for the selection and payment of the order, the robotic bartender to prepare the cocktail and the mobile robot for the cocktail serving at the table. Secondly, focusing on a sequence of main tasks that the robotic bartender must perform for the cocktails preparation, the operator’s control on the simulated robotic system has been reproduced. In fact, the aim of this first experimental phase is to test the interaction between the human operator and the simulated immersive environment for the remote control of the robotic system. Two use cases have been reproduced: the first is related to the recovery from a failure situation such as the fall of a glass, while the second refers to the trajectory training to perform some repeating actions. Six operators (three males and three females), who already knew the taks, with an age between 25 and 40 years and a minimum experience with VR technology for personal entertainment, have been involved in the test phase. For this reason, the paper will finally discuss the perception of the involved operators about the use of the proposed VR architecture in terms of usability and mental workload.

Keywords: CoppeliaSim | HTC VIVE Pro | Remote control | Teleoperation | Virtual Reality

Abstract: Sailing navigation is an activity that requires acquiring and processing information from the surrounding environment. The advancement of technology has enabled sailboats to have an increasing number of onboard sensors that make sailing more user-friendly. However, data provided by these sensors are still visualized on 2D digital displays that imitate traditional analog interfaces. Although these displays are strategically placed on the sailboat, the user needs to divert attention from the primary navigation task to look at them, thus spending a significant amount of cognitive resources. AR-based technologies have the potential to overcome these limitations by displaying information registered in the real environment, but there are no studies in the literature for validating the effectiveness of this technology in the field of sailing. Thus, we designed a head-mounted display AR-based interface to assist users in monitoring wind data to avoid user diversion from the primary task of sailing. We conducted a user study involving 45 participants in an Immersive Virtual Reality simulated environment. We collected objective and subjective measures to compare the AR-based interface with a traditional data visualization system. The AR-based interface outperformed the traditional data visualization system regarding reaction time, cognitive load, system usability, and user experience.

Keywords: Augmented reality | Cognitive load | Human–computer interaction | Nautical | Sailing | User study

Abstract: Research shows that reduced exposure to natural contexts is associated with an increase in psychophysical disorders. Recent evidence suggests that even a brief experience in natural scenarios can positively affect people's health and well-being. However, natural contexts are not always easily accessible. This study investigates the effects of natural and indoor virtual environments (VREs) on psychophysiological and cognitive responses. Following a within-subject design, 34 healthy participants were exposed to two VREs (i.e., a forest and a living room) in a counterbalanced order through a head-mounted display (Oculus Rift). Participants were asked to explore the scenarios and execute a modified version of the Paced Auditory Serial Addition Test. Physiological parameters (heart rate, skin conductance level [SCL], and respiration rate) were recorded during the whole session. After the exposure to VREs, participants filled a set of visual analog scales to rate their subjective experience of presence, relaxation, and stress. Participants reported a higher perceived sense of relaxation in the virtual forest. Moreover, their SCLs were significantly higher in this environment, showing that the forest elicited higher physiological arousal than the living room. Furthermore, their SCLs were significantly higher during the attentional task in the virtual living room. The results suggest that a natural virtual environment can make people feel more relaxed and physiologically engaged than an indoor scenario. The latter instead can be linked to a performing venue, as reported for real contexts. However, these changes were not related to modulations of attentional performance.

Keywords: attentional performance | physiological measurement | relaxation | virtual environments | virtual reality

Abstract: The current study aimed to propose a Deep Learning (DL) based framework to retrieve in real-time the position and the rotation of an object in need of maintenance from live video frames only. For testing the positioning performances, we focused on intervention on a generic Fused Deposition Modeling (FDM) 3D printer maintenance. Lastly, to demonstrate a possible Augmented Reality (AR) application that can be built on top of this, we discussed a specific case study using a Prusa i3 MKS FDM printer. This method was developed using a You Only Look Once (YOLOv3) network for object detection to locate the position of the FDM 3D printer and a subsequent Rotation Convolutional Neural Network (RotationCNN), trained on a dataset of artificial images, to predict the rotations’ parameters for attaching the 3D model. To train YOLOv3 we used an augmented dataset of 1653 real images, while to train the RotationCNN we utilized a dataset of 99.220 synthetic images, showing the FDM 3D Printer with different orientations, and fine-tuned it using 235 real images tagged manually. The YOLOv3 network obtained an AP (Average Precision) of 100% with Intersection Over Unit parameter of 0.5, while the RotationCNN showed a mean Geodesic Distance of 0.250 (σ = 0.210) and a mean accuracy to detect the correct rotation r of 0.619 (σ = 0.130), considering as acceptable the range [r − 10, r + 10]. We then evaluate the CAD system performances with 10 non-expert users: the average speed improved from 9.61 (σ = 1.53) to 5.30 (σ = 1.30) and the average number of actions to complete the task from 12.60 (σ = 2.15) to 11.00 (σ = 0.89). This work is a further step through the adoption of DL and AR in the assistance domain. In future works, we will overcome the limitations of this approach and develop a complete mobile CAD system that could be extended to any object that presents a 3D counterpart model.

Keywords: Augmented reality | CAD assistance | Deep learning | Neural network

Abstract: Understanding user experience (UX) is essential to design engaging and attractive products, so nowadays has emerged an increasingly interest in user-centred design approach; in this perspective, digital technologies such as Virtual Reality (VR) and Mixed Reality (MR) could help designers and engineers to create a digital prototype through which the user feedback can be considered during the product design stage. This research aims at creating an interactive Digital Twin (DT) using MR to enable a tractor driving simulation and involve real users to carry out an early UX evaluation, with the scope to validate the design of the control dashboard through a transdisciplinary approach. MR combines virtual simulation with real physical hardware devices which the user can interact with and have control through both visual and tactile feedback. The result is a MR simulator that combines virtual contents and physical controls, capable of reproducing a plowing activity close to reality. The principles of UX design was applied to this research for a continuous and dynamic UX evaluation during the project development.

Keywords: Digital Engineering | Digital Twin | Human-centered Design | Mixed Reality | User experience design

Abstract: This paper presents a case study regarding the simulation of a robotic workstation to virtually test objects detection and obstacle avoidance. Testing these features inside a virtual environment is useful, especially when human-robot cooperation and interaction are involved. Indeed, it allows users to avoid real dangerous conditions, lowering the possible risks of injuries by the users and cutting down the costs compared to a real testing environment. The work presented here exploits a framework where a virtual environment is connected to a Robot Operating System (ROS) able to simulate the kinematic of the robot and, on the other side, a physical ultrasonic sensor acts as the bridge with the real world. The latter, driven by an Arduino board, allows the virtual robot to recognize static obstacles in the real world, mapping the surrounding environment and computing a suitable trajectory to accomplish the given task. Thanks to the sensing capability, the virtual robot is also able to react to the presence of other obstacles (e.g. humans) entering the workspace at runtime. The seamless connection between the virtual and the physical world makes the framework suitable for the fast testing of new algorithms driving the behavior of the robot when interacting with dynamic environments.

Keywords: Costs | Human robot interaction | Object detection | Ultrasonic applications | Virtual reality

Abstract: Memorization techniques are of primary importance in education. Two relevant and extensively used techniques are the Concept maps and the method of loci. Both methods are based on the visualization of information, which helps memorization and retrieval. For these reasons, they are also considered inclusive learning tools for people with Specific Learning Disability. Augmented Reality is a technology that has gained popularity in many sectors, from industry to the medical one, for its effectiveness in visualizing graphical items on top of real contexts. The paper demonstrates that Augmented Reality can also be beneficial for representing and interacting with Concept maps in a 3D virtual space that is linked to the real world. Specifically, the authors developed an Augmented Reality application in which the key features of both the Concept maps (such as visual-spatial logic and concepts organization) and of the method of loci are integrated with those of Augmented Reality technologies to improve the learning process and the retrieval of information.

Keywords: Augmented Reality | Concept maps | virtual learning

Abstract: The collaboration among humans and machines is one of the most relevant topics in the Industry 4.0 paradigm. Collaborative robotics owes part of the enormous impact it has had in small and medium size enterprises to its innate vocation for close cooperation between human operators and robots. The i-Labs laboratory, which is introduced in this paper, developed some case studies in this sense involving different technologies at different abstraction levels to analyse the feasibility of human-robot interaction in common, yet challenging, application scenarios. The ergonomics of the processes, safety of operators, as well as effectiveness of the cooperation are some of the aspects under investigation with the main objective of drawing to these issues the attention from industries who could benefit from them.

Keywords: human-in-the-loop | human-robot collaboration | obstacle avoidance | virtual reality

Abstract: Operator 4.0 has to deal with a vast amount of product variants and production data especially within the mass customization paradigm, high mental demanding tasks, and smart production systems. Technologies capable of supporting his training and his work become fundamental, such as the extended reality (XR). Its increasing use in industrial applications, however, opens up new challenges related to interface and interaction design, which can determine the success of both the use and development experience. The lack of guidelines for designing interfaces for mixed reality (MR) applications is what this paper aims to address. Design requirements for MR interfaces are presented and applied in the context of operator training in wire harness activities. Different interaction modes and user interfaces have been developed to evaluate the most suitable and user-friendly one for the operator. A pilot test was conducted to assess the applications' usability and potentialities with satisfactory results.

Keywords: Augmented Reality | Human-Computer Interaction | Industry 4.0 | Mixed Reality | Operator training | User Interface | Wire harness

Abstract: The “San Pietro al Monte Abbey project: a virtual tour for everyone” consists in the construction of a high-tech station that allows visitors with mobility limitations to be virtually accompanied by a guide to the Benedictine abbey of San Pietro al Monte along the ancient access route that can only be reached on foot with trekking equipment. The room with the virtual instrumentation is located in the Casa del Pellegrino in Civate (Lecco, Italy). It is a museum structure-based located in a media reception building. From its entrance, it is possible to see the final destination. The virtual tour preserves the dialogue between the environment, the monument, and the ‘virtual pilgrim’. By also acting as an information database, it enhances the use of the basilica of San Pietro al Monte even for visitors equipped with tablets who reach the building on foot. The virtual tour application has been developed with Unity3D. The interactive application has different virtual scenes with photos, 360 ∘ videos, an external digital twin of the abbey, and some interesting internal digital twins of the most important monuments inside the abbey.

Keywords: Digital twin | Photogrammetry | Virtual tour

Abstract: The choice of furniture in a retail store is usually based on a product catalog and simplistic product renderings with different configurations. We present a preliminary field study that tests a Multi-Sensory In-Store Virtual Reality Customer Journey (MSISVRCJ) through a virtual catalog and a product configurator to support furnishings sales. The system allows customers to stay immersed in the virtual environment (VE) while the sales expert changes the colors, textures, and finishes of the furniture, also exploring different VEs. In addition, customers can experience realistic tactile feedback with in-store samples of furniture they can test. The journey is implemented for a furniture manufacturer and tested in a flagship store. Fifty real customers show positive feedback in terms of general satisfaction, perceived realism, and acceptance. This method can increase purchase confidence, reduce entrepreneurial costs, and leverage in-store versus online shopping.

Keywords: customer journey | furniture | in-store | industrial design | multisensory | retailing | virtual reality

Abstract: Objectives: This study aims to evaluate the acceptability of Cinematic VR technology as a novel therapeutic approach supporting Social Skills Training (SST) rehabilitation interventions among patients with schizophrenia. Materials and Methods: We developed an innovative cinematic VR-based platform as a support system for SST rehabilitation of independent living skills and evaluated its acceptance among psychiatric patients in terms of usability, user experience, and use performance. Ten voluntary participants were enrolled in the study. The study inclusion criteria consisted of age 18-65 years, lack of moderate and severe intellectual disability, no substance use disorder, and schizophrenia spectrum disorder pathology according to DSM V. We administered post treatment questionnaires and developed the platform to capture relevant data automatically. Results: Patients rated usability and user experience from good to excellent. We also observed an improvement in the use performance. Conclusions: Cinematic Virtual Reality based applications showed good acceptability among patients with schizophrenia. This result supports further efforts in evaluating its effectiveness as a novel therapeutic approach supporting SST rehabilitation interventions.

Keywords: Cinematic virtual reality | Schizophrenia rehabilitation | Social skills training | Task performance | Usability | User experience

Abstract: Immersive Vimial Reality (IVR) training offers the capability to industrial workers to acquire skills and address complex tasks by immersing them in a safe and controlled virtual environment (VE). However, in the literature, IVR training is mainly based on principles of standardization and efficiency without considering the operators' well-being. A novel design approach consists of the introduction in the VE of Positive Computing to improve workers' well-being by applying the Biophilia hypothesis. In this work, we explored the possibility of introducing biophilic elements in a VE training scenario that would support psychological well-being and human potential. However, the introduction of virtual elements not related to the training task may distract operators, impairing their performance. We selected as a training scenario the assembly of a real truck engine. It is accomplished in a workstation, and operators do not interact with the surrounding VE. Therefore, we placed the training area into four different types of VEs: 3D Minimal (MIN), 3D Minimal Biophilic enriched (MIN+BIO), 3D Realistic (REAL), and 3D Realistic Biophilic enriched (REAL+BIO). We compared the MIN and REAL scenarios with the respective biophilic enriched scenarios. The performance of 40 participants was evaluated in terms of completion time, object fixation time, training task accuracy, knowledge accuracy, cognitive load, and user experience. The results revealed that introducing biophilic elements in a VR training environment attracts users' attention in the idle phase of the training. In contrast, they keep concentrating on the task without worsening their performance during the task accomplishment

Keywords: Biophilia Hypothesis | Industrial Training | Positive Computing | Virtual Reality

Abstract: As noise is a pervasive element of work environments, it could affect workers' performance and wellbeing. In particular environments, such as confined spaces, noise could represent an even greater disruptor because reverberation effects amplify exposure levels and could affect cognitive abilities. The study of the effects of noise has mixed results in the literature and was scarcely investigated with reference to this application scenario. Nowadays, Immersive Virtual Reality (IVR) can simulate in a realistic way the working conditions in these environments and consequently simplify the investigations in this field that otherwise would be expensive and difficult to implement for safety reasons. In this work, we verify the ability of a current high-end IVR system to reproduce the acoustic conditions of a confined space realistically, and we evaluate the effects on user cognitive performance and user-perceived workload of a noise source typical of these industrial working environments.

Keywords: immersive virtual reality | N-back test | noise effects | NOISE-TLX | user experience | user performance

Abstract: Noise has affected aircraft since the dawn of aviation. Aircraft noise reduces comfort for passengers and crew inside the cabin and the cockpit, in addition to the structural problem created by the vibration on the aircraft structures. Acoustic characteristics of an aircraft are traditionally represented with numerical data or are in the form of pressure maps and color maps calculated at particular positions taking into account several parameters. Audializing this data, involving potential users, is possible on physical prototypes at the end of the product development requiring considerable time and cost resources. From the past decade, innovative technologies such as Extended Reality (XR) have paved their way towards digital transformation of the products augmenting Human Centred approach. In this paper, we present a novel adaptation of these technologies in developing a multi-sensory virtual aircraft cabin environment in order to provide realism and improve immersion for a user. A new concept to “auralize” the noise inside the virtual passenger cabin, combining numerical acoustics and XR technology has been proposed to develop a tool to evaluate passenger comfort and wellbeing before the prototypes are manufactured. Different solutions on modeling acoustics in a virtual reality cabin have been studied, developed and discussed.

Keywords: Acoustics | Auralization | Multi-Sensory | Realism | Virtual Reality

Abstract: The rehabilitation process after stroke may exhibit some limits regarding physical therapy adherence and there could be a lack of patients’ motivation and trust impacting on the quality of the rehabilitation procedure. This research work aims at introducing a specific interactive design method to develop rehabilitation tools based on the medical knowledge and VR devices to recover motor skills of patients’ hands after stroke and to increase the patients’ adherence. The method we propose consists of three main steps: medical requirement analysis, identification of ICT tools, and medical data management. A case study related to hand rehabilitation is also presented.

Keywords: Hand-tracking devices | Leap motion device | Stroke rehabilitation | Virtual reality

Abstract: The paper presents a method to develop Virtual Reality (VR) platforms based on serious games for the rehabilitation of severe memory loss. In particular, it is related to retrograde amnesia, a condition affecting patient's quality of life usually caused by brain stroke. Nowadays, the standard rehabilitation process consists in showing pictures of patient's familiar environments in order to recover the memory. Past research works have investigated the use of 3D scanners for the virtualization of real environment and virtual reality for the generation of more immersive interaction to design serious games for neurocognitive rehabilitation. Reached results highlighted a time-consuming development process to interface each new environment with the game logic specifically developed for the serious games. Furthermore, a complete VR platform must also consider the medical monitoring and the data management oriented to a more objective medical assessment. The proposed method allows the design of VR platforms based on patient-specific serious games for memory loss starting from the 3D scanning acquisition of familiar environments. The 3D acquisition is performed using the Occipital Structure Sensor and the Skanect application. A modular procedure has been designed to interface the virtual objects of each acquired environment with the modules of the game-logic developed with Unity. The immersive Virtual Reality is based on the use of the HTC Vive Pro head mounted display. Furthermore, the method permits to associate the patient-specific serious game to a set of software modules for the medical monitoring and the data management for the generation of reports useful for the evaluation. The solution has been evaluated by measuring the time needed to develop a whole VR platform for two different familiar environments. Less than 5 hours are required to complete the design process.

Keywords: 3D scanner | Brain stroke | Memory loss | Rehabilitation | Serious games | Virtual reality

Abstract: Retrograde amnesia is a severe memory loss dramatically affecting patient’s quality of life. Traumatic brain injuries, strokes, degenerative processes or metabolic disorders are the main causes. At present, rehabilitation tries to recover patients’ memory by means of neuro-cognitive exercises guided by a physiotherapist. Unfortunately, the adherence to these rehabilitation exercises drops when patients are discharged from hospital. Furthermore, conventional rehabilitation is usually performed using standard exercises, which are not customized to each patient. The reproducibility of real environments and situations is a crucial feature to guarantee the efficacy of neuro-rehabilitation and it is defined as ecological validity. Ecological validity is important for making the exercises useful to re-learn specific information and for performing daily activities with the lowest effort. Nowadays, the traditional rehabilitation for retrograde amnesia is based on a set of pictures shown to the patient to remember or learn familiar environments, such as his/her home. This approach is very limiting because the patient can see few points of view of the home without learning and memorizing how to move and get into a specific room. Therefore, the traditional approach has a low ecological validity. The advent of innovative technology, like 3D scanners and virtual reality, permit the design of innovative solutions that virtually replicate patient’s home. This research work presents a novel procedure to design serious games for neuro-cognitive rehabilitation for patient with retrograde amnesia. The proposed procedure exploits low-cost and free technologies; in particular, the Occipital Structure sensor has been chosen as 3D scanner to acquire the 3D indoor environments, which are used inside Unity to develop the game logic of the serious games. The HTC Vive Pro head mounted display has been used to interact with the serious games in an immersive way. The designed procedure makes available a set of Unity scripts to develop the serious game for new patients by changing only the 3D environment (i.e., patient’s house). The procedure has been tested by creating three different serious games and the total time to create them can be approximated to a working day. The obtained results have been shown to medical personnel who have evaluated the proposed approach with a high ecological validity and decided to plan future medical tests by involving patients.

Keywords: Blender | Ecological validity | Occipital Structure sensor | Rehabilitation | Retrograde amnesia | Unity | Virtual reality

Abstract: Background: In this study, we promote a global approach to occupational risk perception in order to improve occupational health and safety training programs. The study investigates the occupational risk perception of operating room healthcare workers using an Analytic Hierarchy Process approach. Methods: A pilot study was carried out through a cross-sectional survey in a university hospital in Southern Italy. An ad hoc questionnaire was administered to enrolled medical post-graduate students working in the operating room. Results: Fifty medical specialists from seven fields (anaesthetists, digestive system surgeons, general surgeons, maxillofacial surgeons, thoracic surgeons, urologists, and gynaecologists) were questioned about perceived occupational risk by themselves. Biological, ionizing radiation, and chemical risks were the most commonly perceived in order of priority (w = 0.300, 0.219, 0.210). Concerning the biological risk, gynaecologists unexpected perceived this risk as less critical (w = 0.2820) than anaesthesiologists (w = 0.3354), which have the lowest perception of the risk of ionizing radiation (w = 0.1657). Conclusions: Prioritization methods could improve risk perception in healthcare settings and help detect training needs and perform sustainable training programs.

Keywords: Analytic hierarchy process | Healthcare workers | Occupational risk perception | Prioritization risk methodology

Abstract: The present work has two main objectives: the realization of a Virtual Training system about assembly operations in two immersive virtual environments and the comparison between the features offered. The first solution proposed is the use of commercial software IC.IDO, produced by ESI Group. It is designed for applications throughout all the PLM, offering support both in the design and production phases, as well as after-sales assistance. At the opposition, the second solution proposed is the use of the Open-source software Unity. This graphics engine has found significant success in gaming field in the recent years, as it allows more flexibility to the developer for the implementation of the features but requires greater competence in terms of programming. The following paragraphs firstly address the path outlined from the collection of customer’s requirements to the virtual prototypes’ release, following the Systems Engineering approach. Secondly, a benchmark between IC.IDO and Unity’s features is illustrated to point out their main differences, strengths and weaknesses.

Abstract: Among the key technologies of Industry 4.0, Augmented Reality (AR) is one of the most promising and enabling technologies for supporting factory workers and engineers at the workplace. To this end, the paper proposes a novel AR tool to assist operators during the inspection activities for the detection of production and assembly errors. In fact, thanks to the superimposition of the 3D models, as designed by the technical office, a worker can easily detect the presence of design discrepancies on the final physical assembled product and report them by adding 3D annotations directly on virtual models. This AR tool has been developed by using ARCore™ libraries to ensure, in the first place, its compatibility with commonly used devices for which workers are already trained and, secondly, to take advantage of the hybrid-tracking techniques that combine vision- and sensor-based methods to improve the reliability of the AR visualization. Nevertheless, the proposed AR tool adopts multiple markers to minimize tracking errors and therefore to provide greater freedom of movement to the user, who can use the tool also for the assessment of large-size products. Field experimentations have been carried out on a real case study with end-users in order to assess its usability and perceived mental workload through the SUS (System Usability Scale) and NASA-TLX (Task Load Index) standard questionnaires, respectively. The usability study was performed taking into account also objective metrics, i.e., by analysing user performance in target acquisition tasks while interacting with the AR tool. Statistical analysis proved that the adoption of this AR tool requires low mental demand, and its usability has reached a high level of satisfaction both by the factory workers and engineers involved in the user study.

Keywords: Augmented Reality | Google ARCore | Industry 4.0 | Marker-based tracking | Operator 4.0 | Usability studies

Abstract: A major challenge in the field of Augmented Reality (AR) is the way in which augmented information is presented in a wide range of uncontrollable environmental conditions. In fact, the variability of colours and illumination conditions of the real environment makes it difficult to choose the most suitable appearance properties for augmented contents. In many AR applications, the colours of virtual objects play a crucial role in blending digital information into the real environment, therefore these colours should be selected according to the appearance of the real background. In some use cases, the colours of virtual objects need to be harmonised with the ones of the real environment; in other cases, the colours should be chosen to ensure the visibility (e.g. maximizing the contrast) of the augmented data with respect to the background. To this end, the paper presents a background-aware colourisation technique that allows for selecting virtual objects' colours in accordance with the real environment in real-time. Given an arbitrary real background, virtual objects' colours are automatically chosen according to three different strategies, i.e. harmonic, disharmonic, and balanced. The proposed AR colourisation technique was assessed with a user study that focused on three different case studies. The results were promising and suggest the potential of the proposed technique for many different application areas. In particular, disharmonic and balanced strategies ensured the distinctiveness of virtual objects according to the real background. Instead, the harmonic strategy was less effective in the case of colourful complex AR scenarios.

Keywords: Augmented reality | colour harmonisation | colourisation | image processing | user studies

Abstract: This study investigates the use of augmented reality technology (AR) in the field of maritime navigation and how researchers and designers have addressed AR data visualisation. The paper presents a systematic review analysing the publication type, the AR device, which information elements are visualised and how, the validation method and technological readiness. Eleven AR maritime solutions identified from scientific papers are studied and discussed in relation to previous navigation tools. It is found that primitive information such as course, compass degrees, boat speed and geographic coordinates continue to be fundamental information to be represented even with AR maritime solutions.

Keywords: augmented reality | data visualisation | human-computer interaction | maritime

Abstract: Today’s sailing visualization instruments struggle to cope with the increasing number of onboard sensors, automation, artificial intelligence, and the high dy-namism of the crew. Current solutions scatter multiple displays all over the boat, both inside and outside, potentially reducing usability and increasing costs. This work presents a systematic review of augmented reality (AR) as an integral solution for sailing data visualization, which revealed four scientific papers and eight commercial products. We analyzed the publication type, the AR hardware, what and how information is presented using AR, the validation method (if present), and the technological readiness. We defined the technical requirements needed for the AR device for sailing and distinguished a first generation of commercial solutions based on head-up displays from a second one based on proper augmentation with stereo head-mounted displays. The displayed information elements are limited in number and are commonly 2-D graphics (e.g., text and symbols) with a screen-relative frame of reference (as opposed to body-or world-relative). The most visu-alized elements are heading (10) followed by wind direction (8), boat speed (7) compass (7), and wind speed (7). We also found that most of the solutions lack critical evaluation. We conclude that AR has the potential to integrate sailing data from different systems and to improve accessibility, situation awareness, and safety for a large group of users. However, the main limitations are the lack of AR head-mounted displays suitable or adaptable for sailing conditions, an extensive exploration of 3-D interface elements, and an adequate number of usability studies in the scientific literature.

Keywords: Augmented reality | Data visualization | Human-computer interaction | Nautical instruments | Sailing

Abstract: This paper describes the design and preliminary test of a virtual reality driving simulator capable of conveying haptic and visual messages to promote eco-sustainable driving behavior. The driving simulator was implemented through the Unity game engine; a large street environment, including high-speed and urban sections, was created to examine different driving behaviors. The hardware setup included a gaming driving seat, equipped with a steering wheel and pedals; the virtual scenarios were displayed through an Oculus Rift headset to guarantee an immersive experience. Haptic stimulation (i.e., vibrations) was delivered to the driver through the accelerator pedal, while visual stimuli (i.e., icons and colors) were shown on a virtual head-up display. The sensory feedbacks were presented both alone and in combination, providing information about excessive acceleration and speed. Four different virtual scenarios, each one including a distracting element (i.e., navigator, rain, call, and traffic), were also created. Ten participants tested the simulator. Fuel consumption was evaluated by calculating a mean power index (MPI) in reference to the sensory feedback presentation; physiological reactions and responses to a usability survey were also collected. The results revealed that the haptic and visuo-haptic feedback were responsible for an MPI reduction, respectively, for 14% and 11% compared with a condition of no feedback presentation; while visual feedback alone resulted in an MPI increase of 11%. The efficacy of haptic feedback was also accompanied by a more relaxing physiological state of the users, compared with the visual stimulation. The system’s usability was adequate, although haptic stimuli were rated slightly more intrusive than the visual ones. Overall, these preliminary results highlight how promising the use of the haptic channel can be in communicating and guiding the driver toward a more eco-sustainable behavior.

Keywords: Eco-driving | Haptics | Multisensory | Virtual reality

Abstract: The issue of training operators in the use of machinery is topical in the industrial field and in many other contexts, such as university laboratories. Training is about learning how to use machinery properly and safely. Beyond the possibility of studying manuals to learn how to use a machine, operators typically learn through on-the-job training. Indeed, learning by doing is in general more effective, tasks done practically are remembered more easily, and the training is more motivating and less tiresome. On the other hand, this training method has several negative factors. In particular, safety may be a major issue in some training situations. An approach that may contribute overcoming negative factors is using Virtual Reality and digital simulations techniques for operators training. The research work presented in this paper concerns the development of a multisensory Virtual Reality environment for training operators to properly use machinery and Personal Protective Equipment (PPE). The context selected for the study is a university laboratory hosting manufacturing machinery. It has been developed an application that allows user to navigate the laboratory, to approach a machine and learn about how to operate it and also what PPE to use while operating. Specifically, the paper describes the design and implementation of the application.

Keywords: Multisensory interaction | Virtual reality | Virtual training

Abstract: Design for Sustainability is a research area based on a multidisciplinary approach, which has become increasingly important in recent years. Great attention is paid to the design of products that can impact on users' behaviours, through embedded smart technologies, e.g. Internet of Things (IoT). In fact, IoT systems are able to "dialogue" with the users, supporting the identification of any misbehaviour, and suggesting more sustainable ones. This paper presents a research aiming at supporting users towards more conscious food consumption in their daily life to reduce food waste. As a case study, it has been developed an interactive system in which chicken eggs are used as main communication element. Indeed, the environmental footprint of the egg industry is very heavy, and eggs are one of the main wasted food. The interactive system consists of a physical product, an eggs tray, integrating sensors and actuators for handling the interaction with users. It is accompanied by an interactive application for monitoring eggs consumption, displaying eggs waste statistics, and an Augmented Reality part for children, aimed to improve their awareness about food waste and the impact on their food habits through an "edutainment" approach.

Keywords: Industrial design | Sustainability | Virtual reality

Abstract: This research aims at providing an example of how Virtual Reality technology can support the design and development process of the Human Machine Interaction system in the field of Autonomous vehicles. The autonomous vehicles will be an important role in the future’s daily life, as widely concerned. However, the relationship between the human user and the vehicle changes in the autonomous driving scenario, therefore a new interactive modality should be established to guarantee the operational precision and the comfort of the user. But as an underdevelopment sector, there are no mature guidelines for the interaction design in autonomous vehicles. In the early phase of the autonomous vehicle popularization, the first challenge is to build the trust of the user towards the autonomous vehicle. Keeping high transparency of the autonomous vehicle’s behavior to the user will be very helpful, however, it is not possible to communicate the information that the sensors of the autonomous vehicle are collecting because it can create safety risks. In this research, two hierarchical Human Machine Interaction information systems have been introduced and a virtual reality scenario has been developed, based on the most popular applicating scenario: the autonomous taxi. Possible verification methods are also discussed to apply the tool, considering the current design and development procedure in industry, in order to give constructive help to the researchers and practitioners in the field.

Keywords: Autonomous Vehicle | Fully autonomous vehicle | HMI design | Human Machine Interaction | Virtual Reality | VR

Abstract: The term User Experience (UX) was introduced to define the dynamics of the human-product interaction, and it was thought that design would have been a main recipient of UX research. However, it can be claimed that the outcomes of UX studies were not seamlessly transferred into design research and practice. Among the possible reasons, this paper addresses the fragmentary knowledge ascribable to the field of UX. The authors reviewed the literature analyzing the conceptual contributions that interpret UX, proposing definitions and/or a theoretical framework. This allowed the authors to provide an overview of recurring elements of UX, highlighting their relationships and affecting factors. This research aims to clarify the overall understanding of UX, along with its key components (the user, interaction, the system, and context) and dimensions (ergonomic, affective, and the cognitive experiences). The authors built a semantic construction inspired by the structure of a grammatical sentence to highlight the relationship between those components. Therefore, UX is defined by a subject/user who performs an action-interaction towards an object-system. A complement-context better defines the condition(s) where the action-interaction takes place. This work is expected to lay the foundations for the understanding of approaches and methods employed in UX studies, especially in design.

Keywords: Semantic framework | User experience | UX | UX definitions | UX dimensions | UX fundamental elements | UX influence factors | UX studies

Abstract: The concept of User Experience (UX) dates back to the 1990s, but a shared definition of UX is not available. As design integrates UX, different interpretations thereof can complicate the possibility to build upon previous literature and develop the field autonomously. Indeed, by analysing the literature, UX emerges as a cauldron of related and closely linked concepts. However, it is possible to find recurring attributes that emerge from those definitions, which are ascribable to two foci: the fundamental elements of the interaction (user, system, context) and typologies of experience (ergonomic, cognitive, and emotional). Those are used to build a framework. We have preliminarily investigated how UX is dealt with in design by mapping a sample of UX-related experimental articles published in design journals. We classified UX case studies based on the framework to individuate the UXs that emerge most frequently and the most studied ones in the design field. The two-focus framework allows the mapping of experiments involving UX in design, without highlighting specific favorable combinations. However, comprehensive studies dealing with all elements and UX typologies have not been found.

Keywords: Emotional design | Experience design | Framework | User centred design | User experience

Abstract: The importance of training for operators in industrial contexts is widely highlighted in literature. Virtual Reality (VR) technology is considered an efficient solution for training, since it provides immersive, realistic, and interactive simulations environments where the operator can learn-by-doing, far from the risks of the real field. Its efficacy has been demonstrated by several studies, but a proper assessment of the operator's cognitive response in terms of stress and cognitive load, during the use of such technology, is still lacking. This paper proposes a comprehensive methodology for the analysis of user's cognitive states, suitable for each kind of training in the industrial sector and beyond. Preliminary feasibility analysis refers to virtual training for assembly of agricultural vehicles. The proposed protocol analysis allowed understanding the operators' loads to optimize the VR training application, considering the mental demand during the training, and thus avoiding stress, mental overload, improving the user performance.

Keywords: Cognitive ergonomics | Industrial ergonomics | Training Assessment | Virtual assembly | Virtual Reality

Abstract: In the medical education field, the use of highly sophisticated simulators and extended reality (XR) simulations allow training complex procedures and acquiring new knowledge and attitudes. XR is considered useful for the enhancement of healthcare education; however, several issues need further research. The main aim of this study is to define a comprehensive method to design and optimize every kind of simulator and simulation, integrating all the relevant elements concerning the scenario design and prototype development. A complete framework for the design of any kind of advanced clinical simulation is proposed and it has been applied to realize a mixed reality (MR) prototype for the simulation of the rachicentesis. The purpose of the MR application is to immerse the trainee in a more realistic environment and to put him/her under pressure during the simulation, as in real practice. The application was tested with two different devices: the headset Vox Gear Plus for smartphone and the Microsoft Hololens. Eighteen students of the 6th year of Medicine and Surgery Course were enrolled in the study. Results show the comparison of user experience related to the two different devices and simulation performance using the Hololens.

Keywords: Design methodology | Education | Medical Simulation | Mixed Reality | Training

Abstract: A mixed reality (MR) system, by providing visual, auditory, and haptic feedback to the learner, can offer a high level of immersion and realism, especially in the healthcare context. In medical training through MR simulations, it is particularly important to avoid mental overload, discomfort, fatigue, and stress, to guarantee productive learning. The present work proposes a systematic assessment of stress, cognitive load, and performance (through subjective and objective measures) of students during an MR simulation for the rachicentesis procedure. A specific application has been developed to enhance the sense of realism, by showing, over the skill trainer, a digital patient that responds with auditory and visual feedback, based on the learner’s interaction. A sample of 18 students has been enrolled in the pilot study. Preliminary results suggest the effectiveness of the proposed MR application using Hololens: high performances are achieved, and the cognitive conditions are well balanced.

Keywords: Cognitive load | Medical simulation | Mixed reality | Performance | Stress

Abstract: Users acceptance of innovative product appearance authoring tools based on Spatial Augmented Reality (SAR) is still limited due to their perception of a high technology complexity and a low performance/functionality of the current interaction systems. The integration of SAR technologies in professional design activities is still marginal, though many studies in this field have already proved their potential as supporting tools. To overcome this barrier, efficient means for interacting with the digital images projected onto the surfaces of real objects are essential. The aim of the current study is to respond to this demand by proposing and validating three UI configurations displayed by an unique and portable device embedded with a touch screen. These interface layouts, designed to cooperate with the output of the SAR system and to not affect the well-known benefits of its augmented environment, provide different types of visual feedback to the user by duplicating, extending or hiding the information already displayed by the projected mock-up. The experimental study reported here, performed with a panel of 41 subjects, revealed that accuracy, efficiency and perceived usability of the proposed solutions are comparable with each other and in comparison to standard desktop setups commonly used for design activities. According to these findings, the research simultaneously demonstrates (i) the high performances achieved by the touch device when coupled with a SAR system during the execution of authoring tasks, (ii) the capability of the projected mock-up to behave as an actual three-dimensional display for the real time rendering of the product appearance and (iii) the possibility to freely select - according to the users preference, the design task or the type of product - one of the three UI configurations without affecting the quality of the result.

Keywords: Authoring tool | Human-Computer interface | Interface validation | Spatial Augmented Reality | Touch interaction | Usability evaluation

Abstract: This study is aimed at evaluating the impact of different technical solutions of a virtual reality simulator to support the assessment of advanced human–machine interfaces for hydraulic excavator based on a new coordinated control paradigm and haptic feedbacks. By mimicking the end-effector movements, the control is conceived to speed up the learning process for novice operators and to reduce the mental overload on those already trained. The design of the device can fail if ergonomics, usability and performance are not grounded on realistic simulations where the combination of visual, auditory and haptic feedbacks make the users feel like being in a real environment rather than a computer-generated one. For this reason, a testing campaign involving 10 subjects was designed to discriminate the optimal set-up for the hardware to ensure a higher immersion into the VR experience. Both the audio–video configurations of the simulator (head-mounted display and surround system vs. monitor and embedded speakers) and the two types of haptic feedback for the soil–bucket interaction (contact vs. shaker) are compared in three different scenarios. The performance of both the users and simulator are evaluated by processing subjective and objective data. The results show how the immersive set-up improves the users’ efficiency and ergonomics without putting any extra mental or physical effort on them, while the preferred haptic feedback (contact) is not the more efficient one (shaker).

Keywords: Excavator coordinated control | Haptic control | Human–machine interface | Multi-sensory feedbacks | Virtual reality simulator

Abstract: Enhancing the appearance of physical prototypes with digital elements, also known as mixed prototyping, has demonstrated to be a valuable approach in the product development process. However, the adoption is limited also due to the high time and competence required for authoring the digital contents. This paper presents a content authoring tool that aims to improve the user acceptance by reducing the specific competence required, which is needed for segmentation and UV mapping of the 3D model used to implement a mixed prototype. Part of the tasks related to 3D modelling software, in fact, has been transferred to simpler manual tasks applied onto the physical prototype. Moreover, the proposed tool can recognise these manual inputs thanks to a computer-vision algorithm and automatically manage the segmentation and UV mapping tasks, freeing time for the user in a task that otherwise would require complete engagement. To preliminarily evaluate effectiveness and potential of the tool, it has been used in a case study to build up the mixed prototype of a coffee machine. The result demonstrated that the tool can correctly segment the 3D model of a physical prototype in its relevant parts and generate their corresponding UV maps.

Keywords: Augmented reality | Computational design methods | Industrial design | Mixed prototyping | New product development

Abstract: The main aim of the CASTLE (Cabin System Design Towards Passenger Wellbeing) European project is to deliver innovative interiors solutions that maximize the comfort and wellbeing of passengers in the next future. To achieve such objective, an effective HCD (Human Centred Design) approach has been employed to derive a Human Response Model based on a holistic assessment of comfort. The overall methodology has been conceived to provide different tools and methods to collect data on the impact that the design of each cabin item has on the user from the earliest design stages. One of these tools is represented by using 3D virtual mock-ups to capture data on the user’s perception and to rate the level of appreciation inspired by the specific design. In this paper we present the experimental procedures and the results from a preliminary experimental campaign of Human in the loop simulations in Virtual/Augmented Reality of a Regional Aircraft.

Abstract: The design of an E segment, executive, midsize sedan car was chosen to fill a gap in the market of the Ford brand and to achieve the goal of innovation looking towards the future. Ford has not owned an E-segment flagship sports sedan for years, since the historic 1960s Falcon. Starting from the latter assumption and considering that the major car manufacturers are currently investing heavily in E-segment cars, it is important to design a new model, which has been called the Eagle. This model proposed here is to fill the gap between Ford and other companies that are already producing sport cars for the electric sector and to complete Ford’s proposal. The presented methodology is based on SDE, on which many design tools are implemented, such as Quality Function Deployment (QFD), Benchmarking (BM), and Top Flop Analysis (TPA). A market analysis follows in order to identify the major competitors and their key characteristics considering style and technology. The results are used to design an innovative car. Based on the most developed stylistic trends, the vehicle is first sketched and then drawn in the 2D and 3D environments for prototyping. This result leads to the possibility of 3D printing the actual model as a maquette using the Fused Deposition Modelling (FDM) technology and testing it in different configurations in Augmented Reality (AR). These two final applications unveil the possibilities of Industry 4.0 as enrichment for SDE and in general rapid prototyping.

Keywords: Additive manufacturing | Augmented reality | Benchmarking | Car design | Design engineering | QFD | Stylistic design engineering (SDE)

Abstract: This paper presents a study based on Design for Disassembly (DfD) applied to a hydraulic pump through the Disassembly Geometry Contacting Graph (DGCG) methodology. DfD is today very important to reduce the disposal or maintenance costs foreseeable already in the planning phase. One of the key points in reducing costs is reducing time for disassemble each component. Because of that, the disassembly time was considered respect to other fundamental and optimizable characteristics such as: Disassembly costs, operations to be performed, quantity of material, etc. All the operations have been evaluated using the time measurement units (TMUs). The objective of the paper is to minimize the disassembly times required for an operator to separate each single component from the other. The study of accessibility, positioning, strength, and basic time led to a comparison between different disassembly methods in order to produce the optimal sequence. In the end, the validation of the sequence was carried out in an Augmented Reality (AR) environment in order to predict the manual disassembly understanding the possible issues without the need of building the components. Using AR, it was possible to look at the assembly during the design phase in a 1:1 scale and evaluate the chosen sequence.

Keywords: CAD | Disassembly | Sequence | TMU | Virtual Reality

Abstract: Nowadays, the importance of the concept of “Urban Mining” is growing even more, which consists in searching for raw materials inside objects that have reached the end of their life, instead of “inside nature”. It can be commonly found especially in mechanical and electronic equipment valuable materials, which can be extracted and reused as secondary raw materials. The importance of Design for Disassembly (DfD), that is the central topic of this paper, is increasing because of it brings great advantages in terms of disassembly times of components that have reached the end of life. According to the Disassembly Sequence Planning (DSP), this paper presents an application of several methods derived from literature to a two-way valve, to find optimal disassembly sequences. Different sequences have been compared in terms of disassembly time consuming, by the conversion of operations into disassembly time using accredited methods found in literature. Finally, an application in Augmented Reality is proposed to simulate a practical evaluation of what has been theorised so far.

Keywords: Augmented Reality | CAD modelling | Design for Disassembly | Sequence Planning

Abstract: The life of industrial products is getting shorter due to the rapid evolution of technologies. Because of that, the creation of models that are interested in last part of the product’s life are becoming extremely relevant. In recent years, many investments have been made in the recycling of raw materials and the reuse of End-Of-Life (EOL) products in order to reduce the waste of resources. Strategies of Design for Environment (DfE) have been searched and, for this reason, the Design for Disassembly (DfD) has become a fundamental phase in the product life cycle with the subsequent creation of design techniques aimed precisely at disassembly. Using this methodology, the designer can study and plan the optimal sequence which should be based on countless factors and criteria because there is not a straightforward path or a single combination of operations to follow. This paper describes and compares multiple disassembly methods based on minimum disassembly time with reference to a worm gear reducer. In particular, the component was made entirely on CAD (SolidWorks) and the sequences were pplied in a virtual environment. In this way, it was possible to evaluate different algorithms and obtain the optimal disassembly sequence that minimize the overall disassembly time.

Keywords: CAD | Disassembly | Sequence | TMU | Virtual Reality

Abstract: This paper presents a novel Augmented Reality (AR) interface for Head Mounted Display HMD, specifically for sailing navigation. Compared to literature and the commercial solutions available, the novelty is to use boat-referenced 3D graphics. It allows representing wind direction and intensity, heading with monochrome green elements. Furthermore, we carried out a user validation study. We implemented a virtual simulator including a sailboat, the marine environment (i.e., sea, sky, marine traffic, and sounds), and the presented interface as AR overlay. We evaluated the effectiveness of the wind representation of the AR interface through an online questionnaire based on a video simulation and asking the user to imagine it as the result of an AR visualization. We defined one test scenario with wind variations and a distracting element (i.e., a crossing vessel). 75 sailors (59% experts, with more than 50 sailing days per year) participated in the questionnaire, and most of them (63%) considered the video effective at simulating the AR interface. In addition, 75% are ready to wear an AR device during sailing. Also, the usability (SUS questionnaire) and the user experience (UEQ) results provided positive results.

Keywords: Augmented reality | Human-computer interaction | Nautical | Sailing | user test

Abstract: Starting from the pro-environmental potential of virtual reality (VR), the aim was to understand how different statistical information formats can enhance VR persuasive potential for plastic consumption, recycling and waste. Naturalistic, immersive virtual reality environments (VREs) were designed ad hoc to display three kinds of statistical evidence formats, featured as three different formats (i.e., numerical, concrete and mixed). Participants were exposed only to one of the three formats in VR, and their affect, emotions, sense of presence, general attitudes toward the environment, specific attitudes and behavioral intentions toward plastic, use, waste, recycle, as well as their social desirability proneness were measured. Numerical format was the least effective across all dimensions. Concrete and mixed formats were similar. Social desirability only partially affected participants’ attitudes and behavioral intentions. Numerical format did not increase the persuasive efficacy of statistical evidence displayed in VR, with respect to visual alone. Implications and future directions for designing effective VRE promoting pro-environmental behaviors were discussed.

Keywords: Concrete | Format | Numerical | Plastic | Statistical evidence | Virtual reality

Abstract: The current and future challenges of sustainable development require a massive transformation of habits and behaviors in the whole society at many levels. This demands a change of perspectives, priorities, and practices that can only result from the development of more aware, informed, and instructed communities and individuals. The field of design for sustainable behavior is answering this need through the development of products, systems, and services to support the change of people’s habits and decision-making processes. In this regard, Virtual Reality (VR) is a promising tool: it has already been explored to drive sustainable behavior change in several situations, through a wide range of devices, technologies, and modalities. This variety provides uncountable opportunities to designers, but it comes with a series of ethical, psychological, and technical questions. Hence, VR developers should be able to distinguish and identify possible strategies, delivering suitable solutions for each case study. In this work, we present a framework for the development of VR experiences to support sustainable behavior change, based on a systematic review. We consider the various features to manage and possible alternatives when creating a VR experience, linking them to the behavioral aspects that can be addressed according to the project’s aim. The framework will provide designers with a tool to explore and orient themselves towards possible sets of optimal choices generating tailored solutions.

Keywords: Design framework | Sustainable behavior | Virtual reality

Abstract: This research investigates the application of positive computing principles to Virtual Reality (VR) training scenarios where the Virtual Environment (VE) has not a direct influence on operator learning. We propose to place the 3D models of the only objects needed for the task in a VE consisting of 360° panoramas of natural environments. We made a preliminary evaluation of the user experience which showed that the hedonic quality is significantly higher with this VE than a 3D modeled empty room. However, we also observed a reduction of the pragmatic quality, due to potential distractions. Thus, further research is needed to demonstrate the efficacy of our positive computing approach in training against a traditional one based on the faithful 3D reproduction of the real environment.

Keywords: Industrial Training | Positive Computing | Virtual Reality

Abstract: The work carried out has the purpose of improving and optimizing various industrial technical operations, such as preventive maintenance, taken here as an example of application, using the Design for Disassembly (DfD) technique. Therefore, through four metaheuristic methods that have been chosen among the most widespread in the field (described below) to make a comparison between them, the optimal disassembly sequence is sought, if it exists, in terms of time and then costs in order to extract a target component without damaging the other mechanical parts of the assembly. The hypothesis that has been tested throughout this case study is “a responsible application of DfD, not only from the design process of a product but also during the disassembly procedure, can bring substantial benefits to the company”. Interaction with a hypothetical operator in charge of the work to be performed is implemented with the use of augmented reality. In fact, through an application programmed for an Android device (in this case, a mobile phone, hence a handheld device), the operator can be instructed step-by-step on the disassembly sequence in dynamics as an animation. Finally, two virtual buttons were added in augmented reality with which the operator can start and pause/resume the animation at any time to facilitate the understanding of the different steps established by the sequence.

Keywords: Augmented reality | CAD | DFD | Industrial maintenance | Optimization

Abstract: This article reports the results of a research aimed to evaluate the ability of a haptic interface to improve the user experience (UX) with virtual museum systems. In particular, two user studies have been carried out to (1) compare the experience aroused during the manipulation of a 3D printed replica of an artifact with a pen-like stylus with that aroused during the interaction (visual and tactile) with a 3D rendering application using a haptic interface and PC monitor, and (2) compare the users' perceived usability and UX among a traditional mouse-based desktop interface, haptic interface, and haptic gamified interface based on the SUS scale and the AttrakDiff2 questionnaire. A total of 65 people were involved. The considered haptic application is based on the haptic device Omega 6 produced by Force Dimension, and it is a permanent attraction of the Museo Archeologico Nazionale delle Marche. Results suggest that the proposed haptic interface is suitable for people who commonly use mouse-based computer interaction, but without previous experience with haptic systems, and provide some insights useful to better understand the role of haptic feedback and gamification in enhancing UX with virtual museums, and to guide the development of other similar applications in the future.

Keywords: haptic interface | user experience | Virtual museum | virtual reality

Abstract: The limitations due to the SARS-Cov-2 pandemic affected both interactions between people and work activities; particularly in the educational field. In fact, the students were forced to face the lessons from home, which did not bring problems in the understanding of theoretical aspects, but turned out to be a problem regarding the laboratory activities, since they did not have the possibility to test on real instruments what they learned in theory. In this sense, this article describes the development of a remote laboratory in augmented reality, which allows instruments to be faithfully replicated on any portable device available to the students. Furthermore, by exploiting the protocols typical of the Internet of Things, it is possible to guarantee proper control of the real workstation by interacting with the one in augmented reality.

Keywords: Augmented Reality | Internet of Things | Remote Laboratory | Remote Measurements

Abstract: There is a looming shortage of well-trained professionals in the wood construction workforce. To challenge this shortage, we developed a simulated learning environment that leverages a novel Virtual Reality (VR) system to train novice workers in wooden wall construction. A comprehensive task analysis was first used to best identify training requirements. Then, a virtual building site was modeled and a 3D video tutorial was implemented using a VR Head-Mounted Display (HMD). To evaluate the effectiveness of this tool, participants who learned via the VR training tool were compared with participants who instead only had simple 2-D instructional video training. VR training resulted in better retention, task performance, learning speed, and engagement than the video training counterpart, maintaining system usability. This demonstrates that VR is a viable training tool for the construction sector and can produce benefits beyond those of traditional video training.

Keywords: Human-computer interaction | Virtual reality | Virtual training | Workforce development

Abstract: In recent years information and communication technologies (ICT) have played a significant role in all aspects of modern society and have impacted socioeconomic development in sectors such as education, administration, business, medical care and agriculture. The benefits of such technologies in agriculture can be appreciated only if farmers use them. In order to predict and evaluate the adoption of these new technological tools, the technology acceptance model (TAM) can be a valid aid. This paper identifies the most commonly used external variables in e-learning, agriculture and virtual reality applications for further validation in an e-learning tool designed for EU farmers and agricultural entrepreneurs. Starting from a literature review of the technology acceptance model, the analysis based on Quality Function Deployment (QFD) shows that computer self-efficacy, individual innovativeness, computer anxiety, perceived enjoyment, social norm, content and system quality, experience and facilitating conditions are the most common determinants addressing technology acceptance. Furthermore, findings evidenced that the external variables have a different impact on the two main beliefs of the TAM Model, Perceived Usefulness (PU) and Perceived Ease of Use (PEOU). This study is expected to bring theoretical support for academics when determining the variables to be included in TAM extensions.

Keywords: Agriculture | E-learning | QFD | TAM | Technology acceptance | Virtual reality

Abstract: In the last years, museums have begun to apply new technological solutions to manage their exhibits in a more open, inclusive, and creative way, to improve the visitors' experience to respond to the need to expand the audience. The main goal is to face the increasing competition in an economy referred to as the “Experience Economy”. To this end, Augmented Reality technology seems to represent a good solution for museum guide systems, to improve visitors' learning and enjoyment. In this context, the present paper proposes a museum guide system based on Spatial Augmented Reality powered by dynamic projection. The paper describes the overall HW and SW system architecture and reports in detail the developed process adopted to design and implement a museum guide and entertainment application, in the context of the “Studiolo of Federico da Montefeltro” in the Ducal Palace of Urbino. A preliminary survey has been carried out, which involved a total of 79 subjects, aimed at investigating the quality of visitor's experience, aroused by the proposed application, in terms of the “Four Experience Realms” defined by Pine & Gilmore (1998). Results suggest that the proposed application can be used to stage experiences that satisfy the visitors and may help to enable museums into the Experience Economy.

Keywords: Cultural heritage | Dynamic projection | Experience economy | Museum guide | Spatial augmented reality

Abstract: Virtual museum systems, based on different X-reality technologies, has begun to spread, as they represent decisive tools to promote exhibitions and reaching out to audiences. Although budgetary considerations have so far limited the choice of technologies a wide range of possible technological options are available today at low cost. This paper provides the results of an empirical study, with the aim to determine the most appropriate technologies to satisfy the visitors’ expectation and maximise their likelihood to repeat and recommend the experience. The study focuses on the comparison of the performance of five VM systems for visualise digital reproduction of archaeological finds, based on different technologies (i.e., PC desktop, holographic display, 3D stereoscopic projection, head mounted display and mobile Augmented Reality). The results provide useful insight for the development of VM systems, in order to maximize the visitor experience in terms of presence and ability to activate an experience economy perspective.

Keywords: Augmented reality | Mixed reality | Presence | Virtual Museum | Virtual reality | Visitor experience | X-Reality technologies

Abstract: The fourth industrial revolution is promoting the Operator 4.0 paradigm, originating from a renovated attention towards human factors, growingly involved in the design of modern, human-centered processes. New technologies, such as augmented reality or collaborative robotics are thus increasingly studied and progressively applied to solve the modern operators’ needs. Human-centered design approaches can help to identify user’s needs and functional requirements, solving usability issues, or reducing cognitive or physical stress. The paper reviews the recent literature on augmented reality-supported collaborative robotics from a human-centered perspective. To this end, the study analyzed 21 papers selected after a quality assessment procedure and remarks the poor adoption of user-centered approaches and methodologies to drive the development of human-centered augmented reality applications to promote an efficient collaboration between humans and robots. To remedy this deficiency, the paper ultimately proposes a structured framework driven by User eXperience approaches to design augmented reality interfaces by encompassing previous research works. Future developments are discussed, stimulating fruitful reflections and a decisive standardization process.

Keywords: Augmented reality | Human factors | Human-centered design | Human–robot collaboration | Human–robot interaction | User eXperience

Abstract: Virtual reality (VR) offers a promising set of technologies to digitally simulate industrial processes and interaction between humans and machines. However, the use of immersive VR simulations is still limited in industry due to the uncertainty of benefits in respect with traditional digital tools, and the lack of structured methodologies to effectively implement immersive virtual simulations in practice. This paper deals with the application of VR to create virtual manufacturing simulations with the aim to design assembly lines in compliance with factory ergonomics. It proposes a methodology to allow the virtualization and simulation of assembly tasks using a combination of VR tools by replicating, or rather anticipating, what would happen at the shop floor. The adopted tools are Unity 3D for virtual environment generation, HTC VIVE to immerse the user in the virtual factory layout, Xsens as tracking system, and Leap Motion for gesture recognition. The paper also compares the new VR-based procedure with a more traditional desktop-based digital simulation on industrial cases. Results show that the new methodology is more precise to detect the operator’s comfort angles and more powerful to predict process criticalities and optimize factory layout design. At the same time, it is less sensitive to errors during ergonomic assessment related to the expert’s subjectivity during the analysis.

Keywords: Human-centered design | Industrial ergonomics | Virtual manufacturing | Virtual reality

Abstract: Product and process digitalization is pervading numerous areas in the industry to improve quality and reduce costs. In particular, digital models enable virtual simulations to predict product and process performances, as well as to generate digital contents to improve the general workflow. Digital models can also contain additional contents (e.g., model-based design (MBD)) to provide online and on-time information about process operations and management, as well as to support operator activities. The recent developments in augmented reality (AR) offer new specific interfaces to promote the great diffusion of digital contents into industrial processes, thanks to flexible and robust applications, as well as cost-effective devices. However, the impact of AR applications on sustainability is still poorly explored in research. In this direction, this paper proposed an innovative approach to exploit MBD and introduce AR interfaces in the industry to support human intensive processes. Indeed, in those processes, the human contribution is still crucial to guaranteeing the expected product quality (e.g., quality inspection). The paper also analyzed how this new concept can benefit sustainability and define a set of metrics to assess the positive impact on sustainability, focusing on social aspects.

Keywords: Augmented reality | Humancentered design | Model-based design | Product development | Quality inspection | Social sustainability

Abstract: Virtual reality (VR) training allows companies to train their workforce thanks to virtually simulated environments, leveraging the skills of people before the system production with the final aim to reduce the downtime of productive equipment and improve the global factory efficiency. However, the use of VR immersive training is still limited in industry due to the lack of structured methodologies to effectively implement these simulations. This paper deals with the application of VR technologies to create virtual training simulations addressing assembly or maintenance tasks. It suggests a methodology to create an interactive virtual space in which operators can perform predefined tasks in a realistic way, having dedicated instructions to support the learn-by-doing, based on key training features (KTFs). This methodology was applied to an industrial case study concerning some specific tractor assembly phases. Results show that operators generally appreciate this new training process, enabling faster and more intuitive learning.

Keywords: Smart factory | Virtual assembly | Virtual factory | Virtual reality | Virtual training

Abstract: This paper presents an integrated approach for the design of human-robot collaborative workstations in industrial shop floors. In particular, the paper presents how to use virtual reality (VR) technologies to support designers in the creation of interactive workstation prototypes and in early validation of design outcomes. VR allows designers to consider and evaluate in advance the overall user experience, adopting a user-centered perspective. The proposed approach relies on two levels: the first allows designers to have an automatic generation and organization of the workstation physical layout in VR, starting from a conceptual description of its functionalities and required tools; the second aims at supporting designers during the design of human-machine interfaces (HMIs) by interaction mapping, HMI prototyping and testing in VR. The proposed approach has been applied on two realistic industrial case studies related to the design of an intensive warehouse and a collaborative assembly workstation for automotive industry, respectively. The two case studies demonstrate how the approach is suited for early prototyping of complex environments and human-machine interactions by taking into account the user experience from the early phases of design.

Keywords: Design | Human-machine interface | Human-robot collaboration | Human-robot interaction | Virtual reality

Abstract: Today, virtual reality and augmented reality can allow people to interact with products and places in a very realistic way. In this direction, the use of immersive virtual tours (VTs) can improve the users' experience, their perceptions, attitudes and even intended behaviours as potential or actual consumers. The paper focuses on a traditional Italian cheese product and defines a transdisciplinary, multimodal approach where VT helps the remote customer experience based on a VT application to virtually visit a Parmigiano Reggiano cheese dairy, using cutting-edge virtual reality set-up. The paper describes how to create a virtual tour of industrial plants by mapping the main actions, from the storytelling definition, to the plant digitization, until the creation of the virtual, immersive and multimodal application using Unity3D. The VT combines visual experience with gesture recognition and audio stimulation, adding also olfactory cues, in order to create an interactive and realistic experience.

Keywords: Customer Experience | Multimodal approach | User-centered design | Virtual Reality | Virtual Tours

Abstract: Color schemes play a crucial role in blending virtual objects with the real environment. Good color schemes improve user’s perception, which is of crucial importance for augmented reality. In this paper, we propose a set of novel methods based on the color harmonization methodology to recolor augmented reality content according to the real background. Three different strategies are proposed—harmonic, disharmonic, and balance—that allow for satisfying different needs in different settings depending on the application field. The first approach aims to harmonize the colors of virtual objects to make them consistent with the colors of the real background and reach a more pleasing effect to a human eye. The second approach, instead, can be adopted to generate a set of disharmonious colors with respect to real ones to be associated with the augmented virtual content to improve its distinctiveness from the real background. The third approach balances these goals by achieving a compromise between harmony and good visibility among virtual and real objects. Furthermore, the proposed re-coloring method is applied to three different case studies by adopting the three strategies to meet three different objectives, which are specific for each case study. Several parameters are calculated for each test, such as the covered area, the color distribution, and the set of generated colors. Results confirm the great potential of the proposed approaches to improve the AR visualization in different scenarios.

Keywords: Augmented reality | Color harmonization | Image processing

Abstract: Underwater cultural heritage sites represent an attractive and exciting experience for diving tourists, even if often it is complicated for them to understand the significance and value of the remains that are usually strongly damaged and covered by the marine organisms. Thanks to the recent advancements in technologies that overcome these problems, augmented reality is nowadays possible even in such harsh conditions, opening new possibilities for enhancing the diver's experience. However, no user study has formally evaluated the usefulness and usability of augmented reality in open sea underwater environments. This paper presents two novel solutions for underwater augmented reality: a compact marker-based system for small areas, and a complex acoustic system for large areas. Both of them were deployed at an underwater cultural heritage site and evaluated by ten divers in experiments analyzing their perception and remembrance, interests, and user experience. For comparison, the same study was also performed with non-divers assessing the marker-based system on land. Results show that both systems allow divers to encounter new and exciting moments and provide valuable insights for underwater augmented reality applications.

Keywords: Augmented reality | cultural heritage | sensor fusion | underwater | user experience | user testing

Abstract: Augmented Reality (AR) is worldwide recognized as one of the leading technologies of the 21st century and one of the pillars of the new industrial revolution envisaged by the Industry 4.0 international program. Several papers describe, in detail, specific applications of Augmented Reality developed to test its potentiality in a variety of fields. However, there is a lack of sources detailing the current limits of this technology in the event of its introduction in a real working environment where everyday tasks could be carried out by operators using an AR-based approach. A literature analysis to detect AR strength and weakness has been carried out, and a set of case studies has been implemented by authors to find the limits of current AR technologies in industrial applications outside the laboratory-protected environment. The outcome of this paper is that, even though Augmented Reality is a well-consolidated computer graphic technique in research applications, several improvements both from a software and hardware point of view should be introduced before its introduction in industrial operations. The originality of this paper lies in the detection of guidelines to improve the Augmented Reality potentialities in factories and industries.

Keywords: augmented reality | design for disassembly | factory automation | Industry 4.0 | maintenance | mixed reality | object tracking

Abstract: This paper presents an innovative safety training method based on digital ergonomics simulations and serious games, which are games that focus on education. Digital ergonomics is intended to disseminate the culture of safety among workers, while serious games are used to train the operators on specific safety procedures and verify their skills. The results of the experimentation in a real industrial environment showed that, compared to the traditional training methodology, multimedia contents and quantitative ergonomic analyses improve the level of attention and the awareness of the workers about their own safety. However, serious games turned out to be promising training tools with regard to standard operating procedures that are usually difficult or dangerous to simulate in a real working scenario without stopping production. Practitioner summary: Digital ergonomics and serious games are used to disseminate the culture of safety among the workers and for safety training. Our results show that the proposed methodology improves the level of attention and provides a better feedback about the actual skills of the workers than the standard educational strategies. Abbreviations:.

Keywords: Digital humans | occupational safety | serious games | training methods

Abstract: Most occupational safety regulations and international standards recognize the importance of keeping a corporate document that set all the safety procedures prescribed for a certain workplace. However, experience show that, to be truly effective, any piece of information must be kept updated and correctly delivered to the right recipient. From this point of view, the possibility, given by modern technology, to receive, process and send information in real time using common smartphones is a great opportunity. The authors developed a solution for mobile devices, which is based on augmented reality technologies and indoor positioning algorithms, aimed at speeding up and simplifying the information flow among safety managers, workers and casual users about safety-related content. Safety managers can use it as a support tool for the preparation of the risk assessment documentation, on the workers' side, the same application acts as an informational tool providing safety-related content when and where needed through augmented reality technologies. Preliminary results from in situ testing show that augmented reality may be a powerful tool to improve the occupational safety.

Keywords: Augmented reality | Mobile technologies | Workplace safety

Abstract: Most occupational safety regulations and international standards recognize the importance of keeping a corporate document that set all the safety procedures prescribed for a certain workplace. However, experience show that, to be truly effective, any piece of information must be kept updated and correctly delivered to the right recipient. From this point of view, the possibility, given by modern technology, to receive, process and send information in real time using common smartphones is a great opportunity. The authors developed a solution for mobile devices, which is based on augmented reality technologies and indoor positioning algorithms, aimed at speeding up and simplifying the information flow among safety managers, workers and casual users about safety-related content. Safety managers can use it as a support tool for the preparation of the risk assessment documentation, on the workers’ side, the same application acts as an informational tool providing safety-related content when and where needed through augmented reality technologies. Preliminary results from in situ testing show that augmented reality may be a powerful tool to improve the occupational safety.

Keywords: Augmented reality | Mobile technologies | Workplace safety

Abstract: The research work presents a preliminary study to create a virtual reality platform for the medical assessment of spatial extrapersonal neglect, a syndrome affecting human awareness of a hemi-space that may be caused by cerebral lesions. Nowadays, the extrapersonal neglect is assessed by using real objects positioned in the space around the patient, with a poor capability of repetition and data gathering. Therefore, the aim of this research work is the introduction of a virtual reality solution based on consumer technology for the assessment of the extrapersonal neglect. By starting from the needs of the involved medical personnel, an online serious-game platform has been developed, which permits to perform a test and a real-time evaluation by means of objective data tracked by exploited technologies, i.e. an HTC Vive Pro head mounted display and ad-hoc IT solutions. The test is based on a virtual environment composed by a table on which twenty objects have been placed, ten on the right side and ten on the left side. The whole 3D virtual environment has been developed using low-cost and free development tools, such as Unity and Blender. The interaction with the virtual environment is based on voice recognition technology, therefore the patient interact with the application by pronouncing the name of each object aloud. The VR application has been developed according to an online gaming software architecture, which permits to share the 3D scene by exploiting a Wi-Fi hotspot network. Furthermore, the on-line gaming software architecture allows sending and receiving data between the doctor's laptop and the VR system used by the patient on another laptop. The therapist can see through his/her personal computer a real time faithful replica of the test performed by the patient in order to have a fast feedback on patient’s field of view orientation during the evaluation of 3D objects. A preliminary test has been carried out to evaluate the ease of use for medical personnel of the developed VR platform. The big amount of recorded data and the possibility to manage the selection of objects when the voice commands are not correctly interpreted has been greatly appreciated. The review of the performed test represents for doctors the possibility of objectively reconstructing the improvements of patients during the whole period of the rehabilitation process. Medical feedback highlighted how the developed prototype can already be tested involving patients and thus, a procedure for enrolling a group of patients has been planned. Finally, future tests have been planned to compare the developed solution with the Caterine Bergero Scale to define a future standardization.

Keywords: Extrapersonal neglect | Head mounted display | Neglect assessment | Virtual reality

Abstract: This paper presents a combined subjective and objective evaluation of an application mixing interactive virtual reality (VR) experience with 360◦ storytelling. The hypothesis that the modern immersive archaeological VR application presenting cultural heritage from a submerged site would sustain high levels of presence, immersion, and general engagement was leveraged in the investigation of the user experience with both the subjective (questionnaires) and the objective (neurophysiological recording of the brain signals using electroencephalography (EEG)) evaluation methods. Participants rated the VR experience positively in the questionnaire scales for presence, immersion, and subjective judgement. High positive rating concerned also the psychological states linked to the experience (engagement, emotions, and the state of flow), and the experience was mostly free from difficulties linked to the accustomization to the VR technology (technology adoption to the head-mounted display and controllers, VR sickness). EEG results are in line with past studies examining brain responses to virtual experiences, while new results in the beta band suggest that EEG is a viable tool for future studies of presence and immersion in VR.

Keywords: 360-video storytelling | Cultural heritage | EEG | Immersion | Presence | Virtual reality

Abstract: The use of modeling and ergonomic analysis software is a widespread practice in the industrial sector to effectively improve the operator’s well-being and operating comfort within the workplace. In this context, the paper proposes a Mixed Reality system for the ergonomic assessment of industrial workstations. Specifically, the proposed system integrates motion capture tools, a head-mounted display device, and ergonomic analysis software to simulate and analyse the operations to be carried out within a virtual workplace where some physical components, with which the operator interact, are prototyped through 3D printing technology in order to make the simulation as realistic as possible. The proposed Mixed Reality system in fact increases the realism of the simulation and improves the effectiveness of the ergonomics analysis thanks to the haptic feedback that the user perceives when manipulating the physical objects.

Keywords: Ergonomics | Industry 4.0 | Mixed Reality | Rapid prototyping

Abstract: In the context of Industry 4.0, Augmented Reality occupies an important role thanks to its unique capability to enhance the perception of the real world with virtual information. Taking advantage of this capability, the paper presents a handheld mobile Augmented Reality tool that supports manufacturing and production workers and engineers to easily check on-site the ongoing operations carried out in the manufacturing environment for the tubing and piping assembly. The tool runs on a modern tablet and performs an augmented reality visualization of the 3D models, as defined in the project plan, on the corresponding physical objects. In this manner the user can easily check the presence of assembly errors or detect design discrepancies. The level of acceptance of the proposed handheld mobile Augmented Reality tool has been assessed by means of a preliminary test carried out with representative users on a real case study. Results from the experiment are presented and discussed in the paper.

Keywords: Google ARCore | Industrial augmented reality | Industry 4.0 | Mobile augmented reality | Usability

Abstract: Extended Reality technologies, including Virtual Reality (VR) and Augmented Reality (AR), are being applied in a wide variety of industrial applications, but their use within design practice remains very limited, despite some promising research activities in this area over the last 20 years. At the same time, design practice has been evolving to place greater emphasis on the role of the client or end-user in the design process through ‘co-creative design’ activities. Whilst offering many benefits, co-creative design activities also present challenges, notably in the communication between designers and non-designers, which can hinder innovation. In this paper, we investigate the potential of a novel, projection-based AR system for the creation of design representations to support co-creative design sessions. The technology is tested through benchmarking experiments and in-situ trials conducted with two industrial partners. Performance metrics and qualitative feedback are used to evaluate the effectiveness of the new technology in supporting co-creative design sessions. Overall, AR technology allows quick, real-time modifications to the surfaces of a physical prototype to try out new ideas. Consequently, designers perceive the possibility to enhance the collaboration with the end-users participating in the session. Moreover, the quality and novelty of ideas generated whilst using projection-based AR outperform conventional sessions or handheld display AR sessions. Whilst the results of these early trials are not conclusive, the results suggest that projection-based AR design representations provide a promising approach to supporting co-creative design sessions.

Keywords: Co-creation | Co-design | Design representation | Prototype | Spatial augmented reality

Abstract: This paper presents and discusses a fast and efficient method to study artefact-centric interactions in co-design sessions. The method is particularly useful for exploring the introduction of augmented reality (AR) environments since AR application representations combine both digital content and physical content, transforming the way users interact with the design object. Although protocol analysis is extensively used in cognitive studies of design, it is a time-consuming and cumbersome method and hence unsuitable for extensive analysis in industrial environments. Our real-time coding method makes it possible to perform “on-the-fly” coding of physical interactions in co-design sessions. Focusing on quantifying interaction occurrences, our results are consistent with those obtained with post-session coding. Internal validity was assessed using relevant statistical tests. Based on the data collected in co-design sessions we show how aggregate results, especially timelines and interaction densities, can be displayed rapidly at the end of each session. This research paves the way for a more general implementation of real-time coding of collaborative work sessions in industrial situations.

Keywords: Artefact-centric interaction | Augmented reality | Co-design | Collaborative design | Gesture analysis | Mixed artefact | Real-time coding

Abstract: Within co-design sessions involving designers and non-designers, the type and characteristics of the design representations employed is known to impact the performance of such sessions in terms of idea generation, idea evaluation and communication. This study captures the challenges practitioners face in creating and using design representations for co-design sessions and goes on to investigate the potential of Spatial Augmented Reality (SAR) to overcome those challenges. The advantages of SAR in this application are that, multiple concepts can be represented using one physical model, concepts can be modified live during the session, and additional equipment (such as head mounted displays or handheld devices) is not required, thus eliminating any possible interference with the natural interactions between participants. Interviews with design practitioners and trials with a prototype SAR system are used to identify the key challenges faced by practitioners in their current use of design representations, and to capture the technology requirements for a SAR system for use in co-design sessions. These findings can inform the work of technology developers and researchers working on systems to support co-design sessions.

Keywords: co-creation | Co-design | design representation | prototype | requirements | Spatial Augmented Reality

Abstract: Sailing is a multidisciplinary activity that requires years to master. Recently this sustainable sport is becoming even harder due to the increasing number of onboard sensors, automation, artificial intelligence, and the high performances obtainable with modern vessels and sail designs. Augmented Reality technology (AR) has the potential to assist sailors of all ages and experience level and improve confidence, accessibility, situation awareness, and safety. This work presents our ongoing research and methodology for developing AR assisted sailing. We started with the problem definition followed by a state of the art using a systematic review. Secondly, we elicited the main task and variables using an online questionnaire with experts. Third, we extracted the main variables and conceptualized some visual interfaces using 3 different approaches. As final phase, we designed and implemented a user test platform using a VR headset to simulate AR in different marine scenarios. For a real deployment, we witness the lack of available AR devices, so we are developing one specific headset dedicated to this task. We also envision the possible redesign of the entire boat as a consequence of the introduction of AR technology.

Keywords: Augmented Reality | Human Computer Interaction | Nautical | Sailing | Yacht

Abstract: This paper investigates the Augmented Reality (AR) technology with a novel approach based on patent research. We searched the USPTO for AR-related granted patents in the period 1993–2018, we selected and manually browse a total of 2,373, we classified them in five key technological classes i.e., display device, tracking, user interaction, application, and system, and we finally analyzed the results. The main contribution of this paper is the investigation of the technological trends, with outcomes that can be useful for researchers and developers for technical steering, but also for policymakers, managers and entrepreneurs for technology scouting and forecasting. Our study found that AR technological development has especially increased in the last decade. In particular, we evidenced a remarkable steady of 82 % annual growth rate of the number of granted patents after 2012. From geographical distribution, we found that North America is the leader (68 %); Asia (18 %) and Europe (13 %) are lagging behind despite dedicated Industry 4.0 policies actuated by the governments. Another nontrivial result is the incoherency between the owners of a high quantity of patents and those highly impacting. In fact, only Microsoft Corporation and Amazon Technologies are at the same time in the top 10 of the most patent-intensive organizations and the top 10 of highly impacting organizations. Moreover, the majority of the patents are owned by companies, albeit some of the highly impacting ones come from universities or research centers. These findings provide theoretical, managerial, and policy implications for future research activities in the AR domain.

Keywords: Application | AR systems | Augmented reality | Display device | Geographical distribution | Holograms | Human-machine interaction | Immersive technologies | Industry 4.0 | Interactive technologies | Patent analysis | Presence | Review | Technological innovation | Technological trend | Technology forecasting | Technology scouting | Tracking | User interaction | Visual overlay

Abstract: Technical documentation is evolving from static contents presented on paper or via digital publishing to real-time on-demand contents displayed via virtual and augmented reality (AR) devices. However, how best to provide personalized and context-relevant presentation of technical information is still an open field of research. In particular, the systems described in the literature can manage a limited number of modalities to convey technical information, and do not consider the 'people' factor. Then, in this work, we present a Context-Aware Technical Information Management (CATIM) system, that dynamically manages (1) what information as well as (2) how information is presented in an augmented reality interface. The system was successfully implemented, and we made a first evaluation in the real industrial scenario of the maintenance of a hydraulic valve. We also measured the time performance of the system, and results revealed that CATIM performs fast enough to support interactive AR.

Keywords: Augmented reality | Context-aware | Human-centered design | Human-computer interaction | Industrial | Information manager | Maintenance | Technical documentation

Abstract: Museums have been subjected to important changes in the approach they use to involve visitors. Among the other trends, storytelling and interactive exhibitions are two of the most used approaches used to make exhibitions more interesting for users. Virtual Reality and Augmented Reality methods can be effectively used in the context of a museum exhibition to support both storytelling and interaction. The primary objective of the use of these technologies is to make the visit of museums much more engaging, and suitable for different types of visitors. Among the several museums that are moving in this direction, there is the Museo Astronomico di Brera. The museum mainly consists of a corridor, hosting instruments used by astronomers, and the Cupola Schiaparelli, which is an observatory dome. The aim of the research presented in this paper is to develop an interactive Virtual Reality application to be used for improving the users’ experience of visits to the Museo Astronomico di Brera. Specifically, the paper presents a VR application to virtually visit the Dome. Preliminary tests have been carried out for evaluating the users' sense of presence in the VR environment. An analysis of the collected data is presented in the paper.

Keywords: Augmented Reality | Museum exhibitions | User Experience | Virtual Reality

Abstract: Cultural and educational institutions have been subjected to important changes in the approach they use to involve the public in the last years. For example, museums are more and more playing a pedagogical role, referring not only to exhibitions of pieces of art, but also to exhibitions concerning current topics in cultural and social affairs. Storytelling and interaction are two of the most popular methods used to make exhibitions more interesting for the visitors, and many works have demonstrated that Virtual Reality and Augmented Reality technologies can be effectively used to support these approaches in the context of museum exhibitions. This paper presents a research work aimed to design and develop an interactive multisensory AR application (based on sight, hearing, and olfaction senses), which can be used for improving the users' engagement in exhibitions and generate awareness about the dramatic outcomes of pollution on the environment. Specifically, the paper describes a case study of multisensory Augmented Reality interactive experiences concerning the negative effects of human activities on natural environments.

Keywords: Augmented Reality | Multisensory Perception | User Experience

Abstract: Kandinsky-Experience Book is a multisensory Augmented Reality experience that involves sight, hearing and smell senses and aims at improving the users’ engagement in the Kandinsky’s artworks. Specifically, the aim of the application is to augment the experience of the user creating a journey throughout Kandinsky's work by using an AR application for smartphones integrated with audio and olfactory stimuli, in order to allow him/her to be more immersed in the piece of art. The research project has been inspired by the synesthetic approach of the abstract painter to the theory and the perception of art in his books. Starting from the artist’s considerations about the relationship between different sensorial stimuli in works of art, we decided to amplify some of his theories suggesting a connection between the main pictorial elements and some corresponding olfactory stimuli, grounding our hypotheses on the content of papers concerning the crossmodal synesthetic correspondences between olfactory stimuli and other sensorial modalities. Thanks to the simultaneous presentation of the specifically developed AR contents and the olfactory stimuli, the users’ feelings and emotions during the experience are amplified as a result of the sensory integration. Moreover, by using AR technology and olfactory devices to stimulate visual and olfactory perceptual channels we aimed at increasing the generation of longer-lasting memories in the users' mind.

Keywords: Augmented Reality | Cultural Heritage | Multisensory Perception | User Experience

Abstract: Human beings interact with the external world through the perception that they get by touching, looking at, listening to, tasting and smelling it. Even if this exploration is essential to identify opportunities and dangers, today it is also used to investigate, understand and enjoy objects that surround us and to use them to manage our life, have fun, increase our knowledge, relax, etc. To date, interaction is based primarily on sight, on hearing and on touch. Very little interaction, however, is based on smell, a sense considered very difficult to manage and to use for creating more pleasant and effective experiences. Yet, olfactory stimuli can make the interaction between users and objects more engaging and effective on sub-conscious levels and long-term memory. This can be particular relevant for museums, which are subjected to important changes in the way they use to involve visitors in their exhibitions. This work presents two case studies of olfactory experiences integrated in applications for cultural heritage purposes, and effectively used to enhance the user interaction and experience.

Keywords: Augmented Reality | Interactive exhibitions | Multisensory experiences

Abstract: The research on the use of virtual reality (VR) in the design domain has been conducted in a fragmentary way so far, and some misalignments have emerged among scholars. In particular, the actual support of VR in early design phases and the diffusion of practices involving VR in creative design stages are argued. In the present paper, we reviewed VR applications in design and categorized each of the collected 86 sources into multiple classes. These range from supported design functions to employed VR technologies and the use of systems complementing VR. The identified design functions include not only design activities traditionally supported by VR, such as 3D modelling, virtual prototyping, and product evaluation, but also co-design and design education beyond the early design phases. The possibility to support early design phases by means of VR is mirrored by the attention on products that involve an emotional dimension beyond functional aspects, which are particularly focused on in virtual assemblies and prototypes. Relevant matches between VR technologies and specific design functions have been individuated, although a clear separation between VR devices and supported design tasks cannot be claimed.

Keywords: 3D modelling | Co-design | Early design phases | Engineering design | Industrial design | Product design | Product evaluation | Technological development | Virtual prototyping | Virtual reality

Abstract: The paper investigates the relationship between the forms through which products are represented and the outcomes of evaluations made by observers. In particular, the study focuses on perceived affordances of creative designs, meant as the capability of capturing original elements and corresponding functions, for products presented through static images or videos. Also thanks to the use of Eye Tracking, the experimental results show how dynamic effects that involve salient aspects of products, as well as creative features, are critical to observers' capability of capturing design intentions.

Keywords: design affordance | design creativity | design evaluation | forms of representation | product design

Abstract: An adequate medical education is the key driver of healthcare quality improvement. Technological innovations have led to consistent improvement in learning outcomes but the systematic measurement of students performance and cognitive workload need further research. The aim of this paper is to propose an innovative method for the Design and Development of new advanced learning models, to be used in the training of medical students, which includes also the analysis of students performance and cognitive ergonomics. A web-based survey, on team simulation training and technology role, has been administered to 180 medical students. On the basis of this analysis, a list of guidelines for the design of medical education training has been proposed.

Keywords: Advanced learning technologies - cognitive ergonomics | Augmented reality | Instructional design | Medical simulations | Mental workload | Training

Abstract: As the popularity of winter outdoor sports is increasing, a growing number of users are engaging in activities in uncontrolled terrain, thus training for avalanche rescue is more important than ever. Traditional training takes place through workshops and in field sessions, presenting limitations to the training availability, since they require time, organization and specific weather conditions. This is problematic since the use of transceivers to locate buried individuals is not trivial and requires practice. Virtual Reality (VR) training has shown to be effective in several fields, especially in the context of hazardous conditions and emergencies, which require decision making under time pressure and management of complex tools in uncontrolled or unsafe environments. Examples include disaster medicine, military operations, and other fields in which actions must be performed precisely in short time frame. In this work, we present the development of an immersive VR system for avalanche rescue training as a complementary tool to the traditional techniques in order to prepare the trainee for field training sessions. We discuss the definition of the system requirements, the design and implementation of the tool, and considerations regarding hardware and software. Finally, we discuss possible limitations and future development.

Keywords: Avalanche Rescue Training | Virtual Reality

Abstract: A human mission to Mars has long been advocated. As each year the scientific researches bring mankind closer to establishing human habitats on Mars, the question of how astronauts can sustain themselves whilst away from the blue planet becomes crucial. The project presented in this paper aims at designing and developing the Virtual Reality (VR) simulation of an inflatable modular greenhouse featuring a system that manages the growth of the plants and helps the astronauts control and monitor the whole greenhouse more extensively. The use of VR technology allows simulating an immersive environment of a Mars habitat highlighting its greenhouse overcoming the limitation of physical locations. Wearing the Oculus Rift head-mounted display (HMD) while holding Oculus Rift Touch Controllers, astronauts or Mars exploration enthusiasts could experience the highly interactive and realistic environment. Its goal is to provide training and evaluative simulations of astronauts’ basic tasks and performances in the greenhouse on Mars while testing the growing method of hydroponics equipped with a smart growing controlling and monitoring system.

Keywords: Control | Greenhouse | Hydroponics | Interactive environment | Mars exploration | Mars habitat | Monitor system | Oculus Rift | Virtual Reality

Abstract: Virtual and Augmented Reality systems have been increasingly studied, becoming an important complement to traditional therapy as they can provide high-intensity, repetitive and interactive treatments. Several systems have been developed in research projects and some of these have become products mainly for being used at hospitals and care centers. After the initial cognitive rehabilitation performed at rehabilitation centers, patients are obliged to go to the centers, with many consequences, as costs, loss of time, discomfort and demotivation. However, it has been demonstrated that patients recovering at home heal faster because surrounded by the love of their relatives and with the community support.

Keywords: Cognitive rehabilitation | Gaming | LeapMotion | Oculus rift | VR/AR

Abstract: Spatial Augmented Reality allows users to visualise information onto physical objects by projecting digital contents on them. Product design applications could profitably exploit this feature to create prototypes partially real and partially virtual (mixed prototypes) to be used for the evaluation of products during the development processes. A mixed prototype needs a high visual quality, because design decisions are taken on the base of its aspect, and projected colours should match the colour standards (e.g. Pantone, RAL, etc.) to be able to rely on the visualised colours. The current paper analyzes the effect of a colour calibration method, based on the iteration of comparison and compensation phases, onto the projected images using objective measurements and subjective users’ evaluations. The procedure, whose effectiveness is verified thanks to the presented results, allows to replicate any colour available inside the projector gamut by simply using a physical sample.

Keywords: Colour calibration | Colour fidelity | Product design | Spatial augmented reality

Abstract: The aim of this paper is to describe the application of digital visualization tools to assist air accident investigators, including both their investigation activities and as a training resource. Augmented Reality technology is used to re-create a real aircraft crash scene, both in terms of wreckage distribution and features of the surroundings, in a full-scale 3D representation. A case study shows both potentials and limitations of the approach, and recommendations on how to improve the methodology are also proposed. Overall, it is concluded that Augmented Reality has achieved a maturity stage sufficient to consider it as an effective tool for training of air accident investigators and, to some extent, to support the investigation process itself, although more developments are required to address some current limitations and fully exploit the capabilities of this technology.

Keywords: Augmented Reality | Aviation accident investigations | Aviation safety

Abstract: Digitalization has already permeated most of the design activities, but in spite of this, the generation of visual representations of concepts in the product design domain still relies on analog tools in real world contexts. Despite immersive 3D technologies, such as Virtual Reality, have become widely available and affordable, most designers still make use of pencils and paper sheets, or their digital counterparts, to sketch their initial ideas on 2D supports. This study aims at investigating the reasons behind the mismatch between the rapid growth of immersive technologies and their scarce adoption in the conceptual design activities. Based on the analysis of the state of the art, a classification of the approaches proposing new ways to conduct conceptual representation of products has been drawn. The geometry representation, i.e. parametric or polygonal, and the interaction methods have been taken as metrics to categorize previous works. Weak connections between the modeling paradigm implemented and the interaction methods, lack of spatial faithfulness, ergonomic concerns and the need for quantitative metrics to compare objectively the data resulting from different testing sessions across the various studies are the main issues identified. In order to get concrete evidence of such thoughts, an experimental session has been devised with users from different backgrounds. They were asked to make conceptual sketches of a computer mouse in a traditional fashion, i.e. with pen and paper, and using two off-the-shelf Virtual Reality applications, based on 3D sketching and 3D sculpting respectively. The results are discussed qualitatively by visually comparing the sketches made by the testers, enriched by information deducted by surveying the users before and after the experiments. By comparing the sketches made by each user with the three procedures, preliminary results indicate that VR systems don't bring dramatic improvements compared to traditional 2D sketching tools. Furthermore, despite being enjoyable, VR systems caused physical fatigue, which is a problem that basically does not affect 2D sketching. Despite the size of the sample cannot provide statistical evidence, the outcomes provided good indications about the technology readiness level of Virtual Reality as a conceptual design tool, paving the way for future research directions.

Keywords: Conceptual design | Product design | Virtual reality

Abstract: The ultimate goal of maintenance managers in any industrial firms is to maximize the uptime of the production assets and to keep the downtime to a minimum. These factors affect the capability of an industry to meet the production deadline while still ensuring the good quality product at minimum production cost. To realize this objective, effective maintenance method and innovative tool are required. Previous study has shown that the growing complexity of current manufacturing technologies will necessitate the increasing competent and trained personnel to resolve quickly the interruptions that occur in the shop floor. However, an efficient repair operation is sometimes difficult to achieve especially when the dysfunctional machine involves various possible problems and the assignment of skilled technician and resources to attend to the failed equipment requires more than just the information reported by the operator concerning what was not functioning on site. Augmented Reality (AR) as one of the emerging technologies in the framework of Industry 4.0 provides a way to accelerate the maintenance process and to minimize the recommissioning of maintenance work due to limited maintenance information provided by the operator. This paper presents the application of AR integrated with CMMS on the emerging computing platform Hololens to demonstrate the potential of this integration to optimize the pipeline of maintenance procedure in order to boost the profitability and competitive advantage of an industrial firm.

Keywords: Augmented reality | CMMS | Downtime | Hololens | Maintenance

Abstract: XR is an acronym used to refer to the spectrum of hardware, software applications, and techniques used for virtual reality or immersive environments, augmented or mixed reality and other related technologies. The special thematic session on ‘XR Accessibility’ explores current research and development as well as presenting diverse approaches to meeting real user needs in immersive environments. The contributed research papers range from using spatial sound for object location and interaction for blind users, to alternative symbolic representation of information, Augmented Reality (AR) used in rehabilitation for stroke patients and vocational skills training for students with intellectual disabilities. The session also explores what we can learn from previous research into immersive environments – looks at opportunities for future research and collectively explores how we can together iterate accessibility standards.

Keywords: Accessibility | Augmented Reality | Immersive web | Inclusive design | Rehabilitation | Serious games | Usability | Virtual reality

Abstract: An innovative airport control tower concept based on the use of modern augmented reality technologies has been developed and validated by means of human-in-the-loop experiments in a simulated environment. An optical-based augmented reality interface underpins the proposed concept that consists in providing air traffic control operators in the airport control tower with complete head-up information, as opposed to the current mix of information retrieval through both head-up real view and head-down interfaces. Specific measurement of the time spent by the operator working in either head-up or head-down position, show that the proposal has a clear effect in stimulating the air traffic control operator to work in a head-up position more than in a head-down position, with positive effects on his/her situational awareness and perceived workload, especially when dealing with low visibility conditions operational scenarios.

Abstract: In the last decade, Augmented Reality has become increasingly popular. As improved performances are gathered in terms of mature hardware and software tools, we are observing the stemming of a huge number of applications of this technology both in the entertainment and in the industrial domains. On the one hand, such applications are usually claimed to bring benefits in terms of productivity or enhancement of the human’s capability to perform tasks. On the other hand, researchers and developers seem not to adequately consider the different meanings that AR assumes when implemented through visualization devices that can differ significantly in nature and in their capability to provide a mixed real-virtual scenario. In this paper, we describe a user-centred method based on an integrated QFD-AHP approach to select the best visualization display technology with regard to a specific application context. The aim is to establish a repeatable and documented process for the identification of the technology that best suits and mitigates the acceptability risks of the transition from a legacy working environment to an AR based operational environment. The method has been developed in the framework of the RETINA (Resilient Synthetic Vision for Advanced Control Tower Air Navigation Service Provision) project involving the end users, in this case, air traffic controllers. Nevertheless, it can be generalised and applied to other contexts of use. Furthermore, in order to be resilient to the fast, technological development in AR, it can be used to update the results as improvements arise in the performance level of the display devices in a specific technology.

Keywords: Air traffic control | Analytic hierarchy process (AHP) | Augmented reality | Human machine interface | Quality function deployment (QFD)

Abstract: Augmented Reality is considered one of the most promising technologies for Industry 4.0. Augmented Reality allows to display the needed information at the right time and to locate it in the desired space, superimposing it to the real world. In this way, it could simplify the work of industrial operators in a variety of tasks, from planning and commissioning to assembly and maintenance. Despite the growing research interest and efforts for the development of Augmented Reality applications and technologies for the industrial domain, Augmented Reality is still rarely used in real industrial procedures, often remaining at a conceptual level. In fact, developers fail to completely answer the actual needs of industries and workers because of a lack of knowledge of the context and the absence of guidelines to drive the design and development of Augmented Reality applications. One of the open issues is the choice of the proper visualization methods to display technical information in Augmented Reality, which is not trivial. Both the paper-based documentation, based on text and illustrations, and digital documentation, including CAD models and image or video-based tutorials, present different criticalities when adapted to the Augmented Reality technology. Other visual material has been developed specifically to be used in Augmented Reality, but it is difficult to use because of a lack of standards which are instead present for the former paper-based and digital documentation. In this work, we categorize and compare different Augmented Reality visual assets, illustrating their advantages and disadvantages and providing directions for future research.

Keywords: Augmented Reality | Graphical User Interface | Industry 4.0 | Technical documentation

Abstract: The aim of this work is to present an algorithm used to perform an energetic analysis for an electric vehicle in a Virtual Reality (VR) scenario. This was useful to recognize some patterns of driving behavior considering users and their psychological aspects. The primary aspect enlightened is the description of the experimental setup used to perform tests with 26 users. Through Unity and Matlab software, it was possible to exploit a VR scenario aimed at recreating in the same route both urban and highway paths in condition of real traffic, performed with a Battery Electric Vehicle (BEV). The acquisition part is illustrated in its methodology in two different cases covering disturbance and non-disturbance scenario. Moreover, the population was divided considering gender to establish a characterization which linked energy consumption and associated analysis to psychological traits of the driver.

Keywords: Battery electric Vehicle | dnving behavior | Electric vehicle | energy consumption | psychological aspects | virtual reality

Abstract: Virtual reality is a candidate to become the preferred interface for architectural design review, but the effectiveness and usability of such systems is still an issue. We put together a multidisciplinary team to implement a test methodology and system to compare VR with 2D interaction, with a coherent test platform using Rhinoceros as industry-standard CAD software. A direct and valid comparison of the two setups is made possible by using the same software for both conditions. We designed and modeled three similar CAD models of a 2 two-story villa (1 for the training and 2 for the test) and we implanted 13 artificial errors, simulating common CAD issues. Users were asked to find the errors in a 10 minutes fixed-Time session for each setup respectively. We completed our test with 10 students from the design and architecture faculty, with proven experience of the 2D version of the CAD. We did not find any significant differences between the two modalities in cognitive workload, but the user preference was clearly towards VR. The presented work may provide interesting insights for future human-centered studies and to improve future VR architectural applications.

Keywords: Architecture | design review | evaluation | user preference | Virtual Reality | workload

Abstract: Environmental impact and recycling have been increasingly frequent topics in recent years. At the same time, the life cycle of products has increasingly become shorter, as the escalating competitive market requires new products in smaller pieces. From this perspective, the recovery of parts and products that are produced in this market system for subsequent reuse when they reach the end of their life cycle is essential. For these reasons, it has become critical that companies re-evaluate their product design with a view to the possible recovery of the parts that comprise their products and to create new products for the market. The following discussion was based on the study of design for disassembly (DfD), which is the analysis of industrial products aimed at optimizing disassembly in terms of time and costs. The application of the DfD to a case study of a gearbox has, among its main objectives, the search for the best disassembly sequence in terms of time and money. During the course of the study, augmented reality (AR) was used. Through the use of the Unity software and Vuforia package, it was possible to bring the gearbox back to AR and then simulate the disassembly sequence in AR.

Keywords: Augmented reality | Computer-aided design | Design for disassembly | Disassembly evaluation chart | Disassembly sequence planning | Gearbox | SolidWorks

Abstract: The work here described aims to offer a starting point for improving and making a generic maintenance process more efficient, first of all thanks to the use of a cutting-edge technology such as augmented reality, as a key tool that makes it possible and immediate to communicate to operators which are the fundamental stages of the maintenance process to be followed in the working area. Furthermore, thanks to the use of two methods applied in the context of the Design for Disassembly (later described), we also propose to search for all the possible sequences to get to the removal of a target component to be adjusted—in particular the optimal one (if it exists, in terms of time and costs) to be subsequently applied in an augmented reality “self-disassembly” model that can be viewed and followed by the operator, in a way that is still very little used today.

Keywords: Augmented reality | CAD | DFD | Industrial maintenance | Optimization

Abstract: This study presents an augmented reality-based system for the training of assembly line operators in the context of the high-end footwear industry. The proposed multi-layer software architecture, in combination with the AR viewer (Microsoft HoloLens™), guides operators of the shoe assembly/finishing line during the offline training activities. An evaluation protocol has been defined and preliminary experimentation of the system have been conducted in an Italian company that produces classic and luxury leather shoes.

Keywords: Augmented Reality | Footwear Industry | Manufacturing | Operator Training

Abstract: The study on cognitive workload is a field of research of high interest in the digital society. The implementation of 'Industry 4.0' paradigm asks the smart operators in the digital factory to accomplish more 'cognitive-oriented' than 'physical-oriented' tasks. The Authors propose an analytical model in the information theory framework to estimate the cognitive workload of operators. In the model, subjective and physiological measures are adopted to measure the work load. The former refers to NASA-TLX test expressing subjective perceived work load. The latter adopts Heart Rate Variability (HRV) of individuals as an objective indirect measure of the work load. Subjective and physiological measures have been obtained by experiments on a sample subjects. Subjects were asked to accomplish standardized tasks with different cognitive loads according to the 'n-back' test procedure defined in literature. Results obtained showed potentialities and limits of the analytical model proposed as well as of the experimental subjective and physiological measures adopted. Research findings pave the way for future developments.

Keywords: Cognitive load | Heart rate variability | Information theory model | NASA-TLX | Smart operators

Abstract: Very often historical buildings used as museums are characterized by rooms completely covered with decorations (e.g. frescoes, inlay, etc.) rich in details and symbolic contents. Providing adequate information to the visitors to allow them to fully appreciate the artworks is of paramount importance in this context. X-reality technologies have the potential to provide an effective response to the need to combine the educational mission of museums with the ability to involve visitors emotionally, allowing the public to learn new knowledge in a playful way. This study introduces a Museum Guide System, based on Dynamic Projection, to increase the involvement of visitors to the Studiolo by Federico da Montefeltro at Urbino, minimizing technology intrusiveness by ensuring a complete fusion of digital contents with the physical environment. The system is able to track the visitors, to detect their profile and to offer information and multimedia contents tailored to the characteristics of the audience (e.g., children, adults).

Keywords: Adaptive system | Digital Cultural Heritage | Dynamic projection | Museum Guide System | Spatial Augmented Reality

Abstract: In the context of smart factories, where intelligent machines share data and support enhanced functionalities at a factory level, workers are still seen as spectators rather than active players (Hermann, Pentek, & Otto, 2017). Instead, Industry 4.0 represents a great opportunity for workers to become part of the intelligent system; on one hand, operators can generate data to program machines and optimize the process flows, on the other hand they can receive useful information to support their work and cooperate with smart systems (Romero et al., 2016). Diversely from machines, humans are naturally smart, flexible and intelligent, so putting the operators in the digital loop can bring more powerful and efficient factories. The paper aims at defining a theoretical human-centered framework for Operator 4.0, and testing its feasibility and impact on companies, thanks to the integration of human factors in 4.0 computerized industrial contexts. The proposed framework is based on data collection about the workers’ performance, actions and reactions, with the final objective to improve the overall factory performance and organization. Data are used to assess the workers’ ergonomics performance and perceived comfort and to build a proper knowledge about the human asset of the factory, to be integrated with the knowledge derived from machine data collection. The framework is cased on the adoption of an Operator 4.0 monitoring system, which consists of an eye tracking and a wearable biosensor, combined to a proper protocol analysis to interpret data and create a solid knowledge. Virtual prototypes are used to make the workers interact with the digital factory to conveniently simulate the human–machine interaction (HMI) in order to avoid bottlenecks at the shop floor, to optimize the workflows, and to improve the workstations’ design and layout. The study represents a step toward the design of human-centred industrial systems, including human factors in the digital twin. The research approach has been successfully tested on an industrial case study, developed in collaboration with CNH Industrial, for the re-design of assembly workstations.

Keywords: Digitization | Human factors | Industry 4.0 | Mixed reality | Operator 4.0

Abstract: Background and Objective. We present an original approach to the development of augmented reality (AR) real-time solutions for robotic surgery navigation. The surgeon operating the robotic system through a console and a visor experiences reduced awareness of the operatory scene. In order to improve the surgeon's spatial perception during robot-assisted minimally invasive procedures, we provide him/her with a solid automatic software system to position, rotate and scale in real-time the 3D virtual model of a patient's organ aligned over its image captured by the endoscope. Methods. We observed that the surgeon may benefit differently from the 3D augmentation during each stage of the surgical procedure; moreover, each stage may present different visual elements that provide specific challenges and opportunities to exploit for organ detection strategies implementation. Hence we integrate different solutions, each dedicated to a specific stage of the surgical procedure, into a single software system. Results. We present a formal model that generalizes our approach, describing a system composed of integrated solutions for AR in robot-assisted surgery. Following the proposed framework, and application has been developed which is currently used during in vivo surgery, for extensive testing, by the Urology unity of the San Luigi Hospital, in Orbassano (To), Italy. Conclusions. The main contribution of this paper is in presenting a modular approach to the tracking problem during in-vivo robotic surgery, whose efficacy from a medical point of view has been assessed in cited works. The segmentation of the whole procedure in a set of stages allows associating the best tracking strategy to each of them, as well as to re-utilize implemented software mechanisms in stages with similar features.

Keywords: Augmented reality | Robot-Assisted surgery | Surgical navigation

Abstract: Augmented Reality began to be used in the last decade to guide and assist the surgeon during minimally invasive surgery. In many AR-based surgical navigation systems, a patient-specific 3D model of the surgical procedure target organ is generated from preoperative images and overlaid on the real views of the surgical field. We are currently developing an AR-based navigation system to support robot-assisted radical prostatectomy (AR-RARP) and in this paper we address the registration and localization challenge of the 3D prostate model during the procedure, evaluating the performances of a Successive Quadratic Programming (SQP) non-linear optimization technique used to align the coordinates of a deformable 3D model to those of the surgical environment. We compared SQP results in solving the 3D pose problem with those provided by the Matlab Computer Vision Toolkit perspective-three-point algorithm, highlighting the differences between the two approaches.

Keywords: Augmented Reality | Computed-assisted surgery | Performance evaluation | Prostatectomy | Robotic surgical procedures | Successive Quadratic Programming

Abstract: This work describes a Mixed Reality application useful to modify and cut virtual objects. A digital simulation of surgical operations is presented. Following this approach, surgeons can test all the designed solutions of the preoperative stage in a Mixed Reality environment. High precision in surgery applications can be achieved thanks to the new methodology. The presented solution is hands free and does not need the use of a mouse or computer’s keyboard: it is based on HoloLens, Leap Motion device and Unity. A new cutting algorithm has been developed in order to handle multiple objects and speed up the cut with complex meshes and preserve geometry quality. A case study presents the cut of several bones in order to simulate surgeon’s operations. A reduction in cut time compared to the original method is noticed, together with a high flexibility of the tool and a good fidelity of the geometry. Moreover, all the object fragments generated from the algorithm are available for manipulation and new cuts.

Keywords: cutting algorithm | leap motion | Mixed Reality | unity

Abstract: This work focuses on an innovative training methodology based on the use of Virtual ergonomics and “serious games” in the field of occupational safety. Virtual Ergonomics was chosen as an effective and convincing tool for disseminating the culture of safety among the workers, while a “serious game” was developed to train operators on specific safety procedures and to verify their skills. The results of the experimentation in a real industrial case study showed that, compared to the traditional training methodology, multimedia contents and quantitative ergonomic analyses improve the level of attention and the awareness of the operators about their safety. On the other hand, Serious games turned out as promising tools to train the workers about safe operating procedures that are difficult to implement in a real working environment.

Keywords: Ergonomics | Occupational safety | Serious game | Virtual humans | Virtual reality

Abstract: Although the so-called Industry 4.0 trend is promoting the increasing automation of processes in the factories of the future, manual activities still play an extremely important role within the factory and human factors greatly affect the process performance. However, the analysis of human-machine interaction and the prediction of human performance in industry are difficult but crucial to have an optimized design of workspaces and interfaces, reducing time and cost of implementation, and avoiding late design changes. This research adopts a multimodal human-centered approach for the analysis of human-machine interaction, and proposes a multimodal experimental set-up for the evaluation of the workers' experience to support the design of industrial workstations. The set-up combines virtual mockups, interaction with both physical and virtual objects, and monitoring sensors to track users and analyze their actions and reactions. It allows creating a multimodal environment able to deepen the interaction between humans and systems or interfaces, to support design activities. Indeed, it has been demonstrated that the analysis of the reactions of the users involved, allows to evaluate the quality of the interaction, identify the critical issues, define corrective actions, and propose guidelines for system design or redesign [1]. The paper describes the application of the proposed set-up on two industrial case studies and reports the main results.

Keywords: Digital Manufacturing | Human Factors | Human-Centered Design | Industry 4.0 | Virtual Reality

Abstract: Augmented Reality (AR) is one of the nine key technologies of Industry 4.0 and one of the most promising innovation accelerators that in the next years will bring smart factories to a higher level of efficiency. In this context, the paper presents an AR tool that improves and increases the efficiency of data collection and exchange of information among different professional figures involved in the design and production processes of products for the oil and gas sector. In fact, prototyping and labour-intensive activities usually require modifications and improvements to be made on-site that should be sent as feedback to the technical office. To this end, the proposed AR tool supports workers at the workplace to easily detect and annotate design variations made during their working activities and furthermore to formalize and automate the collecting and transferring of this data to the designers in order to prevent loss of information. Field experimentation has been carried out with end-users to evaluate their acceptance by means usability studies, based on objective and subjective metrics, and personal interviews. Experimental results show that the proposed AR tool provides medium-to-high levels of usability and has been positively accepted by all the participants involved in the study.

Keywords: Augmented reality | Design discrepancies | Industry 4.0 | Technical instructions

Abstract: The Mediterranean Sea has a vast maritime heritage which exploitation is made difficult because of the many limitations imposed by the submerged environment. Archaeological diving tours, in fact, suffer from the impossibility to provide underwater an exhaustive explanation of the submerged remains. Furthermore, low visibility conditions, due to water turbidity and biological colonization, sometimes make very confusing for tourists to find their way around in the underwater archaeological site. To this end, the paper investigates the feasibility and potentials of the underwater Augmented Reality (UWAR) technologies developed in the iMARECulture project for improving the experience of the divers that visit the Underwater Archaeological Park of Baiae (Naples). In particular, the paper presents two UWAR technologies that adopt hybrid tracking techniques to perform an augmented visualization of the actual conditions and of a hypothetical 3D reconstruction of the archaeological remains as appeared in the past. The first one integrates a marker-based tracking with inertial sensors, while the second one adopts a markerless approach that integrates acoustic localization and visual-inertial odometry. The experimentations show that the proposed UWAR technologies could contribute to have a better comprehension of the underwater site and its archaeological remains.

Keywords: Markerless and marker-based tracking | Underwater acoustic localization | Underwater augmented reality | Underwater cultural heritage

Abstract: The paper presents the digital technologies developed in the VISAS project and their application to the underwater archaeological site of Cala Minnola (Levanzo Island, Italy)that preserves the remains of a Roman ship. Following the basic principles defined by UNESCO for the protection of the Underwater Cultural Heritage the VISAS project has led to the development of innovative digital technologies for a more engaging and educational exploitation of the submerged archaeological sites. In particular, the paper describes a virtual diving system that allows users to perform, outside of the submerged environment, a virtual exploration of the Cala Minnola shipwreck site. Moreover, an augmented diving system provides, through an underwater tablet, a geolocalized multimedia guide for the divers that visit the underwater archaeological site. Both digital technologies allow users to perform an entertaining and interdisciplinary learning experience by receiving archaeological, historical, and biological information of the specific submerged site.

Keywords: Underwater 3D reconstruction | Underwater archaeology | Underwater Cultural Heritage | Virtual exploitation | Virtual reality

Abstract: The paper presents the application of some Virtual Reality technologies developed in the Horizon 2020 i-MARECulture project to the case study of the sunken "Villa con ingresso a protiro", dated around the II century AD, and located in the Marine Protected Area - Underwater Park of Baiae (Naples).The i-MARECulture project (www.imareculture.eu), in fact, aims to improve the public awareness about the underwater cultural heritage by developing new tool and techniques that take advantage of the virtual reality technologies to allow the general public to explore the archaeological remains outside of the submerged environment.To this end, the paper details the techniques and methods adopted for the development of an immersive virtual tour that allow users to explore, through a storytelling experience, a virtual replica and a 3D hypothetical reconstruction of the complex of the "Villa con ingresso a protiro".

Keywords: 3D hypothetical reconstruction | digital storytelling | Virtual Diving | Virtual Heritage | Virtual Reality

Abstract: Augmented Reality (AR) is an innovation accelerator for Industry 4.0 that supports the digitalization and improves the efficiency of the industrial sector by providing powerful tools able to enhance the workers’ visual perception by combining the real world view with computer-generated data. In this context, the paper presents a new AR tool and an exploratory test in order to examine how well it supports the user’s tasks for the detection of design discrepancies. In particular, the test aims to evaluate the effectiveness and efficiency of the proposed tool and how it compares to other instruments traditionally adopted for this end, such as technical drawings and CAD systems. The experimental findings show that the proposed AR tool presents similar results with the other instruments in term of effectiveness and very encouraging results about its efficiency.

Keywords: Assistive tool | Augmented reality | Design variations | Industry 4.0

Abstract: In view of the wide scope of challenges concerning Industry 4.0, a variety of enabling digital industrial technologies can support the digitization of the manufacturing sector. Among them, Augmented Reality represents one of the most promising innovation accelerators that will support human workers and bring Smart Factories to a higher level of efficiency. To this end, the paper presents an Augmented Reality tool that provides support at the workplace to easily detect and collect design changes by augmenting virtual 3D models, as defined in the project plan, on the actual design. The proposed tool runs on a consumer smartphone and adopts hybrid tracking techniques to allow workers to formalize and make more efficient the knowledge management of the design changes within the overall design process.

Keywords: Augmented reality | Industry 4.0 | Knowledge-based engineering

Abstract: In this work, we present an Augmented Reality (AR) application for handheld devices that support operators in information retrieval tasks in maintenance procedures in the context of Industry 4.0. Indeed, using AR allows the integration of knowledge-based information, traditionally used by operators and mainly provided in the form of technical drawings, and data available from sensors on the equipment. This approach is suggested by companies, especially Small and Medium-sized Enterprises, that want a gradual introduction of Industry 4.0 technologies within their established practices. We implemented a prototype of the application for the case study of a milling plant. The application augments on a Piping and Instrumentation Diagram (P&ID) of the plant some virtual interactive graphics (hotspots) referenced to specific components drawn. Component data are retrieved, through a user interface, directly from the factory database and displayed on the screen. We evaluated the application through a user study aimed at comparing the AR application with the current practice, based on paper documentation, for an information retrieval task within a maintenance procedure. Results of the study revealed that AR is effective for this task in terms of task time reduction and usability. The AR application was tested both with a tablet and a smartphone, but results revealed that using tablet does not improve user performance in terms of task time, error rate, and usability.

Keywords: Augmented Reality | Industry 4.0 | Information retrieval | Maintenance | User evaluation

Abstract: In this work, we present an Augmented Reality framework for handheld devices that enhance users in the comprehension of plant information traditionally conveyed through printed Piping and Instrumentation Diagrams (P&ID). The proposed framework augments on the P&ID of a plant some virtual interactive graphics (hotspots) referenced to specific components drawn on the P&ID. In this way, it is possible to easily find all the components belonging to the same category (e.g., all the pumps). By tapping, on the tablet screen, on a single hotspot further multimedia information can be displayed: Technical data, 3D CAD model of the component, and 360° images of the plant section. The application is connected to the factory database where all the information associated with the plant components is stored. We used, as a case study, the cleaning section of a milling plant. With the tool presented in this work, technicians will be able to find information updated and in less time, so reducing the intervention time and increasing the accuracy of the operations. Furthermore, the cognitive load associated with the task of understanding the plant is highly reduced through the use of virtual information displayed using Augmented Reality.

Keywords: Augmented Reality | Industrial plant | Industry 4.0 | P&ID | Technical information

Abstract: The use of odors in different areas, such as marketing, entertainment, wellbeing, and arts is nowadays calling increasing attention. Various types of olfactory displays to integrate into Virtual Reality applications have been proposed in recent years. Actually, several characteristics of the sense of smell and of odors still remain not fully understood, and the development of olfactory displays presents a high level of complexity. Therefore, new strategies for the development of more reliable and effective olfactory displays are required.This paper presents a wearable olfactory display, based on solid fragrances, to integrate into multisensory applications for museum exhibitions. The motivation is that the use of odors can be particularly effective for improving the users' involvement, comprehension and experience of art exhibitions. The paper also describes the experimental setup for the evaluation of the display.

Keywords: Multisensory Application | Olfactory Display | Virtual Reality

Abstract: Augmented Reality (AR), is one of the most promising technology for technical manuals in the context of Industry 4.0. However, the implementation of AR documentation in industry is still challenging because specific standards and guidelines are missing. In this work, we propose a novel methodology for the conversion of existing “traditional” documentation, and for the authoring of new manuals in AR in compliance to Industry 4.0 principles. The methodology is based on the optimization of text usage with the ASD Simplified Technical English, the conversion of text instructions into 2D graphic symbols, and the structuring of the content through the combination of Darwin Information Typing Architecture (DITA) and Information Mapping (IM). We tested the proposed approach with a case study of a maintenance manual of hydraulic breakers. We validated it with a user test collecting subjective feedbacks of 22 users. The results of this experiment confirm that the manual obtained using our methodology is clearer than other templates.

Keywords: Augmented reality | Industry 4.0 | Maintenance support | Technical documentation

Abstract: The control shifting between a human driver and a semi-autonomous vehicle is one of the most critical scenarios in the road-map of autonomous vehicle development. This paper proposes a methodology to study driver's behaviour in semi-autonomous driving with physiological-sensors-integrated driving simulators. A virtual scenario simulating take-over tasks has been implemented. The behavioural profile of the driver has been defined analysing key metrics collected by the simulator namely lateral position, steering wheel angle, throttle time, brake time, speed, and the take-over time. In addition, heart rate and skin conductance changes have been considered as physiological indicators to assess cognitive workload and reactivity. The methodology has been applied in an experimental study which results are crucial for taking insights on users' behaviour. Results show that individual different driving styles and performance are able to be distinguished by calculating and elaborating the data collected by the system. This research provides potential directions for establishing a method to characterize a driver's behaviour in a semi-autonomous vehicle.

Keywords: Evaluation | Semi-autonomous vehicle | Simulation | User behaviour | Virtual reality

Abstract: A research area of interest is that one concerning the design of solutions for improving the life conditions of users in extreme environmental situations. An example is the spacecraft environment, where astronauts are subject to particular conditions, due to the extreme environment. The isolated and confined environment influences behaviors and perceptions. This situation can impact both on astronauts’ moods, cause states of depression, and impact on their performance in working activities. A spacecraft can be the Space Station orbiting the Earth, or future means of transportation used for travelling to other planets. In both cases the space should be designed so as to offer the best possible living and working conditions to the astronauts. The research presented in this paper aims at designing and developing a multisensory VR system for the entertainment and the relaxation of astronauts. The use of VR technology allows us to overcome physical and psychological boundaries of the confined space, which is typical in a spacecraft environment. The sense of smell, which is more linked to visceral emotions than the other senses and can affect various aspects of humans’ physiological and psychological conditions, is used to improve astronauts’ productivity and concentration, and also to relieve their stress and anxiety.

Keywords: Entertainment | Multisensory environment | Relaxation | Scents simulation | Virtual Reality

Abstract: The use of collaborative robots in the manufacturing industry has widely spread in the last decade. In order to be efficient, the human-robot collaboration needs to be properly designed by also taking into account the operator’s psychophysiological reactions. Virtual Reality can be used as a tool to simulate human-robot collaboration in a safe and cheap way. Here, we present a virtual collaborative platform in which the human operator and a simulated robot coordinate their actions to accomplish a simple assembly task. In this study, the robot moved slowly or more quickly in order to assess the effect of its velocity on the human's responses. Ten participants tested this application by using an Oculus Rift head-mounted display; ARTracking cameras and a Kinect system were used to track the operator's right arm movements and hand gestures respectively. Performance, user experience, and physiological responses were recorded. The results showed that while humans’ performances and evaluations varied as a function of the robot’s velocity, no differences were found in the physiological responses. Taken together, these data highlight the relevance of the kinematic aspects of robot’s motion within a human-robot collaboration and provide valuable insights to further develop our virtual human-machine interactive platform.

Keywords: Human-robot collaboration | Stress | Virtual reality | Workload

Abstract: The usability of heavy construction equipment is strongly affected by the design of their human-machine interfaces. Lack of confidence with the current input devices is due to their counterintuitive design and the absence of loop feedback between the end effector and human hands. In the last few years, many researchers have been demonstrated that haptic devices, joined with a suitable design of the control levers, could help to face this problem. In this paper, an innovative control logic for hydraulic excavators has been proposed based on the inverse kinematic of the arms of the hydraulic excavator. The aim of this control is to reduce the cognitive effort of the users if compared with the one required by the current control systems. The implementation of this control logic has been based on previous research projects, technical documentations and interviews with experts. The proposed control logic has been evaluated by means of experimental activities with a virtual simulator which test the usability and efficiency of the proposed solution.

Keywords: Coordinated control | Excavator | Haptic device | Human-machine interface | Usability evaluation | Virtual reality

Abstract: The PERVIVAL project aims at developing an interactive system with the preliminary function of explaining a complex museum collection in a simple and immediate way and allowing the visitor to better understand the museum collection he is about to see. In particular, the interactive system aims at enhancing the understanding of the collections of funeral furnishings of Egyptians, which are characterized by a multiplicity of objects of rich symbolism and connected to each other through complex funeral rituals. The idea is to explain the religious creed of ancient Egyptians through the objects placed in the tomb, having in this way a double benefit: enlightening the rituals and placing the objects back in their primary function. In this way, the knowledge of the visitor is not only enlarged through the description of something that is described on papyruses or inscriptions (hence, not comprehensible) but also the proper function of every single object will be explained through the connection among them, as a function of amulets or goods necessary to travel through the World of the Dead. The connection between the different objects allows a much greater understanding of the exposed collection that would be perceived in this way not as a set of single isolated pieces, but as a harmonious set of complementary elements between they represent a specific historical-cultural context.

Keywords: Cultural Heritage | Dissemination | Egyptians | Exhibition | Museum | Storytelling | Virtual Reality

Abstract: The paper broadly addresses how Industry 4.0 program drivers will impact maintenance in aviation. Specifically, Industry 4.0 practices most suitable to aeronautical maintenance are selected, and a detailed exposure is provided. Advantages and open issues are widely discussed and case studies dealing with realistic scenarios are illustrated to support what has been proposed by authors. The attention has been oriented towards Augmented Reality and Additive Manufacturing technologies, which can support maintenance tasks and spare parts production, respectively. The intention is to demonstrate that Augmented Reality and Additive Manufacturing are viable tools in aviation maintenance, and while a strong effort is necessary to develop an appropriate regulatory framework, mandatory before the wide-spread introduction of these technologies in the aerospace systems maintenance process, there has been a great interest and pull from the industry sector.

Keywords: Additive Manufacturing | Aeronautical maintenance | Augmented Reality | Industry 4.0

Abstract: One of the technologies that is showing the most potential in an always widening range of applications, from entertainment to design and even healthcare, is Augmented Reality (AR). The most defining characteristic of AR consists of the possibility to overlap virtual object to be projected by a dedicated device upon a real environment. Throughout this work, the development of an application for the visualization of medical data in AR environments will be discussed. Nowadays, the most appropriate devices for such applications are Head Mounted Displays (HMDs). Investigating current visualization procedures of medical data deriving from CT or MRI scans, AR tools and features have the potential to streamline common activities, as diagnostic and surgery planning precesses, providing a much more immersive user experience. In this paper, an efiective visualization approach merging these new technologies and biomedical images is proposed. Using a cutting edge HMD device, an application that takes as input CT or MRI scans and gives as output an immersive visualization of patient's internal body structures has been developed. The range of possible use scenarios is very wide. In essence, the goal is to recreate the illusion for the user to be able to see through the patient's skin with no visual distortion, providing a much improved perception of the relative position of the inner body structures. Present techniques make use of DICOM data mostly in the diagnostic stage, whereas it is possible to integrate AR tools within the whole medical workfiow and even to further stages of the disease treatment, including surgery planning and training activities. Additionally, the possibility to connect a multitude of HMD devices together that are able to display the same scene from difierent angles, depending on each own position, opens up to a great variety of collaborative working or learning situations. Although there is still a lot of scope for improvements, the paper details the potential of AR for a medical implementation, outlining specifications needed to evolve into something actually usable in a real-life scenario.

Keywords: Abdominal Aortic Aneurysm | Augmented Reality | Biomedical Data | CT

Abstract: Realistic visualization of products is now a must-have for all companies facing worldwide and highly competitive market. Despite Virtual Reality technologies are appealing, its industrial use is still limited to conceptual design and prototyping activities. One of the reason is that generating Virtual Reality (VR) environment is a complex and time-consuming task, especially for complex products or systems. Many technical data are involved in their design and conflguration. A meaningful example is the preliminary conflguration of assembly lines devoted to deliver a quotation to the customer. To be competitive, the quotation should be completed in tight time and contain variants of the conflgured system ranging different costs. Moreover, high-impact and successful quotation goes beyond the merely technical aspect. In this view, the automatic generation of a virtual reality environment can foster the adoption of this technology in industry, since its setup time is short and doesn’t require any skills. In this paper, the integration of a VR module in product conflguration and quotation process is proposed. The framework is a Knowledge-based Engineering (KBE) system that, taken the customer requirements as input is able to automatically generate a bunch of different solutions. Starting from technical data coming from a KBE system, a virtual environment is generated automatically fltting the features of the conflgured solution. Furthermore, the immersivity of the VR scene is enhanced by integrating the animation of the objects, like robots and pallets. After a brief description of the KBE system, the paper details the information is involved in, the implementation of the VR module and its integration within the KBE framework.

Keywords: Assembly line | Knowledge-based engineering | Quotation | Virtual reality

Abstract: The development of systems for supporting neuro-rehabilitation is of primary importance, due to the high number of people in need of rehabilitation and the limited effectiveness of most of the current developed systems. Our research work aims at developing more engaging interaction modalities for neuro rehabilitation systems, through virtual reality, music based on harp therapy and fragrance feedback modalities and which are also fun and motivational for the patients. The proposed interaction modalities consist of a set of virtual immersive environments which includes an olfactory feedback, where odours are used to increase the sense of presence and the attention of the patients during the execution of the exercises. While the patient performs the rehabilitation exercise, the harp therapist plays the harp accordingly to the patient emotional condition. The system shows a virtual scenario, including virtual objects and/or 360 videos used to increase his sense of presence in the scenario. Odours are associated with virtual scenarios.

Abstract: Increasing the level of automation in air traffic management is seen as a measure to increase the performance of the service to satisfy the predicted future demand. This is expected to result in new roles for the human operator: he will mainly monitor highly automated systems and seldom intervene. Therefore, air traffic controllers (ATCos) would often work in a supervisory or control mode rather than in a direct operating mode. However, it has been demonstrated how human operators in such a role are affected by human performance issues, known as Out-Of-The-Loop (OOTL) phenomenon, consisting in lack of attention, loss of situational awareness and de-skilling. A countermeasure to this phenomenon has been identified in the adaptive automation (AA), i.e., a system able to allocate the operative tasks to the machine or to the operator depending on their needs. In this context, psychophysiological measures have been highlighted as powerful tool to provide a reliable, unobtrusive and real-time assessment of the ATCo’s mental state to be used as control logic for AA-based systems. In this paper, it is presented the so-called “Vigilance and Attention Controller”, a system based on electroencephalography (EEG) and eye-tracking (ET) techniques, aimed to assess in real time the vigilance level of an ATCo dealing with a highly automated human–machine interface and to use this measure to adapt the level of automation of the interface itself. The system has been tested on 14 professional ATCos performing two highly realistic scenarios, one with the system disabled and one with the system enabled. The results confirmed that (i) long high automated tasks induce vigilance decreasing and OOTL-related phenomena; (ii) EEG measures are sensitive to these kinds of mental impairments; and (iii) AA was able to counteract this negative effect by keeping the ATCo more involved within the operative task. The results were confirmed by EEG and ET measures as well as by performance and subjective ones, providing a clear example of potential applications and related benefits of AA.

Keywords: adaptive automation | air traffic control | electroencephalography | eye-tracking | human–machine interface | Out-Of-The-Loop | passive brain–computer interface | vigilance

Abstract: In the recent past a growing attention to the passenger is emerging overall in the transport domain. Hence, maximising the quality of travelling from the human’s point of view is a new challenge especially in those fields, such as aeronautics, in which technical efficiency, capacity and sustainability have traditionally driven the design process of systems and subsystems. In this context it is crucial to implement an efficient human centred design process in order to foresee the capability of a specific cabin interiors design of meeting the user’s expectations, including the needs related to comfort and well being. By using virtual reality technologies as a vehicle/platform, it allows the users/passengers to experience the interior environment of the cabin long before the actual development and manufacturing of the full size demonstrator. Due to the complex nature of aerospace programmes, typically taking ‘many’ years to develop and productionise, technologies which help reduce programme risk and potential delays are hugely beneficial to all partners involved. In this paper we present the results of a virtual reality based evaluation campaign specifically conceived for the collection of potential users’ feedback in the design of innovative and breakthrough solutions for the business jet industry. The main issues have regarded the identification of the expectation for such an elitist population and the creation of a Virtual Environment to explore the entire cabin as a holistic approach and innovative passenger experience. The work has been performed in the framework the Horizon 2020 project CASTLE (Cabin Systems Design Toward Passenger Well-being).

Keywords: Aircraft design | Cabin interiors | Comfort | Human centred design | Virtual reality

Abstract: Variations in alpha rhythm have a significant role in perception and attention. Recently, alpha decrease has been associated with externally directed attention, especially in the visual domain, whereas alpha increase has been related to internal processing such as mental arithmetic. However, the role of alpha oscillations and how the different components of a task (processing of external stimuli, internal manipulation/representation, and task demand) interact to affect alpha power are still unclear. Here, we investigate how alpha power is differently modulated by attentional tasks depending both on task difficulty (less/more demanding task) and direction of attention (internal/external). To this aim, we designed two experiments that differently manipulated these aspects. Experiment 1, outside Virtual Reality (VR), involved two tasks both requiring internal and external attentional components (intake of visual items for their internal manipulation) but with different internal task demands (arithmetic vs. reading). Experiment 2 took advantage of the VR (mimicking an aircraft cabin interior) to manipulate attention direction: It included a condition of VR immersion only, characterized by visual external attention, and a condition of a purely mental arithmetic task during VR immersion, requiring neglect of sensory stimuli. Results show that: (1) In line with previous studies, visual external attention caused a significant alpha decrease, especially in parieto-occipital regions; (2) Alpha decrease was significantly larger during the more demanding arithmetic task, when the task was driven by external visual stimuli; (3) Alpha dramatically increased during the purely mental task in VR immersion, whereby the external stimuli had no relation with the task. Our results suggest that alpha power is crucial to isolate a subject from the environment, and move attention from external to internal cues. Moreover, they emphasize that the emerging use of VR associated with EEG may have important implications to study brain rhythms and support the design of artificial systems.

Abstract: Automation has been introduced more and more into the role of air traffic control (ATC). As with many other areas of human activity, automation has the objective of reducing the complexity of the task so that performance is optimised and safer. However, automation can also have negative effects on cognitive processing and the performance of the controllers. In this paper, we present the progress made at AUTOPACE, a European project in which research is carried out to discover what these negative effects are and to propose measures to mitigate them. The fundamental proposal of the project is to analyse, predict, and mitigate these negative effects by assessing the complexity of ATC in relation to the mental workload experienced by the controller. Hence, a highly complex situation will be one with a high mental workload and a low complex situation will be one in which the mental workload is low.

Keywords: Air-traffic controllers | Automation | Mental workload

Abstract: Today's market drives companies to change, adapt, and compete. Many consumers are increasingly looking at price, without sacrificing quality. In order to be attractive to the customer, companies must be able to offer the required quality at the lowest possible price. The life cycle of many products has been shortened compared to the past because now technologies are evolving faster. For these reasons, it is important that companies reevaluate all the operations that are carried out within them, to optimize them and eventually adopt new technologies if they offer interesting opportunities. In this discussion, we first study the design for disassembly, a technique that can bring several advantages during the life cycle of a component, offering the possibility of reducing time and cost of disassembling a product, and better reuse of the different materials of which it is composed. Subsequently, augmented reality is discussed, and how this technology is exploited in the world, especially in the industrial sector. During the work, we discuss a case study, with the gearbox being the object of analysis. This allows us to apply the theoretical concepts illustrated in a concrete way, allowing for a better understanding of the topics.

Keywords: Augmented reality | Computer aided design (CAD) | Design for disassembly | Gearbox

Abstract: The paper proposes a design method for the configuration of customized workplaces supported by the use of VR tool. The method allows to consider end-users anthropological features and configuration aspects related to the workplace (e.g. equipment). The adoption of VR technology allows supporting the configuration process, engaging end-users in the final customization. A yachts’ ship bridge console is proposed as a case study and a VR-enhanced configuration tool has been developed for the equipment configuration. The adoption of this tool in this specific field shows different advantages such as efficiency in configuration and time saving for the development of workplaces design alternatives. Another benefit of this approach is the automatic generation of an associated BoM and its management through PLM tools.

Keywords: Anthropological aspects | Configuration | Design | Ergonomics | Ship bridge system | VR

Abstract: Shape-based design activities analyse specific shapes to highlight functions that products shaped that way could implement. These design activities encompass a clearly interactive approach; designers have direct, personal confrontations with shapes. It is not to say that the effectiveness of these interactions depends on the way the shapes, along with the environments where their evaluations take place, are represented. Different representations may imply different interaction paradigms that, in turn, could be more or less suitable to highlight implicit problems to solve or foster innovative design solutions. The research described in this paper exploits an existing classification of representations to examine in depth the influence of representations on shape-based design activities. Metrics like quantity, novelty, variety, etc., are involved to analyse the results of these activities from the quantitative point of view. All of this confirms the existence of the influence and allows highlighting some direct relationships between representations and design results. These relationships, in turn, are available to select the best representations to lead design efforts towards specific targets time by time.

Keywords: Augmented reality | Interactive approaches to design | Representations | Shape-based design | Virtual reality

Abstract: Augmented reality is considered one of the enabling technologies of the fourth industrial revolution, within the Industry 4.0 program and beyond. Indeed, augmented reality solutions can increase the working quality and the productivity and allow a better use of the human resources. This technology can help the operator in the industrial applications during the crucial phases of the processes. Since the quality assessment of the surfaces is recognized to be a key phase in the polishing process, in this paper we propose a novel method that exploits augmented reality to support the operators during this phase. The metrology data measured by a surface measurement system are directly projected on the polished component through an augmented reality headset worn by the operators and used to assess the quality of the worked surfaces. Rather than imagine how a certain parameter change can affect the result achieved, the information is directly there on the component's surface. Users can see from the data where refinements are required and make better and faster decisions, which is compelling for its potential beyond industrial polishing. The proposed method is implemented and validated on an industrial cell, where the robot automatically perform the polishing task and move the head of the surface measurement system along the surface to measure the metrology parameters. Thanks to the proposed approach, the end-user and the operator can directly see on the component if the quality reached satisfies the specifications or if some parts of the surface require further refinements through additional polishing steps.

Keywords: Augmented reality | Industrial robotic solutions | Robotic polishing

Abstract: As the potentials of technology grow, the embedding of IT advances in different fields and applications increases. A recent example is virtual reality and in particular the virtual product. The possibility of having a product in a virtual form allows creators and designers to efficiently manage the cycle of a product generation and evolution. The key advantage of the “virtual” is to have the product in advance, even in the conceptualization phase, with clear benefits in terms of consumptions of resources and, hence, sustainability. A potential customer could thus interact with a product-to-be and provide feedback about its look and feel, its usability, and, most of all, give an emotional response. In this context, the interaction between the virtual product and the future customer becomes a core point for the new approaches related to user-centred and user experience design, giving birth to a design methodology called “emotional design”. In particular, the study of facial expressions seems to be the more reliable and attractive aspect of it.

Keywords: 3D | Concept design | Emotional design | Facial expression recognition | PLM | Virtual reality

Abstract: SAR provides an unobtrusive implementation of AR and enables multiple stakeholders to observe and interact with an augmented physical model. This is synonymous with co-design activities and hence, there is a potential for SAR to have a significant impact in the way design teams may set-up and run their co-design activities in the future. Whilst there are a growing number of studies which apply SAR to design activities, few studies exist that examine a particular element of a design activity in a controlled manner. This paper will begin to fill this gap through the controlled study of SAR and its effects on the communication between participants of a co-design activity. To do so the paper compares a controlled design session, using more traditional methods of design representations (3D models on a screen), to sessions run using SAR. The sessions are then analysed to gather information on the gestures used by the participants as well as the overall efficiency of the participants at completing the set design task. The paper concludes that the data gathered tentatively supports a link between the use of SAR and improved communication between design session participants.

Keywords: Augmented Reality | Collaborative design | Communication | Spatial Augmented Reality | Virtual reality

Abstract: Manufacturing ergonomics refers to the application of ergonomic principles and human factors analysis to the design of manufacturing tasks with the final aim to optimize the workers’ wellbeing and guarantee the expected process performance. Traditional design approaches are based on the observation of individual workers performing their jobs, the detection of unnatural postures (e.g., bending, twisting, overextending, rotating), and the definition of late corrective actions according to ergonomic guidelines. Recently, computer-integrated simulations based on virtual prototypes and digital human models (DHMs) can be used to assess manufacturing ergonomics on virtual manikins operating in digital workplaces. Such simulations allow validating different design alternatives and optimizing the workstation design before the creation, and pave the way to a new approach to manufacturing system design. The present paper aims at comparing different computer-integrated set-ups to support the design of human-centred manufacturing workstations. It defines a protocol analysis to support workstation design by analysing both physical and cognitive aspects, and applies the protocol within different digital set-ups. In particular, the study investigates a 2D desktop set-up using standardized DHMs and a 3D immersive mixed reality set-up based on motion capture of real workers’ acting into a mixed environment, comparing them with the traditional approach. An industrial case study focusing on design optimization of a manufacturing workstation in the energy industry is used to test the effectiveness of the two digital set-ups for the definition of re-design actions.

Keywords: Digital human models | Human factors | Human-centred design | Manufacturing ergonomics | Mixed reality

Abstract: In this paper, we present a solution for the photorealistic ambient light render of holograms into dynamic real scenes, in augmented reality applications. Based on Microsoft HoloLens, we achieved this result with an Image Base Lighting (IBL) approach. The real-time image capturing that has been designed is able to automatically locate and position directional lights providing the right illumination to the holograms. We also implemented a negative "shadow drawing" shader that contributes to the final photorealistic and immersive effect of holograms in real life. The main focus of this research was to achieve a superior photorealism through the combination of real-time lights placement and negative "shadow drawing" shader. The solution was evaluated in various Augmented Reality case studies, from classical ones (using Vuforia Toolkit) to innovative applications (using HoloLens).

Keywords: Augmented reality | Holographic shadow | Image processing | Light mapping | Rendering techniques

Abstract: Interactive learning experiences are becoming the standard for today’s ‘tech-savvy’ generation of students and an important issue for research in instructional technology. The design and implementation of higher education, incorporating interactive technologies, can be difficult and often requires high levels of design knowledge. Our intent is to assist researchers, instructors and designers in identifying an effective methodology to design interactive learning contents that use recent interactive technologies, in particular 360° video, and encourage greater student engagement. In this study, 360° videos have been designed and implemented in an engineering program but the design methodology we suggest can be apply in any industrial or educational context. Then, 360° videos have been evaluated by the students as highly immersive and engaged environments that surround them and offer them an increased sense of presence, giving them a 360-degree view of the environment. In this type of video, viewers no longer only look at a single screen, they can point the camera lens wherever they want, allowing viewers to watch the video from multiple perspectives (active), rather than only from the director’s point of view (passive).

Keywords: 360° video | Engineering education | Student engagement | Virtual reality

Abstract: The use of Augmented Reality (AR) to support assembly tasks has been an area of interest from its origins in the 90s. Since then, the benefits that this technology could bring to assembly-related tasks have been shown. And, although several advances have been done in different areas such as software, hardware, and human interaction, there are still some problems that have not allowed AR to expand and reach its full potential. Thereby, authors propose a real-time vibrotactile guidance method based on the Gestalt continuity principle and developed a Haptic Augmented Reality application with a low-cost configuration to evaluate the support of the proposed method in assembly tasks. Thus, it potentially overcomes the existing visual issues of AR allowing the user to focus on the task and avoid over-reliance into the technology. The proposed system recognizes the different parts and sub-assemblies, generates the instructions to perform the assembly based on the target position and rotation of each part and verifies the assembly. Additionally, a test was conducted to guide the user in positioning a part, obtaining a high accuracy of rotation and location placement. Also, different functions of the application were tested and the results are suitable for supporting the user guidance.

Keywords: Assembly guidance | Augmented Reality | Haptic | Object recognition | User interaction

Abstract: Virtual Reality (VR) storytelling enhances the immersion of users into virtual environments (VE). Its use in virtual cultural heritage presentations helps the revival of the genius loci (the spirit of the place) of cultural monuments. This paper aims to show that the use of actors in VR storytelling adds to the quality of user experience and improves the edutainment value of virtual cultural heritage applications. We will describe the Baiae dry visit application which takes us to a time travel in the city considered by the Roman elite as 'Little Rome (Pusilla Roma)' and presently is only partially preserved under the sea.

Keywords: Baiae | Interactive digital storytelling | Virtual cultural heritage | Virtual reality | VR storytelling

Abstract: Augmented reality can be deployed in various application domains, such as enhancing human vision, manufacturing, medicine, military, entertainment, and archeology. One of the least explored areas is the underwater environment. The main benefit of augmented reality in these environments is that it can help divers navigate to points of interest or present interesting information about archaeological and touristic sites (e.g., ruins of buildings, shipwrecks). However, the harsh sea environment affects computer vision algorithms and complicates the detection of objects, which is essential for augmented reality. This paper presents a new algorithm for the detection of fiducial markers that is tailored to underwater environments. It also proposes a method that generates synthetic images with such markers in these environments. This new detector is compared with existing solutions using synthetic images and images taken in the real world, showing that it performs better than other detectors: it finds more markers than faster algorithms and runs faster than robust algorithms that detect the same amount of markers.

Keywords: Augmented reality | Cultural heritage | Generating synthetic images | Marker-based tracking | Real time

Abstract: HoloLens is the most recent and advanced forms of wearable Mixed Reality (MR) technology. It enables the user wearing a head-mounted device to experience 3D holographic objects “inside” the visualization of the real environment where he or she is located. Existing HoloLens applications have been developed in domains such as data visualization, entertainment, industrial training, education, and tourism, but the use of this technology in the arena of mental health is largely unexplored. The paper presents a HoloLens-based system called MemHolo that addresses persons with mild Alzheimer’s Disease (AD). AD is associated to a chronic progressive neurodegenerative process that severely affects cognitive functioning (especially memory) and some motor functions. MemHolo is intended to be used as a cognitive training tool to practice short-term and spatial memory in a safe and controlled virtual environment, and to mitigate the effects of mental decline. The paper discusses the design process of MemHolo, and describes three evaluation studies on progressive prototypes. To our knowledge, MemHolo is the first HoloLens application designed natively for persons with AD. Our empirical work sheds a light on how these people experience HoloLens applications, highlights some challenges and potential benefits of using MR technology in the AD arena, and may pave the ground towards new forms of treatment.

Keywords: Alzheimer disease (AD) | Augmented reality | Cognitive training | Elderly | HoloLens | Mixed reality

Abstract: HoloLearn is a Mixed Reality (MR) application that exploits Microsoft HoloLens to help people with Cognitive Disability improve autonomy in everyday life. Using HoloLearn, the user is immersed in a MR environment based on the surrounding space, in which s/he can learn simple daily tasks in an engaging way, with the help of a virtual assistant if needed.

Keywords: Augmented Reality | Cognitive Disability | Holograms | HoloLens | Mixed Reality | Virtual Assistant

Abstract: The objective of the work is to describe the design and the realization of a virtual simulator of a metropolitan railway cockpit, aimed at improving the perception of safety by means of tests made by users in Virtual Reality, analysed through statistical methodologies. The user lives the experience of a driver in an immersive and interactive Augmented Reality session, interacting with the train dashboard and all its control and signalling devices. In particular, the user is proposed to test different dashboards, different configurations of the controls and different signalling and safety devices in order to compare different concept and select the optimum in terms of perception of dangerous situation, reaction to an event and cognitive response in different situations of the rail vehicle driving. The simulator consists of a simulacrum integrating different technologies, physically composed of a dashboard of the cockpit of a metropolitan train and a real seat. The geometry of the dashboard has been acquired through Reverse Engineering techniques from a real train dashboard. The user’s immersion in the virtual environment during the simulation is guaranteed by the scene displayed on the Augmented Reality device, while, simultaneously, the stereoscopic projection on a screen above the dashboard makes available the experience even to users not directly involved, seeing the scene from the driver’s point of view. The immersive Augmented Reality is realized through a Head-Mounted Display (HMD) by which the user, protagonist of the driving experience, sees the configuration of the virtual control devices (CAD geometries) overlapped with the physical dashboard in order to naturally interact into the immersive environment. The interaction between user and simulator happens through the NUI (Natural User Interfaces) based on markerless tracking of parts of the user’s body.

Keywords: Augmented reality | Head-mounted display | IDEAinVR | Natural user interfaces | Preventive safety | Railway design | Reverse engineering | Train dashboard | Vehicle driving | Vitual simulator

Abstract: The use of Augmented Reality (AR) technologies is the new challenge of management models born under the “Industry 4.0” paradigm. The aim of the work is to evaluate the usability of two types of AR devices (tablet and see-through) employed in the training and information activities of workers according to the ISO/IEC 9126 and ISO 9241 standards. Starting from the state of the art, evaluating market and competitors and developing different concepts of interfaces, a dedicated application was programmed and, then, the usability of such devices for the professional figures involved was evaluated through experimental tests. Two reference scenarios were defined, the Department of Industrial Engineering of University of Naples Federico II and INAIL (National Institute for Insurance against Accidents at Work) laboratories, an user interface was designed and developed, as an aid in the drafting of the document for risk evaluation and subsequent training of workers. The activity is part of the IDEE Project (Interactive Design for Ergonomics), born by the collaboration between Joint Lab IDEAS and Contarp-INAIL-Regional Management for Campania. The data analysis allowed to evaluate the goodness of the devices and the degree of satisfaction in their use on the basis of the sample of users who conducted the tests. The use of AR devices produces better results than paperwork in terms of efficiency and effectiveness, but not all devices produce appreciable results in terms of user satisfaction. Although AR technologies are mature, the tasks need to be carefully defined to avoid rejection phenomena. The strong expectation, that they generate in potential users, risks to remain disappointed today for some usability limits found in currently available devices. It is necessary to start testing in pilot applications in various industrial fields in order to capture in time and adequately support this opportunity of innovation in Italy.

Keywords: Augmented reality | Risk assessment | Usability

Abstract: Over recent years, various virtual prototyping technologies have been developed to innovate apparel industry. For each step of the garment design process one can find dedicated tools (from body acquisition to garment modelling and simulation) with the aim of making the process easier and faster. However, most of them are based on expensive solutions both for hardware and software systems. In this paper, we focus the attention on the first step of the made-to-measure garment design, i.e. customer’s measures acquisition. We present a plug-in, named Tailor Tracking, which permits to get the measurements by interacting with the customer’s avatar using hands as in the traditional way. Tailor Tracking has been developed using low cost devices, such as Microsoft Kinect sensor, Leap motion device and Oculus Rift, and open source libraries, such as Visualisation Toolkit (VTK) and Qt. The proposed approach is based on the use of multiple Kinect v2 to simultaneously acquire both customer’s body and motion. This permits to emulate the customer’s postures required to take the correct measurements. In addition, a virtual measuring tape is made available to replicate the one commonly used by the tailor. A men shirt has been considered as case study and a tailor and 14 people with no skills in garment design and different levels of experience in virtual reality technology have been involved to preliminary test Tailor tracking. Finally, tests as well as results reached so far are presented and discussed. Results have been considered quite good; however, some critical measures have been identified as well as future developments. Anyway, Tailor Tracking can represent an alternative solution to the existing approaches that automatically extract anthropometric measures from the customer’s avatar.

Keywords: Clothing design | garment measurements | hand-tracking device | head mounted display | Kinect sensors | motion capture | virtual reality

Abstract: Tactile sensors are essential components for the implementation of complex manipulation tasks using robot grippers, allowing to directly control the grasping force according to the object properties. Virtual Reality represents an effective tool capable of visualizing complex systems in full details and with a high level of interactivity. After the implementation of cost-effective tactile arrays on a 3-finger Robotiq® gripper using an ARDUINO controller, it is presented an innovative VR interface capable of visualizing the pressure values at the fingertips in a 3D environment, providing an effective tool aimed at supporting the programming and the visualization of the gripper VR.

Keywords: Calibration | Robot | Virtual reality

Abstract: Underwater manipulation is a key technology for marine industries and exploration that can be efficiently adopted in other application fields, such as underwater archaeology, biological manipulation, scientific expedition, as well as offshore construction in the Oil and Gas industry. It is performed remotely by expert pilots thanks to the visual feedbacks provided by one or more cameras but without any information about the distance between the end-effector and the target. To this end, the paper presents a novel system based on a sensorized robotic arm, stereoscopic 3D perception and augmented reality visualization to support ROV's pilots in underwater manipulation tasks. The system, thanks to the adoption of an optical-stereo camera, provides a visual feedback of the underwater scene on which a depth map of the underwater workspace is augmented on. In particular, combining the kinematics of the robotic arm and the standard photogrammetric model of the stereo camera, it is possible to generate a depth map that shows to the pilots the distances of the surface of the scene objects from the end-effector's pose. Experimental tests carried out in the context of the CoMAS (In-situ conservation planning of Underwater Archaeological Artefacts) project have demonstrated the effectiveness of the proposed system.

Keywords: Augmented reality | Forward kinematics | Optical-stereo camera | Underwater manipulation

Abstract: The paper presents a virtual diving system based on a virtual reality (VR) application for the exploitation of the Underwater Cultural Heritage. The virtual diving experience has been designed to entertain users, but its added pedagogical value is explicitly emphasized too. In fact, the ludic activities, consisting in the simulation of a real diving session from the point of view of a scuba diver, are following a storyline described by a virtual diving companion who guides users during the exploration of the underwater archaeological site. The virtual diving system provides general and historical-cultural contents, but also information about the flora and fauna of the specific submerged site to the users. The results collected through user studies demonstrate that the proposed VR system is able to provide a playful learning experience, with a high emotional impact, and it has been well appreciated by a large variety of audiences, even by younger and inexperienced users.

Keywords: Serious games | Underwater archaeological sites | Underwater Cultural Heritage | Virtual diving system | Virtual reality

Abstract: Nowadays, the adoption of virtual reality (VR) exhibits is increasingly common both in large and small museums because of their capability to enhance the communication of the cultural contents and to provide an engaging and fun experience to its visitors. The paper describes a user-centered design (UCD) approach for the development of a VR exhibit for the interactive exploitation of archaeological artefacts. In particular, this approach has been carried out for the development of a virtual exhibit hosted at the “Museum of the Bruttians and the Sea” of Cetraro (Italy). The main goal was to enrich the museum with a playful and educational VR exhibit able to make the visitors enjoy an immersive and attractive experience, allowing them to observe 3D archaeological artefacts in their original context of finding. The paper deals with several technical issues commonly related to the design of virtual museum exhibits that rely on off-the-shelf technologies. The proposed solutions, based on an UCD approach, can be efficiently adopted as guidelines for the development of similar VR exhibits, especially when very low budget and little free space are unavoidable design requirements.

Keywords: Human–computer interaction | User interface design | User-centered design | Virtual museum systems | Virtual reality

Abstract: One of the most effective strategies that can be adopted to make successful cultural heritage expositions consists in attracting the visitors’ attention and improving their enjoyment/engagement. A mid-air gesture-based Natural User Interface was designed, through the user-centric approach, for the navigation of virtual tours in cultural heritage exhibitions. In detail, the proposed interface was developed to “visit” Murgia, a karst zone lying within Puglia, very famous for its fortified farms, dolines, sinkholes, and caves. Including an “immersive” gesture-based interface was demonstrated to improve the user's experience thus giving her/him the sensation of “exploring” in a seamless manner the wonderful and rather adventurous sites of Murgia. User tests aimed at comparing the implemented interface with a conventional mouse-controlled one confirmed the capability of the proposed interface to enhance the user engagement/enjoyment and to make “more” natural/real, the virtual environment.

Keywords: Gesture vocabulary design | Natural user interface | User-centric approach | Virtual tour

Abstract: The evolution of technical documentation in the age of Industry 4.0 is going towards the use of visual manuals, in particular exploiting Augmented Reality (AR) technology. Traditional manuals are rich of text instructions that in AR applications are not advisable. In fact text occludes the real scene behind and it is an issue for the translation. For this reason, we propose to create and adopt a controlled and exhaustive vocabulary of graphical symbols, to be used in AR to represent maintenance instructions. In particular, in this work we identified the most frequent maintenance actions used in manuals, and converted them into graphical symbols. Then, we made an elicitation of the symbols designed and created different candidate vocabularies of symbols basing on the criteria found in literature of guessability and homogeneity. Moreover, the vocabularies had to respect two constraints: conflict set and reversibility. Finally, we identified the best of symbols and integrated this one in a real AR application for remote maintenance.

Keywords: Augmented Reality | Industry 4.0 | Maintenance

Abstract: The present work proposes a new approach for defining an interactive user manual in complex assemblies, using a new enabling technology of Industry 4.0, i.e. Augmented Reality. The AR environment supports the user in step-by-step assembly on-the-fly. The study of this method, suitable for the assembly of parts, is a stimulating engineering mission, which takes advantage of the latest innovations in imaging technologies and computer graphics. In the present paper, a proposal for an innovative method based on Augmented Reality used to support the components’ assembly is suggested. The methodology is based on a four steps process: (1) the designer performs the assembly structure through a CAD system; (2) an inexperienced user assembles the same parts without any suggestion, and the differences between the two assembly sequences are documented and broken down in order to distinguish critical points in the assembly; (3) a virtual user manual is shaped in an Augmented Reality environment; and (4) the assembly is then performed by the same inexperienced user, guided by the AR tool. When the end-user employs the instrument, the location of the item to assemble is perceived by tracking the finger position of the user itself. In order to help the end-user in the assembly procedure, a series of symbols and texts is added to the external scene. In this paper, a case study based on the assembly of a scale model has been developed to evaluate the methodology. After an evaluation process, the procedure seems to be feasible and presents some advantages over the state-of-the-art methodologies proposed by literature.

Keywords: Assembly | Augmented Reality | Marker | Task automation | User manual

Abstract: The paper discusses how Industry 4.0 could impact practitioners performing maintenance in aviation. The attention has been on Augmented Reality and Additive Manufacturing, which can support maintenance tasks and spare parts production respectively. Advantages and open issues are widely discussed and couple of case studies dealing with realistic scenarios are presented to support what has been proposed by the authors. The intention is to demonstrate that AR and AM are viable tools in aviation maintenance, even if effort is necessary to develop an appropriate regulatory framework, required before the introduction of these technologies in the maintenance process. Once applied to real maintenance tasks by airline companies, the practitioning community can develop best practices and the necessary regulation pertaining to maintenance and repair of aerospace systems using AR and AM technologies.

Keywords: Additive Manufacturing | Aeronautical Maintenance | Augmented Reality | Industry 4.0

Abstract: In a highly competitive global aviation environment, European regional aircraft requires continuous improvements in cabin comfort. This paper describes the design for comfort process proposed in an industrial research project, aimed at the definition of innovative design approaches to measure the affective impact on a potential user when living in and interacting with the cabin. The study is performed in the framework of CASTLE (CAbin Systems design Toward passenger welLbEing). CASTLE is a project granted under the Horizon 2020 EU's research programme in the framework of the Clean Sky 2 initiative. Requirements and recommendations for the comfort aspects to be taken into account to develop a regional aircraft's cabin were provided by Leonardo's Aircraft Division. The methodological approach is set up in a Virtual / Augmented Reality Environment for the definition of a Human Centered Response Model for the design for comfort of regional aircraft interiors. In this context, special attention has been paid to the improvements that can be brought to the cabin interiors, and specifically to the experience that passengers can live in the aircraft of the future. The main objective of the project is to conceive, develop, prototype and test cabin interiors solutions following a HCD (Human Centered Design) methodology. The paper analyzes the approach toward the definition of the design for comfort according to the considered cabin items and design requirements. The proper comfort metrics are selected and linked to an experimental protocol analysis for their assessment. A Virtual Reality environment has been set up to support the comfort assessment in aircraft cabin interiors, from their preliminary design.

Keywords: Aircraft interiors Design | Comfort | Emotional Design | Human Centered Design | Virtual Reality

Abstract: The Resilient Synthetic Vision for Advanced Control Tower Air Navigation Service Provision (RETINA) project is one of the selected Single European Sky ATM Research (SESAR) exploratory research projects on High Performing Airport Operations and it investigates the potential and applicability of Virtual/Augmented Reality (V/AR) technologies for the provision of Air Traffic Control (ATC) service by the airport control tower. The project assesses whether those concepts that stand behind tools such as Head-Mounted Displays (HUDs), Enhanced Vision Systems (EVSs) and Synthetic Vision Systems (SVS) can be transferred to ATC with relatively low effort and substantial benefits for controllers' Situational Awareness (SA). In doing so, two different augmented reality systems are investigated: Spatial Displays (SD) that, potentially, can be made to coincide with the tower windows and See-Through Head-Mounted Displays (ST-HMD). In this context the RETINA concept will enable the Air Traffic Controller to have a head-up view of the airport traffic even in low visibility conditions, similar to the vision currently provided in the cockpit with Head-Up displays. In the two-year project, the RETINA concept was developed, implemented and validated by means of human-in-the-loop simulations where the external view is provided to the user through a high fidelity 3D digital model in an immersive environment.

Keywords: Air Traffic Control | Airport Control Tower | Augmented Reality | Synthetic Vision

Abstract: Awe is a little-studied emotion with a great transformative potential. Therefore, the interest toward the study of awe's underlying mechanisms has been increased. Specifically, researchers have been interested in how to reproduce intense feelings of awe within laboratory conditions. It has been proposed that the use of virtual reality (VR) could be an effective way to induce awe in controlled experimental settings, thanks to its ability of providing participants with a sense of "presence," that is, the subjective feeling of being displaced in another physical or imaginary place. However, the potential of VR as awe-inducing medium has not been fully tested yet. In the present study, we provided an evidence-based design and a validation of four immersive virtual environments (VEs) involving 36 participants in a within-subject design. Of these, three VEs were designed to induce awe, whereas the fourth VE was targeted as an emotionally neutral stimulus. Participants self-reported the extent to which they felt awe, general affect and sense of presence related to each environment. As expected, results showed that awe-VEs could induce significantly higher levels of awe and presence as compared to the neutral VE. Furthermore, these VEs induced significantly more positive than negative affect. These findings supported the potential of immersive VR for inducing awe and provide useful indications for the design of awe-inspiring virtual environments.

Keywords: Awe | Emotion inducti | Emotions | Presence | Virtual reality

Abstract: Augmented reality (AR) is a key technology for the development of smart manufacturing. One of the main advantages of AR is that it can help workers to accomplish several tasks, making it possible the shift from mass production to mass customization. However, it is still not clear how these promises can be fulfilled in an industrial scenario. In particular, the question about which display solutions fit better the industrial constraints remains open. Based on the literature overview, laboratory experiments, and feedbacks from industrial companies, we supported the use of spatial augmented reality (SAR), designing a prototype intended to be used for manual working stations of the future smart factories. This work presents the evaluation of the effectiveness of conveying technical instructions with this SAR prototype as compared to paper manual. We run a within-subjects experiment with 16 participants to measure user task performance (completion times and error rates) and to collect subjective evaluation. We projected technical information on a motorbike engine during a seven-task maintenance procedure. Our results proved that SAR technology improves the operators’ performance with respect to a paper manual and that users well accept it. We found that SAR is more effective for difficult tasks than for simple ones and that the main advantage of SAR is related more to the reduction of error rates than to completion times. These results confirm the goodness of our design choices; then our prototype can be a valid candidate solution for a smart manufacturing application.

Keywords: Assembly | Industry 4.0 | Maintenance | Projection | Spatial augmented reality | Technical instructions

Abstract: Spatial Augmented reality (SAR) represents a key technology for the development of smart manufacturing as it is barrier free, does not require the use of Head Mounted Displays or any other wearable devices and it fits most of the industrial constraints. The paper presents a novel SAR-based system to support the manual work in future smart factories. It conveys technical instructions during assembly, provides alerts in case of risks for humans' safety and finally identifies which postures can bring to muscoloscheletric problems if repeated. Experiments with 30 participants demonstrated the effectiveness of the proposed SAR-based system as compared LED monitor-based system and the overall achieved usability. The results proved that SAR technology improves the operators' performance with respect to a LED monitor-based system and that users well accept it. We found that SAR is more effective for difficult tasks than for simple ones.

Keywords: Augmented Reality | Ergonomic assessmnt | In-Situ projection | Musculoskeletal Disorders evaluation | task guidance

Abstract: The visualization and analysis of mosaics and pavements are often compromised by their large sizes, which do not enable the observer to perceive their whole arrangement or to focus on details placed in farthest areas from its boundaries. Moreover, the usual precarious state of conservation of these artefacts, often with damaged or missing areas, makes it difficult to perceive their original aesthetic value. To overcome these limitations, we propose an application of augmented reality able to support the observer in two ways: first, the application completes the missing surface of the mosaic or pavement by integrating the existent surface with a virtual reconstruction; second, it enables the analysis of the geometric pattern of the mosaic/pavement by overlaying virtual lines and geometric figures in order to explicit its geometric arrangements. The result is achieved via a custom Android application able to recognize and track the mosaic figure pattern and extra marker board, obtaining in that way a coordinate system used to render in real-time the reconstruction of the mosaic. Such rendering is overlaid to the video stream of the real scene. The application runs on a standard smartphone embedded in a Google Cardboard-compatible viewer and therefore is extremely affordable. As a case study, in order to reconstruct its aspects and to analyse its geometric pattern, we chose the roman mosaic re-found in Savignano sul Panaro (near Modena, Italy) in 2011, after 115 years from its first discovery, which is preserved less than half of its original 4.5 x 6.9 m surface.

Keywords: Augmented Reality | Cultural heritage | Geometric pattern | Photogrammetry | Real-time visualization | Roman mosaic

Abstract: This paper proposes the integration of photogrammetric reconstruction, 3D modelling and augmented reality application in order to achieve the complete visualization of a stone sculpture even if highly damaged or fragmentary. The first part of the research aims to the reconstruction of the original aspect of an incomplete sculpture, by using photogrammetry techniques based on standard resolution photos and free software in order to obtain a first model; then, we integrate this model with other 3D digital data (from other sculptures of the same period) or with 3D modelling based on historical sources and views from historians, aiming to achieve the original aspect of the sculpture. The second part of the research consists of the embedding of the obtained model in a custom application able to render in real-time the 3D reconstruction of the lion. Then, the rendering is overlaid to the video stream of the real scene and, as a result, a complete 3D digital model of the sculpture is achieved and could be visualized through a VR viewer. As a case study, we focus on a Roman stone sculpture of a male lion conserved in the Museo Estense of Modena (Italy), which lacks of its head and its four legs. The original aspect of the lion may be achieved by integrating the damaged sculpture with other photogrammetric reconstructions of lions sculptures of the same period and with 3D model based on historical sources. Finally, the lion is visualized through an augmented reality application which digitally overlays the reconstructed models on the original one.

Keywords: Augmented Reality | Cultural heritage | Photogrammetry | Real-time visualization | Virtual modeling

Abstract: In the Industry 4.0 and digital revolution era, the world of manufacturing industry is experiencing an innovative reconfiguration of design tools and methodologies, with a different approach to the production processes organization. The design philosophy is changing, integrating to engineering contribution interpretative aspects (design thinking), executive practices (design doing) and cognitive aspects (design cultures). The design becomes human-centered. The new Virtual Reality technologies allow to validate performances of designed products and production processes by means of virtual prototypes in a virtual simulated environment. This approach generates several benefits to the companies, in terms of costs and time, and allows optimizing the assembly line design and related workplaces, by improving workers' benefits too. This paper proposes an innovative method to validate the design of workplaces on automotive assembly lines in a virtual environment, based on ergonomic approach, according to ERGO - Uas system, applied by FCA (Fiat Chrysler Automobiles) groups, that integrates UAS method for measurement and EAWS method for biomechanical effort evaluation. Creating 3D virtual scenarios allows to carry on assembly tasks by virtual manikins in order to be evaluated from different points of view. In particular, data coming from the simulation can be used to assess several ergonomic indexes, improving safety, quality and design. The analysis is supported by the use of a motion capture system, developed by the University of Campania and composed of wearable inertial sensors, that estimates the attitude of fundamental human segments, using sensor fusion algorithms based on Kalman filtering. In this way, it is possible to make a further design validation, assessing the EAWS index basing on posture angles trends evaluated. This method can represent an innovation for human-centered design of workplace in developing new products, reducing costs and improving job quality.

Keywords: design | ergonomics | manufacturing | motion capture | product feasibility | simulation | Virtual reality

Abstract: Underwater augmented reality is a very challenging task and amongst several issues, one of the most crucial aspects involves real-time tracking. Particles present in water combined with the uneven absorption of light decrease the visibility in the underwater environment. Dehazing methods are used in many areas to improve the quality of digital image data that is degraded by the influence of the environment. This paper describes the visibility conditions affecting underwater scenes and shows existing dehazing techniques that successfully improve the quality of underwater images. Four underwater dehazing methods are selected for evaluation of their capability of improving the success of square marker detection in underwater videos. Two reviewed methods represent approaches of image restoration: Multi-Scale Fusion, and Bright Channel Prior. Another two methods evaluated, the Automatic Color Enhancement and the Screened Poisson Equation, are methods of image enhancement. The evaluation uses diverse test data set to evaluate different environmental conditions. Results of the evaluation show an increased number of successful marker detections in videos pre-processed by dehazing algorithms and evaluate the performance of each compared method. The Screened Poisson method performs slightly better to other methods across various tested environments, while Bright Channel Prior and Automatic Color Enhancement shows similarly positive results.

Keywords: Augmented reality | Dehazing | Image restoration | Markers | Tracking | Underwater images

Abstract: Our research explores the potential of wearable Mixed Reality (MR) for people with Neuro-Developmental Disorders (NDD). The paper presents HoloLearn, a MR application designed in cooperation with NDD experts and implemented using HoloLens technology. The goal of HoloLearn is to help people with NDD learn how to perform simple everyday tasks in domestic environments and improve autonomy. An original feature of the system is the presence of a virtual assistant devoted to capture the user's attention and to give her/him hints during task execution in the MR environment. We performed an exploratory study involving 20 subjects with NDD to investigate the acceptability and usability of HoloLearn and its potential as a therapeutic tool. HoloLearn was well-accepted by the participants and the activities in the MR space were perceived as enjoyable, despite some usability problems associated to HoloLens interaction mechanism. More extensive and long term empirical research is needed to validate these early results, but our study suggests that HoloLearn could be adopted as a complement to more traditional interventions. Our work, and the lessons we learned, may help designers and developers of future MR applications devoted to people with NDD and to other people with similar needs.

Keywords: Augmented Reality | Holo Lens | Holograms | Mixed Reality | Neuro-developmental Disorders | Virtual assistant

Abstract: Nowadays digital substitutes of human beings (digital humans), capable of interacting with digital mock-ups in Virtual Reality, are widely used in many fields of engineering (e.g. ergonomics, product design, maintenance, and training). Nevertheless, the animation process of digital humans is still a time-consuming task, and its accuracy and reliability strongly depend on the experience and the skills of the operator. This paper presents an innovative algorithm capable of significantly speeding up the animation process of digital humans, allowing the operator to focus only on the so-called "task-related control points". This approach allows also to easily conduct biomechanical analyses. The algorithm has been tested with reference to several application scenarios in Virtual Reality.

Keywords: Algorithm | Digital humans | Kinematics | Virtual reality

Abstract: In fashion industry, 2D and 3D CAD systems to design garments already exist; however, some tasks of the process are neglected. We refer to made-to-measure garments and focus the attention on the first step of garment design, i.e. acquisition of customer’s measurement. In this paper we present an application based on mixed reality, named Tailor LABoratory (TLAB), which permits to take measures for clothing design as traditionally done by the tailor. TLAB has been developed using open source libraries (e.g., VTK and Blender) and low cost devices, such as Microsoft Kinect v2 to scan the human body, Oculus Rift v2 to create the 3D virtual reality and Leap Motion device to track hands motion. In particular, a virtual tape measure is made available to take measures interacting with the human avatar. To replicate the customer’s posture with her/his digital model, Blender has been adopted. It permits to manage body animations and automatic association of an animation to the 3D human avatar. Finally, preliminary tests are illustrated as well as results reached so far and future development.

Keywords: Leap Motion | mixed reality | Oculus Rift | tailor’s work | virtual garments design

Abstract: This paper presents an overview of main open source software, low-cost devices and related SDKs (Software Development Kits) that can be used to develop custom applications based on virtual and augmented reality. At present, the high modularity of the open source software for computer graphics allows developing custom applications with high quality for several research and industrial fields. To this end, we introduce a general-purpose software framework, which permits to manage the synchronization among the SDKs of different low-cost devices. Mentioned devices and software modules have been exploited to develop three applications in different fields.

Keywords: Low-cost devices | Open-source software | Virtual reality

Abstract: The current trend in manufacturing is to obtain a flexible work cell in which human and robot can safely interact and collaborate. Virtual Reality (VR) represents an effective tool capable of simulating such complex systems with a high level of immersion. In order to take advantage of VR technologies to study Human-Robot Cooperation (HRC), a digital model of a redundant manipulator (KUKA LBR iiwa) has been developed starting with kinematic modeling and then coupled with the real robot. This approach allows simulating HRC in several scenarios, to reproduce the safe behavior on the real robot, as well as to train operators.

Keywords: Human-robot collaboration | Modelling | Simulation | Virtual reality

Abstract: Analysis of human-related aspects is fundamental to guarantee workers’ wellbeing, which directly limits errors and risks during task execution, increases productivity, and reduces cost [1]. In this context, virtual prototypes and Digital Human Models (DHMs) can be used to simulate and optimize human performances in advance, before the creation of the real machine, plant or facility. The research defines a human-centred methodology and advanced Virtual Reality (VR) technologies to support the design of ergonomic workstations. The methodology considers both physical and cognitive ergonomics and defines a proper set of metrics to assess human factors. The advanced virtual immersive environment creates highly realistic and interactive simulations where human performance can be anticipated and assessed from the early design stages. Experimentation is carried out on an industrial case study in pipe industry.

Keywords: Digital Human Model | Ergonomics | Human-Centred Design | Sustainable Manufacturing | Virtual Reality

Abstract: Virtual museum (VM) systems are a very effective solution for the communication of cultural contents, thanks to their playful and educational approach. In fact, these appealing technological systems have demonstrated their usefulness and value in science centres and traditional museums all over the world, thanks to the fact that visitors can view digitized artworks and explore reconstructed historical places by means of VM-hosted installations. This paper presents a methodology, based on user studies, for the comparative evaluation of different design alternatives related to the user interaction with VM systems. The methodology has been validated by means of a testbed related to a VM system hosted at the “Museum of the Bruttians and the Sea” of Cetraro (Italy). The results of the user study demonstrate that this methodology can be effectively adopted in the development process of VM systems to optimize its outcomes in terms of usability and potential for entertainment and education.

Keywords: User study | User-centered design | Virtual museum systems | Virtual reality

Abstract: The paper presents the application of the technologies and methods defined in the VISAS project for the case study of the underwater archaeological site of Cala Minnola located in the island of Levanzo, in the archipelago of the Aegadian Islands (Sicily, Italy). The VISAS project (http://visas-project.eu) aims to improve the responsible and sustainable exploitation of the Underwater Cultural Heritage by means the development of new methods and technologies including an innovative virtual tour of the submerged archaeological sites. In particular, the paper describes the 3D reconstruction of the underwater archaeological site of Cala Minnola and focus on the development of the virtual scene for its visualization and exploitation. The virtual dive of the underwater archaeological site allows users to live a recreational and educational experience by receiving historical, archaeological and biological information about the submerged exhibits, the flora and fauna of the place.

Keywords: Underwater 3D reconstruction | Underwater cultural heritage | Virtual exploitation | Virtual reality

Abstract: The paper describes a user-centered design (UCD) approach that has been adopted in order to develop and build a virtual museum (VM) system for the “Museum of the Bruttians and the Sea” of Cetraro (Italy). The main goal of the system was to enrich the museum with a virtual exhibition able to make the visitors enjoy an immersive and attractive experience, allowing them to observe 3D archaeological finds, in their original context. The paper deals with several technical and technological issues commonly related to the design of virtual museum exhibits. The proposed solutions, based on an UCD approach, can be efficiently adopted as guidelines for the development of similar VM systems, especially when very low budget and little free space are unavoidable design requirements.

Keywords: Human-computer interaction | User interfaces design | User-centered design | Virtual museum systems

Abstract: This article explores what it takes to make interactive computer graphics and VR attractive as a promotional vehicle, from the points of view of tourism agencies and the tourists themselves. The authors exploited current VR and human-machine interface (HMI) technologies to develop an interactive, innovative, and attractive user experience called the Multisensory Apulia Touristic Experience (MATE). The MATE system implements a natural gesture-based interface and multisensory stimuli, including visuals, audio, smells, and climate effects.

Keywords: computer graphics | gesture controls | human-machine interface | multisensory virtual environment | natural user interfaces

Abstract: Books are the tools used for reading novels and stories, but also for educational purposes. Conventional books have undergone a radical transformation in recent years due to the use of new technologies. However, even today the technological devices used for reading e-books are still poorly exploited, despite the fact that they represent a fundamental tool to make the reading experience more immersive by using a complete multisensory approach. In this perspective, one sense that represents an important element of human perception is the sense of smell. Consequently, authors make the hypothesis that the introduction of odours during reading sessions could increase the user experience and the learning performances. In order to demonstrate these hypotheses, the authors have defined and carried out several experimental testing sessions. The analysis of the collected data proved that the introduction of odour does not disturb the reader during reading activities but, on the contrary, can actually make the experience more immersive. Similarly, odours do not disturb studying activities, but they can instead increase the level of concentration and people's learning performance.

Keywords: Augmented reality | Multisensory environment | Olfactory technologies | User experience

Abstract: This paper presents an immersive virtual reality system (IVRS) that has been designed for unilateral amputees in order to reduce the phantom limb pain (PLP). The patient's healthy limb is tracked by using a motion sensor. Data of the limb in motion are used as input parameters to move the phantom limb in the immersive virtual reality system. In this way, the patient has the illusion of moving the phantom limb while moving the real and contra-lateral limb. The system has been implemented by using low cost and open technologies, and combines the Oculus Rift SDK2 device, the LeapMotion device, a motion sensor, and an engine for interactive 3D content and gaming generation (Unity 3D). The Oculus Rift head mounted display is used to provide the immersive experience.

Abstract: Due to the Industry 4.0 initiative, Augmented Reality (AR) has started to be considered one of the most interesting technologies companies should invest in, especially to improve their maintenance services. Several technological limitations have prevented AR to become an effective industrial tool in the past. Now some of them have been overcome, some others not yet by off-the-shelf technologies. In this paper, we present a solution for remote maintenance based on off-the-shelf mobile and AR technologies. The architecture of the application allows us to remotely connect a skilled operator in a control room with an unskilled one located where the maintenance task has to be performed. This application, which has been initially described in a previous work, has been improved on the basis of feedback received by industrial partners. We describe the important features we have added and the rationale behind them to make the remote communication more effective.

Keywords: Augmented Reality | Industry 4.0 | Remote Maintenance

Abstract: This paper describes the beneficial impact of an augmented reality based technique on the 3D printing process monitoring within additive manufacturing machines. A marker is applied in a fixed point of the rapid prototyping machine, integral with the component being manufactured; as an alternative, a markerless approach can be followed too. A virtual model of the object to be printed is superimposed to the real one. In this way, the shape of the object in different printing stages can be viewed. An interactive comparison between real and virtual model can be carried out both in manual and automatic mode. If manufacturing errors are detected, the building process can be stopped. Augmented reality technique allows an intuitive shape check of a part being printed with rapid prototyping technologies. In case of complex objects it helps the operator in the detection of possible errors along the manufacturing process; stopping the machine as soon as an error appears avoids waste of machining time and material. The average precision of the augmented reality is useful to find significant geometrical errors; geometrical deviations less than 1 mm can hardly be assessed both in manual and in automatic mode, and further studies should be carried out to increase the technique precision and range of application. To the best of the authors’ knowledge it is the first time where experiments on the integration between augmented reality and rapid prototyping to interactively monitor 3D parts’ printing have been investigated and reported in literature.

Keywords: 3D printing | Additive Manufacturing | Augmented Reality | Design | Rapid Prototyping

Abstract: The mounting attention to environmental issues requires adopting better disassembly procedures at the product's End of Life. Planning and reckoning different disassembly strategies in the early stage of the design process can improve the development of sustainable products with an easy dismissing and recycling oriented approach. Nowadays many Computer Aided Process Planning software packages provide optimized assembly or disassembly sequences, but they are mainly based on a time and cost compression approach, neglecting the human factor. The environment we developed is based upon the integration of a CAD, an Augmented Reality tool, a Leap Motion Controller device, see-through glasses and an algorithm for disassembly strategies evaluation: this approach guarantees a more effective interaction with the 3D real and virtual assembly than an approach relying only on a CAD based disassembly sequence planning. In such a way, the operator may not test in a more natural and intuitive way automatic disassembly sequences, but he/she can also propose different strategies to improve the ergonomics. The methodology has been tested in a real case study to evaluate the strength points and criticalities of this approach.

Keywords: Augmented Reality | CAD | Design for Disassembly | Disassembly Sequence Optimization

Abstract: The study of the methodologies useful to support the assembly of parts is a challenging engineering task which can benefit of the most recent innovations in computer graphics and visualization technologies. This paper presents a proposal for an innovative methodology based on Virtual and Augmented Reality useful to support the components’ assembly. The herein introduced strategy is based upon a four stages procedure: at first the designer conceives the assembly sequence using a CAD system, visualizing the scene wearing an immersive Virtual Reality device. In the second stage, the same sequence is developed by an unexperienced user using the same equipment: the differences between two assembly sequences are recorded and exploited to detect critical points in the assembly sequence and to develop a Knowledge Based System. Finally, a virtual user manual is produced in Augmented Reality. When the final user uses the tool, the position of the object to assemble is detected by tracking the finger position of the user itself. A series of symbols and writings is added to the external scene to help the end-user in the assembly procedure. A test case based on the assembly of a scale model has been developed to evaluate the methodology. After an evaluation process, the procedure seems to be feasible and presents some advantages over the state-of-the-art methodologies proposed by literature.

Keywords: Assembly | Augmented Reality | Marker | Task automation | Virtual Reality

Abstract: HoloLens technology enables mixed reality experiences that integrate holographic objects into the real world in which the mixed reality head-mounted device is used. Since HoloLens' launch in March 2016, some mixed-reality applications of this technology have been announced or showcased, addressing different fields, including education, data visualization, tourism, entertainment, and professional training e.g., in medicine, architecture, manufacturing, and engineering. Still, a limited number of reported research provide examples of user experience designs and evaluations for applications using HoloLens. We are interested in the use of HoloLens as therapeutic tool for people with the Alzheimer's Disease. The paper describes a set of therapeutic activities that have been designed in cooperation with neurologists and aim at stimulating short term memory and spatial memory in this target group. We also report a preliminary study of the usability of these activities among the elderly subjects.

Keywords: Alzheimer's | Augmented reality | Cognitive training | Hologram | HoloLens | Memory | Mixed reality | Rehabilitation

Abstract: In this work, we proposed a method to reduce text in technical documentation, aiming at Augmented Reality manuals, where text must be reduced as much as possible. In fact, most of technical information is conveyed through other means such as CAD models, graphic signs, images, etc. The method classifies technical instructions into two categories: instructions that can be presented with graphic symbols and instructions that should be presented with text. It is based on the analysis of the action verbs used in the instruction, and makes use of ASD Simplified Technical English (STE) for remaining text instructions and let them easier to translate into other languages.

Keywords: Augmented Reality | Graphic symbols | Industry 4.0 | Simplified Technical English | Technical Documentation | Text reduction | Visual

Abstract: Text legibility in augmented reality with optical see-through displays can be challenging due to the interaction with the texture on the background. Literature presents several approaches to predict legibility of text superimposed over a specific image, but their validation with an AR display and with images taken from the industrial domain is scarce. In this work, we propose novel indices extracted from the background images, displayed on an LCD screen, and we compare them with those proposed in literature designing a specific user test. We collected the legibility user ratings by displaying white text over 13 industrial background images to 19 subjects using an optical see-through head-worn display. We found that most of the proposed indices have a significant correlation with user ratings. The main result of this work is that some of the novel indices proposed had a better correlation than those used before in the literature to predict legibility. Our results prove that industrial AR developers can effectively predict text legibility by simply running image analysis on the background image.

Abstract: This paper presents a structured User Centered Design (UCD) method to design and develop a highly usable smart home platform to manage the energy consumption of connected appliances. It exploits advanced Tangible Augmented Reality (TAR) technologies to virtually prototype the conceived design solutions and carry out usability testing with sample users. Usability tests are carried out both on traditional high fidelity prototypes and on an innovative Tangible Augmented Reality prototype. Experimental results prove the efficiency of the UCD approach supported by virtual prototypes, instead of traditional ones, the reliability of TAR prototypes to detect usability problems and assess user satisfaction, and its high interaction quality. Advantages obtainable by implementing the proposed structured UCD approach for web interface design, in the context of smart home, are discussed.

Keywords: Human Centered Computing | Human Computer Interaction | User Centered Design | User Interface Design | Virtual Prototyping | Virtual Reality

Abstract: Sustainable Manufacturing (SM) traditionally focused on optimization of environmental and economic aspects, by neglecting the human performance. However, the industrial plant's costs, productivity and process quality highly depend on the individual human performance (e.g., comfort perceived, physical and mental workload, simplicity of actions, personal satisfaction) and how much hazardous positions and uncomfortable tasks finally cost to the company. The present paper defines a human-centred virtual simulation environment to optimize physical ergonomics in workstation design and demonstrates its benefits on an industrial case study in pipe industry. The proposed environment aims at overcoming traditional approaches, where analysis are carried out at the shop-floor when the plant is already created, by providing a virtual environment to easily test and verify different design solutions to optimize physical, cognitive and organizational ergonomics.

Keywords: Digital Human Models (DHM) | Human-Centred Design (HCD) | Manufacturing Ergonomics | Virtual Reality | Workstation design

Abstract: In spite of the potentially harmful effects of vibrations on the human body, a new path was recently opened for the use of these mechanical means in the therapeutic field. The stimulation of proprioceptive and exteroceptive sensitivity is the main target in both peripheral (diabetes type 1 and type 2) and central (stroke, Parkinson's disease multiple sclerosis) nervous system disorders, particularly for the recovery and maintenance of functional state. By the way the response to the treatment is highly variable from subject to subject. Our experimental apparatus consists of a virtual reality system "LEAP Motion" which involves the patient in the execution of visuo-manual tasks in a virtual environment while receiving vibrotactile stimulation. We also used a modular 36 channels EEG system and a vibratory stimulation system able of delivering vibratory stimuli perpendicular and tangential to the body surface area.The study evaluation of motor performance and the ability to perform the tasks of visuomotor task assigned, in the presence and absence of vibratory stimulation and in real time, evoked potentials in the cortex.The vibration frequency extended from 5 to 200 Hz and with accelerations between 0.3G and 1,5G with displacement amplitude of about 0.5 mm applied on the affected limb hand. As the frequency, the amplitude and the direction of the vibration may vary we studied the relationship between the characteristics of the stimulus and the perception in the cerebral cortex, or other levels of the nervous system, studying potential models of elicitation of the somatosensory system. In this regard, our study took into account patients with Parkinson's disease and in particular evoked potentials N18 N20 N24 N30 particularly related to tactile stimulus, and indicative of the level of perception and processing in the brain of the Parkinson's patient.

Keywords: EEG | Evoked potentials | Local vibration | Occupational medicine | Parkinson disease | Virtual reality

Abstract: Medical devices play a crucial role in all diagnosis, care and treatment of patients and the number and variety of medical devices is huge and increasing annually. Healthcare professionals handle a wide range of devices every day. In this paper 3D Web-based interactive learning applications, which provide a helpful background, reminder and practical training to medical devices, have been developed. This article presents the production cycle of these interactive learning objects and the study conducted to measure the students’ perception of the objects’ effectiveness for learning. Three 3D Web-based interactive medical devices (sphygmomanometer, an electrocardiogram and a defibrillator) are described. Each of these applications introduces elements of interactivity with the learning content, and provides a practical training to the use of the medical devices. This e-learning package is aimed at all medical, nursing and allied healthcare professionals working in all healthcare settings and environments. It is also for all pharmacists in any environment and all community care workers in nursing and private healthcare settings. This training will also be useful to all staff working in a patient or patient supporting area, either someone new to the role or to a long standing professional as a reminder of best practice.

Keywords: 3D Web-based interactive medical devices | Interactive learning environments | Learning and training applications | Medical education | Virtual reality

Abstract: This paper presents a Mixed Reality environment, named Virtual Orthopedic LABoratory (VOLAB), which permits to emulate an orthopedic lab and design lower limb prosthesis, in particular, the socket component. The proposed solution is based on low cost devices (e.g., Microsoft Kinect) and open source libraries (e.g., OpenCL and VTK). In detail, the hardware architecture consists of three Microsoft Kinect v2, Oculus Rift for 3D environment visualization and Leap Motion device for hand/fingers tracking. The software development has been based on the modular structure of the prosthetic CAD system, named Socket Modelling Assistant (SMA) and modules have been developed to guarantee the communication among the devices and the performance. Finally, preliminary tests are illustrated as well as results reached so far and future development.

Keywords: Augmented Interaction | Human body devices | Low Cost Hand-Tracking devices | Mixed Reality | Prosthesis design | Socket Modelling Assistant

Abstract: This paper presents an application, named TLAB (i.e., Tailor’s LABoratory) to support virtual clothing design. In particular, it focuses the attention to the first step of garment design process, i.e., customers’ measure acquisition. TLAB is based on low-cost innovative technology (e.g., Oculus Rift SDK 2, Leap Motion device and Microsoft Kinect v2) to permit the interaction by hand and emulate the work traditionally done by tailors to manufacture a garment. In this paper, firstly we present the technology used for creating TLAB by describing hardware and software solutions adopted. Then, the design of a Natural User Interface (NUI) is depicted. The NUI design allows simplifying the interaction with hands through the use of the Leap Motion device as hand-tracking device. Finally, preliminary tests are discussed and conclusion presented.

Keywords: Augmented interaction | Garment design | Mixed reality

Abstract: The paper presents an outcome of the VISAS project (www.visas-project.eu) that concerns a virtual reality application for the exploitation of the underwater cultural heritage. The VR system takes advantage of novel 3D reconstruction techniques to provide geolocated and multi-resolution textured 3D models of underwater archaeological sites. Within the virtual underwater sites users live a recreational and educational experience by receiving historical, archaeological and biological information and contents about the submerged exhibits and structure of the site. Furthermore, the VR system allows divers to make a detailed planning of the operations and itinerary that will be later performed in the underwater environment.

Keywords: underwater 3D reconstruction | Underwater cultural heritage | VR systems

Abstract: The underwater cultural heritage is an immeasurable archaeological and historical resource with huge, but yet largely unexploited, potentials for the maritime and coastal tourism. In this regard, in the last years, national and international government authorities are supporting and strengthening research activities and development strategies, plans and policies to realize a more sustainable, responsible and accessible exploitation of the underwater cultural heritage. To this end, the paper presents the architecture of a new system that, taking advantage of the modern virtual and augmented reality technologies, allows diver and non-diver tourists to make a more engaging and educational experience of the underwater archaeological sites. This system has been developed and tested in the VISAS project (www.visasproject. eu) that aims to the enhancement of the cultural and tourist offer related to the underwater archaeology through innovation of modes of experience, both on site and remote, of the underwater environments of archaeological interest.

Keywords: AR systems | Underwater 3D reconstruction | Underwater cultural heritage | VR systems

Abstract: We present a novel interaction method for augmented industrial maintenance based on a "magic mirror" interface and virtual motion buttons. The system includes a video camera for object tracking, a video\depth camera for capturing user gestures, a projector for displaying technical instruction to the operator and a LCD monitor providing feedback of the virtual buttons. The operator can trigger maintenance commands by directional swift of the hands in regions sensitive to motion speed and direction. The main advantage of the presented interface is that it can work in realistic industrial conditions: (i) operators wearing gloves, (ii) operators handling tools, (iii) presence of moving machinery and personnel in the background. We measured the performances of the system with a laboratory test and we proved the feasibility with an automotive inspection test case. We calculated an average interaction time below 2 seconds and an error rate lower than 5%. However, we found some performances limitations if the operator is handling tools.

Keywords: Augmented reality | Computer assisted maintenance | Human computer interfaces | Motion buttons

Abstract: We present the design and a prototype of a projective AR workbench for an effective use of the AR in industrial applications, in particular for Manual Working Stations. The proposed solution consists of an aluminum structure that holds a projector and a camera that is intended to be mounted on manual working stations. The camera, using a tracking algorithm, computes in real time the position and orientation of the object while the projector displays the information always in the desired position. We also designed and implemented the data structure of a database for the managing of AR instructions, and we were able to access this information interactively from our application.

Abstract: This work evaluates the impact of augmented reality (AR) technology to support operators during manual industrial tasks. The work focuses specifically on monitor-based augmented reality as a solution to provide instructions to the operator. Instructions are superimposed directly onto a video representation of the physical workspace and are displayed on a standard monitor. In contrast to previous AR solutions proposed to support manual tasks, the current work is more industrially acceptable because it meets most of the industrial requirements and it is cost effective for a deployment in an industrial environment. The developed prototype is described and evaluated by means of a user study to compare the monitor-based augmented reality solution to provide instructions and the traditional method. The test shows that AR can be a valid substitute to support the operators during manual tasks because it allows them to be more time efficient and it reduces their mental workload compared to traditional instructional manuals.

Keywords: Augmented reality | Comparative test | Instructional media | Manual task

Abstract: Abstract: The product design process is based on a sequence of phases where the concept of the shape of a product is typically represented through a digital 3D model of the shape, and often also by means of a corresponding physical prototype. The digital model allows designers to perform the visual evaluation of the shape, while the physical model is used to better evaluate the aesthetic characteristics of the product, i.e. its dimension and proportions, by touching and interacting with it. Design and evaluation activities are typical cyclical, repeated as many times as needed in order to reach the optimal and desired shape. This reiteration leads to an increase of the development time and, consequently, of the overall product development cost. The aim of this research work is to develop a novel system for the simultaneous visual and tactile rendering of product shapes, thus allowing designers to both touch and see new product shapes already during the product conceptual development phase. The proposed system for visual and tactile shape rendering consists in a Tactile Display able to represent in the real environment the shape of a product, which can be explored naturally through free hand interaction. The device is designed in order to be portable, low cost, modular and high performing in terms of types of shapes that can be represented. The developed Tactile Display can be effectively used if integrated with an Augmented Reality system, which allows the rendering of the visual shape on top of the tactile haptic strip. This allows a simultaneous representation of visual and tactile properties of a shape. By using the Tactile Display in the initial conceptual phases of product design, the designers will be able to change the shape of a product according to the tactile evaluation, before the development of the physical prototype. This feature will lead to a decrease of the number of physical prototypes needed, thereby reducing, both cost and overall time of the product development process.

Keywords: augmented reality | shape rendering | Tactile display | virtual prototyping

Abstract: A novel Augmented Reality (AR) tool for structural health monitoring is illustrated in this work. It provides maintenance operators with the results of an impact detection methodology. It interacts with an eyepiece allowing the inspector to see the estimated impact position on the structure. Electric signals are collected by a network of piezosensors bonded on the structure to be monitored. Dispersive propagation compensation is performed to improve estimation robustness. Hyperbolic beamforming is exploited to locate the impact. Real-time impact data are finally fed to the AR eyepiece. The proposed approach is tested on a Cessna 150 engine cowling. Experimental results confirm the feasibility of the method and its exploitability in maintenance practice.

Keywords: Augmented reality | Guided waves | Structural health monitoring

Abstract: As a matter of fact, airports are considered as the bottleneck to increasing the capacity of the overall Air Traffic Management (ATM) system. While augmenting throughput in high performing airport operations, attention has rightly been placed on doing it in a safe manner. Many of the advances in airport operational safety come in the form of visualization tools for tower controllers. The increasing interest in Synthetic Vision (SV) and Augmented Reality (AR) technologies has led various analysts to positively esteem the adoption of new tools enabling both pilots and controllers to seamlessly operate under Visual Meteorological Conditions and Instrument Meteorological Conditions. This paper presents the motivations, the objectives, the proposed methodology and the expected impacts of the RETINA (Resilient Synthetic Vision for Advanced Control Tower Air Navigation Service Provision) project that has recently been granted by the SESAR (Single European Sky Air Traffic Management Research) Joint Undertaking. The two-years exploratory research project will investigate the potential and applicability of SV tools and Virtual/Augmented Reality (V/AR) display techniques for the Air Traffic Control (ATC) service provision by the airport control tower.

Keywords: Air traffic control | Augmented reality | Synthetic vision | Virtual reality

Abstract: The SESAR (Single European Sky Air Traffic Management Research) Joint Undertaking has recently granted the Resilient Synthetic Vision for Advanced Control Tower Air Navigation Service Provision project within the framework of the H2020 research on High Performing Airport Operations. Hereafter, we describe the project motivations, the objectives, the proposed methodology and the expected impacts, i.e. the consequences of using virtual/augmented reality technologies in the control tower.

Keywords: Air Traffic Control | Airport tower | Synthetic vision | Virtual/augmented Reality

Abstract: The purpose of the air traffic management system is to accomplish the safe and efficient flow of air traffic. However, the primary goals of safety and efficiency are to some extent conflicting. In fact, to deliver a greater level of safety, separation between aircrafts would have to be greater than it currently is, but this would negatively impact the efficiency. In an attempt to avoid the trade-off between these goals, the long-range vision for the Single European Sky includes objectives for operating as safely and efficiently in Visual Meteorological Conditions as in Instrument Meteorological Conditions. In this respect, a wide set of virtual/augmented reality tools has been developed and effectively used in both civil and military aviation for piloting and training purposes (e.g., Head-Up Displays, Enhanced Vision Systems, Synthetic Vision Systems, Combined Vision Systems, etc.). These concepts could be transferred to air traffic control with a relatively low effort and substantial benefits for controllers’ situation awareness. Therefore, this study focuses on the see-through, head-tracked, head-up display that may help controllers dealing with zero/low visibility conditions and increased traffic density at the airport. However, there are several open issues associated with the use of this technology. One is the difficulty of obtaining a constant overlap between the scene-linked symbols and the background view based on the user’s viewpoint, which is known as ‘registration’. Another one is the presence of multiple, arbitrary oriented Head-Up Displays (HUDs) in the control tower, which further complicates the generation of the Augmented Reality (AR) content. In this paper, we propose a modified rendering pipeline for a HUD system that can be made out of several, arbitrary oriented, head-tracked, AR displays. Our algorithm is capable of generating a constant and coherent overplay between the AR layer and the outside view from the control tower. However a 3D model of the airport and the airport’s surroundings is needed, which must be populated with all the necessary AR overlays (both static and dynamic). We plan to use this concept as a basis for further research in the field of see-through HUDs for the control tower.

Keywords: Air traffic control tower | Augmented reality | Head-up display

Abstract: In this work a study about the application of the augmented reality in steelmaking area is presented. Augmented reality systems add virtual computer-generated material to the surrounding physical world. The augmented reality systems use see-through headworn displays to overlay graphics and sounds on a person's natural vision and hearing. As the person moves about, the position and orientation of the head is tracked, allowing the overlaid material to remain tied to the physical world. An application and description of prototypes to the training and maintenance of continuous casting machine is described.

Keywords: Augmented reality | Ironmaking | Maintenance | Prototypes | Steelmaking

Abstract: In this paper, we propose a forearm rehabilitation system based on a serious game in Augmented Reality (AR). We designed and developed a simplified AR arcade brick breaking game to induce rehabilitation of the forearm muscles. We record the electromyographic signals using a low cost device to evaluate the applied force. We collected and analysed data in order to find a relationship between the applied force and the difficulty of the game. This research focuses on the dehospitalization of subjects in the middle or final stages of their rehabilitation where the new technologies, like Virtual and Augmented Reality, may improve the experience of repetitive exercises. The results achieved prove that the force applied by the user to hit the virtual sphere with real cardboard cube is related to sphere speed. In a rehabilitation scenario the results could be used to evaluate the improvements analysing the performance history.

Keywords: Brick serious games | Rehabilitation | Virtual and Augmented Reality

Abstract: This paper presents a novel application of natural interaction techniques for technical documentation navigation in augmented reality maintenance. The objective is to provide technical information where and when needed in order to reduce the time spent in retrieving procedural data from manuals. The system integrates a video and depth camera for user gesture acquisition, a webcam for marker-based tracking and a see through head mounted display for digital information overlay. We designed a novel hand free gesture-based interface to access technical documentation while performing maintenance tasks. We implemented a specific flexible system to prototype natural interfaces using UML configurable state machines. It supports 2D and 3D content navigation, CAD assembly explosion, real time 3D sectioning and operation browsing. We differentiate the interaction metaphors for 2D documents and 3D CAD models in order to be intuitive and easy to learn. A case study and a user test demonstrated the feasibility and the good acceptance of the presented gesture interface.

Keywords: 3D hand gestures | Augmented reality | Digital technical documentation | Industrial maintenance | Natural interface

Abstract: Augmented Reality (AR) technology is becoming more available for everyday applications, providing advanced information about the observed objects. In this paper we present a system combining optical tracking and augmented reality with applications in local large areas. We use two different tracking technologies for positioning objects: the infrared marker-based motion capture system OptiTrack and the Vuforia software platform, which enables augmented reality app across the real world environment. We present a robust solution for communication between a client (mobile platform) and a server that computes all global tracking data. This technology has high potential for end users and can be of great benefit.

Keywords: Augmented reality | Client-server architecture | Mobile device | Network streaming | Tracking systems

Abstract: Numerous companies all around the world are shifting from traditional products to product-service solutions, thanks to the increased 'intelligence' and ' connectivity' of modern products and the more deep integration among mechanics, electronics, Information and Commutation Technologies (ICT) and Internet of Things (IoT). Such Product-Service Systems (PSSs) are usually designed and developed by considering product and service as separated entities with the consequent increase of design and validation difficulties. In addition, a final physical prototype has to be realized to validate the overall solution. In this context, Virtual Prototyping can support PSS design to reduce process iterations and time to market. However, actual virtual prototypes are usually conceived for product validation, and are not so effective for PSS. The paper defines a set of requirements for PSS simulation on digital models, and defines a set of tools for successful PSS prototyping.

Keywords: Model-in-the-loop | Product-Service System | Virtual Prototyping | Virtual Reality

Abstract: A Product-Service System (PSS) is an innovation strategy, shifting the business focus from designing physical products only, to designing a system that combines tangible products, intangible services, supporting network and infrastructure, which are jointly capable of fulfilling specific customer needs. Due to the widespread of this paradigm, the present research provides a methodological framework and related tools to support the design of PSSs. The aim is to propose a user-centered approach to involve end-users during the different stages of PSS development.

Keywords: Hardware-in-the-Loop | Product-Service System | User-Centered Design | Virtual reality

Abstract: The need for more flexible tools and reduction of time and cost has led to the implementation of augmented reality (AR) and virtual reality (VR) techniques in the product design and development process. Those techniques have already been used in the conceptual, manufacturing and assembly stages of product design instead of or in extension of the physical prototypes. Such virtual applications have demonstrated superior performance in assembly process design and evaluation of activities that present information about different assembly states in real time, thanks to their flexibility in manipulating and creating new working scenarios. Here, the development of an AR application, called PoliART, aimed at the visual evaluation of assembly sequences at early stages of design is presented. At an industrial level this allows collaborative work between designers and manufacturing engineers from the very beginning in order to consider assembly devices, times and resources, with a short implementation time and reduced costs.

Keywords: ASA | Assembly process | Assembly sequence analysis | Augmented reality | Authoring tool | Computer aided engineering | Evaluation tool

Abstract: Today smells are used for communicating information about products as household cleaners and food. However, smells can be also applied to any kind of products. Several researches have focused on integrating smells in virtual environments. The research questions addressed in this work concern whether Virtual Prototypes, including the sense of smell, can be used for evaluating products as effectively as studies performed in real environments, and whether smells can increase the users' sense of presence in the virtual environment. For this purpose, an experimental framework including a wearable olfactory display has been developed, and experimental tests have been performed.

Keywords: Olfactory Display | Presence | Virtual Reality | Wearable Device

Abstract: ABSTRACT: The paper describes an interactive Finite Elements Analysis (FEA) tool that aims to improve the learning of mechanical behavior of materials in industrial engineering schools. We implemented a “user in the loop” approach where students can explore the mechanical behavior of virtual specimens selected from a library of standard elements (cantilever beam, IPE beams etc.). The users can apply forces or displacements interactively by mouse or haptic device, and visualize and “feel” the structures stress configurations. We extended our previous work and compared this novel approach with respect to traditional FEA learning techniques. A test with twenty engineering students showed that learners following the interactive approach are faster in completing the given assignment showing a reduced error rate.

Keywords: engineering education | haptics | real-time finite element analysis | virtual reality

Abstract: ABSTRACT: This paper proposes a new interactive Augmented Reality (AR) system, which has been conceived to allow a user to freely interact with virtual objects integrated in a real environment without the need to wear cumbersome equipment. The AR system has been developed by integrating the Fog Screen display technology, stereoscopic visualization and the Microsoft Kinect. The user can select and manage the position of virtual objects visualized on the Fog Screen display by using directly his/her hands. A specific software application has been developed to perform evaluation testing sessions with users. The aim of the testing sessions was to verify the influence of issues related to tracking, visualization and interaction modalities on the overall usability of the AR ‘system. The collected experimental results demonstrate the easiness of use and the effectiveness of the new interactive AR system and highlight the features preferred by the users.

Keywords: augmented reality | design review | gesture-based interface

Abstract: Although 3D models are useful to preserve the information about historical artefacts, the potential of these digital contents are not fully accomplished until they are not used to interactively communicate their significance to non-specialists. Starting from this consideration, a new way to provide museum visitors with more information was investigated. The research is aimed at valorising and making more accessible the Egyptian funeral objects exhibited in the Sforza Castle in Milan. The results of the research will be used for the renewal of the current exhibition, at the Archaeological Museum in Milan, by making it more attractive. A 3D virtual interactive scenario regarding the "path of the dead", an important ritual in ancient Egypt, was realized to augment the experience and the comprehension of the public through interactivity. Four important artefacts were considered for this scope: Two ushabty, a wooden sarcophagus and a heart scarab. The scenario was realized by integrating low-cost Virtual Reality technologies, as the Oculus Rift DK2 and the Leap Motion controller, and implementing a specific software by using Unity. The 3D models were implemented by adding responsive points of interest in relation to important symbols or features of the artefact. This allows highlighting single parts of the artefact in order to better identify the hieroglyphs and provide their translation. The paper describes the process for optimizing the 3D models, the implementation of the interactive scenario and the results of some test that have been carried out in the lab.

Keywords: 3D modelling | Cultural Heritage | Leap Motion | Oculus Rift | Unity | Virtual Reality | Visualisation

Abstract: Traditional User/Maintenance Manuals provide useful information when dealing with simple machines. However, when dealing with complex systems of systems and highly miniaturized technologies, like UAVs, or with machines with millions of parts, a commercial aircraft is a case in point, new technologies taking advantage of Augmented Reality can rapidly and effectively support the maintenance operations. This paper presents a User/Maintenance Manual based on Augmented Reality to help the operator in the detection of parts and in the sequence to be followed to assemble/disassemble systems and subsystems. The proposed system includes a handheld device and/or an head mounted display or special goggles, to be used by on-site operators, with software management providing data fusion and overlaying traditional 2D user/maintenance manual information with an augmented reality software and appropriate interface. This device is connected by internet to a maintenance centre located in the aircraft manufacturer facilities. The on-site operator can directly access to multimedia content and historical information and can be helped or guided remotely by expert engineers residing at the manufacturer company offices. This resource may exploit Computer Aided Design and Product Data Management PDM remote facilities to prepare additional and specific 3D graphic content, supported also by a video and audio streaming from the camera and microphone of the on-site operator's handheld device. The proposed solution has revealed a number of significant advantages compared to the currently used operations: there is no need for preparing animations and graphic content for all the required maintenance sequences. The expert engineers and designers can both be involved directly in the maintenance tasks, a useful mean of feedback to evaluate the design for further projects or for project improvement. Additionally, the sensitive data is not shared outside the company since data is transmitted for visual display but it is stored on a secured location.

Abstract: In this work we propose an approach based on IT tools to improve all the clinical activities related to Abdominal Aortic Aneurism (AAA) detection and treatment. In particular, the approach is focused on a 3D CAD model of the AAA and CFD simulations to improve diagnosis by evaluation of rupture risk indicators and Spatial Augmented Reality (SAR) to simulate endovascular repair (EVAR) of the pathology. Geometric model of the vascular wall is build from Computer Tomography (CT) data by using and customizing algorithms and tools implemented in Vascular Modeling ToolKit (VMTK) software library. Four methodologies of geometry initialization are compared and we choose those able to describe the vascular disease excluding any foreign tissue (i.e., bones, internal organs, and muscles). An evaluation of the problems connected to the relative parameters of reconstruction, their influence for the correct geometry representation, focusing the attention on segmentation level and the smoothing of the surface, are presented. In particular, we point out the effect of the smoothing by the use of the Hausdorff distance. We propose a standardized process able to guide users in the modeling of this type of vessels; moreover, SAR can dramatically improve the efficacy of AAA visualization for some different clinical aspects. The 3D geometry of the AAA can be used to carry out CFD analysis, to calculate parameters of the blood flow and evaluate the rupture risk indicator, like Oscillatory Shear Index (OSI). The availability of risk indicators facilitates the physician in the diagnosis and prognosis. Finally, we present a methodology and tools to simulate EVAR on a specific patient based on geometry reconstruction and CFD simulations; this type of procedure can effectively improve training and facilitate endovascular surgery.

Keywords: Abdominal aortic aneurysm | CFD | Computer-aided diagnosis | Spatial augmented reality

Abstract: The present work describes the interactive prototype and the preliminary evaluation results of a tool dedicated to the light General Aviation pilot’s community. The tool’s interface has been developed through an Android tablet application and aims at supporting the pilots in the task of staying “well-clear” from the surrounding traffic by presenting them the long-term prediction of the flights. The initial results and the approach of a heuristic evaluation conducted with five experts coming from the fields of user-experience, aviation and automotive are discussed along with the improvements in the design of the user-interface focusing on the trajectory depictions.

Keywords: Heuristic evaluation | Light general aviation | Trajectory prediction visualization | User-interface design

Abstract: Augmented Reality is a promising technology for the product lifecycle development, but it is still not established in industrial facilities. The most relevant issues to be addressed relate to the ergonomics: avoid the discomfort of Head-Worn Displays, allow the operators to have free hands and improve data visualization. In this work we study the possibility to use projection-based Augmented Reality (projected AR), as optimal solution for technical visualization on industrial workbenches. In particular, text legibility in projected AR is difficult to optimize since it is affected by many parameters: environment conditions, text style, material and shape of the target surface. This problem is poorly addressed in literature and in the specific industrial field. We analyze the legibility of a set of colors prescribed by international standards for the industrial environments, on six widely used industrial workbenches surfaces. We compared the performance of 14 subjects using projected AR, with that using a traditional LCD monitor. We collected about 2500 measurements (times and errors) through the use of a test application, followed by qualitative interviews. The results showed that, as regards legibility, projected AR can be used in place of traditional monitors in most of the cases. Another not trivial finding is that the influence on legibility of surface irregularities (e.g., grooves, prominences) is more important than that of surface texturization. A possible limitation for the use of projected AR is given by the blue color, whose performance turned out to be lower than that of other colors with every workbench surface.

Keywords: Industrial applications | Spatial Augmented Reality | Text legibility | Visualization

Abstract: Text readability with augmented reality head-worn displays is critical and at present time, there are no standard guidelines to follow. The readability depends mainly on background lighting, display technology (i.e., OST: optical see-through or VST: video see-through), and text style (e.g., plain text, outline or billboard). In this work, we addressed the readability limits for industrial activities. We experimented the effects of two background illuminances levels (1,000 lx for very fine basic industrial tasks and 4,000 lx for fine machining), two commercially available head-worn display technologies, variable outline widths and contrast polarity of text. We analyzed the performance of 12 subjects by collecting about 3,400 measurements using a specific test application and followed by qualitative interviews. With high illuminances, VST performed better than OST, regardless of contrast polarity and outline width. We found that negative contrast polarity is preferable with VST, and that just a minimum outline (1 px) around black text is optimal. On the contrary, positive contrast polarity should be used with OST and outline is not effective. Therefore, we evaluated the usage limits of the OST by sampling its contrast sensitivity function.

Keywords: Augmented reality | Contrast sensitivity function | Industrial lighting | Optical see-through | Video see-through

Abstract: In the Industrie 4.0 vision, the creation of leading-edge options for interaction between people and technology occupies a key role. In this context, augmented reality (AR) is one of the most suitable solutions. However, it is still not ready to be effectively used in industry. A crucial problem is the legibility of text seen through AR head-worn displays (HWDs). AR interface designers have no standard guidelines to follow, especially for these devices. Literature and anecdotal evidence suggest that legibility depends mainly on background, display technology (that is, see-through optical or video HWDs), and text style (for example, plain text, outline, or billboard). Furthermore, there are constraints to consider in industrial environments, such as standard color-coding practices and workplace lighting. The authors examine aspects affecting text legibility with an emphasis on deriving guidelines to support AR interface designers. Their results suggest that enhancing text contrast via software, along with using the outline or billboard style, is an effective practice to improve legibility in many situations. If one text style is needed for both types of HWD, their results suggest that colored billboards (with neutral white text) are effective. When color coding is not mandatory, white text and blue billboard are more effective than other styles tested.

Keywords: augmented reality | computer graphics | head-worn displays | human-computer interface | Industrie 4.0 | mixed reality | style guides | usability engineering | vision I/O

Abstract: A remote control station for Unmanned Aerial Vehicles (UAV) based on oculus Rift-style headsets and joysticks is proposed in this paper. With this solution situation awareness and distraction can be controlled and measured during the flight. With Virtual Augmented Reality (VAR) software it is possible reproduce accurately both the cockpit and the external view thanks to the helmet tracking system. Also the head-up display (HUD) and up-to-date flight instruments can be reproduced. In this way the PF (Pilot Flyng) station can be reduced to helmet, throttle/stick joysticks with force feedback and a few additional LCDs. Another main advantage of VAR headsets is the possibility of reconfiguring the cockpit via software and to use it for several different UAVs. In Figure-5 it is possible to see a logical schema of a VAR station: the pilot inputs via helmets (line of sight direction), flight controls (stick and throttle) and switches on joysticks the data in the AVCS software (Aircraft Visualization and Control System): The Aircraft Visualization and Control System take the data from the aerial vehicles, elaborates them and outputs the external view (external visual system) and the view of instruments (instrument visualization system). These two "images" are overlapped and mixed in a highly hierarchical visualization system, where only the relevant objects are depicted. To do so the external camera images from the aerial vehicle are analyzed and cleaned of all non-relevant data. The data from the sensors are also to be included in the synthesizing process. The application of these ideas as discussed in this paper consists of the realisation of a VAR display system for a remotely piloted aerial vehicle. All the instruments are modelled via Head Up Display (HUD) while the external scenery is analyzed and only relevant elements for mission accomplishment or collision avoidance are represented. The PF have the possibility of a 360° field of view. Sound realism and true situation awareness can be then achieved. Software for distraction control and situation awareness can be easily implemented in the system. A synthetic audio interrogation system can keep track of the current state of alert of the PF.

Keywords: Augmented reality | Remote control | UAV

Abstract: In the last years introducing measures to face age discrimination and increasing work safety in production environments have become crucial goals. The present research proposes an innovative user interface exploiting Augmented Reality techniques to support frail people, mainly elderly, in everyday work on complex automated machines. It adapts its functionalities according to the user skill, tasks, age, and cognitive and physical abilities thanks to a set of knowledge-based configuration rules. A case study is described to illustrate the methodology to manage the complexity of configuration rules and the resulting developed platform.

Keywords: Accessibility | Adaptation | Augmented reality | Human-computer interaction | User-centred design

Abstract: The Visuo-Haptic Mixed Reality (VHMR) is a branch of the Mixed Reality (MR) that is acquiring more and more interest in the recent years. Its success is due to the ability of merging visual and tactile perceptions of both virtual and real objects with a collocated approach. Like any emerging technology, the development of the VHMR systems is accompanied by challenges that, in this case, deals with the efforts to enhance the multi-modal human perception with the user-computer interface and interaction devices at the moment available. This paper deals with two of the typical problems related to VHMR systems, that are device obtrusion and tool-hand misalignment, and suggests solutions whose effectiveness has been tested by means of user studies. First, the paper analyzes the obtrusion problem and the benefits that users may gain performing task in a mixed environment with unobstructed haptic feedback, performed by means of a novel technique. Secondly, it investigates the effects of tool-hand misalignment on user perception and verifies the efficacy of a proposed misalignment correction technique by means of a comparative user test. Experimental results show that users would benefit from using the proposed unobstructed visuo-haptic approach and the misalignment compensation technique. These enhancements demonstrate the efficacy of the proposed solutions and at the same time get stronger the awareness that obtrusion and misalignment problems are fundamental issues to take into account for producing a realistic perception of a visuo-haptic mixed environment. © 2014 Elsevier Ltd.

Keywords: Haptic interaction | Occlusion handling | User evaluation | Visuo-Haptic Mixed Reality

Abstract: In this work, a method for monitoring fatigue crack growth in a metal to composite bonded joint based on the strain field is proposed and applied in the framework of a visualization tool based on Augmented Reality (AR). This tool superimposes some virtual objects, which are the data acquired by the sensors and the crack length, directly on top of the specimen under inspection and in real time. By finite element (FE) analyses, a good correlation between the crack tip position and the strain field in a single lap specimen is found and this feature is exploited to monitor the crack length during fatigue tests and to feed the AR system to virtually visualize the crack on the real specimen. An array of electrical resistance strain gauges is bonded to the surface of one adherend. A Matlab function collects values from the strain gauges mounted on the specimen under investigation analyses them on the basis of the FE analysis and finally feeds the AR system. The validation of this process is done by measuring the crack by optical microscope. This procedure is also tested with the use of Fiber Bragg Gratings (FBG) optical strain gauges. Copyright © 2014 Taylor & Francis Group, LLC.

Keywords: Aluminum and alloys | Augmented reality | Composites | Fatigue | Fiber Bragg Grating

Abstract: This paper describes a system that combines haptic, virtual reality and game technologies in order to assist repetitive performances of manual tasks to patients, which are recovering from neurological motor deficits. These users are able to feel virtual objects by using a haptic device, which acts as a virtual guide taking advantages of its force feedback capabilities. A virtual environment is used forming a haptic interface between the patient and the game. The haptic device is driven under the users movements and assisted through the Magnetic Geometry Effect (MGE). Preliminary evaluation has been performed in order to validate the system in which two different tasks have been performed (throw down bricks in an hexagonal tower without and with haptic assistance) with the aim to obtain more information related to the accuracy of the device. © 2014 Springer International Publishing.

Keywords: Gaming | Haptic interface | Post-stroke Rehabilitation | Virtual Reality

Abstract: The research activity presented in this paper aim at extending the traditional planar navigation, which is adopted by many desktop applications for searching information, to an experience in a Virtual Reality (VR) environment. In particular, the work proposes a system that allows the user to navigate in virtual environments, in which the objects are spatially organized and sorted. The visualization of virtual object has been designed and an interaction method, based on gestures, has been proposed to trigger the navigation in the environment. The article describes the design and the development of the system, by starting from some considerations about the intuitiveness and naturalness required for a three-dimensional navigation. In addition, an initial case study has been carried out and consists in using the system in a virtual 3D catalogue of furniture. © 2014 Springer International Publishing.

Keywords: Gestures | Natural User Interfaces | Navigation | Virtual Catalogue | Virtual Reality

Abstract: The paper proposes a framework for supporting maintenance services in industrial environments through the use of a mobile device and Augmented Reality (AR) technologies. 3D visual instructions about the task to carry out are represented in the real world by means of AR and they are visible through the mobile device. In addition to the solutions proposed so far, the framework introduces the possibility to monitor the operator's work from a remote location. The mobile device stores information for each maintenance step that has been completed and it makes them available on a remote database. Supervisors can consequently check the maintenance activity from a remote PC at any time. The paper presents also a prototype system, developed according to the framework, and an initial case study in the field of food industry. © 2014 Springer International Publishing.

Keywords: Augmented Reality | Framework | Maintenance tasks | Remote Supervision

Abstract: This paper presents a novel transportable and cost-effective Augmented Reality system developed to support interior design activities in the contract design sector. The main functioning principles and technical information about its implementation are provided to show how this system allows overcoming some of the issues related to the use of the Augmented Reality in interior design activities. The effectiveness of this system is verified through two different testing sessions based on a case study, which relates to the contract design sector. The testing sessions involved many interior designers with the intent of demonstrating the possibility of integrating this Augmented Reality system in the "everyday" interior design practice in the specific context of the contract design. © 2014 CAD Solutions, LLC.

Keywords: augmented reality | interior design | virtual prototyping

Abstract: This paper illustrates a Human-Machine Interface based on Augmented Reality (AR) conceived to provide to maintenance operators the results of an impact detection methodology. In particular, the implemented tool dynamically interacts with a head portable visualization device allowing the inspector to see the estimated impact position on the structure. The impact detection methodology combines the signals collected by a network of piezosensors bonded on the structure to be monitored. Then a signal processing algorithm is applied to compensate for dispersion the acquired guided waves. The compensated waveforms yield to a robust estimation of guided waves difference in distance of propagation (DDOP), used to feed hyperbolic algorithms for impact location determination. The output of the impact methodology is passed to an AR visualization technology that is meant to support the inspector during the on-field inspection/diagnosis as well as the maintenance operations. The inspector, in fact, can see interactively in real time the impact data directly on the surface of the structure. Here the proposed approach is tested on the engine cowling of a Cessna 150 general aviation airplane. Preliminary results confirm the feasibility of the method and its exploitability in maintenance practice. © 2014 SPIE.

Keywords: Aircraft Maintenance | Augmented Reality | Guided waves | Impact localization | Nondestructive Testing | Warped Frequency Transform

Abstract: Cabin interiors design can be considered one of the key enablers to face the societal challenges of future air transport. The cabin is the interface between the aircraft and the passenger and it strongly impacts on the user's experience. The quality of the travelling depends on the quality of the cabin. It could be measured through several and different dimensions, such as passenger's efficiency during and after the flight and the level of comfort that he/she experiences. Therefore, reading or working while flying or feeling restored when leaving the cabin should be the key performance indicators of a comfortable cabin. In this paper we propose an innovative design tool, that is an objective method to measure the affective impact that interior design provokes to passengers, with particular regards to seat design. Such method is based on the use of large virtual environments, coupled with BCIs (Brain Computer Interfaces). We propose a novel and original framework to correlate the BCIs affective scales to a set of comfort indicators. Experimental results are presented and discussed.

Keywords: Aircraft cabin layout | Aircraft design | Brain computer interfaces | Virtual reality

Abstract: We present an empirical study that evaluates the effectiveness of technical maintenance assisted with interactive augmented reality instructions. Our approach consists in an augmented visualization on a large screen and a combination of multiple fixed and mobile cameras. We used commercially available solutions. In our test, 14 participants completed a set of 4 maintenance tasks based on manual inspections of a motorbike engine. Tool selection, removal of bolts, and part dis\assembly, are supported by visual labels, 3D virtual models and 3D animations. All participants executed similar operations in two modalities: paper manuals and augmented instructions. Statistical analyses proved that augmented instructions reduced significantly participants' overall execution time and error rate. © 2013 Elsevier B.V. All rights reserved.

Keywords: Augmented reality | Computer aided task guidance | Large screen instruction | Maintenance

Abstract: Efficient text visualization in head-worn augmented reality (AR) displays is critical because it is sensitive to display technology, text style and color, ambient illumination and so on. The main problem for the developer is to know the optimal text style for the specific display and for applications where color coding must be strictly followed because it is regulated by laws or internal practices. In this work, we experimented the effects on readability of two head-worn devices (optical and video see-through), two backgrounds (light and dark), five colors (white, black, red, green, and blue), and two text styles (plain text and billboarded text). Font type and size were kept constant. We measured the performance of 15 subjects by collecting about 5,000 measurements using a specific test application and followed by qualitative interviews. Readability turned out to be quicker on the optical see-through device. For the video see-through device, background affects readability only in case of text without billboard. Finally, our tests suggest that a good combination for indoor augmented reality applications, regardless of device and background, could be white text and blue billboard, while a mandatory color should be displayed as billboard with a white text message. © 1995-2012 IEEE.

Keywords: Augmented reality | optical see-through | style guides | video see-through | vision I/O

Abstract: Quality of life of various types of people can strongly benefit of a design process developed to take into account needs and requirements of end users. In this context the paper present a study on the cognitive and physical abilities of elderly persons, to design a friendly kitchen, that is considered one of the most complex home environment for the provided functionalities and involved human capabilities. A robust inclusive design approach is conceived to make simple and intuitive the interaction between humans and the systems installed in the kitchen environment. An investigation of virtual prototyping techniques is proposed to find the best Virtual Reality system to create a living lab to involve elderly in user-based assessment.

Keywords: home environment | mixed reality | user-centred-design | virtual prototyping

Abstract: Modern airliners are really UAV (Unmanned Aerial Vehicle) that fly with a very limited contribution from the pilots. UAV modern technology is ready to turn to full automatic passenger transportation. The barrier is purely physiological and not technical. This technology is available since a few years. However, the main critical safety issue is still emergency. When everything seems to go wrong, manual control may be the last resource. The emergency, back up, pilot can be comfortably seated several thousands of miles away. His awareness about aircraft situation may be null. He has to grasp immediately the situation and take a proper corrective action. Time is the essential factor. Still, the remote pilot may have not logged many active flying hours. Now, in everyday flight, the normal configuration is AF (Autopilot Flying) + PNF1 (Pilot Not Flying 1) + PM/PNF2 (Pilot Monitoring/Pilot Not Flying 2). The pilot(s) still log(s) these hours as active flying hours, even if, for most of the time, it is the autopilot that is in charge for flying. The pilot training problem is now very serious. Pilots, remote or not, are interfaced to the aircraft system through MFD (Multifunctional Flight Displays): these are Computer Graphic Displays with or without side buttons. What it is seen by the pilot is the GUI (Graphic User Interface): it should then be easy and familiar to the pilots especially when they are called to work. GUI modern WYSIWYG (What You See Is What You Get) Augmented Reality and hierarchical techniques should be used, the more familiar to the average person the better. Basic manual training should be provided to pilots to make them ready to handle critical situations. The actual training based on normal automatic procedures and on air traffic handling is now obsolete. In fact, automatic systems and air traffic can be easily handled even without any human intervention. This fact had been demonstrated by the famous US air traffic controller strike back in the Reagan era (1981). In this paper, several issues about the interface are discussed. An additional, non-certified, graphic interface focused on non-standard or emergency condition flight is proposed in this paper. This augmented reality interface is designed for a smooth transition between automatic and human controlled flight, being human control extraordinary for nowadays airliners.

Keywords: Air transportation safety | Augmented reality | Interface systems

Abstract: The present paper describes the development of a modular and easily configurable simulation platform for ground vehicles. This platform should be usable for the implementation of driving simulators employed both in training purposes and in vehicle components testing. In particular, the paper presents a first architectural model for the implementation of a simulation platform based on the Functional Digital Mock-Up approach. This platform will allow engineers to implement different kinds of simulators that integrate both physical and virtual components, thus achieving the possibility to quickly reconfigure the architecture depending on the hardware and software used and on specific test case needs. The platform has been tested by developing a case study that integrates a motion platform, some I/O devices and a simple dynamic ground vehicle model implemented in OpenModelica. © 2013 Springer-Verlag France.

Keywords: Driving simulator | Functional Digital Mock-up | OpenModelica | Real time simulation | Virtual reality

Abstract: The ISO 9001: 2000 requires the phase of design validation in order to demonstrate that the design output are able to satisfy specified or forecasted user requirements; for this reason, generally, physical prototype are realized in order to evaluate real product performances and their correlation with simulated ones. In this paper, the use of virtual inspection probes to validate playground equipment design is studied, pointing out limits and defining the optimal test strategies. Through the use of Robust Design techniques the authors show that the combined use of real time shadows and positional sound feedback allows to reduce the percentage of wrong inspections in the validation phase of playground equipment design. The authors develop an inspection probe simulation tool in virtual immersive environment. In particular, this tool ensures the collision detection through visual and sound feedback and increases the realism of the immersive environment. Finally, an experimental session, using a physical prototype of playground equipment, is carried out in order to compare these results with the ones coming out from the VR experiments. For a standard entrapment test, the authors assess the dependences of false and missed alarm by the diameter of the openings to evaluate and establish an acceptability threshold for the usability of the virtual probes. © 2013 Springer-Verlag France.

Keywords: Design validation | Grasping techniques | Playground equipment | Usability | Virtual reality

Abstract: This paper concerns a research project that aims at developing an innovative platform to design lower limb prosthesis. The platform is centered on the virtual model of the amputee and is based on a computer-aided and knowledge-guided approach. In particular, the paper focuses on the module, named Socket Modeling Assistant-SMA, conceived to design the socket, the most critical component of the whole prosthesis. The underlining idea is to experiment low-cost devices, such the Leap Motion, to manipulate the 3D virtual model of the socket using hands as traditional done by the prosthetist. The goal is to make available a modeling tool that permits to replicate/emulate manual operations usually performed by the prosthetist during the traditional development process. First, we first describe the traditional socket development process; then the SMA software architecture and the guidelines used to develop the interaction algorithms (integrated within SMA) that exploit the Leap Motion and Falcon devices. Finally preliminary tests and results will be illustrated. © 2013 ACM.

Keywords: 3D modeling | hand tracking | haptic interaction | lower limb prosthesis

Abstract: This paper focuses on a structured methodology that uses virtual reality (VR) and digital human modeling (DHM) to study maintenance procedures of industrial products. VR technologies help to highlight the most critical aspects of maintenance operations, while DHM tools allow detailing working sequences. Data coming from these analyses are then used to draw up a multimedia maintenance manual based on digital video animations, audio comments, explanatory images and written recommendations. Information is available to maintenance personnel directly on the working site through portable electronic devices. Further, web-based multimedia manuals can be updated on-line and help to shorten learning time and maintenance downtimes. © 2013 Springer-Verlag France.

Keywords: Design for maintainability | Digital human modelling | Multimedia maintenance manuals | Virtual design review

Abstract: Mixed prototyping (MP) is an emerging approach for usability testing, thanks to its multimodal environment, which is able to involve sight, hearing and touch thus improving the ability to analyze the inter-relationships between the physical form and the behavior of the industrial products. This paper presents a method to perform usability tests in a mixed reality (MR) environment for analyzing human performance in target acquisition tasks while interacting with household appliances. The proposed method is based on the use of different kinds of digital and physical prototypes and, moreover, it introduces an experimental physical archetype for mixed prototyping that contributes to increase the efficiency of the usability evaluation process. Through this archetype the design of a user interface can be easily changed by the adoption of plug-and-play moving components (knobs and buttons) that allow to model in a few seconds any kind of control panel for washing machines, thus reducing the prototyping costs and enlarging the variety of MR interfaces that can be evaluated. The paper proposes also a validation of the use of the physical archetype through a case study in which three different control panel alternatives have been evaluated in a competitive usability study. The competitive testing allows to gather user behaviors with a broad range of design options before the development of a new control panel refined through iterative design. Experimental results show that the proposed method based on the physical archetype can be an effective support to improve the usability of the product interface. © 2012 Elsevier B.V.

Keywords: Design review | Interaction design | Mixed prototyping | Usability test

Abstract: The paper describes a methodological approach specifically developed to capture and transform the qualitative User Experience (UX) of a consumer product into quantitative technical specifications. Merging the potentialities of Virtual Prototypes (VPs) and Digital Mock-Ups (DMU), a flexible design scenario is built to interpret users' desires. Visual, sound and haptic stimuli are reproduced in order to let users live a realistic multisensory experience interacting with the virtual replica of the product. Parametric models are defined to acquire users' preferences while optimization algorithms are used to transform them into technical specifications. The aim of the approach is to propose a robust technique to objectify users' desires and enable their direct and active participation within the product development process. The methodology is derived merging insights coming from four case studies as well as indications available in literature. Specifically the paper describes how to design the multisensory UX with household appliance doors and drawers with a specific focus on the haptic/force feedback objectification. © 2013 The Design Society.

Keywords: Experience design | Haptic feedback | Human in the loop | User centred design | Virtual reality

Abstract: The paper describes the results of an on-going research activity whose aim is to allow companies, operating in the consumer goods market, to design the multisensory experience of their products. In case of the household appliances market, which is the research context of this study, the user experience derives from the interaction with specific product features such as the door, buttons, and drawers. Designing a good multisensory experience is complex since it means taking into account a combination of visual, hearing and haptic feedbacks a user perceives when interacting with the product. Virtual Reality offers the technologies to design and test that experience thought virtual prototypes, even if to date there is a lack of methodological approaches to practically guide and support this design activity. Relying on the results of previous authors' researches, the paper describes further methodological advances focused on making usable the proposed approach in the current design practice. The case study chosen to demonstrate the effectiveness of the method is a dishwasher door and the paper describes how to re-engineer the haptic feedback of a commercial model in order to make it more perceptually appealing at the moment of purchase. © 2013 CAD Solutions, LLC.

Keywords: Haptic interaction | Reverse engineering | User experience design | Virtual prototyping

Abstract: The paper describes a methodology that can be employed to perform the analysis of aspects related to human interaction with consumer products during the Product Development Process, thanks to the use of mixed prototypes. The methodology aims at helping designers to take decisions earlier compared to the current practice based on not easily modifiable physical prototypes. Authors' method considers the interaction with adaptable mixed prototypes as a possible validating procedure for product interaction-enabling features: a multimodal environment is created to perform these validations, integrating three sensorial modalities such as vision, hearing and touch. The paper firstly describes the requirements for the creation of the multimodal environment. Then it focuses on the opportunity of using an approach based on mixed prototypes rather than on completely virtual ones: the intent is to increase the level of "realism" of the simulation by overcoming limitations of actual technologies for the sense of touch. Finally, a case study is discussed, starting from the analysis of a commercial consumer product up to the interaction with the developed Mixed Prototype. The expected benefits for the product development process are highlighted. © 2013 CAD Solutions, LLC.

Keywords: Haptic interaction | Product virtualization | Virtual and mixed prototyping

Abstract: The Space Planning (SP) is a process that allows making an environment more ergonomic, functional and aesthetically pleasing. The introduction of Computer Aided tools for this kind of practice led to an increase of the quality of the final result thanks to some versatile support used for the generation of different options to consider for the evaluation. In particular, those based on Augmented Reality (AR) technologies allow evaluating several options directly in a real room. In this paper, an AR system, developed with the aim of supporting Space Planning activities, is proposed. The system has been developed in order to overcome some problems related to the tracking in wide environments and to be usable in different typologies of Space Planning environments. The paper also presents a qualitative evaluation of the AR system in three different scenarios. The positive results obtained through these evaluation tests show the effectiveness and the suitability of the system in different Space Planning contexts. © 2013 Springer-Verlag Berlin Heidelberg.

Keywords: Augmented Reality | HCI | Space Planning design

Abstract: This paper proposes an augmented reality (AR) strategy in which a Lamb waves based impact detection methodology dynamically interacts with a head portable visualization device allowing the inspector to see the estimated impact position (with its uncertainty) and impact energy directly on the plate-like structure. The impact detection methodology uses a network of piezosensors bonded on the structure to be monitored and a signal processing algorithm (the Warped Frequency Transform) able to compensate for dispersion the acquired waveforms. The compensated waveforms yield to a robust estimation of Lamb waves difference in distance of propagation (DDOP), used to feed hyperbolic algorithms for impact location determination, and allow an estimation of the uncertainty of the impact positioning as well as of the impact energy. The outputs of the impact methodology are passed to a visualization technology that yielding their representation in Augmented Reality (AR) is meant to support the inspector during the on-field inspection/diagnosis as well as the maintenance operations. The inspector, in fact, can see interactively in real time the impact data directly on the surface of the structure. To validate the proposed approach, tests on an aluminum plate are presented. Results confirm the feasibility of the method and its exploitability in maintenance practice. © 2013 Luca De Marchi et al.

Abstract: This paper describes the design and development of a virtual environment conceived to support flight operations of an Unmanned Air Vehicle (UAV) used for wind mapping in the proximity of existing or planned wind farms. The virtual environment can be used in pre-flight briefings aiming to define a trajectory from a list of waypoints, to change and eventually re-plan the mission in case of intersection with no fly zones, to simulate the mission, and to preview images/videos taken from the UAV on-board cameras. During flight, the tool can be used to compute the wind speed along the trajectory by analyzing the data streaming from the UAV. The integration of Augmented Reality (AR) techniques in the flight environment provides assistance in remotely piloted landings, and allows visualizing flight and environmental information that are critical to the mission. For example, when spatial and temporal knowledge of the wind speed is required, AR can be used to overlap wind speed vectors to the external real environment. Eventually, wind vectors and UAV position and attitude can be visualized in a Virtual Reality systems based on Cave Automatic Virtual Environments (CAVE) or stereoscopic view. Tests shows that the proposed tool and methodology can effectively support wind speed detection missions since it can improve operational safety and contribute to the accomplishment of mission goals. Copyright © 2013 SAE International.

Abstract: Aeronautical transport system is rapidly growing and more demanding. It has become a total system of complex systems in which the human is recognized as the decisional point that is asked to act quickly and safely. In this context, innovative technologies provide the challenge to design revolutionary Human Machine Interfaces for the people involved.In this paper a report of recent works in Human Machine Interfaces in aeronautics developed at the University of Bologna is presented. Synthetic visualization and task automation are the main commonalities in these projects. What we can expect from the future is then explored through an insight into technological aspects. © 2013 IEEE.

Keywords: Human Machine Interface | Synthetic Displays | Virtual Reality

Abstract: The contribution regards the surveying of two statues of famous contemporary sculptors that have been placed in the central zone of Cosenza, which has been transformed in an open air museum. To realize a 3-D representation of the museum, different methodologies have been used, based on classical surveying (total stations and GNSS), image data and range data. The increasing performances of the new models of Time Of Flight (T.O.F.) laser scanners allow to build accurate models also for medium-size objects; on the other hand, the recent techniques of 3D modeling enable the processing of large amount of data and the effective removal of noises. Thus, if an extreme accuracy is not required, one can think to use the T.O.F. laser scanner, also for the surveying of statues. For the acquisition of the surfaces of the statues, two different types of laser scanning have been used: the Leica Scan StationC10, based on Time Of Flight, and the Minolta VIVID 300 triangulation scanner. In the paper, the comparison between the results obtained by using the different techniques is described.

Keywords: 3D modelling | Augmented reality | City model | Cultural heritage | Laser scanning | Multiresolution | Representation

Abstract: The interaction metaphor, based on mouse, monitor and keyboard, presents evident limits in the engineering design review activities, when real and virtual models must be explored and compared, and also in "outside-the-office" environments, where the desk is not available. The presented research aims to explore a new generation of gesture-based interfaces, called "natural interfaces", which promise an intuitive control using free hands and without the desk support. We present a novel natural design review workspace which acquires user motion using a combination of video and depth cameras and visualizes the CAD models using monitor-based augmented reality. We implemented a bimanual egocentric pointer paradigm by a virtual active surface in front of the user. We used a XML configurable approach to explore bimanual gesture commands to browse, select, dis/assembly and explode 3D complex models imported in standard STEP format. Our experiments demonstrated that the virtual active surface is able to effectively trigger a set of CAD specific commands and to improve technical navigation in non-desktop environments: e.g. shop floor maintenance, on site quality control, etc. We evaluated the feasibility and robustness of the interface and reported a high degree of acceptance from the users who preferred the presented interface to a unconstrained 3D manipulation. © 2012 Springer-Verlag France.

Keywords: 3D manipulation | Augmented reality | CAD | Depth camera | Human computer interfaces | Natural interfaces

Abstract: The validation of a product interface is often a critical issue in the design process. Virtual reality and mixed reality (MR) are able to enhance the interactive simulation of the product human-machine interface (HMI), as these technologies allow engineers to directly involve end users in the usability assessment. This paper describes a MR environment specifically addressed to the usability evaluation of a product interface, which allows the simulation of the HMI behaviour using the same models and the same software employed by engineers during the design phase. Our approach is based on the run-time connection between the visualisation software and the simulators used for product design and analysis. In particular, we use Matlab/Simulink to model and simulate the product behaviour, and Virtools to create the interactive MR environment in which the end user can test the product. Thanks to this architecture, any modification done on the behaviour models is immediately testable in MR. © 2012 Springer-Verlag London.

Keywords: Finite state machine | Functional behaviour simulation | Interactive virtual prototype | Mixed reality | Product design evaluation

Abstract: Visuo-haptic mixed reality consists of adding to a real scene the ability to see and touch virtual objects. It requires the use of see-through display technology for visually mixing real and virtual objects, and haptic devices for adding haptic interaction with the virtual objects. Unfortunately, the use of commodity haptic devices poses obstruction and misalignment issues that complicate the correct integration of a virtual tool and the user's real hand in the mixed reality scene. In this work, we propose a novel mixed reality paradigm where it is possible to touch and see virtual objects in combination with a real scene, using commodity haptic devices, and with a visually consistent integration of the user's hand and the virtual tool. We discuss the visual obstruction and misalignment issues introduced by commodity haptic devices, and then propose a solution that relies on four simple technical steps: color-based segmentation of the hand, tracking-based segmentation of the haptic device, background repainting using image-based models, and misalignment-free compositing of the user's hand. We have developed a successful proof-of-concept implementation, where a user can touch virtual objects and interact with them in the context of a real scene, and we have evaluated the impact on user performance of obstruction and misalignment correction. © 2012 IEEE.

Keywords: haptic interfaces | image-based rendering | Mixed reality | occlusion handling | visuo-haptic mixed reality

Abstract: The paper deals with the description of a new methodology for addressing the modelling for static and dynamic simulation of the cross-axis flexural pivot. The proposed methodology is based on the use of the dynamic spline formulation for describing the deformation of the structure using reference points. By using this approach, the very large displacement of the compliant pivot can be modelled using a reduced number of variables. The methodology has been formulated to be also suitable for an integration with an augmented reality interactive design environment. The results coming from the simulations (both static and dynamic) of the proposed model have been compared to those of an equivalent finite element model and show very good accordance. The proposed methodology is able to take into account the nonlinear aspects and it is suitable for real-time computation. An example of implementation in an augmented reality interactive design environment has been successfully implemented. © 2012 Springer-Verlag France.

Keywords: Augmented reality | Cross-axis flexural pivot | Dynamic spline | Interactive design | Simulation

Abstract: In this work we apply an innovative participative design approach for the quality evaluation of virtual prototypes of new industrial products (i.e. concept designs), by adopting statistical procedures and carrying out tests in an immersive VR environment. This methodology has been fully exploited through a case study concerning the choice of the optimal design for the interiors of a new regional train. Following this approach, the optimal concept design is defined at the end of a process consisting of five phases: identification of the quality elements of the concept design, classification of the quality elements, generation and quality evaluation of product concepts and, finally, definition of the optimal concept. According to the applied methodology after the identification of the customer's needs, a structured set of quality elements has been defined and, successively, classified according to Kano's theory. Following the approach of conjoint analysis, the design factors have been combined according to an experimental plan to form product virtual concepts. During the concepts generation phase we have explored those product architectures that integrate design characteristics correlated to the set of quality elements. The concepts have been created according to comfort, ergonomic and safety criteria. In particular we have considered the ergonomics of places and furniture dimensions, through the use of virtual manikins. The evaluation of the quality of the different concepts has been carried out in the VR laboratory (named "VRTest") of the Competence Centre for the Qualification of Transportation Systems founded by Regione Campania according to an original statistical procedure and has involved a group of experts in train's interiors design and a group of common users of regional trains. Copyright © 2012 by ASME.

Keywords: Concept design | Participative design | Quality engineering | Railway engineering | Virtual Reality

Abstract: The design of a new product requires a series of validations before its approval and manufacture. Virtual prototyping based on mixed reality technology seems a promising technique, especially when applied to the design review of products that are characterised by interaction with users. This paper presents a new methodology that allows the collaborative design review and modification of some components of automotive interiors. Professionals can work together through a mixed reality distributed design platform by interacting intuitively and naturally with the virtual prototype of the product. The methodology has been validated by means of tests with users, aiming at assessing the effectiveness of the approach, and at identifying potential usability issues. © 2012 Copyright Taylor and Francis Group, LLC.

Keywords: automotive design | collaborative design | design methodology | mixed reality | virtual reality

Abstract: Advantages of using HUDs in aviation have been already proven and this technology is widely being applied both in civil and military flight operations. HUDs are based on displaying data on a transparent layer allowing the pilot to simultaneously look at them and the outside world by means of a collimated projection. Commercial HUDs differ in the nature of the data they display and in the design of the interface. A considerable interest is currently being focused on the possibility of displaying conformal symbology, intended as geo-referenced symbols (as opposed to nonconformal navigation data), representing data that are usually displayed onto HDDs (Head Down Displays). While being associated with real features in the flight scenarios, such conformal symbols are typically based on twodimensional graphics. The aim of this work is to provide a simulator to investigate the use of threedimensional objects in the HUDs (Head Up Displays) to be installed in future cockpits. A Virtual Reality based HUD simulator has been conceived. It exploits a stereoscopic visualization screen where a double viewport has been created to overlay a computer generated HUD onto a synthetic flight scenario. The exploitation of such virtual prototyping systems in cockpit design is expected to give advantages in terms of an improved capability of following the evolution of safety and efficiency requirements.

Keywords: Cockpit design | HMI Human Machine Interface | HUD Head Up Display | Virtual reality

Abstract: Highly usable human-system interfaces can have a large benefit on the quality of life for the elderly and disabled. New emerging product design technologies, such as Virtual Reality (VR) and Augmented Reality (AR), give many opportunities to evaluate and improve system usability in the early design stages. In this way different design alternatives can be evaluated in terms of physical and cognitive performance. In this context the present paper describes a systematic approach for designing highly usable home environments and optimizing human-machine interaction. VR/AR technologies are adopted for user interface evaluation. The approach will be applied to the design of a new kitchen.

Keywords: Augmented Reality | Home environment usability | Inclusive design | User interfaces | Virtual Reality

Abstract: The use of haptic devices in Virtual Reality applications makes the interaction with the digital objects easier, by involving the sense of touch in the simulation. The most widespread devices are stylus-based, so the user interacts with the virtual world via either a tool or a stylus. These kinds of devices have been effectively used in several virtual prototyping applications, in order to allow the users to easily interact with the digital model of a product. Among the several open issues related to these applications, there is the choice of the set-up and of the techniques adopted to combine the visual and the haptic stimuli. This paper presents the comparison of three different solutions specifically studied for virtual prototyping applications and in particular for usability assessment. The first is a simple desktop configuration where the user looks at a screen, and visual and haptic stimuli are presented in a de-located manner. The second is a HMD based set-up where the user has a more natural first-person immersive interaction. The third requires a video-see-trough HMD in order to augment the virtual scene with the visualization of the real user's hand. The test realized with the users on these three different setups have been finalized to study the effect of two different factors that are crucial for the effectiveness and the user-friendliness of the interaction. One is the perception of the visual and haptic stimuli in a collocated manner; the other is the visualization of his/her own hand during the interaction with the virtual product. Copyright © 2011 by ASME.

Abstract: The choice of the interaction menu to use is an important aspect for the usability of an application. In these years, different solutions, related to menu shape, location and interaction modalities have been proposed. This paper investigates the influence of haptic features on the usability of 3D menu. We have developed a haptic menu for a specific workbench, which integrates stereoscopic visualization and haptic interaction. Several versions of this menu have been developed with the aim of performing testing sessions with users. The results of these tests have been discussed to highlight the impact that these features have on the user's learning capabilities. © 2011 IFIP International Federation for Information Processi.

Keywords: Haptic Interaction | Haptic Menu | Mixed Reality

Abstract: The paper describes a collaborative platform to support the development and the evaluation of cars interior by using a Mixed Prototyping (MP) approach. The platform consists of two different systems: the 3D Haptic Modeler (3DHM) and the Mixed Reality Seating Buck (MRSB). The 3DHM is a workbench that allows us to modify the 3D model of a car dashboard by using a haptic device, while the MRSB is a configurable structure that enables us to simulate different driving seats. The two systems allow the collaboration among designers, engineers and end users in order to get, as final result, a concept design of the product that satisfies both design constraints and final users' preferences. The platform has been evaluated by means of several testing sessions, based on two different scenarios, so as to demonstrate the benefits and the potentials of our approach. © 2011 Springer-Verlag.

Keywords: Collaborative design | Ergonomic assessment | Haptic modeling | Mixed Reality | Virtual Prototype

Abstract: Several haptic devices have been developed in recent years in order to reproduce the sensation of physical contact with virtual objects. Many of these devices are point-based, and some haptic interfaces behave like small surfaces that conform to a virtual shape. None of these allow a full-hand contact with the shape, and they are, in general, too small to render big surfaces. The simulation of tasks, such as the exploration of aesthetic surfaces made by industrial designers in order to check the quality of prototypes, require full-hand contact with the shape on a one-to-one scaled representation of the object. These explorations follow trajectories that can be approximated with planar or geodesic curves. In this paper, we describe the design and implementation of a linear haptic device that is able to render these trajectories. The device is part of a multimodal system including stereoscopic visualization that allows visual representation of the entire surface. Industrial designers use the system for checking the quality of shapes while exploiting their manual and visual skills. The system has been tested by industrial designers and the results are reported in this paper. © 2011 IEEE.

Keywords: curve rendering | Haptic strip | industrial design | mixed reality | multimodal interfaces | virtual prototyping

Abstract: Over the past decade, researchers have investigated AR as a promising candidate technology for building advanced interfaces for maintenance personnel. Nevertheless, the low usability of cumbersome hardware, the need to use markers, and the complexity of creating digital content seem to hinder AR's effective implementation in industry. This prototype system aims to demonstrate that you can implement AR to support aircraft maintenance personnel. To meet a daily inspection procedure's operational requirements, the system employs markerless pose estimation. © 2006 IEEE.

Keywords: aircraft maintenance | augmented reality | computer graphics | graphics and multimedia | human-computer interaction

Abstract: Virtual reality as the way to display digital models and to interact with them has flourished in industrial contexts some years ago, both for design and marketing reasons. However, some specific sectors, e.g. furnishings and garments, would prefer to evaluate their products in a real environment, where their models could be easily placed, and where the interaction with them could take place in a natural way. These requirements suggested the design of an application, based on the augmented reality, which allows users placing digital models of pieces of furniture in real domestic environments, verifying their dimensional and aesthetic compatibility with the existing context, and interacting with them to test functional behavior and usability issues. Such a project would result interesting both for possible customers and for designers, because some important design hints could come from its adoption. An application prototype has been developed and tested in the field in a couple of case studies. Copyright © 2011 by ASME.

Keywords: Augmented reality | Furnishing configuration | Interaction design

Abstract: This work presents a novel Augmented Realty (AR) application to superimpose interactive Product Manufacturing Information (PMI) onto paper technical drawings. We augment drawings with contextual data and use a novel tangible interface to access the data in a natural way. We present an optimized PMI data visualization algorithm for CAD models in order to avoid model and annotation cluttering. Our algorithm ranks the model faces with technical annotations according to angle, distance, occlusion and area. The number of annotations visualized on 3D model is chosen following the cognitive perception theory to avoid information overload. We also extended the navigation metaphor adding the concept of tangible model navigation and flipping using the duplex drawing. As case studies we used annotated models from ASME standards. By using PC hardware and common paper drawings, this approach can be integrated at low-cost in existing industrial processes. Copyright © 2011 by ASME.

Keywords: Augmented reality | PMI annotations | Technical drawing

Abstract: The paper explores the potentialities of virtual reality (VR) to improve the learning process of mechanical product design. It is focused on the definition of a proper experimental VR-based set-up whose performance matches mechanical design learning purposes, such as assemblability and tolerances prescription. The method consists of two main activities: VR technologies benchmarking based on sensory feedback and evaluation of how VR tools impact on learning curves. In order to quantify the performance of the technology, an experimental protocol is defined and an testing plan is set. Evaluation parameters are divided into performance and usability metrics to distinguish between the cognitive and technical aspects of the learning process. The experimental VR-based set up is tested on students in mechanical engineering through the application of the protocol. © 2011 Springer-Verlag.

Keywords: Experimental protocol | Learning curve | Mechanical product design | Mechanics education | Virtual reality

Abstract: In this chapter the authors discuss several approaches in order to integrate computeraided engineering instruments into Augmented Reality environment. Engineers and designers often develop their creative ideas in front of a computer monitor using mouse and keyboard. Although the integration between numerical computation and graphics leads to the generation of very realistic digital mock-ups, they are still far from the real context and the user has limited interaction with them. The purpose is to illustrate how recent development in computer graphics and image processing can improve the realism and interactivity with digital mock-ups. Starting from the interactive modeling of 3d shapes, the chapter presents some examples about the integration of real-time mechanism motion simulation, structural and fluid dynamics analysis post-processing. © 2010 by Nova Science Publishers, Inc. All rights reserved.

Keywords: Augmented reality | Computer-aided design | Simulation | Virtual engineering

Abstract: The design review process of new products is time consuming, requires the collaboration and synchronization of activities performed by various experts having different competences and roles, and is today performed using different tools and different product representations. In order to improve the performances of the overall product design process, it would be beneficial the availability of Computer Aided tools supporting both conceptual design and analysis activities within an integrated environment based on a multi-disciplinary model paradigm. The paper presents an environment named PUODARSI that allows product designers to intuitively modify the shape of a product through haptic interaction and to test in real-time the structural and fluid-dynamics impact of these changes. The research work shows that a smooth and effective integration of modeling tools based on haptic interfaces, fluid-dynamics analysis tools, and Virtual Reality visualization systems is feasible in real-time through the use of a proper data model exchange. © 2010 Springer-Verlag.

Keywords: Design review | Haptic interaction | Interactive simulation | Multi-disciplinary model | Virtual design

Abstract: The paper describes the development of a seating buck system for ergonomic evaluation of the driver's cab. Seating buck is a configurable structure that, thanks to the mixed reality technologies, allows us to simulate different driving seat and to perform different evaluation tests. In particular we are interested to evaluate the ergonomics of car's dashboard with its knobs, buttons, display and other control systems. For this reason, we studied and developed a seating buck system that addresses these issues. In particular, we investigated on the possibility of changing in real time the position of some components of the car dashboard with the aim of analysing different layouts. The effectiveness of the system has been subsequently validated through some test sessions with users. © Organizing Committee of TMCE 2010 Symposium.

Keywords: Ergonomic analysis | Haptic devices | Mixed reality | Rapid prototyping | Seating buck

Abstract: The product development process, of industrial products, includes a phase dedicated to the design review that is a crucial phase where various experts cooperate in selecting the optimal product shape. Although computer graphics allows us to create very realistic virtual representations of the products, it is not uncommon that designers decide to build physical mock-ups of their newly conceived products because they need to physically interact with the prototype and also to evaluate the product within a plurality of real contexts. This paper describes the hardware and software development of our Augmented Reality design review system that allows to overcome some issues related to the 3D visualization and to the interaction with the virtual objects. Our system is composed by a Video See Through Head Mounted Display, which allows to improve the 3D visualization by controlling the convergence of the video cameras automatically, and a wireless control system, which allows us to create some metaphors to interact with the virtual objects. During the development of the system, in order to define and tune the algorithms, we have performed some testing sessions. Then, we have performed further tests in order to verify the effectiveness of the system and to collect additional data and comments about usability and ergonomic aspects. © 2010 Copyright SPIE - The International Society for Optical Engineering.

Keywords: 3D interaction | Augmented reality | Design review | HMD | Stereoscopic visualization

Abstract: This work presents the results of a research project that evaluated the possibility to carry out ergonomic analyses on virtual prototypes that permitted tactile interaction. We propose an approach based on Virtual Reality (VR) and haptics: the former to improve visual rendering of a digital model, the latter to permit tactile interaction. The products considered were control boards. Atomic components such as knobs, sliders and buttons are employed. The paper presents the architecture of an ergonomic workstation and its first implementation based on commercial systems and ad hoc haptic devices specifically developed for the purpose. Major problems related to a VR environment, such as visualisation and human body tracking, are discussed; the architecture of haptic devices and technical solutions to achieve a satisfactory haptic rendering are presented. Finally, the paper proposes a test procedure and presents the results of tests on the behaviour of ergonomic workstation components. Work in progress and future developments conclude the paper. Copyright © 2010 Inderscience Enterprises Ltd.

Keywords: Control boards | Ergonomic tests | Haptic | Virtual prototype | Virtual reality

Abstract: The increasing complexity of the air traffic system is pushing towards the development of innovative and more automated tools conceived to manage it. In this scenario, an important role is played by HCI (human-computer interfaces) used by air traffic controllers and operators to visualize and interact with air traffic data. Currently, information about three-dimensional (3D) scenery is displayed with a two-dimensional representation. This paper presents the design, development, and evaluation of an innovative interface for air traffic control (ATC) based on a four-dimensional (4D) (3D space+time) visualization display. The proposed interface allows the operator to perceive all the information, including meteorological conditions, that is useful for TWR/APP (ToWeR/APProach) control in a unique 4D synthetic reconstruction of the airport area. Particular attention is dedicated to the fourth variable, time, which is a fundamental parameter in ATC. A simple and fast trajectory prediction algorithm has been implemented in order to provide the operator with an effective 'user assistance' tool in conflict detection activities. The interface has been evaluated by performing test simulations and surveys were used to collect results and useful advice for its future development.

Keywords: air traffic control | human-computer interfaces | synthetic environment | virtual reality

Abstract: In this work we integrate augmented reality technology in a product development process using real technical drawings as a tangible interface for design review. We present an original collaborative framework for Augmented Design Review Over Network (ADRON). It provides the following features: augmented technical drawings, interactive FEM simulation, multimodal annotation and chat tools, web content integration and collaborative client/server architecture. Our framework is intended to use common hardware instead of expensive and complex virtual or augmented facilities. We designed the interface specifically for users with little or no augmented reality expertise proposing tangible interfaces for data review and visual editing for all the functions and configurations. Two case studies are presented and discussed: a real-time "touch and see" stress/strain simulation and a collaborative distributed design review session of an industrial component. © 2008 Elsevier Ltd. All rights reserved.

Keywords: 3D model annotation | Augmented reality | Collaborative design review | FEM data visualization

Abstract: Virtual Reality systems can impact on quality of collaboration design processes. In the present work is defined a structured method to classify, qualify and select VR-based tools for supporting co-design activities by adopting a set of benchmarking metrics. Attention is focused on collaboration scopes and requirements, participants' behavior and exploited interaction modalities. The method has been applied to synchronous and remote collaboration that actually represents the most critical communication in industry. Three different types of collaborative VR tools have been investigated and compared. Experimental results highlight how the proposed method is able to identify the main collaboration requirements by selecting the most proper supporting technology and show also the contribution to collaboration success. © 2010 Springer-Verlag London Limited.

Keywords: Benchmarking | Collaborative design | Virtual reality

Abstract: Mixed prototyping is an industrial practice that combines virtual and real components in order to realize a prototype of a product used to evaluate and assess the design choices. Recently, mixed prototypes have been also used to assess the usability of products interface. This particular application arises several problems related to the devices and the interaction techniques that, better than others, allow a natural interaction with the mixed prototype. This paper presents a mixed reality environment for usability evaluation that deals with two specific problems of this kind of application: the occlusion between real and virtual objects and the interpretation of the user's gestures while he/she is interacting with the elements of the product interface. In particular we propose a technique able to manage both the problems by using only commodity hardware and video processing algorithms, thus avoiding the use of expensive datagloves and tracking devices. The proposed approach has been validated through a user study addressed to establish whether and to what extent the augmented reality devices and the techniques proposed may distort the usability assessment of the product. Moreover, the user study compares the mixed reality environment adopted in this study with a classical virtual reality set-up. Copyright © 2010 by ASME.

Abstract: VR (Virtual Reality) is a powerful tool for the simulation of virtual prototypes, because it allows engineers to enhance the analysis and validation of the digital product before manufacturing any physical mock-up. Unfortunately, VR software tools are not able to fully simulate the behaviour of a virtual product, because they are mainly conceived to reproduce the visual appearance of the product: the functional simulation is limited to basic behaviours related to the animation of the objects in the virtual world (e.g.: part movements). This paper describes an innovative approach to create functional behaviour simulations in VR using the same models and the same software employed by the engineers in the design phase. Our approach is based on the run-time connection between the VR software and the simulators used for product design and analysis. This means that there is no need to write code for describing the product behaviour, and any modification done on the behaviour models is immediately testable in VR. It is apparent that these advantages allow to reduce the time needed to implement the virtual prototyping, thus achieving a more efficient design process. © Organizing Committee of TMCE 2010 Symposium.

Keywords: Finite state machine | Functional behaviour simulation | Interactive virtual prototype | Product design evaluation

Abstract: The usability of the user interface is a key aspect for the success of several industrial products. This assumption has led to the introduction of numerous design methodologies addressed to evaluate the user-friendliness of industrial products. Most of these methodologies follow the participatory design approach to involve the user in the design process. Virtual Reality is a valid tool to support Participatory Design, because it facilitates the collaboration among designers and users. The present study aims to evaluate the feasibility and the efficacy of an innovative Participatory Design approach where Virtual Reality plays a 'double role': a tool to evaluate the usability of the virtual product interface, and a communication channel that allows users to be directly involved in the design process as co-designers. In order to achieve these goals, we conducted three experiments: the purpose of the first experiment is to determine the influence of the virtual interface on the usability evaluation by comparing "user-real product" interaction and "user-virtual product" interaction. Subsequently, we tested the effectiveness of our approach with two experiments involving users (directly or through their participation in a focus group) in the redesign of a product user interface. The experiments were conducted with two typologies of consumer appliances: a microwave oven and a washing machine. © 2009 Elsevier Ltd. All rights reserved.

Keywords: Participatory design | Product interface design | Usability | Virtual reality

Abstract: The use of Mixed Reality in the product development process is emerging as a promising solution that combines the advantages of virtual and rapid prototyping. A mixed prototype is usually based on a physical mock-up on which the visual appearance of the product is superimposed thanks to the augmented reality technologies. An open issue in mixed prototyping is the reduction of time and efforts required to generate and update the behaviour model of the product. This paper presents a Mixed Reality environment in which the product behaviour is simulated using the same models and the same software employed by the engineers in the design phase. This approach guarantees the reliability of the simulation and allows a strong reduction of the time needed to develop the digital prototype. Moreover the paper presents an innovative technique, specifically studied for the simulation of electrical appliances, that aims to make the user able to naturally interact with the mixed prototype. This technique is able to manage the occlusion between real and virtual objects and the interpretation of the user's gestures while he/she is interacting with the elements of the product interface. The novelty of the technique is that it does not require any specific device like data-gloves or tracking systems. © 2010 by ASME.

Abstract: For nearly two decades, virtual reality (VR) technologies have been employed in the field of cultural heritage for various purposes. The safeguard, the protection and the fruition of the remains of the past have gained a powerful tool, thanks to the potentialities of immersive visualization and 3D reconstruction of archaeological sites and finds. VR applications based on videogame technologies are known for their realism and fluid interactivity, but the choice of the fittest technologies remains a complex task because there is an ample number of hardware devices and software development kits. Moreover the design of a VR application for cultural heritage requires several different professional skills and presents a certain complexity in coordination and management. This paper presents strategies to overcome these problems, by suggesting some guidelines for the development of VR systems for cultural heritage. It illustrates a complete methodology to create a virtual exhibition system, based on realistic high-quality 3D models of archaeological finds (reconstructed using a 3D Scanner and a high definition camera) and a low-cost multimedia stereoscopic system called MNEME, which allows the user to interact in a free and easy way with a rich collection of archaeological finds. The solution we propose is intended to be easy to transport and fully usable by different user typologies, without any external assistance or supervision. © 2009 Elsevier Masson SAS. All rights reserved.

Keywords: 3D acquisition | 3D scanner | Archaeological finds | Laser scanner | MNEME | Stereoscopic vision | Virtual archaeology | Virtual museum | Virtual reality

Abstract: In this paper, the authors discuss a methodology to enhance multibody systems simulations using Augmented Reality (AR) implementation. The AR deals with the use of live video imagery which is digitally processed and augmented by the addition of computer generated graphics. The purpose is to illustrate how recent developments in computer-aided design and augmented reality can improve the realism and interactivity when simulating the movement of digital mock-ups. The paper discusses hardware and software implementations and an overview of several illustrative examples. The basic idea is described starting from a simple simulation of a falling body subjected to gravity with the initial conditions set interactively by the user. Then, a more complex interactive simulation of the kinematics of a robot whose end-effector can be grabbed and moved by the user is presented. Finally, the real time dynamic simulation of a slider crank mechanism is discussed. The integration between AR and multibody simulation has revealed to be very useful for didactical purposes and collaborative design.

Keywords: Augmented Reality | Interactive simulation | Multibody

Abstract: In recent years, especially in the city environment, the mobility demand has favoured the increment of the transport on two wheels. This increase is due to the small size of the motorcycle vehicles that allows both to fit easily in tight spaces and to guarantee a certain level of flexibility to the users in the congested traffic. In this trend higher attention must be paid on the training of future riders. So, the research work undertaken is devoted to give a possible solution for the problem by means of the study and the realization of a motorcycle simulator. The first step of the research, reported in this paper, has been focused on the study of the visualization system of the simulator. To improve the riding experience simulated the visualization system has been characterized by the tracking of an head mounted display (HMD) that permits to calculate the rotation of the user's head and so of the point of view in the virtual environment, This has been initially used to conduct different tests in order to evaluate the user's perception level of road signs. The test has been performed in two different environment (urban and rural) with different weather condition (sun, rain and fog) to modify the depth perception. The relative simple conditions of riding permit to ignore the ride ability of the user thus evaluating only the visual perception. Measuring the time perception and the number of perceived signs it was possible to demonstrate the efficiency of the tested visualization system despite of a traditional one like a monitor. © Organizing Committee of TMCE 2010 Symposium.

Keywords: Perception of road signs | Riding simulator | Tracked head mounted display | Virtual reality

Abstract: The paper aims at providing a methodological contribution to the concept design of train interior in order to improve the quality perceived by users in compliance with railway standards. Indeed, the combined use of advanced CAD tools, experimental statistical methods and Virtual Reality tools allows developing, selecting and experimentally evaluating new concepts. The design cycle starts both from designers' proposal and the identification of user's needs; then, it makes use of datum-based CAD models in order to generate virtual concepts that satisfy railway standards; the cycle proceeds with the immersive evaluation of virtual prototypes, performed by potential and expert users in Virtual Reality. The identification of the optimal concept closes the design process. This procedure can be iterated in order to improve the quality of train interiors, evaluated thanks to the user's involvement in the design cycle. In this work a case study on seat design of a regional train is presented, developed at the Virtual Reality laboratory, named, of the Regional Centre for the qualification of transportation systems set up by Campania Regional Authority. © 2009 Springer-Verlag.

Keywords: CAD models | Concept design | Designfor quality | Kano methodology | Virtual reality

Abstract: Virtual reality (VR), even if it does not represent any more a novel technology, is one of the most powerful tool to help designers during the development of new projects. This is proved by very numerous research activities related to this field. In this research, we have studied a new way to approach the development of a product. We present the ongoing development of a system, called VirDe, acronym of virtual design, which can allow the designers to perform the whole design process, from the modelling phase to the finite element method (FEM) simulation analysis, in a virtual reality environment. This new method allows remarkable time and money saving in the overall product design process, but the most important contribution of VirDe is that, as far as we concerned, there is no known similar approach which has been studying the simultaneous combination of CAD, FEM and virtual environment (VE). © Springer Verlag France 2008.

Keywords: 3Dinput device | CAD modelling | FEM analyses | Simulation | Virtual reality

Abstract: Market rules show that most of the times the aesthetic impact of a product is an important aspect that makes the difference in terms of success among different products. The product shape is generally created and represented during the conceptual phase of the product and the last trends show that the use of haptic devices allows users to more naturally and effectively interact with 3D models. Nevertheless the shape needs to satisfy some engineering requirements, and its aesthetic and functional analysis requires the collaboration and synchronization of activities performed by various experts having different competences and roles. This paper presents the description of an environment named PUODARSI that allows designers to modify the shape of a product and engineers to evaluate in real-time the impact of these changes on the structural and fluid dynamic properties of the product, describing the choice of the software tools, the implementation and some usability tests. © 2009 Springer Berlin Heidelberg.

Keywords: Haptic interaction | Interactive simulation | Mixed reality

Abstract: The paper presents a reference framework for applications based on the mixed prototyping practice and mixed reality techniques and technologies. This practice can be effectively used for rapid design assessment of new products. In particular, the paper addresses applications regarding the use of mixed prototyping practice for the design review of information appliances. Various methods and technologies can be used according to the product aspects to validate and test. The paper describes mixed prototyping applications for positioning information appliances within systems and for the evaluation of ergonomics aspects of interactive devices. © Springer-Verlag 2009.

Keywords: Haptic devices | Mixed prototyping | Mixed reality | Product assessment | Reference framework

Abstract: Current state of the art technology offers various solutions for developing virtual prototyping applications that also allow the interaction with the real environment. In particular, Augmented Reality (AR) technologies include tracking systems, stereoscopic visualization systems, photorealistic rendering tools, hi-resolution video overlay systems that allow us to create various types of applications where the virtual prototype is contextualized within the real world. One application domain is product design: AR technologies allow designers to perform some evaluation tests on the virtual prototype of industrial products without the necessity to produce a physical prototype. This paper describes the development of a new Video See-Through Head Mounted Display (VST-HMD) that is high-performing and based on stereoscopic visualization. The developed display system overcomes some issues concerning the correct visualization of virtual objects that are close to the user's point of view. The paper also presents the results of some tests about an AR application developed for product design assessment. © 2009 Springer Berlin Heidelberg.

Keywords: Augmented Reality | Design Assessment | Head Mounted Display | Video See-Through HMD

Abstract: Recent analyses on the uninhabited aerial vehicle (UAV) accidents revealed that several kinds of human-system control problems occur in current UAV missions. Therefore, a design of the man-machine interface that allows for an efficient and effective interaction between the operator and the remote vehicle becomes one of the challenges in the development of more reliable UAVs. This paper presents a first implementation of an advanced interface for UAV ground control station based on a touch screen, a 3D virtual display, and an audio feedback message generator. The touch screen is used to send high level commands to the vehicle, the 3D virtual display provides a stereoscopic and augmented visualization of the complex scenario in which the vehicle operates, and the audio feedback message generator informs the operator about any change in operational scenario. The hardware/software architecture of the interface also includes a planning algorithm and a generic vehicle model. The interface has been tested by simulating several UAV missions. The results have shown that the interface requires an adequate level of workload to command the vehicle and allows the operator to build a good level of awareness of the state of the vehicle under his or her control, as well as of the environment in which it operates. © 2009 by the Massachusetts Institute of Technology.

Abstract: This paper presents a novel 'touch and see' approach for interactive leaching of dynamic stress/strain distribution in engineering education. Our Augmented Reality application visualizes Finite Element Method (FEM) results overlaid over the real model. The user can interactively change the boundary conditions of the simulation and then evaluate the stress distribution in real time. Marker based video tracking is used to measure displacements while COMSOL Multiphysics solves the structural FEM analysis. A cantilever test case has been implemented and evaluated We describe the optimization solutions needed to achieve real-time simulation and precise and stable tracking. The presented system demonstrates significant educational benefits making the student's experience more attractive and effective. © 2009 TEMPUS Publications.

Keywords: Active learning | Augmented Reality | FEM | Tangible interfaces

Abstract: Virtual Reality (VR) has quickly evolved over the last years in terms of technological and applicative dimensions, Human-Computer Interaction is particularly meaningful in the design activities involving multidisciplinary teamwork, collaborating to achieve a common task. It influences users behaviors, representational and communication modalities. A successful Collaborative Virtual Environment has to naturally support cognitive design actions while reducing time and costs. In this context, our research goal is to evaluate performances of different human-scale virtual environments in design situations involving multiple specialists with different knowledge and expertise. We proposed a protocol to highlight the main interaction styles in collaborative environments in order to assess how VR systems affect multidisciplinary cooperation. Experimental test cases are used to compare performances of virtual and physical prototypes in design reviews activities.

Keywords: Collaborative design | CVE | Multimodal interaction | Virtual Reality

Abstract: In recent years, the interest of small and medium sized enterprises towards Virtual Reality (VR) systems is strongly increased thanks both to the improvement of VR tools effectiveness and to the cost reduction of technologies implementation. Due to the growing number of installed systems, many SMEs (Small Manufacturing Enterprises) companies require robust methods for evaluating technology performance. In this context, the present paper presents a metrics-based approach in order to analyze the VR system performance. It is specifically dedicated to the design review process during styling product design. The evaluation parameters are related to the effective communication and preservation of design intent. Metrics are classified in two main classes. The first one is related to the product, the process and the characteristics of VR technology. The second one is related to the design intent meanings preservation along the design process. Two experimental case studies are reported in order to test the approach in different operative fields. Copyright © 2009 by ASME.

Keywords: Design intent | Human computer interaction | Metrics | Virtual reality

Abstract: Visuo-haptic mixed reality consists of adding to a real scene the ability to see and touch virtual objects. It requires the use of see-through display technology for visually mixing real and virtual objects, and haptic devices for adding haptic interaction with the virtual objects. However, haptic devices tend to be bulky items that appear in the field of view of the user. In this work, we propose a novel mixed reality paradigm where it is possible to touch and see virtual objects in combination with a real scene, but without visual obtrusion produced by the haptic device. This mixed reality paradigm relies on the following three technical steps: tracking of the haptic device, visual deletion of the device from the real scene, and background completion using image-based models. We have developed a successful proof-of-concept implementation, where a user can touch virtual objects in the context of a real scene. ©2009 IEEE.

Keywords: H.5.1 [information interfaces and presentation]: multimedia information systems - artificial, augmented, and virtual realities

Abstract: Virtual reality (VR) became a very common mean during the development of the industrial products. One of the main applications of VR in the industrial field is the validation of virtual prototypes (VP). A virtual prototype should be able to reproduce, as realistically as possible, the behaviour of the product from any point of view. In this paper we propose an unexploited approach to the simulation of a VP in VR. A high level software library for the inter-process communication has been developed to let the multi-body solver communicate with the VR environment. Such approach allows the designer to use different software frameworks for the simulation and the visualisation. The test case provided regards an excavator machine. It is possible to simulate the action of the actuators to move arms and bucket, and also perform visibility analyses discovering the viewing volume of the operator. © 2008 Springer-Verlag London Limited.

Keywords: Multi-body analysis | Numerical analysis | Virtual reality | Visibility analysis

Abstract: This paper describes a new methodology that, making use of a haptic device, can simulate the palpation, a diagnostic manoeuvre aiming to verify the condition of internal organs or anatomical formations. In the developed application, that has the purpose to pick out an anatomical formation and understand its characteristics, the palpation of a soft tissue has been taken in consideration. Particularly the fingertip, the skin and an anatomical formation have been simulated. The user, handling the haptic system at disposal can feel the contact with the skin but also perceiving the presence, the shape and the dimension of the subcutaneous formation (invisible to the operator), that has been modelled as a rigid sphere (like a nodule). The evaluation of the skin deformations, following from the palpation, has been performed through a massspring based algorithm, which allows to obtain results in real time.

Keywords: Haptic | Palpation simulation | Virtual reality

Abstract: The design of solutions for robotic extenders of wheelchairs must take into account both objective and subjective metrics for everyday activities in human environments. Virtual Reality (VR) constitutes a useful tool to effectively test design ideas and to verify performance criteria. This paper presents the development of a simulation environment, where three different manipulators to be mounted on a commercially available wheelchair have been considered. Experimental results are discussed in a significant case study, based upon users' feedback. © The Eurographics Association 2008.

Keywords: Human robot interaction | Kinematics | Rehabilitation robotics | Virtual Reality

Abstract: The present work deals with the re-designing of a locomotive, according to in force European standards, in the field of active and passive safety. The paper illustrates the use and the management of heterogeneous product information (2D drawings, technical documentation, photos), Virtual Reality tools and digital human models, for the re-designing of a locomotive, using a collaborative approach with a total absence of the reference digital models. The project development has been organised using a top-down approach in a collaborative environment. Finally, by means of the digital prototype of locomotive, a series of aesthetic, functional and ergonomic analyses, in virtual environment, has been performed. © Springer Verlag France 2007.

Keywords: Collaborative design | Ergonomics | Top-downapproach | Virtual reality

Abstract: The efficacy of virtual reality (VR) as a design support technique is widely recognised by industries. However, the efficiency of the routine employment of VR into the product development process (PDP) still finds an obstacle in the poor integration of the tools employed. The use of VR still needs long and quite hard procedures to work effectively; the models have to be converted into a format that is compatible with VR systems and each task requires an effort to prepare the virtual environment or to post-process the results that depend on the complexity of the task. This work analyses some VR applications into the PDP and describes some ideas to effectively support the operator that prepares the virtual environment. These ideas have been tested by developing four software interfaces, able to create an easy data exchange link between VR and other design tools like CAD, CAE and computer aided control engineering (CACE). © 2007 Springer-Verlag London Limited.

Keywords: CACE | CAD | CAD-VR integration | CAE | Virtual prototyping | Virtual reality

Abstract: Virtual Reality (VR) technologies provide novel modes of human computer interaction that can be used to support industrial design processes. The integration can be successful if supported by a method to qualify, select and design the VR technologies according to the company's requirements in order to improve collaboration in extended enterprises and timesaving. The aim of the present work is the definition of a method to translate the company's expectations into heuristic values that allow the benchmarking of VR systems. The method has been tested on a real test case. Copyright © 2007 by ASME.

Keywords: Benchmarking criteria | Design review | Virtual reality

Abstract: The increasing complexity of Air Traffic System is pushing toward the development of innovative and more automated tools conceived to manage it. In this scenario an important role is assumed by HCI (Human Computer Interfaces) used by air traffic controllers and operators to visualize and interact with air traffic data. Currently, information about 3D scenery are displayed with a two-dimensional representation. This paper presents the design, development and evaluation of an innovative interface for ATC (Air Traffic Control) based on a 4D (3D space + time) visualization display. The proposed interface allows the operator to perceive all the information, included meteorological conditions, useful for TWR/APP (ToWeR/APProach) control in a unique 4D synthetic reconstruction of the airport area. A particular attention is dedicated to the fourth variable, time, which is a fundamental parameter in ATC. A simple and fast trajectory prediction algorithm has been implemented in order to provide the operator with an effective "user assistance" tool in conflict detection activities. The interface has been evaluated performing test simulations and collecting results and useful advices for future developments by means of questionnaires.

Keywords: ATC | HCI | Synthetic environment | Virtual reality

Abstract: The current development of an "European Air Traffic Management Network" is emphasizing the role of training activities for controllers. The most advanced training centers are equipped with tower, radar and flight simulators in which all the phases of air traffic control process (including the pilot's condition) can be reproduced with an high degree of realism. Therefore, these systems mostly operate stand alone not exploiting the challenge of connecting single platforms in a unique distributed environment. In this paper a distributed training system is designed and experimented. It includes traffic simulation, flight simulation and real time voice communications. Air traffic in approach, landing and ground operations is simulated in a virtual air field and displayed on a table-top interface. One of the flight animations is performed in real time connecting the table-top with a FFS (Fixed Flight Simulator). The other planes which animate the environment follow recorded paths. Once the simulation is performed training activities continue using an off-line virtual debriefing tool of the simulated ATC process. The whole system aims at providing the controller with the awareness about the tasks performed by pilots and their consequences on the ATC scenario development. Copyright © 2008 by the American Institute of Aeronautics and Astronautics, Inc.

Keywords: ATC | Distributed simulation | Synthetic environments | Virtual reality

Abstract: Computed tomography is a medical instrument that can be useful not only for diagnostic purposes, but also for surgical planning, thanks to the fact that it offers volumetric information which can be translated in three dimensional models. These models can be visualized, but also exported to Rapid Prototyping (RP) systems, that can produce these structures thanks to the rapidity and versatility of the technologies involved. The literature reports various cases of stereolithographic models used in orthopedic, neurological, and maxillo-facial surgery. In these contexts, the availability of a copy of the real anatomy allows not only planning, but also the practical execution of surgical operations, within the limitations of the materials. Nevertheless, the Rapid Prototyping model also presents some disadvantages that can be reduced if practical simulation is accompanied by virtual simulation, performed on a digital model. The purpose of this work is to examine and present the use of Virtual Reality (VR) and Rapid Prototyping for surgical planning in Maxillo-Facial surgery. ©2008Muntaz B. Habal, MD.

Keywords: Osteogenesis distraction | Rapid prototyping | Reverse engineering | Surgical planning | Virtual reality

Abstract: We present a flexible framework for multimodal engineering applications using augmented reality. Our goal is to simplify the configuration procedures and to provide a higher grade of flexibility in multimodal interfaces. The system architecture is based on an extensible plug-in approach. A specific component has been designed to manage and synchronize the different multimodal inputs. A configurable XML based layer manages user preferences in a hierarchical way. We tested two engineering applications: a structural component re-design case and an industrial plant visualization. Industrial experts were positively impressed by the augmented visualization and by the usability of the interface. Most of them agree that the multimodal interface surpasses the desktop based interface.

Keywords: Augmented reality | Computer aided engineering | Multimodal interaction

Abstract: This study explores the potentialities of Virtual Reality for improving the learning process of mechanical design principles. It is focused on the definition of a proper experimental VR-based set-up whose performances match functional design, assembly design and geometrical tolerances prescription learning purposes. Benchmarking of VR technologies is based on the analysis of perception and on the usability and presence provided by the assessed systems. An experimental plan is defined and evaluation metrics are set.

Keywords: Experience-based learning environment | Mechanical design | Virtual reality

Abstract: In this paper an integration between a computer aided 3D modeller and an augmented reality environment is presented. The system is based on an high resolution web cam to acquire video stream from the real world and an electromagnetic tracking system (Flock of Bird by Ascension) which allows the user to interact with real and virtual objects in the augmented scene. The software to manage user interaction and data flow is implemented in Visual C++ and it makes use of the Artoolkit libraries, the OpenGL libraries and the Flock of Birds libraries. The purpose of the system is to speed up reverse engineering and prototyping processes, because the user can relate real object features in the scene to model its virtual entities or acquire geometrical features of existing parts. Moreover, the user can export the virtual models into CAD system or import external models to see how they fit in their real environment.

Keywords: Augmented reality | CAD | Motion tracking

Abstract: Although the 3D shape recovery of a real object have been greatly improved in the last few years, modeling a complex virtual object by starting from the real prototype is still a very time-consuming activity. In this paper an originally conceived method and testing software to recover a CAD model from a real object is presented. The developed software tool joins a professional Computer-Aided Design (CAD) and a Mixed Reality (MR) tool in the same interface, enabling the operator to use standard CAD tools and features together with a camera, which provides external image streaming displayed in the workspace background. Moreover, a special programmed library performs a real-time calculation of camera position and other parameters with respect to standard markers in order to drive the CAD 3D virtual camera and align it to external world. In that way rendered virtual models may be superimposed to external images of reality grabbed by the video camera. Thus with MR-CAD tool the operator may easily recover a complex shape directly from the external views of a real object or may start the object re-design from the previous reconstructed geometry. Furthermore the interface is totally integrated in a CAD environment, both avoiding to work with unfamiliar new software and exploiting CAD geometry database and tools. Finally, MR-CAD can be considered a significant step ahead in the bi-directional interaction of virtual and real models, reducing also the gap between real prototypes and CAD data.

Keywords: Augmented reality | Mixed-reality | Reverse engineering | Virtual prototyping

Abstract: Design for Usability (DFU) is a well known concept in computer science, since it refers to an important topic for the Human-Computer-Interaction research community. In the industrial design field this term is not so widely employed although the product usability is considered one of the most important factors for a product to be successful on the market. The importance of usability addresses the relationship between the interface of a product and its users. The success of several products on the market depends on complex interfaces, which require the user's intense interaction. Often such devices, which the designer sees as being extremely efficient, are too complex and incomprehensible to the user. Usability should be one of the peculiar aims of the design, and thus assessed throughout the design itself. It should work on the assumption that final users should be involved during all the phases of the development of a product. The present study proposes a methodology of participatory design based on the user-product interaction in a virtual environment, and developed by involving users in the definition of the interface of an interactive product for household use.

Keywords: Participatory design | Usability | Virtual reality

Abstract: The early identification of the optimal concept is a critical task of the design process in order to increase the chances of satisfying customers. The challenging aspect of the approach proposed in this work relies in the quality evaluation of virtual prototypes of new industrial products (i.e. concept designs) by adopting a statistical procedure previously applied to service industries. Following this approach, the optimal concept design is defined at the end of a process consisting of five phases: identification of the quality elements of the concept design, classification of the quality elements, generation and quality evaluation of product concepts and, finally, definition of the optimal concept. Currently, virtual reality (VR) environment offers the opportunity to evaluate the characteristics of different virtual prototypes by involving experts and/or customers, overcoming the need for several physical prototypes. On the other side, the dynamics of simulation and the stereoscopic visualization in VR environment provides a more realistic and impressive interaction with virtual prototypes than in CAD environment. The proposed methodology is fully exploited through two case studies: the choice of the optimal design for a traditional Neapolitan coffee maker, addressed by the Italian designer Riccardo Dalisi, and for a subassembly of a new minicar. © 2006.

Keywords: Concept design | Quality engineering | Virtual reality

Abstract: The aim of the paper is to make up a virtual showroom and work-through of a train model in order to allow railway companies showing new trains prototypes, in phase of concept, and present their new design in more exhaustive way than simply technical documentation. The possibility of applying Virtual Reality (VR) methodologies to make a scene more realistic as possible is a great advantage for the effectiveness of the presentation, in order to increase their competitivity. Shader technology allows the programmers to have control over shape, appearance (such as colour, lighting, reflection) and animation of objects, in order to make very realistic real-time rendering. In the paper the authors describe the use of shader technology in Virtual Design 2 (VD2) for realistic presentation of train prototypes in VR. The software VD2 is an extensive tool that allows following many phases of product development, from the creation of showroom for realistic presentations supporting shader technology to the assembly simulation or ergonomics analysis. Moreover, the possibility of interfacing with a wide range of input/output devices and the possibility to access to the API made this software to be chosen for Virtual Reality applications in the VR laboratory of the Competence Center for the Qualification of Transportation Systems founded by Campania Region (www.centrodicompetenzatrasporti.unina.it).© 2006 The Eurographics Association.

Keywords: Concept design | Shader technology | Train design | Virtual Reality

Abstract: This paper presents an innovative application of Augmented Reality (AR) techniques in the field of industrial engineering in which the user explores data from numerical simulations or the results of measurements and experiments, superimposed to the real object that they refer to. The user observes the object through a tablet PC, used as a video see-through handheld display. Data are visualized superimposed to the real object that represents a spatial reference relative to which the user can refer to, so the exploration is more natural compared to a traditional visualization software. Moreover, we have developed a new framework, called VTK4AR, that provides a set of useful software classes for the rapid development of AR applications for scientific visualization. VTK4AR is built on top of VTK (an open source API for scientific visualization), so it will be possible to employ a wide range of visualization techniques in many application fields, and moreover, it is possible to interactively manipulate data-sets in order to achieve a more effective way of visualization. © 2006 The Visualization Society of Japan and Ohmsha, Ltd.

Keywords: Augmented reality | Interaction techniques | Scientific visualization

Abstract: In the last ten years many Augmented Reality (AR) applications for Scientific Visualization have been developed, attesting the effectiveness of this technique for data visualization and interaction. In all these applications, a software framework for scientific visualization was used to process data to be visualized, while an AR system was employed to display these data within an AR context. Hence, everyone who intended to approach the development of such applications should become necessarily familiar with the scientific visualization framework and the augmented reality one. This is of course an hurdle for the applications development, and the idea behind this work is exactly to provide a software framework that simplifies the development of such applications. With this in mind, we extended an existing and powerful open source library for scientific visualization (VTK) with few but useful classes for the interfacing with an existing AR library (ARToolKit) to easily handle the video see-through and the video-tracking functionalities. The resulting software tool, called VTK4AR, can be considered as an all in one software framework specific for scientific visualization in AR. Moreover, since it is built on top of VTK, it will be possible to employ a wide range of visualization techniques in many application fields. In particular, it has been tested in two AR applications: one for displaying data relative to a CFD simulation of a flow past a helmet, and another for displaying the forming error obtained prototyping an ankle support with the incremental forming technique. © 2006 The Eurographics Association.

Abstract: The majority of Virtual Reality applications developed today are either specific product oriented, not flexible enough to be implemented in every industrial design process to achieve complex real world tasks. In this context the adoption of VR systems, designed and customized on the need of the specific company, can be accepted only if it is possible to quantify the achievable benefits in terms of time, quality and cost. Our research focuses on how these benefits can be objectively measured. A benchmarking program and related metrics to explore advantages and disadvantages connected with the new design technology have been studied. Our main goal is to measure the performance of the VR-based design review processes by meaningful test cases.

Keywords: Design review | Industrial design | Virtual reality

Abstract: The aim of this work is an efficient methodology development for a real-time control of human assembling sequences of mechanical components. The method involves a CAD environment, an hardware system, referred to as a PAA (Personal Active Assistant), and a set of Mixed Reality features. The whole scheme is targeted to positively influence the connection between CAD and Mixed Reality in order to proficiently reduce the gap between engineers and manual operators. The system is based on a CAD assembly module and on an Mixed Reality wearable equipment. It can be used to improve several activities in the industrial field, such as operator professional training, optimal assembly sequence seeking or on-field teleconferencing (suitable for remote collaboration or for full exploitation of Concurrent Engineering suggestions during design and set up stages). The main characteristic of PAA is a real-time wireless linkage to a remote server or designer workstation, where project geometric database is stored. The Mixed Reality wearable equipment consists of an optical see-through display device and a PAA head-mounted camera. The user can freely operate in the mixed environment, while the camera can record the human driven assembly sequence and check the efficiency and correctness via object recognition: an incrementally sub-assembly detection algorithm has been developed in order to achieve complex dataset monitoring. Conversely, designer or assembly planner can exploit the peculiarities of Mixed Reality-based assembly: a straightforward interaction with the assembly operator can be obtained by sending vocal advices or by displaying superimposed visual information on the real scene. In the paper a new method for CAD models and Mixed Reality environment integration will be presented and discussed in order to improve and simplify personnel training or warehouse part seeking. © 2006 - IOS Press and the auther(s). All rights reserved.

Keywords: Assembly | CAD | Concurrent engineering | Mixed reality

Abstract: The paper describes an industrial application of VR techniques in the field of Digital Mock-Up (DMU) analyses, reporting the results obtained with the development of a software application that allows Elasis engineers to evaluate important design parameters in the serviceability studies. These analyses can be conducted during the early stages of the design process using DMUs and finding the difficulties related to the serviceability tasks. Using VR techniques it is possible to simulate the entire operative context in which the human operator works during the assembly/disassembly task. In this way a designer can directly verify some potential difficulties in component reachability, in posture and visibility.

Keywords: Automotive | Design for maintenance | Serviceability | Virtual reality

Abstract: Gestures, besides speech, represent the mostly used means of expression by humans. For what regards the product design field, designers have multiple ways for communicating their ideas and concepts. One of them concerns the model making activity, where designers make explicit their concepts by using some appropriate tools and specific hand movements on plastic material with the intent of obtaining a shape. Some studies have demonstrated that visual, tactile and kinesthetic feedbacks are equally important in the shape creation and evaluation process [1]. The European project "Touch and Design" (T'nD) (www.kaemart.it/touch-and-design) proposes the implementation of an innovative virtual clay modeling system based on novel haptic interaction modality oriented to industrial designers. In order to develop an intuitive and easy-to-use system, a study of designers' hand modeling activities has been carried out by the project industrial partners supported by cognitive psychologists. The users' manual operators and tools have been translated into corresponding haptic tools and multimodal interaction modalities in the virtual free-form shape modeling system. The paper presents the project research activities and the results achieved so far. Copyright 2005 ACM.

Keywords: Haptic interaction | Haptic modeling | Virtual prototyping

Abstract: We have developed a pen device for CAD applications in virtual reality which provides novel features compared to existing systems. The SenStylus consists of a wireless pen designed to be ergonomically handled by the user for spatial interaction using a six degree of freedom optical tracking. In addition to the classic digital button(s) input, it provides analog multi-axial control, and a dual-rumble feedback output. We have integrated the device into an existing virtual reality CAD environment and extended the application functionalities with new devicespecific features. The SenStylus vibration feedback improves perception in the virtual world by controlling frequency, amplitude, and duration of the feedback, simulating a variety of responses during collisions and selection tasks. This capability enforces the visual depth sensitivity, which is critical when wo rking with complex CAD models. The multi-axial analog input provides a natural interaction paradigm to the user, thus simulating pen pressure and angle as in real world sketching and in real clay modeling. Dynamic tool-tip dimensioning and shaping are implemented as extra features. We present some applications to prove the added value of the SenStylus. The evaluation of the device received positive feedback by designers and engineers alike. The new features offered by this device can easily be extended to other VR applications using the API provided. Copyright UNION Agency - Science Press.

Keywords: 3D interaction | CAD | User interface hardware | Virtual reality

Abstract: The use of 3D tracking systems in a Virtual Reality environment may definitely change CAD interfaces and free-form surface modelling. In this paper an original method for full 3D interactive surface shaping and modifying is described. VISM (Virtual Integrated Surface Modeller) has been developed with the leading idea that 3D tracking system can dramatically speed up modelling sessions. On the opposite of a W-I-M-P (Windows-Icons-Menu-Pointer), paradigm common to most current CAD systems, VISM demonstrates that Virtual Reality devices can manage all types of surface in a unique shape generative action. Unlike "Virtual Clay" based and "Metaball" modelling techniques, VISM both wants to give to engineers and designers a more intuitive and natural tool to get 3D shapes. Based on Polhemus Fastrak and stereoscopic vision, VISM doesn't provide icons to (he designer, leading to a "null icons" and "null menu" full 3D interface. The new interface is fully implemented on bi-manual input system on top of a Virtual Reality environment. The entity grabbing is also supported by pinch-enabled gloves. The designer exploits a NURBS curve tool to deform a NURBS surface and extracts drive curve direction from his right hand movement. The curve tool may be also real-time deformed with left hand through node-control point repositioning. Furthermore the modeller is fully implemented using NURBS curves and surfaces and a fast surface-over-curve positioning and deformation has been implemented, replacing both traditional snapping and picking activities.

Keywords: 3D interface | CAD | Spatial Tracking | Surface modelling | Virtual Reality

Abstract: In this paper we introduce a virtual integrated surface modeller (VISM) equipped with two advanced tools for 3D surface modelling in virtual reality (VR). VISM has been implemented thinking that the immediate visual feedback greatly improve complex surface modelling and also creativity. The proposed new tools are progressive skinning (PS) and curve-over-surface shaping (COSS), both implemented on a bimanual virtual environment. PS lets the designer generate a new surface by interactively and automatically adding section curves to a non-uniform rational B-spline surface. COSS is based on curve-over-a-surface progressive constraining in order to deform an area of an existent surface. Efficient numerical solutions are proposed to achieve true interactive modelling sessions.

Keywords: 3d geometric modelling | Human-machine interaction | Non-uniform rational B-spline | Real-time interaction | Virtual reality

Abstract: An integrated environment based on CAD assembly software and on an Augmented Reality wearable system is used to improve the overall integration between engineering design and real prototypes manufacturing. The environment following called - Personal Active Assistant (PAA) - exploits a CAD tool connection to remarkably improve object recognition, best assembly sequence optimization, and operator instructions generation. PAA is real-time and wirelessly linked to a remote server or designer workstation where project geometric database is stored. The PAA head-mounted camera is also able to acquire the human-driven assembly sequence and check the efficiency and correctness via object recognition: an incremental sub-assembly detection algorithm has been developed in order to achieve complex dataset monitoring. On the other hand, the Augmented Reality-based assembly evaluation tool allows engineers to interact directly with the assembly operator while manipulating the real and virtual prototype components. Information from the assembly planner can be displayed, directly superimposed, on the real scene by using a see-through head-mounted display. Thus the new combined software and hardware equipment may be considered a step ahead in the support of true concurrent engineering and remote collaboration, strongly improving this latter through a better heterogeneous task integration. Several tests have been performed also to achieve personnel training and warehouse part seeking.

Keywords: Assembly | Augmented reality | CAD | Concurrent engineering | Virtual reality

Abstract: This paper outlines new trends in geometric modelling, showing how systems are moving from a geometry-based approach to a physically-based approach. The possibility to simulate the actual behaviour of a product is at the basis of Virtual Prototypes that are becoming a common practice in today's product development process. As a consequence, also interaction modalities and techniques have to be improved in order to satisfy new system requirements and functionalities. The research work described in this paper shows how haptic interaction techniques represent an evolution of current interaction technologies providing intuitive and realistic modalities for interacting with virtual applications. In particular, the paper describes a research work we have carried out integrating haptic technologies together with physically-based modelling and simulation techniques. © 2001 Springer Science+Business Media New York.

Keywords: Haptic interaction | Human-computer interaction | Non-rigid material modelling | Physically-based modelling | Virtual prototyping

Abstract: A prst step towards a semi-immersive Virtual Reality (L!R) interface for Finite Element Analysis (FEA) is presented in this paper. During recent years, user interfaces of FM solvers have matured from characterbased command-line driven implementations into easy-touse graphical user interfaces (GUS). This new generation of GUIs provides access to intuitive and productive tools for the management and analysis of structural problems. Many pre-And post-processors have been implemented targeting the simplification of the manmachine interface in order to increase the ease of use and provide better visual analysis of FEA solver results. Nevertheless, none of these packages provides a real 3Denabled interface. The main objective of this project is to join state-of-The-Art visualization technology, VT devices, and FM solvers into the integrated development environment VRFM.

Keywords: 3D Modeling | Finite Element Analysis | Interactive Modeling | Simulation. | Virtual Reality

Abstract: Virtual Environments (VEs) have the potential to revolutionize traditional product design by enabling the transition from conventional CAD to fully digital product development. The presented prototype system targets closing the "digital gap" as introduced by the need for physical models such as clay models or mockups in the traditional product design and evaluation cycle. We describe a design environment that provides an intuitive human-machine interface for the creation and manipulation of three-dimensional (3D) models in a semi-immersive design space, focussing on ease of use and increased productivity for both designer and CAD engineers.

Keywords: 3D sculpting | Computer aided geometric design (CAGD) | Immersive environments | Virtual reality