Image Processing & Applications

Within the framework of the assisted systems for automotive applications, considerable research has been employed to monitoring the driver's attention level in order to assess the risk level of the driving scenario. In this context, physiological monitoring of the driver's condition has emerged as a relevant approach to enhance driving assistance without having an invasive approach. According to these premises, the authors have developed a driving assistance system capable to employ a dedicated bio-sensor for capture the driver's photoplethysmographic (PPG) signal, which is closely linked to their level of attention. This PPG signal is then processed by a dedicated deep learning architecture to reconstruct the driver's attention level. Meanwhile a separate automotive-grade intelligent vision-based system has been developed to quantify the risk level of the driving scenario by means of a video saliency analysis technique. The effectiveness of this comprehensive system has been validated through experimental results.

Keywords: ADAS , automotive , Deep Learning , risk assessment

Detecting and identifying production defects in the semiconductor industry are crucial for maintaining quality control during manufacturing. The use of new materials, such as Silicon and Silicon Carbide, highlights the need for a reliable wafer defect detection system. The Electrical Wafer Sorting (EWS) stage, which involves an electrical analysis of defect maps, is effective in spotting anomalies and defect patterns on wafers. This phase is time-consuming but enables semiconductor companies to improve and optimize their manufacturing processes, including the integration of advanced deep learning techniques. The proposed pipeline aims to meet the demand for a fully automated system to identify manufacturing defects in wafers, utilizing intelligent analysis of EWS wafer maps in combination with a Deep Convolutional Neural Network and an unsupervised subsystem. The stringent level of intelligent control at the EWS stage is necessary because the devices produced from the analyzed wafers are mainly power devices used in the power and inverter systems of the latest generation of conventional and electric cars. Experimental results have confirmed the effectiveness of this approach.

Keywords: Deep learning , Defect Analysis , Pattern recognition , Silicon , Silicon Carbide

RGB-D cameras have been widely used in various research fields and applications in recent years. With the growing availability of commercial products, choosing the most suitable sensor for a specific application has become more complex. The lack of tools to evaluate and compare the metrological performance of these sensors was addressed by the recent ISO 10360-13:2021 standard, which defines the standards for the metrological characterization of 3D optical coordinate measuring systems. This paper applies the ISO 10360-13 methodology to characterize and compare four RGB-D cameras: three Intel® RealSense™ (D415, D435i, D455) and one Microsoft® Azure Kinect DK. The procedures described in the standard were integrated with three tests: the analysis of systematic errors measured acquiring a planar surface at increasing distances; a test to analyze the resolution performances with a specifically devised artifact; the evaluation of 3D reconstruction performances on two objects with different geometric characteristics. The results highlighted the strengths and weaknesses of each device. The D415 showed better performance in reconstructions from close-range acquisitions. The Azure Kinect DK had the best results for systematic error and resolution. The D455 generally achieved the best performance in the standard tests, followed by the D435i.

Keywords: device characterization , Intel RealSense , ISO 10360-13:2021 standard , Microsoft Azure Kinect DK , performance evaluation

This study contributes to understanding semi-automated ergonomic risk assessments in industrial manufacturing environments, proposing a practical tool for enhancing worker safety and operational efficiency. In the Industry 5.0 era, the human-centric approach in manufacturing is crucial, especially considering the aging workforce and the dynamic nature of the entire modern industrial sector, today integrating digital technology, automation, and sustainable practices to enhance productivity and environmental responsibility. This approach aims to adapt work conditions to individual capabilities, addressing the high incidence of work-related musculoskeletal disorders (MSDs). The traditional, subjective methods of ergonomic assessment are inadequate for dynamic settings, highlighting the need for affordable, automatic tools for continuous monitoring of workers' postures to evaluate ergonomic risks effectively during tasks. To enable this perspective, 2D RGB Motion Capture (MoCap) systems based on computer vision currently seem the technologies of choice, given their low intrusiveness, cost, and implementation effort. However, the reliability and applicability of these systems in the dynamic and varied manufacturing environment remain uncertain. This research benchmarks various literature proposed MoCap tools and examines the viability of MoCap systems for ergonomic risk assessments in Industry 5.0 by exploiting one of the benchmarked semi-automated, low-cost and non-intrusive 2D RGB MoCap system, capable of continuously monitoring and analysing workers' postures. By conducting experiments across varied manufacturing environments, this research evaluates the system's effectiveness in assessing ergonomic risks and its adaptability to different production lines. Results reveal that the accuracy of risk assessments varies by specific environmental conditions and workstation setups. Although these systems are not yet optimized for expert-level risk certification, they offer significant potential for enhancing workplace safety and efficiency by providing continuous posture monitoring. Future improvements could explore advanced computational techniques like machine learning to refine ergonomic assessments further.

Keywords: Computer vision , Deep learning , Ergonomic monitoring , Ergonomic Risk Assessment , Motion capture , Work-related MSDs

This chapter explores the challenging topic of emotion recognition by affective computing. The importance of considering and understanding people's emotions in interaction design is discussed, focusing on the role of human emotions in the entire life cycle of human–system interaction as a means to innovate products and services. The measurement of emotions is also analyzed, including the classification of human emotions and recognition methods, as well as current techniques for measuring emotional responses. An emotional-based approach and related technologies are considered in managing the entire life cycle of human–system interaction as an innovation driver. This chapter also presents how to use affective computing in cross-transversal applications, concentrating on potential applications and different case studies. This chapter concludes with a look towards a world of emotional intelligence, where affective computing plays a crucial role in collecting and analyzing emotional data to support innovative product and service experiences.

Growth monitoring of crops is a crucial aspect of precision agriculture, essential for optimal yield prediction and resource allocation. Traditional crop growth monitoring methods are labor-intensive and prone to errors. This study introduces an automated segmentation pipeline utilizing multi-date aerial images and ortho-mosaics to monitor the growth of cauliflower crops (Brassica Oleracea var. Botrytis) using an object-based image analysis approach. The methodology employs YOLOv8, a Grounding Detection Transformer with Improved Denoising Anchor Boxes (DINO), and the Segment Anything Model (SAM) for automatic annotation and segmentation. The YOLOv8 model was trained using aerial image datasets, which then facilitated the training of the Grounded Segment Anything Model framework. This approach generated automatic annotations and segmentation masks, classifying crop rows for temporal monitoring and growth estimation. The study's findings utilized a multi-modal monitoring approach to highlight the efficiency of this automated system in providing accurate crop growth analysis, promoting informed decision-making in crop management and sustainable agricultural practices. The results indicate consistent and comparable growth patterns between aerial images and ortho-mosaics, with significant periods of rapid expansion and minor fluctuations over time. The results also indicated a correlation between the time and method of observation which paves a future possibility of integration of such techniques aimed at increasing the accuracy in crop growth monitoring based on automatically derived temporal crop row segmentation masks.

Keywords: automatic annotation , grounded SAM , grounding detection transformer with improved denoising anchor boxes (grounding DINO) , growth monitoring , segment anything model (SAM)

The recent trend in the textile setting highlights the need for more effective and affordable processes to make the garment market more sustainable. Secondhand garment reuse and fabric recycling are two virtuous pathways toward material saving and sustainable transition. Sorting processes are fundamental enablers of this transition. The main operations involved are identification, classification, and quality assessment. This work proposes a collaborative robotic solution for the first identification and classification of garments to support human experts, along with the following assessment operations for garment and fabric quality and the identification of textile fibers. The proposed approach is based on a collaborative system integrating robots, machine vision models, and dedicated gripper systems to perform the picking actions required to identify and classify garment categories. The first experiments demonstrate the feasibility of the proposed approach as well as the self-learning capability of the system using the detection model continuous training.

Keywords: Garment detection models , Garment sorting , Gripper for garment picking , Human-robot collaborative system

Digital image correlation (DIC) is widespread in many research fields to achieve full-field displacement measurements. The main idea behind the DIC method is the computation of the correlation factor between image subsets before and after deformation to assess the rough displacement. The correlation map, which is computed at the pixel level, is then refined to subpixel accuracy through interpolation methods or non-linear optimization. The aim of this research is to develop two digital image correlation algorithms for vibration measurements. The first will be based on the conventional approach, which computes the correlation factor in the spatial domain by convoluting the pixel-wise grey levels to obtain the correlation map. The second will be based on the computation of the correlation factor in the “frequency” domain. The algorithms will be evaluated in terms of performance when applied to the same image set, by comparing the computational time, and the sensitivity to small amplitude values, which are typical of vibrations.

This paper focuses on the experimental investigation of the effects of blade-root contact on turbine blade dynamics. Since under operating conditions the blade undergoes both cyclic loads and centrifugal loads, the nonlinear response of a dummy blade is investigated by means of a specially designed test rig, which allows tuning normal load and high-frequency excitation acting on the sample. To avoid interfering with the measurements, it is essential to avoid adding damping and external masses by attaching measuring instruments to the vibrating blade. Accordingly, Stereo Digital Image Correlation (Stereo-DIC) was used in this work to obtain full-field, non-contact measurements of specimen deformation. In this work, tests are performed exciting at a frequency higher than the maximum frame rate of the high-resolution cameras employed, and then a down-sampling technique, called Smoothed Harmonics Analysis (SHA), is implemented to avoid aliasing error in the detection of displacement signals.

Digital Image Correlation (DIC) stands as a promising non-contact method for assessing the complete dynamics of vibrating structures. Utilizing one or two cameras, DIC can capture either 2D or 3D dynamics. Through subsequent post-processing of images, structural points are correlated across different time frames to derive displacement information. Given the abundance of camera options available, spanning frame rate, resolution, versatility, and quality, the selection process is dictated by specific application needs and financial considerations. This study seeks to explore the feasibility of utilizing the cameras integrated into a ubiquitous device of everyday life: the smartphone. The proposed approaches offer straightforward procedures suitable for educational purposes or early investigation of structure dynamics. The paper presents case studies and discusses simple experiments that students or DIC beginners can conduct on simple vibrating structures. Initially, a simple 2D dynamics was investigated by recording the vibration of a clamped beam with moving constraints using just a single smartphone. The L-type cross section of the beam emphasizes its in-plane motion. The frames were post-processed with open-source software and the deformed shapes were extracted. This assignment serves as an introductory exercise in DIC fundamentals. Subsequently, the focus shifts to investigating the 3D dynamics of the beam. Given the challenges associated with synchronizing multiple smartphone video acquisitions, the image of the beam was instead reflected by two mirrors, and these reflected images were recorded by a single smartphone. Then, the recorded images were split and processed to obtain the full-field 3D dynamics of the beam.

Position-controlled servomechanisms are the core elements of flexible manufacturing plants, primarily utilized to actuate robotic systems and automated machines. To match specific torque and costs requirements, typical servomechanism arrangements comprise precision reducers, which introduce motion errors that heavily limit the final performance achievable. Such errors are complex to model and depend from speed, dynamic loading conditions and temperature. Accurate characterization is fundamental to develop digital twins and advanced control strategies aimed at their active prediction and compensation. To properly assess the servomechanisms behavior and elaborate high-fidelity virtual models, instrumented test rigs have been proposed which can replicate the time-varying working conditions encountered in real industrial environments. In this context, the present paper reports about a novel engineering method for developing an active loading apparatus, namely a programmable mechatronic device that can deliver custom loads in a highly dynamic manner. The proposed system, consisting of a secondary servomotor and related rotating vector reducer, is integrated and synchronized within an existing instrumented test rig and is controlled in torque mode via a programmable logic controller. The paper mainly focuses on the description of the implemented closed-loop control and on the related tuning and calibration processes, demonstrating that the proposed solutions avoid important measurement errors that could compromise the final effectiveness of the system. The study finally explores the potential benefits of introducing a filter to further enhance system performance. At last, to prove the importance of stabilizing the rig and demonstrate the influence of the control parameters on its measurements, a standard test aimed at assessing the reducer transmission error is conducted adopting different parameter settings.

Keywords: Loading system , Mechanical transmission , Servomechanisms , Stabilization , Test rig , Torque control

Italian Sign Language (LIS) is the primary form of communication for many members of the Italian deaf community. Despite being recognized as a fully fledged language with its own grammar and syntax, LIS still faces challenges in gaining widespread recognition and integration into public services, education, and media. In recent years, advancements in technology, including artificial intelligence and machine learning, have opened up new opportunities to bridge communication gaps between the deaf and hearing communities. This paper presents a novel educational tool designed to teach LIS through SIGNIFY, a Machine Learning-based interactive serious game. The game incorporates a tutorial section, guiding users to learn the sign alphabet, and a classic hangman game that reinforces learning through practice. The developed system employs advanced hand gesture recognition techniques for learning and perfecting sign language gestures. The proposed solution detects and overlays 21 hand landmarks and a bounding box on live camera feeds, making use of an open-source framework to provide real-time visual feedback. Moreover, the study compares the effectiveness of two camera systems: the Azure Kinect, which provides RGB-D information, and a standard RGB laptop camera. Results highlight both systems' feasibility and educational potential, showcasing their respective advantages and limitations. Evaluations with primary school children demonstrate the tool's ability to make sign language education more accessible and engaging. This article emphasizes the work's contribution to inclusive education, highlighting the integration of technology to enhance learning experiences for deaf and hard-of-hearing individuals.

Keywords: gamification , gesture recognition , hand sign alphabet , machine learning , serious game , social inclusion

Femur fractures are a significant worldwide public health concern that affects patients as well as their families because of their high frequency, morbidity, and mortality. When employing computer-aided diagnostic (CAD) technologies, promising results have been shown in the efficiency and accuracy of fracture classification, particularly with the growing use of Deep Learning (DL) approaches. Nevertheless, the complexity is further increased by the need to collect enough input data to train these algorithms and the challenge of interpreting the findings. By improving on the results of the most recent deep learning-based Arbeitsgemeinschaft für Osteosynthesefragen and Orthopaedic Trauma Association (AO/OTA) system classification of femur fractures, this study intends to support physicians in making correct and timely decisions regarding patient care. A state-of-the-art architecture, YOLOv8, was used and refined while paying close attention to the interpretability of the model. Furthermore, data augmentation techniques were involved during preprocessing, increasing the dataset samples through image processing alterations. The fine-tuned YOLOv8 model achieved remarkable results, with 0.9 accuracy, 0.85 precision, 0.85 recall, and 0.85 F1-score, computed by averaging the values among all the individual classes for each metric. This study shows the proposed architecture\'s effectiveness in enhancing the AO/OTA system\'s classification of femur fractures, assisting physicians in making prompt and accurate diagnoses.

Keywords: Computer assisted diagnosis , Convolutional neural networks , Deep learning , Femur fracture

Industry 4.0 conceptualizes the automation of processes through the introduction of technologies such as artificial intelligence and advanced robotics, resulting in a significant production improvement. Detecting defects in the production process, predicting mechanical malfunctions in the assembly line, and identifying defects of the final product are just a few examples of applications of these technologies. In this context, this work focuses on the detection of ultrasound probes' surface defects, with a focus on Esaote S.p.A.'s production line probes. To date, this control is performed manually and therefore biased by many factors such as surface morphology, color, size of the defect, and by lighting conditions (which can cause reflections preventing detection). To overcome these shortfalls, this work proposes a fully automatic machine vision system for surface acquisition of ultrasound probes coupled with an automated defect detection system that leverage artificial intelligence. The paper addresses two crucial steps: (i) the development of the acquisition system (i.e., selection of the acquisition device, analysis of the illumination system, and design of the camera handling system); (ii) the analysis of neural network models for defect detection and classification by comparing three possible solutions (i.e., MMSD-Net, ResNet, EfficientNet). The results suggest that the developed system has the potential to be used as a defect detection tool in the production line (full image acquisition cycle takes ~ 200 s), with the best detection accuracy obtained with the EfficientNet model being 98.63% and a classification accuracy of 81.90%.

Keywords: Artificial intelligence , Image processing , Inspection system , Product characterization , Virtual prototyping

The aim of this study was to assess the value of the use of software for the preoperative evaluation of cranial bone thickness in syndromic patients undergoing Le Fort III osteotomy. Four patients were evaluated preoperatively to determine whether they were eligible for distraction osteogenesis. Data from the computed tomography scans was evaluated using advanced reverse engineering tools to determine the temporal bone thickness. Three patients showed adequate values for the positioning of a rigid external distractor device (average thickness values >3.5 mm), while one patient showed insufficient bone thickness (average value <2.5 mm) and therefore underwent midface advancement according to the traditional technique. Adequate midface advancement was obtained in the three patients who underwent distraction osteogenesis. No complications related to the rigid external distractor were observed. A shorter skeletal advancement was obtained in the patient who underwent Le Fort III osteotomy according to the traditional technique. A cerebrospinal fluid fistula was observed after the removal of the plates, requiring surgical repair. Software evaluation of the cranial bone thickness is a useful tool in the surgical planning of Le Fort III osteotomy in patients affected by syndromic craniosynostosis.

Keywords: Distraction osteogenesis , Le Fort III osteotomy , Syndromic craniosynostosis

No abstract available

Objectives: The research's purpose is to develop a software that automatically integrates and overlay 3D virtual models of kidneys harboring renal masses into the Da Vinci robotic console, assisting surgeon during the intervention. Introduction: Precision medicine, especially in the field of minimally-invasive partial nephrectomy, aims to use 3D virtual models as a guidance for augmented reality robotic procedures. However, the co-registration process of the virtual images over the real operative field is performed manually. Methods: In this prospective study, two strategies for the automatic overlapping of the model over the real kidney were explored: the computer vision technology, leveraging the super-enhancement of the kidney allowed by the intraoperative injection of Indocyanine green for superimposition and the convolutional neural network technology, based on the processing of live images from the endoscope, after a training of the software on frames from prerecorded videos of the same surgery. The work-team, comprising a bioengineer, a software-developer and a surgeon, collaborated to create hyper-accuracy 3D models for automatic 3D-AR-guided RAPN. For each patient, demographic and clinical data were collected. Results: Two groups (group A for the first technology with 12 patients and group B for the second technology with 8 patients) were defined. They showed comparable preoperative and post-operative characteristics. Concerning the first technology the average co-registration time was 7 (3–11) seconds while in the case of the second technology 11 (6–13) seconds. No major intraoperative or postoperative complications were recorded. There were no differences in terms of functional outcomes between the groups at every time-point considered. Conclusion: The first technology allowed a successful anchoring of the 3D model to the kidney, despite minimal manual refinements. The second technology improved kidney automatic detection without relying on indocyanine injection, resulting in better organ boundaries identification during tests. Further studies are needed to confirm this preliminary evidence.

Keywords: artificial intelligence , kidney cancer , nephron-sparing surgery , renal cell carcinoma , robotic surgery , three-dimensional imaging

Autologous ear reconstruction is a surgical procedure aimed at total or partial reconstruction of the ear of patients with microtia, which is a malformation or absence of the anatomical element. This procedure involves harvesting cartilage material from the patient at the costal level, cut and sculpt the tissue to create the skeleton of the ear, which is then inserted into a subcutaneous pocket in the malformed region. The sculping procedure takes inspiration from the geometry of the patient's contralateral healthy ear and can be very complex given the particularity of the elements to be reconstructed. Taking advantage of Reverse Engineering and Additive Manufacturing technologies, the authors, in collaboration with hospital staff, proposed patient-specific three-dimensional cutting guides that can facilitate the surgeon in sculpting the ear structure. In this work, a fully automated procedure for creating the printable digital model of such guides is proposed: at first is performed the segmentation of ear elements, which represent the starting data for the automated CAD modelling of the surgical guides. The automatic CAD modelling procedure was tested and validated by medical personnel.

Keywords: CAD Modelling , Ear Reconstruction , Reverse Engineering , Surgical Guide

Lattice structures are receiving a renewed interest in many areas such as biomedical and industrial fields, due to the capabilities of additive manufacturing technologies which allow for the fabrication of very complex shapes. Currently, several methods and tools are described in the scientific literature and some commercial software are introducing dedicated packages to reduce designer efforts for lattice structure design and optimization. However, by using commercial CAD/CAM tools in the fabrication of components filled by lattice structure, several critical issues remain and need to be taken into consideration. This work aims at manufacturing variable-density lattice structures via fused deposition modeling deriving the density map from a grayscale or color image. In the proposed approach, the shell-based lattice model is not achieved by CAD tools, but only during the CAM process, while the lattice relative density is computed by editing the G-code, modifying the extrusion flow according to the local grayscale of a volumetric CAD model, defined from an image. The main advantages are related to the absence of a graded lattice geometric model and the consistency of the toolpath. The method is tested on various images and patterns, and can find applications in artworks, embedded information on components, and functional 3D printed parts, such as the replication of the density map of a bone derived from a DICOM grayscale image.

Keywords: Additive manufacturing , Fused deposition modeling , Graded lattices , Heterogeneous objects

Due to the increasing number of people with severe obesity, the demand for patient-specific modelling in bariatric surgery (BS) is increasing because its potentialities in the improvement of surgical planning, optimization of outcomes and prediction of the mechanical response of the stomach. However, the patient-specific anatomical reconstruction is a pivotal and often time-consuming step due to the lack of efficient and fully automatized tools. Ongoing studies on multi-organ segmentation methods based on neural networks for magnetic resonance images (MRI) are currently available, but they have still several limits, mainly due to both the highly flexible individuals anatomical properties, and convolutional neural networks (CNN) trained only in the detection of physiological stomachs. The aim of this work is to perform a convenient transfer learning from a general-purpose CNN, able to improve the performance in automatically detecting the stomach region of patients with severe obesity. The proposed approach represents the basis for the development of pre- and post-surgical computational models for rapid clinical analysis, especially to boost the mechanical stimulation of gastric receptors. The segmentation masks and the corresponding 3D models were compared with the corresponding manual MRI segmentation as ground truth. Intersection Over Union (IOU) and DICE coefficients (DICE) were used to evaluate 2D masks segmentation, while the Relative Volume Error (RVE), mean surface distance (MSD), standard deviation and the Normalised Hausdorff distance (NHD) were applied to assess the obtained 3D results.

Keywords: Bariatric surgery , Image segmentation , MRI , Stomach