Image Processing & Applications

Nowadays, manufacturing plants are required to be flexible to respond quickly to customer demands, adapting production and processes without affecting their efficiency. In this context, Industrial Robots (IRs) are a primary resource for modern factories due to their versatility which allows the execution of flexible, reconfigurable, and zero-defect manufacturing tasks. Even so, the control and programming of the commercially available IRs are limiting factors for their effective implementation, especially for dynamic production environments or when complex applications are required. These issues have stimulated the development of new technologies that support more efficient methods for robot control and programming. The goal of this research is to identify and evaluate the main approaches proposed in scientific papers and by the robotics industry in the last decades. After a critical review of the standard IR control schematic, the paper discusses the available control alternatives and summarizes their characteristics, range of applications, and remaining limitations.

Keywords: industrial robots , instruction streaming , open controller , robot control , robot programming , trajectory streaming

Indeterminate lung nodules detected on CT are common findings in the clinical practice, and the correct assessment of their size is critical for patient evaluation and management. We compared the stability of three definitions of nodule diameter (Feret's mean diameter, Martin's mean diameter and area-equivalent diameter) to inter-observer variability on a population of 336 solid nodules from 207 subjects. We found that inter-observer agreement was highest with Martin's mean diameter (intra-class correlation coefficient = 0.977, 95% Confidence interval = 0.977–0.978), followed by area-equivalent diameter (0.972, 0.971–0.973) and Feret's mean diameter (0.965, 0.964–0.966). The differences were statistically significant. In conclusion, although all the three diameter definitions achieved very good inter-observer agreement (ICC > 0.96), Martin's mean diameter was significantly better than the others. Future guidelines may consider adopting Martin's mean diameter as an alternative to the currently used Feret's (caliper) diameter for assessing the size of lung nodules on CT.

Keywords: Computed tomography , Inter-observer variability , Lung nodules , Size assessment

In situations requiring high levels of customization and limited production volumes, additive manufacturing (AM) is a frequently utilized technique with several benefits. To properly configure all the parameters required to produce final goods of the utmost quality, AM calls for qualified designers and experienced operators. This research demonstrates how, in this scenario, artificial intelligence (AI) could significantly enable designers and operators to enhance additive manufacturing. Thus, 48 papers have been selected from the comprehensive collection of research using a systematic literature review to assess the possibilities that AI may bring to AM. This review aims to better understand the current state of AI methodologies that can be applied to optimize AM technologies and the potential future developments and applications of AI algorithms in AM. Through a detailed discussion, it emerges that AI might increase the efficiency of the procedures associated with AM, from simulation optimization to in-process monitoring.

Keywords: Additive manufacturing , Artificial intelligence , Deep learning , Machine learning , Optimization , Review

The current study presents a multi-task end-to-end deep learning model for real-time blood accumulation detection and tools semantic segmentation from a laparoscopic surgery video. Intraoperative bleeding is one of the most problematic aspects of laparoscopic surgery. It is challenging to control and limits the visibility of the surgical site. Consequently, prompt treatment is required to avoid undesirable outcomes. This system exploits a shared backbone based on the encoder of the U-Net architecture and two separate branches to classify the blood accumulation event and output the segmentation map, respectively. Our main contribution is an efficient multi-task approach that achieved satisfactory results during the test on surgical videos, although trained with only RGB images and no other additional information. The proposed multi-tasking convolutional neural network did not employ any pre- or postprocessing step. It achieved a Dice Score equal to 81.89% for the semantic segmentation task and an accuracy of 90.63% for the event detection task. The results demonstrated that the concurrent tasks were properly combined since the common backbone extracted features proved beneficial for tool segmentation and event detection. Indeed, active bleeding usually happens when one of the instruments closes or interacts with anatomical tissues, and it decreases when the aspirator begins to remove the accumulated blood. Even if different aspects of the presented methodology could be improved, this work represents a preliminary attempt toward an end-to-end multi-task deep learning model for real-time video understanding.

Keywords: bleeding detection , CNN , laparoscopic surgery , multi-task convolutional neural network , semantic segmentation

BACKGROUND: Rehabilitation plays a key role in the recovery of upper extremity function after breast cancer surgery. Motion capture (mocap) systems for serious gaming have shown the potential to enable home-based rehabilitation, but clinical accuracy needs to be examined. OBJECTIVES: Validation of markerless mocap systems for telerehabilitation after breast cancer surgical intervention. METHODS: The accuracy of the markerless mocap device Azure Kinect in detecting compensatory movements and postural disturbances has been compared to a gold standard Optitrack system in five volunteers. Subsequently, a serious game for mocap-based shoulder exercises has been developed and integrated into a telerehabilitation platform. RESULTS: The Azure Kinect shows good reliability for scapular elevation (ICC >0.80; MAE <2.1 cm) and trunk tilt (ICC=0.88; MAE=5°), moderate reliability for rounded shoulders (ICC=0.51; MAE=2.6cm) and poor reliability for kyphosis angle (ICC=0.22; MAE=18°). CONCLUSION: The Azure Kinect provides reasonable performance for shoulder rehabilitation. The proposed telerehabilitation platform has been tested by rehabilitation specialists and received positive feedback.

Keywords: Breast Neoplasms , Exercise Therapy , Exergaming , Motion Capture , Telerehabilitation

An area of interest in orthopaedics is the development of efficient customized neck orthoses, considered that pathologies which affect the neck area are widespread. Advanced acquisition and modelling approaches combined with Additive Manufacturing (AM) can potentially provide customized orthoses with improved performance and complexity. However, in the design of these devices, besides functional and structural requirements, benefit and comfort of the patient should be a main concern, in particular, at the early stage of design during the acquisition of the body's part, and while using the printed orthosis. In this paper, a scanning system with three sensors was developed which allows a fast, about 5 s, and accurate acquisition of the neck area with minimum discomfort for the patient. A neck orthosis with a ventilation pattern obtained by Topology Optimization (TO), lightened by about 35%, was also established. In fact, a main role for comfort is played by the ventilation pattern which contributes both to lightness and breathability. Its structural and comfort performance was evaluated in comparison with an orthosis with a ventilation pattern configured by Voronoi cells. Structural assessment was carried out by means of finite element analysis under main loading conditions. An evaluation of neck temperatures in relation to wearing 3D printed prototypes, manufactured with Hemp Bio-Plastic® filament, was finally conducted by means of a thermal imaging camera. TO orthosis prototype showed a better performance regarding thermal comfort, with a maximum increase of neck temperature less than 1 °C, which makes the proposed configuration very promising for user's comfort.

Keywords: Additive manufacturing , CAD , Orthosis modelling , Reverse engineering , Thermal comfort , Topology optimization

The aim of this review is to provide a comprehensive overview of the existing literature concerning the applications of positron emission tomography (PET) radiomics in lung cancer patient candidates or those undergoing immunotherapy. Materials and Methods: A systematic review was conducted on databases and web sources. English-language original articles were considered. The title and abstract were independently reviewed to evaluate study inclusion. Duplicate, out-of-topic, and review papers, or editorials, articles, and letters to editors were excluded. For each study, the radiomics analysis was assessed based on the radiomics quality score (RQS 2.0). The review was registered on the PROSPERO database with the number CRD42023402302. Results: Fifteen papers were included, thirteen were qualified as using conventional radiomics approaches, and two used deep learning radiomics. The content of each study was different; indeed, seven papers investigated the potential ability of radiomics to predict PD-L1 expression and tumor microenvironment before starting immunotherapy. Moreover, two evaluated the prediction of response, and four investigated the utility of radiomics to predict the response to immunotherapy. Finally, two papers investigated the prediction of adverse events due to immunotherapy. Conclusions: Radiomics is promising for the evaluation of TME and for the prediction of response to immunotherapy, but some limitations should be overcome.

Keywords: immunotherapy , lung cancer , PD-L1 , PET , response to therapy

Mechanical characterization of soft materials is a complicated inverse problem that includes nonlinear constitutive behavior and large deformations. A further complication is introduced by the structural inhomogeneity of tested specimens (for example, caused by thickness variations). Optical methods are very useful in mechanical characterization of soft matter, as they provide accurate full-field information on displacements, strains and stresses regardless of the magnitude and/or gradients of those quantities. In view of this, the present study describes a novel hybrid framework for mechanical characterization of soft membranes, combining (i) inflation tests and preliminary in-plane equi-biaxial tests, (ii) a one-shot projection moiré optical setup with two symmetric projectors that project cross-gratings onto the inflated membrane, (iii) a mathematical model to extract 3D displacement information from moiré measurements, and (iv) metaheuristic optimization hybridizing harmony search and JAYA algorithms. The use of cross-gratings allows us to determine the surface curvature and precisely reconstruct the shape of the deformed object. Enriching metaheuristic optimization with gradient information and elitist strategies significantly reduces the computational cost of the identification process. The feasibility of the proposed approach wassuccessfully tested on a 100 mm diameter natural rubber membrane that had some degree of anisotropy in mechanical response because of its inhomogeneous thickness distribution. Remarkably, up to 324 hyperelastic constants and thickness parameters can be precisely identified by the proposed framework, reducing computational effort from 15% to 70% with respect to other inverse methods.

Keywords: materials characterization , metaheuristic optimization , projection moiré , soft matter

The alternation of dry and wet contact causes a complex interaction of various damage mechanisms in railway wheels. To study the damage evolution in such condition, rolling-sliding contact tests in alternated dry-wet contact were carried out by a bi-disc machine on specimens extracted from railway wheels. The effect of different durations of the dry and wet sessions was investigated. Two kinds of tests were done: in one case coupling two cylindrical specimens, in the other one coupling a cylindrical specimen with a crowned one. The surface appearance during the tests was acquired by a vision system; the acquired images were elaborated to extract quantitative damage indexes. At the end of the tests, the specimens were cut and analysed, to evaluate the subsurface damage and the strain hardening. In cylindrical-cylindrical coupling, the damage increased in alternated dry-wet sessions as far as the cycle number increased, whereas it was stabilized in long dry sessions. The prevailing phenomenon, occurring during the wet sessions, was fatigue, promoted by the pressurization of the water entrapped in the cracks previously formed in dry contact. In crowned-cylindrical couplings, the damage tended to decrease as far as the cycles increased, in all the test conditions. This was correlated to the widening of the contact patch caused by wear, which led to an overall stress relaxation, this way mitigating the damage. The results highlighted the double role of wear, which alters the optimum wheel profile, but at the meantime relaxes the overall stress state and mitigates rolling contact fatigue.

Keywords: Damage monitoring , Dry-wet contact , Railway wheel steel , Rolling contact tests , Vision system

This chapter provides an overview of eXtended Reality (XR), a term that encompasses technologies such as Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR). XR allows the real world to be enriched with virtual data, objects, and information, offering varying levels of immersion and interaction. The chapter explores the components of XR and discusses classification frameworks, including Milgram's Reality-Virtuality Continuum and the 3I model. It further delves into the developments and technologies of VR, highlighting stereoscopic vision, VR headsets and immersive walls. The chapter also explores AR, its accessibility through everyday devices, and its applications. The importance of considering all sensory modalities for creating immersive XR experiences is emphasized, and the role of multimodal interaction in enhancing user experiences is discussed. The chapter highlights the significance of a multisensory approach in achieving a sense of presence and explores examples of early immersive and multisensory technology. Finally, it discusses the impact of touch in human interaction and the incorporation of tactile sensations in VR applications, as well as of olfaction, which plays a significant role in human perception and memory, as certain smells can evoke strong emotional responses and trigger vivid recollections of past experiences. Incorporating odors into XR applications can enhance the overall sense of presence and realism by providing users with scent cues that align with the virtual environment. Olfactory displays are devices that release specific odors or scents to accompany virtual content.

Keywords: Behavioral research , Display devices , Mixed reality , Odors , Stereo image processing

Manufacturing process (MP) selection systems require a large amount of labelled data, typically not provided as design outputs. This issue is made more severe with the continuous development of Additive Manufacturing systems, which can be increasingly used to substitute traditional manufacturing technologies. The objective of this paper is to investigate the application of image processing for classifying MPs in an unsupervised approach. To this scope, k-means and hierarchical clustering algorithms are applied to an unlabelled image dataset. The input dataset is constructed from freely accessible web databases and consists of twenty randomly selected CAD models and corresponding images of machine elements: 35% additively manufactured parts and 65% manufactured with traditional manufacturing technologies. The input images are pre-processed to have the same colour and size. The k-means and hierarchical clustering algorithms reported 65% and 60% accuracy, respectively. The algorithms show comparable performance, however, the k-means algorithm failed to predict the correct subdivisions. The research shows promising potential for MP classification and image processing applications.

Keywords: Additive Manufacturing , Computer Aided Design (CAD) , Image Processing , Machine learning , Unsupervised Learning

Approximately 10% of hospitalized patients develops decubitus ulcers that quickly degenerates into chronic illness that reduces the quality of life and requires expensive clinical management. The use of an anti-decubitus active mattress, that automatically redistributes the pressure loads, reduces the occurrence of new lesions and promotes the healing of the pre-existing ones. The aim of this work is to design and develop two tools to support the design of an anti-decubitus active mattress. Almost all the systems found in literature are based on the classification of pressure maps through machine learning and are difficultly usable in the design context. This work proposes a pressure map Classifier and an Interactive Simulator of the mattress, based on a simpler logic, by integrating image processing techniques and functioning simulations. The Classifier can recognize the patient's pressure maps and classify them according to six reference sleep postures. The Interactive Simulator allows to understand the operating mechanisms of the mattress and to test the controller and the various control logics in the absence of a physical prototype.

Keywords: Active Anti-decubitus Mattress , Images Classifier , New product development , Simulation , User centred design

Purpose: Left atrial appendage occlusion (LAAO) is a structural interventional cardiology procedure that offers several possibilities for the application of additive manufacturing technologies. The literature shows a growing interest in the use of 3D-printed models for LAAO procedure planning and occlusion device choice. This study aims to describe a full workflow to create a 3D-printed LAA model for LAAO procedure planning. Design/methodology/approach: The workflow starts with the patient's computed tomography diagnostic image selection. Segmentation in a commercial software provides initial geometrical models in standard tessellation language (STL) format that are then preprocessed for print in dedicated software. Models are printed using a commercial stereolithography machine and postprocessing is performed. Findings: Models produced with the described workflow have been used at the Careggi Hospital of Florence as LAAO auxiliary planning tool in 10 cases of interest, demonstrating a good correlation with state-of-the-art software for device selection and improving the surgeon's understanding of patient anatomy and device positioning. Originality/value: 3D-printed models for the LAAO planning are already described in the literature. The novelty of the article lies in the detailed description of a robust workflow for the creation of these models. The robustness of the method is demonstrated by the coherent results obtained for the 10 different cases studied.

Keywords: 3D Printing , Additive manufacturing , Heart model , Left atrial appendage occlusion , Left atrium , Stereolithography , Structural heart surgery , Surgical planning

In automotive and industrial application domains, the "health monitoring"or predictive reliability of electronic devices plays a key-role in ensuring reliability of the electrical components. Health monitoring regards a collection of methods aimed at monitoring the operational features and performance of an electronic components in order to schedule preventive/predictive maintenance. Similarly, predictive reliability makes it possible to test electronic devices in an intelligent manner, i.e. by identifying predictive patterns of degradation or anomalous behavior in order to effectively characterize the monitored systems. Specifically, the authors of the present work will show the excellent results obtained by applying artificial intelligence techniques to the monitoring by optical microscopy of the power modules in Silicon Carbide technology usually placed in the traction inverter of electric vehicles. The Silicon-Carbide Power Module is a key component embedded in the traction inverter-system of the modern electric cars, and its functional degradation or unusual behavior may have a significant negative impact on the car's safety. As with any electronic components, the power module is subjected to electro-mechanical stresses which impact its functionality over time, highlighting progressive states of physical degradation. One technique for detecting defects in Silicon-Carbide Power Modules is optical microscopy. Through ad-hoc deep learning system based on the use of an enhanced YOLOv5 architecture, the captured optical microscopic image of the analyzed Silicon-Carbide Power Module will be automatically processed providing in output a bounding-box segmentation of the most impactful and significant defect patterns for the functioning of the monitored power device. Several tests have been performed on different power modules of the ACEPACKTM DRIVE series delivered by STMicroelectronics. The collected performance (identification and classification accuracy close to 93%) confirmed the effectiveness of the proposed solution.

Keywords: Deep Learning , Optical Microscopy , Power Modules

Automotive domain is making rapid growth in next generation cars development embedding higher levels of autonomy and intelligent assistance. Although the general advanced driver assistance system (ADAS) architectures are widely debated in the global automotive market, limited interaction between driver and these intelligent solutions sometimes make these solutions inefficient. For these reasons, the authors started an investigation about driver's feedback with respect to the intelligent assistance inputs provided by the ADAS technologies. In this context, the goal of this proposal is to show the implemented intelligent system which learns from the analysis of the car driver's eyes saccadic movements, the correlated level of attention towards the salient driving scene. With this approach, the authors were able to collect a kind of visual-feedback signal which learns the driver eye's fixing dynamic associated to the analyzed driving scene. Through ad-hoc enhanced motion magnification technique, the authors were able to amplify the mentioned saccadic dynamics to allow a downstream deep classifier to associate this physiological behavior with the corresponding level of the driver attention. The collected performances (near to 97)

Keywords: ADAS , Deep Learning , Driver eye saccadic movements , Motion Magnification

Driver posture and micro movements are main indicators of his attention and situation awareness, as well as of his capability to suddenly take control if necessary. Therefore, the real-time detection of wrong postures is essential to mitigate the risk of accidents. In this work we want to show that, by using a custom Convolutional Neural Network (CNN) for image processing, a very accurate driver posture recognition system can be realized by using a limited number of pressure sensors, grouped in a small carpet placed only on the seat of the driver, regardless of its shape. Data from the sensor carpet are converted in images reproducing the different pressure regions of the driver's body, so that the CNN can extract features and classify 8 postures with an average accuracy of 98.81% in real-time. According to the edge computing paradigm, the CNN implements an end-to-end classification by exploiting a quantization scheme for weights and binarized activations to reduce the number of required resources and allow a compact and low-power HW implementation on a small FPGA. When implemented with a Xilinx Artix 7 FPGA, the CNN consumes less than 7 mW of dynamic power at an operation frequency of 47.64 MHz. Such frequency is compatible with a sensor Output Data Rate (ODR) of 16.50 kHz, fundamental in critical applications, requiring a continuous monitoring and real-time action. Results of a 130 nm CMOS standard cells synthesis have also been reported.

Keywords: Driver safety , Neural Network , Posture recognition , Real-time image processing

Ergonomics focuses on the analysis of the interaction between human beings and their working environment. During the riding of a motorbike, ergonomics studies the rider's posture on the motorbike. An incorrect posture can lead to physical and psychological discomfort, and can affect the perception of risk and the handling of the motorcycle. It is important for motorcyclists to adopt a good riding posture, for their health and road safety. The aim of this work is to propose a fast, cheap, and sufficiently robust method for the 3D reconstruction of the posture assumed by a motorcyclist. The stereo vision and the application of OpenPose made it possible to obtain a 3D reconstruction of the key points, and their evolution over time. The evaluation of the distances between the 3D key points, which represent the length of the various parts of the body, appears to remain sufficiently stable over time, and faithful to the real distances, as taken on the motorcyclist themself. The 3D reconstruction obtained can be applied in different fields: ergonomics, motorsport training, dynamics, and fluid dynamics analysis.

Keywords: 3D human posture , computer vision , digital twin , ergonomics , motorsport , OpenPose , posture assessment

Several emotion-adaptive systems frameworks have been proposed to enable listeners' emotional regulation through music reproduction. However, the majority of these frameworks has been implemented only under in-Lab or in-car conditions, in the second case focusing on improving driving performance. Therefore, to the authors' best knowledge, no research has been conducted for mobility settings, such as trains, planes, yacht, etc. Focusing on this aspect, the proposed approach reports the results obtained from the study of relationship between listener's induced emotion and music reproduction exploiting an advanced audio system and an innovative technology for face expressions' recognition. Starting from an experiment in a university lab scenario, with 15 listeners, and a yacht cabin scenario, with 11 listeners, participants' emotional variability has been deeply investigated reproducing 4 audio enhanced music tracks, to evaluate the listeners' emotional “sensitivity” to music stimuli. The experimental results indicated that, during the reproduction in the university lab, listeners' “happiness” and “anger” states were highly affected by the music stimuli and highlighted a possible relationship between music and listeners' compound emotions. Furthermore, listeners' emotional engagement was proven to be more affected by music stimuli in the yacht cabin, rather than the university lab.

Keywords: Affective Computing , Audio Enhancement , Deep Learning , Facial Emotion Recognition

The research on technologies and methodologies for (accurate, real-time, spontaneous, three-dimensional…) facial expression recognition is ongoing and has been fostered in the past decades by advances in classification algorithms like deep learning, which makes them part of the Artificial Intelligence literature. Still, despite its upcoming application to contexts such as human–computer interaction, product and service design, and marketing, only a few literature studies have investigated the willingness of end users to share their facial data with the purpose of detecting emotions. This study investigates the level of awareness and interest of 373 potential consumers towards this technology in the car insurance sector, particularly in the contract drafting phase, with a focus on differentiating the respondents between generation Y and Z. Results show that younger people, individuals with higher levels of education, and social network users feel more confident about this innovative technology and are more likely to share their expressive facial data.

Keywords: Facial expression recognition , Insurance market , Survey , Technology acceptance , Technology propensity , Young consumers

Human computer models represent a useful tool for investigating the human body response to external static/dynamic loads or for human-centred design. Articulated Total Body (ATB) models are the simplest human multibody models, where body segments are represented by ellipsoids joined at skeletal articulations. Over the years, regression models on both living subjects' and cadavers' data have been developed to predict body segments properties. These models are affected by two main limitations: the only inputs are the subject's weight and height, not considering that for the same combination different morphologies can exist; secondly, regression analyses were performed over a specific population not including peculiar morphologies (under-weight or obese). A novel methodology for developing anthropomorphic ATB models is here presented: a statistical shape model able to predict the external geometry of the human body from a limited set of anthropometric measurements was implemented and body segments were obtained by segmentation; the respective inertial properties were computed from volumes, assuming a constant density value. The properties of this new anthropomorphic ATB model were compared to those calculated by GEBOD (Generator of Body Data), a well-known programme for ATB data calculation. A virtual population of twenty subjects was analysed: with reference to the inertial properties the most relevant differences occurred at the abdomen and the thighs segments (60% relative error), while the trunk, the shoulder and the calves represent the most critical areas for the geometry reconstruction (50 mm average error). The significance of these outcomes was investigated performing multibody simulations with various scenarios.

Keywords: Accident , Articulated total body model , CAESAR database , Forensic biomechanics , Multibody modelling , Principal component analysis