Image Processing & Applications

Preface and Acknowledgements (Editorial)

Keywords: Editorial

This paper describes the design of a novel fabric–based antagonistic pneumatic actuator with multiple chambers that can be used for the development of soft continuum manipulators for collaborative tasks. The concept consists of three pneumatic chambers of fabric material capable of being actuated independently. By connecting multiple actuators of this kind, it is possible to obtain soft continuum manipulators capable of complex movements and able to change the stiffness of their elements. In this work we highlight the design and prototyping of the soft actuator and we present the preliminary experiments in terms of motion and stiffening capabilities.

Keywords: Fabric pneumatic artificial muscles , Soft actuators , Soft continuum manipulators , Soft robotics

Objective and reliable assessment of motor functions, such as dexterity, is a key point for evaluating worker's abilities. In this context, the proposed work presents a tool for objective automatic assessment of the Minnesota Dexterity Test using cameras with depth sensors. Typical performance measurements (i.e., total time and associated percentiles) were estimated using custom algorithms. In addition, the possibility to identify the qualifiers for the code d440 of the International Classification of Functioning, Disability and Health was implemented in the developed algorithms. The proposed tool can also identify the mistakes most frequently committed by the subjects. To prove the capabilities of the proposed method, a series of experimental trials was conducted with 10 healthy young volunteers. Results showed that the developed tool helps clinicians to obtain performance feedback and evaluate patients' dexterity quickly without bias.

Keywords: Automatic assessment , Biomechanics , Depth cameras , Manual dexterity , Motion capture

Post-processing pipeline for image analysis in reverse engineering modelling, such as photogrammetry applications, still asks for manual interventions mainly for shadows and reflections corrections and, often, for background removal. The usage of Convolutional Neural Network (CNN) may conveniently help in recognition and background removal. This paper presents an approach based on CNN for background removal, assessing its efficiency. Its relevance pertains to a comparison of CNN approaches versus manual assessment, in terms of accuracy versus automation with reference to cultural heritage targets. Through a bronze statue test case, pros and cons are discussed with respect to the final model accuracy. The adopted CNN is based on the U-NetMobilenetV2 architecture, a combination of two deep networks, to converge faster and achieve higher efficiency with small datasets. The used dataset consists of over 700 RGB images used to provide knowledge from which CNNs can extract features and distinguish the pixels of the statue from background ones. To extend CNN capabilities, training sets with and without dataset integration are investigated. Dice coefficient is applied to evaluate the CNN efficiency. Results obtained are used for the photogrammetric reconstruction of the Principe Ellenistico model. This 3D model is compared with a model obtained through a 3D scanner. Moreover, through a comparison with a photogrammetric 3D model obtained without the CNN background removal, performances are evaluated. Although few errors due to bad light conditions, the advantages in terms of process automation are consistent (over 50% in time reduction).

Keywords: Close range photogrammetry , CNN , Cultural heritage preservation , MobilenetV2 , Reverse engineering , U-Net

Nowadays, 10–34% of patients undergone Total Knee Arthroplasty (TKA) continues to have pain and reduced mobility. New technologies in TKA have been developed to reproduce the native anatomy, respecting the natural joint line, by means of customized implants or following the kinematic alignment. In this context, the present research aims at defining a method to create customized implants. In particular, three different situations have been studied. The first condition is the healthy knee, that is used as reference for further analysis. For the second situation, an off-the-shelf prosthesis has been virtually implanted, following the kinematic joint line. In the third solution, a custom-made knee implant has been created. In all the three cases, FEA has been performed to study how load transmission and stability change after TKA. To reach the goal, high resolution Magnetic Resonance (MR) images of a healthy knee have been employed. Three-dimensional models of the knee have been reconstructed through a segmentation process, starting from DICOM images. Hence, the three situations have been studied. Distribution of pressure and stress are comparable in the two solutions, since they both maintain the natural joint line. Improving the kinematic function is crucial to increase patient satisfaction. According to the patient's anatomy, the surgeon can choose between the standard and the personalized prosthesis. The studied customized approach allows to overcome the limits of conventional TKA since it permits to create geometries, which accommodate a variety of anatomical variations.

Keywords: 3D modeling , Customized knee implants , FEA , Kinematic alignment

Collaborative robots (cobots) are designed to directly interact with human beings within a shared workspace. To minimize the risk of musculoskeletal disease for the workers, a physical ergonomic assessment of their interaction is needed. Virtual reality (VR) and motion capture (Mocap) systems can aid designers in building low-hazard collaborative environments. This work presents a framework based on VR and Mocap systems for the ergonomic evaluation of collaborative robotic workstations. Starting from the 3D models of the cobot and workstation components, a virtual environment is built in Unity and ROS is employed to manage the cobot behavior. The physical ergonomics is evaluated by means of RULA methodology, exploiting the body tracking capabilities of the device Kinect Azure, a low-cost markerless Mocap system. The framework has been tested by building a virtual environment for collaborative control of flanges with different diameters. The worker interacts with a six-axis Nyro One to move parts on the workstation. The ergonomic assessment is performed in real-time, and a report is generated for later uses and evaluations. The proposed framework fosters the design of collaborative robotics workstations based on an objective assessment of ergonomics. The results of this research work allow planning future development steps for the emulation of more complex workstations with cobots and the use of augmented reality to evaluate how to modify existing workstations to introduce a cobot.

Keywords: Collaborative robots , Ergonomics , Motion capture , Virtual reality

In this paper, a semi-automatic procedure to perform point clouds registration is presented. The method was developed for upper limb 3D scanning. During the acquisition, several frames are acquired from different points of view, to obtain a full 360° acquisition of the arm. Each frame stores both the point clouds coordinates and the corresponding RGB image. During post-processing, the RGB image is elaborated through a neural network, to detect relevant key points of the hand, which are then projected to the point clouds. The corresponding key points detected from different acquisitions are then used to automatically obtain a rough 3D rotation that aligns the point clouds corresponding to different perspectives in a common reference frame. Finally, the registration is refined through an iterative closest point algorithm. The method was tested on actual arm acquisitions, and the registration results are compared with the conventional fully manual 3-2-1 registration procedure, showing promising results of the proposed method.

Keywords: Neural network , Semi-automatic registration , Upper limb 3D scan

Nowadays, Search and Rescue operations can be performed using manned or unmanned Aerial Vehicles. In this latter case, compact cameras are mounted onboard and a bird's eye view is available to find the missing person. However, the analysis of the video frames can be very challenging and dull for the operators. In this context, the use of graphical methodologies can boost the searching operations and improve the process. In this study, a methodology based on the object detector Yolov5 is introduced: the performances in detecting small objects such as persons in aerial images are evaluated. These algorithms implement shallow layers of the feature extractor to increase the spatial-rich features and help the detector to find small objects. Finally, detection algorithms are tested using a video simulating a scenario for Search and Rescue operations. The filtering of frames containing false positives, is carried out using a classical graphical tool such as the Hamming distance.

Keywords: Aerial images , Graphical methodologies , Image analysis , Object detection , SAR operations

Conventionally, visitors of exhibitions cannot directly interact with artworks but remain mainly passive. This work presents a new way to discover paintings through an extended reality application with the aim of enhancing visitors' engagement. The workflow consists of digitally recreating a painting. All its constitutive elements are contoured and then removed by the artwork. These elements will be unveiled by the visitors with gesture interaction. The discovery of each element is followed by additional information on the subject. The application also proposes an alternative experience where the user paints the components of the artwork with gestures. The hands' position was tracked by the Leap Motion Controller, an optical sensor by Ultraleap. The process has been applied to one of the Tintoretto's pieces of art: “L'Annunciazione del Doge Grimani”. The extended reality application has been experienced by over twenty thousand visitors at the dedicated exhibition. Numerous positive comments received from both visitors and guides are encouraging.

Keywords: Cultural heritage , Exhibition , Extended reality , User interaction

Anomaly detection is the identification of any event that falls outside what is considered ‘acceptable behaviour'. This work investigates anomaly detection for automated visual inspection in the context of industry automation (‘Industry 4.0'). For this task we propose a machine vision procedure based on visual feature extraction and one-class k nearest neighbours classification. The method requires only samples of normal (non-defective) instances for the training step. We benchmarked our approach using seven traditional (‘hand-designed') colour texture descriptors and five pre-trained convolutional neural networks (CNN) ‘off-the-shelf'. Experimenting on nine image datasets from seven classes of materials (carpet, concrete, fabric, layered fused filament, leather, paper and wood), each containing normal and abnormal samples, we found overall accuracy in the range 82.0%–90.2%. Convolutional networks off-the-shelf performed generally better than the traditional methods, although – interestingly – this was not true for all the datasets considered. No visual descriptor clearly emerged as the all-purpose best option.

Keywords: Anomaly detection , Colour , Convolutional neural networks , Texture , Visual descriptors

In the field of medical image processing, the resolution capacity exhibited by the initial diagnostic investigations is becoming increasingly important. With respect to them, in fact, the row image set is subjected to three-dimensional reconstruction analysis, by partitioning the regions of interest, as well as to local investigations, aimed, for example, at the extrapolation of topological information, relating to the morphology of the object that needs to be investigated. The accuracy of these functions is, however, difficult to quantify, due to the lack of three-dimensional models that act as a reference Gold Standard. The reproduction of CT-type diagnostic acquisitions, starting from a virtual scanning procedure of a starting known three-dimensional geometry is used. To do this, triangular tessellated three-dimensional models of various geometries were examined. These were broken down into cubic elements, equal in size to those of a common voxel, thus resulting in a volume scan simulation of the original region considered. The structure thus obtained was then subjected to skeletonization and medial axis algorithms to evaluate the effectiveness of some of the most commonly used functions in medical processing. A virtual scanning model of this type can be an extremely effective evaluation analysis tool in discriminating the resolutive quality of the medical image processing functions. From a qualitative comparison of this type, it is possible to optimize automated anatomical investigation algorithms, making a significant contribution in the refinement of the techniques, now more and more demanding, of image processing in the biomedical field.

Keywords: Medical image acquisition , Thinning and Medial Axis , Voxelization

Anterior Cruciate Ligament (ACL) is one of the most injured ligaments. Various researches have studied the relationship between ACL rupture and the bony morphology attempting to understand why ACL rupture occurs. The most investigated predisposing factors in the scientific literature are mainly related to the femoral intercondylar notch and the tibial plateau. The aim of the present research is to create a semi-automatic tool for assessing the risk of developing an ACL injury. A package, named ACL Morsky, has been created in 3D Slicer to follow the entire workflow. The process starts with the load of DICOM data. Three-dimensional models are reconstructed by means of segmentation. Then, five parameters, considered risk factors for the ligament rupture, are measured both in 2D images and on 3D models. A specific algorithm compares the measured values with the range in literature to evaluate the probability of developing an ACL injury. The physician is provided with a report for the patient and his/her risk factors. Knowing in advance the risk of rupture can help in reducing it with preventive plans.

Keywords: 3D Slicer , Knee Injury , Knee Risk Assessment , Ligament Risk Factors , Morphological Parameters

The comparison of the finished product's color to the color demanded by a client or provided by a catalogue is one of the most critical steps of the production of carded fibers. This is currently determined by calculating the colorimetric distance between the finished article, which is made by mixing some colored raw fibers to achieve a homogenous color, and the desired one. When this colorimetric distance exceeds a value specified by the customer, the recipe must be modified to close the gap between the end product's color and the required one. The above mentioned trial and error procedure used to determine the appropriate quantities of colored fibers for achieving a desired final color of a fiber lacks a computer-aided tool to assist colorists. Therefore, it is a bottleneck in the process of creating carded fabrics. The present work offers a novel method for predicting the spectrophotometric response of a carded fiber based on the spectrophotometric response of the raw components used to make the fiber. The method is described step-by-step. • The method is applicable to predict the spectrophotometric response of a fabric composed by any number of pre-colored fibers.

Keywords: Colour mixing , Method for predicting the spectrophotometric response of a carded fabr iccomposed by differently colored raw materials , Predictive assessment , Reflectance factors , Spectrophotometry

The 6D pose estimation of an object from an image is a central problem in many domains of Computer Vision (CV) and researchers have struggled with this issue for several years. Traditional pose estimation methods (1) leveraged on geometrical approaches, exploiting manually annotated local features, or (2) relied on 2D object representations from different points of view and their comparisons with the original image. The two methods mentioned above are also known as Feature-based and Template-based, respectively. With the diffusion of Deep Learning (DL), new Learning-based strategies have been introduced to achieve the 6D pose estimation, improving traditional methods by involving Convolutional Neural Networks (CNN). This review analyzed techniques belonging to different research fields and classified them into three main categories: Template-based methods, Feature-based methods, and Learning-Based methods. In recent years, the research mainly focused on Learning-based methods, which allow the training of a neural network tailored for a specific task. For this reason, most of the analyzed methods belong to this category, and they have been in turn classified into three sub-categories: Bounding box prediction and Perspective-n-Point (PnP) algorithm-based methods, Classification-based methods, and Regression-based methods. This review aims to provide a general overview of the latest 6D pose recovery methods to underline the pros and cons and highlight the best-performing techniques for each group. The main goal is to supply the readers with helpful guidelines for the implementation of performing applications even under challenging circumstances such as auto-occlusions, symmetries, occlusions between multiple objects, and bad lighting conditions.

Keywords: 6D position estimation , Computer vision , Deep learning , RGB Input

Analyzing pathological movements can substantially help neurologists in the diagnosis and treatment improvement for patients with Parkinson's disease (PD). A linkage between the intensity and characteristics of moving and walking disorders and the stage and types of PD can be actually established. The main aim of this study is to develop an effective methodology that allows to evaluate, in real time and / or in deferred time, movements and posture of PD patients in their usual living environments. For this purpose, a wearable suit with Inertial Measurement Unit (IMU) sensors was designed; it has made it possible to acquire linear and angular signals of displacement, velocity and acceleration of the most relevant body points of the patients. The filtered and integrated signals were then used to animate a human parametric multibody model that virtually reproduces in real time and / or in deferred patient's movements and posture. Serving as the patient's “avatar”, the multibody model enables the neurologist to carry out an accurate assessment of the patient's movements and posture (freezing, festination, postural balance) as well as to measure disease progression and response to interventions. If compared to traditional 3D video-based motion analysis systems, the proposed method has the advantage of providing a more accurately measurable patients movements analysis and comparison performed in their usual living environments in real-world conditions.

Keywords: 3D posture analysis , Human parametric multibody model , Inertial Measurement Unit sensors , Motion recognition algorithms , Parkinson's disease movements

Introduction: In the last decade, two new radionuclide-based therapies, 223Radichloride and radioligand therapy (RLT) targeting prostate-specific membrane antigen (PSMA), have been approved by the regulatory authorities for the management of castrate-resistant prostate cancer (PCa). Areas covered: The basic principles of PCa molecular imaging are illustrated, with a particular attention to the combined use of diagnosis and therapy in a unique approach (the so-called theranostics) for response prediction and assessment in patients submitted to 223Radichloride or PSMA targeted therapies. In this perspective, the potential of radiomics, an emerging discipline based on the extraction of quantitative features from medical images, is covered. Expert opinion: Theranostic metabolic and molecular probes have been successfully applied to predict and monitor response to radionuclide-based therapies. In particular, both 99mTc-MDP and 18F-NaF resulted useful tools for personalized dosimetry and prognostic stratification before 223Ra-therapy, while PSMA-ligands, alone or in combination with 18F-FDG, provided valuable information to select patients who are more likely to benefit from RLT and getting information on PCa grade of differentiation and aggressiveness. In spite of its high potential, PET-radiomics for PCa is still at an embryonic phase and needs further validation.

Keywords: artificial intelligence , positron emission computed tomography , precision medicine , Prostate cancer , radiomics , single photon emission tomography , targeted therapy , theranostics

The precision livestock farming (PLF) has the objective to maximize each animal's performance while reducing the environmental impact and maintaining the quality and safety of meat production. Among the PLF techniques, the personalised management of each individual animal based on sensors systems, represents a viable option. It is worth noting that the implementation of an effective PLF approach can be still expensive, especially for small and medium-sized farms; for this reason, to guarantee the sustainability of a customized livestock management system and encourage its use, plug and play and cost-effective systems are needed. Within this context, we present a novel low-cost method for identifying beef cattle and recognizing their basic activities by a single surveillance camera. By leveraging the current state-of-the-art methods for real-time object detection, (i.e., YOLOv3) cattle's face areas, we propose a novel mechanism able to detect the ear tag as well as the water ingestion state when the cattle is close to the drinker. The cow IDs are read by an Optical Character Recognition (OCR) algorithm for which, an ad hoc error correction algorithm is here presented to avoid numbers misreading and correctly match the IDs to only actually present IDs. Thanks to the detection of the tag position, the OCR algorithm can be applied only to a specific region of interest reducing the computational cost and the time needed. Activity times for the areas are outputted as cattle activity recognition results. Evaluation results demonstrate the effectiveness of our proposed method, showing a mAP@0.50 of 89%.

Keywords: Cattle identification , Computer vision , Deep learning , Low-cost sensors , Precision livestock farming

The development of eco-sustainable systems for the textile industry is a trump card for attracting expanding markets aware of the ecological challenges that society expects in the future. For companies willing to use regenerated wool as a raw material for creating plain, colored yarns and/or fabrics, building up a number of procedures and tools for classifying the conferred recycled materials based on their color is crucial. Despite the incredible boost in automated or semi-automated methods for color classification, this task is still carried out manually by expert operators, mainly due to the lack of systems taking into account human-related classification. Accordingly, the main aim of the present work was to devise a simple, yet effective, machine vision-based system combined with a probabilistic neural network for carrying out reliable color classification of plain, colored, regenerated wool fabrics. The devised classification system relies on the definition of a set of color classes against which to classify the recycled wool fabrics and an appositely devised acquisition system. Image-processing algorithms were used to extract helpful information about the image color after a set of images has been acquired. These data were then used to train the neural network-based algorithms, which categorized the fabric samples based on their color. When tested against a dataset of fabrics, the created system enabled automatic classification with a reliability index of approximately 83%, thus demonstrating its effectiveness in comparison to other color classification approaches devised for textile and industrial fields.

Keywords: color , fabrics , machine vision , neural networks

This paper aims to explore the potential offered by emotion recognition systems to provide a feasible response to the growing need for audience understanding and development in the field of arts organizations. Through an empirical study, it was investigated whether the emotional valence measured on the audience through an emotion recognition system based on facial expression analysis can be used with an experience audit to: (1) support the understanding of the emotional responses of customers toward any clue that characterizes a staged performance; and (2) systematically investigate the customer's overall experience in terms of their overall satisfaction. The study was carried out in the context of opera live shows in the open-air neoclassical theater Arena Sferisterio in Macerata, during 11 opera performances. A total of 132 spectators were involved. Both the emotional valence provided by the considered emotion recognition system and the quantitative data related to customers' satisfaction, collected through a survey, were considered. Results suggest how collected data can be useful for the artistic director to estimate the audience's overall level of satisfaction and make choices about the specific characteristics of the performance, and that emotional valence measured on the audience during the show can be useful to predict overall customer satisfaction, as measured using traditional self-report methods.

Keywords: artificial intelligence , customer experience , customer satisfaction , emotion recognition , facial expression recognition

The experimental evaluation of welding-induced distortion is a topic of great interest to both the scientific and industrial communities. With the aim of addressing a specific need in an industrial context, this paper investigates the capabilities of a stereo–Digital Image Correlation (stereo–DIC) system to measure the weld-induced distortion of the front-plate of a bogie train bolster subassembly. Currently, the deviation from planarity of this surface is measured at less than five points using a CMM in the post-weld cooled state. An additional machining process is then used to bring the surface within the tolerance required to join the welded assembly to the train body through a threaded flange. The paper shows that DIC can provide accurate full-field distortion and strain maps over the entire 588 mm × 308 mm surface of the front plate. The distinct strength of DIC over the currently used inspection technique is its ability to provide highly spatially dense data that are unaffected by rigid body motion. This can be advantageous in terms of saving time in the post-weld inspection and reducing errors in the finishing process. In addition, DIC capabilities revealed important information that was not available from the CMM, such as the full-surface map of the initial deviation of the plate from its nominal geometry and its asymmetric deformation after welding. Finally, the full-field nature of the experimental data obtained allows for seamless integration with FE thermo-mechanical simulations for numerical model validation, stress calculation, and optimization of geometry and technological processes.

Keywords: digital image correlation , full-field distortion and strain map , traction frame , welding-induced distortion