Image Processing & Applications

This paper introduces a system that enable the collection of relevant data related to the emotional behavior and attention of both student and professor during exams. It exploits facial coding techniques to enable the collection of a large amount of data from the automatic analysis of students and professors faces using video analysis, advanced techniques for gaze tracking based on deep Learning, and technologies and the principles related to the Affective Computing branch derived from the research of Paul Ekman. It provides tools that facilitates the interpretation of the collected data by means of a dashboard. A preliminary experiment has been carried out to investigate whether such a system may help in assessing the evaluation setting and support reflection on the evaluation processes in the light of the different situations, so as to improve the adoption of inclusive approaches. Results suggest that information provided by the proposed system can be helpful in assessing the setting and the evaluation process.

Keywords: Affective computing , Deep learning , E-leaning , Emotion recognition , Gaze tracking

Experimental studies on rolling-element bearing lubrication represent an ambitious engineering challenge. In the present study, the design of a dedicated test rig to perform Particle Image Velocimetry (PIV) measurements on the lubricant inside a tapered roller bearing is presented. Through the usage of a sapphire outer ring, it has been possible to optically access the inside of a bearing without compromising its kinematics. PIV measurements allowed the reconstruction of the lubricant velocity field in the cavity between the cage and the outer ring at different rotational speeds. Results show that as the speed of rotation increases, bubbles due to aeration phenomena tend to appear and modify the behavior of the lubricant considerably.

Keywords: Lubrication , PIV , Tapered Roller Bearing , Test Rig

Facial appearance is one prominent feature in analyzing several aspects, e.g., aesthetics and expression of emotions, and face analysis is crucial in many fields. Face analysis requires measurements that can be performed by different technologies and typically relies on landmarks identification. Recently, low-cost customer grade 3D cameras have been introduced in the market, enabling an increase of application at affordable cost with nominal adequate performances. Novel cameras require to be thoroughly metrologically characterized to guarantee these performances. Cameras are calibrated following a standard general-purpose procedure. However, the specificity of facial measurements requires a task-based metrological characterization to include typical influence factors. This work outlines a methodology for task-based metrological characterization of low-cost 3D cameras for facial analysis, consisting of: influence factor identification by ANOVA, related uncertainty contribution assessment, uncertainty propagation, landmarking uncertainty estimation. The proposed methodology is then demonstrated on a customer grade state-of-the-art 3D camera available on the market.

Keywords: Depth cameras , Face analysis , Human-machine interaction , Machine vision , Measurement uncertainty , Soft tissue landmarks

Introduction: The current study presents a deep learning framework to determine, in real-time, position and rotation of a target organ from an endoscopic video. These inferred data are used to overlay the 3D model of patient's organ over its real counterpart. The resulting augmented video flow is streamed back to the surgeon as a support during laparoscopic robot-assisted procedures. Methods: This framework exploits semantic segmentation and, thereafter, two techniques, based on Convolutional Neural Networks and motion analysis, were used to infer the rotation. Results: The segmentation shows optimal accuracies, with a mean IoU score greater than 80% in all tests. Different performance levels are obtained for rotation, depending on the surgical procedure. Discussion: Even if the presented methodology has various degrees of precision depending on the testing scenario, this work sets the first step for the adoption of deep learning and augmented reality to generalise the automatic registration process.

Keywords: abdominal , Kidney , prostate

A low-speed stereo-camera DIC setup was used in this paper to measure the down-sampled bandpass vibration signal of a plate in a given frequency range, which is much higher than the available frame rate. Down-sampled vibration measurements are well known in the literature. However, they are always subject to the strong hypothesis of having a single frequency component in the excitation source (e.g., pure sinusoidal excitation). In this scenario, several approaches can be found in the literature to reconstruct the actual response from the down-sampled data. In this paper, a data postprocessing algorithm is newly introduced to properly reconstruct the target response in the case of a frequency band excitation, thus relaxing the single frequency excitation constraint and allowing to explore a given frequency range with a single measurement. Additionally, a custom excitation signal is presented in this paper to achieve a constant intensity in the studied frequency range. The proposed approach is presented and experimentally validated by measuring a cantilever plate vibration in the kHz range using a 100 fps acquisition. A single acquisition of 98 frames allowed to describe the deformed shapes at 49 different frequency values in the range 1110–1160 Hz, highlighting two resonance peaks. The comparison with the results of multiple conventional single-frequency tests and LDV measurements confirmed the effectiveness of the proposed approach.

Keywords: Digital image correlation , Down-sampled bandpass measurement , Frequency band excitation , Low-speed camera , Vibration measurement

The digital image correlation (DIC) was used in this paper to obtain full-field measurements of a target vibrating at a frequency higher than the maximum cameras' frame rate. The down-sampling technique was implemented to compensate for the cameras' moderate frame rate, thus getting an accurate displacement acquisition even at 6.5 kHz. Two innovative methods to support the DIC application were introduced. The use of fringe projection (or structured light), initially applied on the sample at rest, reduced the effort and time required for the stereo matching task's solution and improved this setting's accuracy and reliability. Additionally, a new time-domain image filtering was proposed and tested to improve the quality of the obtained DIC maps. In combination with the down-sampling, the effect of this filtering technique was tested in this work at (approx.) 2500 and 6500 Hz by measuring the response of a bladed disk to sinusoidal excitation. Evidence of improved results was observed for both frequencies for amplitudes in the range of 10 µm.

Keywords: Bladed disk , Digital image correlation , Down-sampling approach , Low-speed camera , Reverse engineering , Vibration measurement

Augmented reality robot-assisted partial nephrectomy (AR-RAPN) is limited by the need of a constant manual overlapping of the hyper-accuracy 3D (HA3D) virtual models to the real anatomy. To present our preliminary experience with automatic 3D virtual model overlapping during AR-RAPN. To reach a fully automated HA3D model overlapping, we pursued computer vision strategies, based on the identification of landmarks to link the virtual model. Due to the limited field of view of RAPN, we used the whole kidney as a marker. Moreover, to overcome the limit of similarity of colors between the kidney and its neighboring structures, we super-enhanced the organ, using the NIRF Firefly fluorescence imaging technology. A specifically developed software named “IGNITE” (Indocyanine GreeN automatIc augmenTed rEality) allowed the automatic anchorage of the HA3D model to the real organ, leveraging the enhanced view offered by NIRF technology. Ten automatic AR-RAPN were performed. For all the patients a HA3D model was produced and visualized as AR image inside the robotic console. During all the surgical procedures, the automatic ICG-guided AR technology successfully anchored the virtual model to the real organ without hand-assistance (mean anchorage time: 7 seconds), even when moving the camera throughout the operative field, while zooming and translating the organ. In 7 patients with totally endophytic or posterior lesions, the renal masses were correctly identified with automatic AR technology, performing a successful enucleoresection. No intraoperative or postoperative Clavien >2 complications or positive surgical margins were recorded. Our pilot study provides the first demonstration of the application of computer vision technology for AR procedures, with a software automatically performing a visual concordance during the overlap of 3D models and in vivo anatomy. Its actual limitations, related to the kidney deformations during surgery altering the automatic anchorage, will be overcome implementing the organ recognition with deep learning algorithms.

Magnetic resonance imaging (MRI) plays an important role in disease diagnosis. The noise that appears in MRI images is commonly governed by a Rician distribution. The bendlets system is a second-order shearlet transform with bent elements. Thus, the bendlets system is a powerful tool with which to represent images with curve contours, such as the brain MRI images, sparsely. By means of the characteristic of bendlets, an adaptive denoising method for microsection images with Rician noise is proposed. In this method, the curve contour and texture can be identified as low-frequency components, which is not the case with other methods, such as the wavelet, shearlet, and so on. It is well known that the Rician noise belongs to a high-frequency channel, so it can be easily removed without blurring the clarity of the contour. Compared with other algorithms, such as the shearlet transform, block matching 3D, bilateral filtering, and Wiener filtering, the values of Peak Signal to Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM) obtained by the proposed method are better than those of other methods.

Keywords: adaptive algorithm , bendlet transform , magnetic resonance imaging , Rician noises

In various tasks of machine vision, image resolution is one of the important factors that affect the performance of the model. Generally, crop images with low resolution and lack of detail informa-tion may be collected. The picture is not good for the accuracy of yield prediction and crop pest identification. In this paper, tomato leaves are used as the target image, and the super-resolution reconstruction process takes advantage of the sharpness of the image, which has the characteristic of Scale invariance. Firstly, each image block is classified by using clustering algorithm accord-ing to the sharpness value of the image, and then wavelet transform is used to extract image features from each class of image blocks to get wavelet subbands respectively, subbands of each class not only train a union dictionary, but also learn a separate mapping function. Joint dictionary training and separate mapping matrix learning are helpful to optimize the high resolution and low resolution sparse coefficients. In the Reconstruction Stage: in order to reduce the image reconstruction time, the wavelet transform is only applied to the image blocks with a certain sharpness value, while the image reconstruction performance is basically unchanged, then the high-resolution image blocks are reconstructed by using the mapping function, coupled dictionary and the sparse representation coefficients of the image blocks. When the sharpness of the image block is lower than a certain sharpness value, the high and middle resolution image blocks will be superimposed to finally get the high resolution image.In the various tasks of machine vision, image resolution is one of the important factors that affect average PSNR value of the algorithm in this paper is 3.94 dB, 3.54 dB, 3.36 dB, 3.23 dB, 3.01 dB and 1.51 dB higher respectively.

Keywords: Clustering , Coupled dictionary , Sharpness value , Sparse representation

Locust slice image is a kind of cartoon-like images, in which the texture possesses the property of self-similarity. Both of the texture and the noises belong to the high-frequency signals, and so it is difficult to tell the difference between them for most denoising methods. Aim to the problem, we propose a novel denoising method by combining the patch reordering with the shearlet transform. In the reordering process, the patches are divided into smooth and texture components. The filters obtained from the training set are employed to process the patches in smooth regions and the shearlet transform are employed to process the texture regions. The experiments show that the values of PSNR and SSIM of the processed images obtained by the proposed method are better than the common methods.

Keywords: Image denoising , Patch reordering , Shearlet transform , Texture preserving

Shearlet is a multi-dimensional function used for sparse representation, which has many excellent characteristics such as multi-resolution and multi-direction. It can detect the position of singular points and the direction of singular curves, and is more sensitive to the geometric structure of the image. Therefore, this paper introduces the shearlet transform and its application in image processing, and introduces the bendlet transform proposed on this basis.

Keywords: Bendlet , Image processing , Shearlet , Transform

Emotion recognition through machine learning techniques is a widely investigated research field, however the recent obligation to wear a face mask, following the COVID19 health emergency, precludes the application of systems developed so far. Humans naturally communicate their emotions through the mouth; therefore, the intelligent systems developed to date for identifying emotions of a subject primarily rely on this area in addition to other anatomical features (eyes, forehead, etc.). However, if the subject is wearing a face mask this region is no longer visible. For this reason, the goal of this work is to develop a tool able to compensate for this shortfall. The proposed tool uses the AffectNet dataset which is composed of eight class of emotions. The iterative training strategy relies on well-known convolutional neural network architectures to identify five sub-classes of emotions: following a pre-processing phase the architecture is trained to perform the task on the eight-class dataset, which is then recategorized into five classes allowing to obtain 96.92% of accuracy on the testing set. This strategy is compared to the most frequently used learning strategies and finally integrated within a real time application that allows to detect faces within a frame, determine if the subjects are wearing a face mask and recognize for each one the current emotion.

Keywords: Artificial intelligence , COVID19 , Emotion recognition , Facial Expression Recognition , Grad-CAM , Non-verbal communication

This paper introduces a web-platform system that performs semi-automatic compute of several risk indexes, based on the considered evaluation method (e.g., RULA—Rapid Upper Limb Assessment, REBA—Rapid Entire Body Assessment, OCRA—OCcupational Repetitive Action) to support ergonomics risk estimation, and provides augmented analytics to proactively improve ergonomic risk monitoring based on the characteristics of workers (e.g., age, gender), working tasks, and environment. It implements a body detection system, marker-less and low cost, based on the use of RGB cameras, which exploits the open-source deep learning model CMU (Carnegie Mellon University), from the tf-pose-estimation project, assuring worker privacy and data protection, which has been already successfully assessed in standard laboratory conditions. The paper provides a full description of the proposed platform and reports the results of validation in a real industrial case study regarding a washing machine assembly line composed by 5 workstations. A total of 15 workers have been involved. Results suggest how the proposed system is able to significantly speed up the ergonomic assessment and to predict angles and perform a RULA and OCRA analysis, with an accuracy comparable to that obtainable from a manual analysis, even under the unpredictable conditions that can be found in a real working environment.

Keywords: Ergonomics risk assessment , Extended reality , Human-centered manufacturing , Machine learning , Motion capture

Qualitative and quantitative analysis of surface defects on aluminum die-cast parts is important for both quality assurance and process monitoring. In addition to the functionality and durability of the parts, the appearance of a die-cast component can be of crucial importance during the inspection of incoming goods by customers in order to ensure their functionality. Nowadays, many of the detection operations of surface defects are performed by specialized operators, but this approach is far from being sustainable for high production rates. In this context, research has focused on machine vision systems for automatic defect detection based on artificial intelligence (AI) and artificial neural networks. However, several obstacles have so far hindered a full-scale application of these intelligent systems. Images of the aluminum surface can contain a large amount of noise due to surface reflectivity and, in addition, vibrations, improper/variable ambient lighting can make automatic analysis of surface defects, which usually have an irregular shape, very difficult. This led to combining the potential of 3D scanning and measuring systems with 2D machine vision as an acquisition technology to detect surface defects. Nevertheless, the two acquisition processes present strengths and weaknesses, which mostly depend on geometrical aspects of the parts to be detected. The present paper illustrates a first attempt to combine the mentioned 2D and 3D systems into an industrial production environment. The paper presents the developed system to allow multiple acquisitions and explains how these sources of information are fed to an AI system.

Keywords: Artificial intelligence , Deep learning , High-pressure die-casting , Machine vision systems , Quality control , Surface defects analysis

Radiological examination of pulmonary nodules on CT involves the assessment of the nodules' size and morphology, a procedure usually performed manually. In recent years computer-assisted analysis of indeterminate lung nodules has been receiving increasing research attention as a potential means to improve the diagnosis, treatment and follow-up of patients with lung cancer. Computerised analysis relies on the extraction of objective, reproducible and standardised imaging features. In this context the aim of this work was to evaluate the correlation between nine IBSI-compliant morphological features and three manually-assigned radiological attributes – lobulation, sphericity and spiculation. Experimenting on 300 lung nodules from the open-access LIDC-IDRI dataset we found that the correlation between the computer-calculated features and the manually-assigned visual scores was at best moderate (Pearson's r between -0.61 and 0.59; Spearman's ρ between -0.59 and 0.56). We conclude that the morphological features investigated here have moderate ability to match/explain manually-annotated lobulation, sphericity and spiculation.

Keywords: Lung cancer , Morphological features , Pulmonary nodules , Radiomics

Image registration has widespread application in fields like medical imaging, satellite imagery and agriculture precision as it is essential for feature detection and extraction. The extent of this paper is focussed on analysis of intensity and feature-based registration algorithms over Blue and RedEdge multispectral images of wheat and cauliflower field under different altitudinal conditions i.e., drone imaging at 3 m for cauliflower and handheld imaging at 1 m for wheat crops. The overall comparison among feature and intensity-based algorithms is based on registration quality and time taken for feature matching. Intra-class comparison of feature-based registration is parameterized on type of transformation, number of features being detected, number of features matched, quality and feature matching time. Intra-class comparison of intensity-based registration algorithms is based on type of transformation, nature of alignment, quality and feature matching time. This study has considered SURF, MSER, KAZE, ORB for feature-based registration and Phase Correlation, Monomodal intensity and Multimodal intensity for intensity-based registration. Quantitatively, feature-based techniques were found superior to intensity-based techniques in terms of quality and computational time, where ORB and MSER scored highest. Among intensity-based methods, Monomodal intensity performed best in terms of registration quality. However, Phase Correlation marginally scored less in quality but fared well in terms of computational time.

Keywords: Control points (CP) , Feature transform (SIFT) , Feature-based , Intensity-based , KAZE , Maximally Stable Extremal Regions (MSER) , Modified difference local binary (MDLB) , Monomodal , Multi spectral (MS) , Multimodal , Mutual information (MI) , Normalized cross correlation , Oriented Fast and Rotated Brief (ORB) , Phase correlation , Scale-invariant (SI) , Speeded Up Robust Features (SURF) , Sum of squared differences (SSD)

Small-scale experimental tests were carried out by a bi-disc machine to reproduce the phenomena occurring in shoe-braked railway wheels. Wheel specimens of three different steels were subjected first to a braking step against cast iron brake-block specimens in full sliding condition, then to a running step against a rail steel in rolling-sliding condition. To monitor the damage occurring at the wheel specimen surface, a vision system, based on a high-speed camera acquiring the surface images during the tests, was implemented. An image analysis procedure was elaborated to evaluate the surface state by means of quantitative synthetic indexes. The correlation of such indexes with the damage was validated by means of comparison with the subsurface state, obtained by optical microscope observation of cut specimens. Finally, the synthetic indexes were used to analyse the evolution of the damage on a long-duration test, proving to be effective as a non-destructive damage monitoring tool.

Keywords: Bi-disc tests , Damage analysis , Railway wheel steels , Shoe-braking , Vision system

Purpose: We evaluate the ability of Artificial Intelligence with automatic classification methods applied to semi-quantitative data from brain 18F-FDG PET/CT to improve the differential diagnosis between Alzheimer Disease (AD) and Mild Cognitive Impairment (MCI). Procedures: We retrospectively analyzed a total of 150 consecutive patients who underwent diagnostic evaluation for suspected AD (n = 67) or MCI (n = 83). All patients received brain 18F-FDG PET/CT according to the international guidelines, and images were analyzed both Qualitatively (QL) and Quantitatively (QN), the latter by a fully automated post-processing software that produced a z score metabolic map of 25 anatomically different cortical regions. A subset of n = 122 cases with a confirmed diagnosis of AD (n = 53) or MDI (n = 69) by 18–24-month clinical follow-up was finally included in the study. Univariate analysis and three automated classification models (classification tree –ClT-, ridge classifier –RC- and linear Support Vector Machine –lSVM-) were considered to estimate the ability of the z scores to discriminate between AD and MCI cases in. Results: The univariate analysis returned 14 areas where the z scores were significantly different between AD and MCI groups, and the classification accuracy ranged between 74.59% and 76.23%, with ClT and RC providing the best results. The best classification strategy consisted of one single split with a cut-off value of ≈ −2.0 on the z score from temporal lateral left area: cases below this threshold were classified as AD and those above the threshold as MCI. Conclusions: Our findings confirm the usefulness of brain 18F-FDG PET/CT QL and QN analyses in differentiating AD from MCI. Moreover, the combined use of automated classifications models can improve the diagnostic process since its use allows identification of a specific hypometabolic area involved in AD cases in respect to MCI. This data improves the traditional 18F-FDG PET/CT image interpretation and the diagnostic assessment of cognitive disorders.

Keywords: Alzheimer disease , artificial intelligence , automatic classification , brain 18F-FDG PET/CT , mild cognitive impairment

Detection and identification of production defects in semiconductor industry is a key process to allow effective quality control of manufacturing. The advent of new technologies including Silicon and Silicon Carbide, highlights the necessity to have a robust detection system of wafer defects.The Electrical Wafer Sorting (EWS) stage based on defects map electrical analysis is suitable to spot anomalies and defect patterns in the wafer. This time consuming phase enables semiconductor companies to optimize and improve manufacturing process also through the usage of modern deep learning approaches. The proposed pipeline addresses the need to have a full-automatic wafer manufacturing defects identification system based on the EWS wafer map intelligent analysis and by using an approach based on a Deep Convolutional Neural Network combined with an unsupervised sub-system. The collected experimental results confirmed the robustness of the proposed approach.

Keywords: Deep learning , Machine learning , Pattern recognition , Silicon , Silicon Carbide

Tests in alternated dry-wet contact were carried out by a bi-disc machine on specimens extracted from railway wheels. Some of them were cylindrical, other ones were crowned: cylindrical-cylindrical and crowned-cylindrical couplings were tested. The surface appearance was acquired by a vision system; the acquired images were elaborated to extract quantitative damage indexes. In cylindrical-cylindrical coupling, the damage increased in alternated dry-wet sessions as far as the cycle number increased, whereas it was stabilized in long dry sessions. In crowned-cylindrical couplings, the damage tended to decrease as far as the cycles increased, in all the test conditions. The contact patch width in crowned-cylindrical coupling increased as far as the cycle number increased: this led the overall stress state to decrease, this way mitigating the damage.

Keywords: damage monitoring , dry-wet contact , Railway wheel steel , rolling contact tests , vision system