Image Processing & Applications

The paper presents the development of a Virtual Reality application to perform dives in the Christoforos Shipwreck. The huge wreck lies in the Panormos Bay of the Skopelos island (Greece) at a depth of 45 meters. Based on a well-known methodology designed for ancient shipwrecks, the 3D reconstruction of the underwater site has been carried out by adjusting the workflow to survey modern shipwrecks. In particular, the methodology, based on photogrammetry, is capable to provide a highly detailed 3D reconstruction of the shipwreck in few dives, considering issues like working depth, elevations, and thin elements modelling. Moreover, the paper describes the optimization of the 3D model and the software to be executed in low-performance HMD devices. The resulting Virtual Reality (VR) application, realized for touristic purposes, recreates the exact ambient conditions inside and outside the water simulating the flora and fauna of the place, the coastline, allowing users to live a recreational and educational experience by virtual diving in the underwater site.

Keywords: 3D modeling , History , Image reconstruction , Ships , Three dimensional computer graphics

The left ventricle (LV) constantly changes its shape and function as a response to patho-logical conditions, and this process is known as remodeling. In the presence of aortic stenosis (AS), the degenerative process is not limited to the aortic valve but also involves the remodeling of LV. Statistical shape analysis (SSA) offers a powerful tool for the visualization and quantification of the geometrical and functional patterns of any anatomic changes. In this paper, a SSA method was devel-oped to determine shape descriptors of the LV under different degrees of AS and thus to shed light on the mechanistic link between shape and function. A total of n = 86 patients underwent computed tomography (CT) for the evaluation of valvulopathy were segmented to obtain the LV surface and then were automatically aligned to a reference template by rigid registrations and transformations. Shape modes of the anatomical LV variation induced by the degree of AS were assessed by principal component analysis (PCA). The first shape mode represented nearly 50% of the total variance of LV shape in our patient population and was mainly associated to a spherical LV geometry. At Pearson's analysis, the first shape mode was positively correlated to both the end-diastolic volume (p < 0.01, R = 0.814) and end-systolic volume (p < 0.01, and R = 0.922), suggesting LV impairment in patients with severe AS. A predictive model built with PCA-related shape modes achieved better perfor-mance in stratifying the occurrence of adverse events with respect to a baseline model using clinical demographic data as risk predictors. This study demonstrated the potential of SSA approaches to detect the association of complex 3D shape features with functional LV parameters.

Keywords: Aortic valve stenosis , Left ventricle , Statistical shape analysis

Convolutional neural networks are increasingly used in the medical field for the automatic segmentation of several anatomical regions on diagnostic and non-diagnostic images. Such automatic algorithms allow to speed up time-consuming processes and to avoid the presence of expert personnel, reducing time and costs. The present work proposes the use of a convolutional neural network, the U-net architecture, for the segmentation of ear elements. The auricular elements segmentation process is a crucial step of a wider procedure, already automated by the authors, that has as final goal the realization of surgical guides designed to assist surgeons in the reconstruction of the external ear. The segmentation, performed on depth map images of 3D ear models, aims to define of the contour of the helix, antihelix, tragus-antitragus and concha. A dataset of 131 ear depth map was created;70% of the data are used as the training set, 15% composes the validation set, and the remaining 15% is used as testing set. The network showed excellent performance, achieving 97% accuracy on the validation test.

A major challenge in the field of Augmented Reality (AR) is the way in which augmented information is presented in a wide range of uncontrollable environmental conditions. In fact, the variability of colours and illumination conditions of the real environment makes it difficult to choose the most suitable appearance properties for augmented contents. In many AR applications, the colours of virtual objects play a crucial role in blending digital information into the real environment, therefore these colours should be selected according to the appearance of the real background. In some use cases, the colours of virtual objects need to be harmonised with the ones of the real environment; in other cases, the colours should be chosen to ensure the visibility (e.g. maximizing the contrast) of the augmented data with respect to the background. To this end, the paper presents a background-aware colourisation technique that allows for selecting virtual objects' colours in accordance with the real environment in real-time. Given an arbitrary real background, virtual objects' colours are automatically chosen according to three different strategies, i.e. harmonic, disharmonic, and balanced. The proposed AR colourisation technique was assessed with a user study that focused on three different case studies. The results were promising and suggest the potential of the proposed technique for many different application areas. In particular, disharmonic and balanced strategies ensured the distinctiveness of virtual objects according to the real background. Instead, the harmonic strategy was less effective in the case of colourful complex AR scenarios.

Keywords: Augmented reality , colour harmonisation , colourisation , image processing , user studies

Colour and texture are two perceptual stimuli that determine, to a great extent, the appearance of objects, materials and scenes. The ability to process texture and colour is a fundamental skill in humans as well as in animals; therefore, reproducing such capacity in artificial (‘intelligent') systems has attracted considerable research attention since the early 70s. Whereas the main approach to the problem was essentially theory-driven (‘hand-crafted') up to not long ago, in recent years the focus has moved towards data-driven solutions (deep learning). In this overview we retrace the key ideas and methods that have accompanied the evolution of colour and texture analysis over the last five decades, from the ‘early years' to convolutional networks. Specifically, we review geometric, differential, statistical and rank-based approaches. Advantages and disadvantages of traditional methods vs. deep learning are also critically discussed, including a perspective on which traditional methods have already been subsumed by deep learning or would be feasible to integrate in a data-driven approach.

Keywords: Colour , Deep learning , Texture , Visual recognition

We present our latest research on skin characterizations by using Contact Capacitive Imaging and High-Resolution Ultrasound Imaging with Machine Learning algorithms. Contact Capacitive Imaging is a novel imaging technology based on the dielectric constant measurement principle, with which we have studied the skin water content of different skin sites and performed image classification by using pre-trained Deep Learning Neural Networks through Transfer Learning. The results show lips and nose have the lowest water content, whilst cheek, eye corner and under-eye have the highest water content. The classification yields up to 83.8% accuracy. High-Resolution Ultrasound Imaging is a state-of-the-art ultrasound technology, and can produce high-resolution images of the skin and superficial soft tissue to a vertical resolution of about 40 microns, with which we have studied the thickness of different skin layers, such as stratum corneum, epidermis and dermis, around different locations on the face and around different body parts. The results show the chin has the highest stratum corneum thickness, and the arm has the lowest stratum corneum thickness. We have also developed two feature-based image classification methods which yield promising results. The outcomes of this study could provide valuable guidelines for cosmetic/medical research, and methods developed in this study can also be extended for studying damaged skin or skin diseases. The combination of Contact Capacitive Imaging and High-Resolution Ultrasound Imaging could be a powerful tool for skin studies.

Keywords: Capacitive imaging , High-resolution ultrasound , Machine learning , Skin image analysis , Skin texture , Skin thickness , Skin water content

Computer-assisted analysis of three-dimensional imaging data (radiomics) has received a lot of research attention as a possible means to improve the management of patients with lung cancer. Building robust predictive models for clinical decision making requires the imaging features to be stable enough to changes in the acquisition and extraction settings. Experimenting on 517 lung lesions from a cohort of 207 patients, we assessed the stability of 88 texture features from the following classes: first-order (13 features), Grey-level Co-Occurrence Matrix (24), Grey-level Difference Matrix (14), Grey-level Run-length Matrix (16), Grey-level Size Zone Matrix (16) and Neighbouring Grey-tone Difference Matrix (five). The analysis was based on a public dataset of lung nodules and open-access routines for feature extraction, which makes the study fully reproducible. Our results identified 30 features that had good or excellent stability relative to lesion delineation, 28 to intensity quantisation and 18 to both. We conclude that selecting the right set of imaging features is critical for building clinical predictive models, particularly when changes in lesion delineation and/or intensity quantisation are involved.

Keywords: Computed tomography , Intensity quantisation , Lesion delineation , Lung nodules , Radiomics , Stability , Texture features

Background: Accurate segmentation of pulmonary nodules on computed tomography (CT) scans plays a crucial role in the evaluation and management of patients with suspicion of lung cancer (LC). When performed manually, not only the process requires highly skilled operators, but is also tiresome and time-consuming. To assist the physician in this task several automated and semi-automated methods have been proposed in the literature. In recent years, in particular, the appearance of deep learning has brought about major advances in the field. Methods: Twenty-four (12 conventional and 12 based on deep learning) semi-automated-'one-click'- methods for segmenting pulmonary nodules on CT were evaluated in this study. The experiments were carried out on two datasets: A proprietary one (383 images from a cohort of 111 patients) and a public one (259 images from a cohort of 100). All the patients had a positive transcript for suspect pulmonary nodules. Results: The methods based on deep learning clearly outperformed the conventional ones. The best performance [Sørensen-Dice coefficient (DSC)] in the two datasets was, respectively, 0.853 and 0.763 for the deep learning methods, and 0.761 and 0.704 for the traditional ones. Conclusions: Deep learning is a viable approach for semi-automated segmentation of pulmonary nodules on CT scans.

Keywords: Computed tomography (CT) , Deep learning , Lung cancer (LC) , Pulmonary nodules , Segmentation

This study shows the application of a system to monitor the state of damage of railway wheel steel specimens during rolling contact fatigue tests. This system can make continuous measurements with an evaluation of damage without stopping the tests and without destructive measurements. Four tests were carried out to train the system by recording torque and vibration data. Both statistical and spectral features were extracted from the sensors signals. A Principal Component Analysis (PCA) was performed to reduce the volume of the initial dataset; then, the data were classified with the k-means algorithm. The results were then converted into probabilities curves. Metallurgical investigations (optical micrographs, wear curves) and hardness tests were carried out to assess the trends of machine learning analysis. The training tests were used to train the proposed algorithm. Three validation tests were performed by using the real-time results of the k-means algorithm as a stop condition. Metallurgical analysis was performed also in this case. The validation tests follow the results of the training test and metallurgical analysis confirms the damage found with the machine learning analysis: when the membership probability of the cluster corresponding to the damage state reaches a value higher than 0.5, the metallurgical analysis clearly shows the cracks on the surface of the specimen due to the rolling contact fatigue (RCF) damage mechanism. These preliminary results are positive, even if reproduced on a limited set of specimens. This approach could be integrated in rolling contact fatigue tests to provide additional information on damage progression.

Keywords: Damage assessment , Machine learning , Metallurgical analysis , Rolling contact fatigue tests

A vision-based experimental methodology was developed for monitoring the surface state evolution of specimens during twin-disc rolling contact tests, aimed at providing information for identifying the damage phenomena. The system is based on a high-speed camera and three laser pointers for illuminating the specimen surface. Images of the specimen surface are acquired and processed, allowing the definition of synthetic surface state indexes, as well as the section profiles of the surface. The vision system was applied to alternated dry–wet rolling–sliding contact tests on railway wheel steel specimens, highlighting its effectiveness in the damage evaluation. The potential of the section profile reconstruction as a tool for surface topology analysis was shown.

Keywords: damage assessment , image processing , Online monitoring , rolling contact fatigue , wear

The research work presents a specific procedure to evaluate the performance of soccer goalkeepers by means of a marker-less motion capture (Mocap) system based on a set of 8 GoPro active cameras. The layout of the sensors and the calibration phase have been defined to be easily replicated and not to interfere with athletes and trainers during the exercises. The proposed solution has been tested in collaboration with goalkeeper coaches, who made available the knowledge about conventional training exercise. The Mocap system has been tested by involving two goalkeepers and a coach. The results have been elaborated and the outcomes have been considered interesting by experts. Furthermore, the adopted technology for motion capture and elaboration could be evaluated for an application to the rehabilitation process of an athlete after an injury, to determine his stage of recovery and the most suitable work to be done.

Keywords: Active camera , Goalkeepers training , Kinematic data analysis , Marker-less motion capture , Outdoor motion capture , Sports technology

The study of the human body and its movements is still a matter of great interest today. Most of these issues have as their fulcrum the study of the balance characteristics of the human body and the determination of its Centre of Mass. In sports, a lot of attention is paid to improving and analysing the athlete's performance. Almost all the techniques for determining the Centre of Mass make use of special sensors, which allow determining the physical magnitudes related to the different movements made by athletes. In this paper, a markerless method for determining the Centre of Mass of a subject has been studied, comparing it with a direct widely validated equipment such as the Wii Balance Board, which allows determining the coordinates of the Centre of Pressure. The Motion Capture technique was applied with the OpenPose software, a Computer Vision method boosted with the use of Convolution Neural Networks. Ten quasi-static analyses have been carried out. The results have shown an error of the Centre of Mass position, compared to that obtained from the Wii Balance Board, which has been considered acceptable given the complexity of the analysis. Furthermore, this method, despite the traditional methods based on the use of balances, can be used also for prediction of the vertical position of the Centre of Mass.

Keywords: 3D motion capture , Convolution neural networks , Open Pose

In this paper, an efficient and robust Cartesian Mesh Generation with Local Refinement for an Immersed Boundary Approach is proposed, whose key feature is the capability of high Reynolds number simulations by the use of wall function models, bypassing the need for accurate boundary layer discretization. Starting from the discrete manifold model of the object to be analyzed, the proposed model generates Cartesian adaptive grids for a CFD simulation, with minimal user interactions; the most innovative aspect of this approach is that the automatic generation is based on the segmentation of the surfaces enveloping the object to be analyzed. The aim of this paper is to show that this automatic workflow is robust and enables to get quantitative results on geometrically complex configurations such as marine vehicles. To this purpose, the proposed methodology has been applied to the simulation of the flow past a BB2 submarine, discretized by non-uniform grid density. The obtained results are comparable with those obtained by classical body-fitted approaches but with a significant reduction of the time required for the mesh generation.

Keywords: Cartesian adaptive grids , Immersed boundaries , LES simulation

This paper introduces a low-cost and low computational marker-less motion capture system based on the acquisition of frame images through standard RGB cameras. It exploits the open-source deep learning model CMU, from the tf-pose-estimation project. Its numerical accuracy and its usefulness for ergonomic assessment are evaluated by a proper experiment, designed and per-formed to: (1) compare the data provided by it with those collected from a motion capture golden standard system; (2) compare the RULA scores obtained with data provided by it with those obtained with data provided by the Vicon Nexus system and those estimated through video analysis, by a team of three expert ergonomists. Tests have been conducted in standardized laboratory conditions and involved a total of six subjects. Results suggest that the proposed system can predict angles with good consistency and give evidence about the tool's usefulness for ergonomist.

Keywords: Ergonomic risk assessment , Industrial ergonomics , Motion capture , Postural analysis , RULA

This paper introduces an e-learning platform for the management of courses based on MOOCs, able to continuously monitoring student's behavior through facial coding techniques, with a low computational effort client-side, and to provide useful insight for the instructor. The system exploits the most recent developments in Deep Learning and Computer Vision for Affective Computing, in compliance with the European GDPR. Taking as input the video capture by the webcam of the device used to attend the course, it: (1) performs continuous student's authentication based on face recognition, (2) monitors the student's level of attention through head orientation tracking and gaze detection analysis, (3) estimates student's emotion during the course attendance. The paper describes the overall system design and reports the results of a preliminary survey, which involved a total of 14 subjects, aimed at investigating user acceptance, in terms of intention to continue using such a system.

Keywords: Affective Computing , Deep Learning , E-leaning , Facial Coding , Facial Recognition

Pattern recognition system developers have looked in multiple directions over the years and designed a broad spectrum of methodologies for face identification and verification, both in 2D and 3D. These techniques rely on sound methods and experimentations, and currently give high to excellent recognition rates in terms of performance. Nonetheless, it seems that the most performing face recognition system, especially when familiar faces are involved, is still the human being, able to detect known faces in the wild, in presence of occlusions or extreme light contrast, caricatures, sketches, partial views, blurred images. This is one of the manifold reasons why the human visual system at eye and brain level and face perception techniques are currently being studied by neuroscientists and psychologists, with the aim to uncover the processes underneath the human vision. The purpose of this work is to review the current literature about perception foundations and related biologically-inspired methodologies for face recognition.

Keywords: Biologically-inspired computing , Face expression recognition , Face perception , Face recognition , Human vision , Visual system

Patients with severe facial deformities present serious dysfunctionalities along with an unsatisfactory aesthetic facial appearance. Several methods have been proposed to specifically plan the interventions on the patient's needs, but none of these seem to achieve a sufficient level of accuracy in predicting the resulting facial appearance. In this context, a deep knowledge of what occurs in the face after bony movements in specific surgeries would give the possibility to develop more reliable systems. This study aims to propose a novel 3D approach for the evaluation of soft tissue zygomatic modifications after zygomatic osteotomy; geometrical descriptors usually involved in face analysis tasks, i.e., face recognition and facial expression recognition, are here applied to soft tissue malar region to detect changes in surface shape. As ground truth for zygomatic changes, a zygomatic openness angular measure is adopted. The results show a high sensibility of geometrical descriptors in detecting shape modification of the facial surface, outperforming the results obtained from the angular evaluation.

Keywords: 3D face analysis , Geometrical descriptors , Malar augmentation , Orthognathic surgery , Soft tissue prediction , Zygomatic osteotomy

Background: The aim of this prospective study is to objectively assess 3D soft tissue and bone changes of the malar region by using the malar valgization osteotomy in concomitant association with orthognatic surgery. Materials and methods: From January 2015 to January 2018, 10 patients who underwent single stage bilateral malar valgization osteotomy in conjunction with maxillo-mandibular orthognatic procedures for aesthetic and functional correction were evaluated. Clinical and surgical reports were collected and patient satisfaction was evaluated with a VAS score. For each patient, maxillofacial CT-scans were collected 1 month preoperatively (T0) and 6 months after the operation (T1). DICOM data were imported and elaborated in the software MatLab, which creates a 3D soft tissue model of the face. 3D Bone changes were assessed importing DICOM data into iPlan (BrainLAB 3.0) software and the superimposition process was achieved using autofusion. Descriptive statistical analyses were obtained for soft tissue and bone changes. Results: Considering bone assessment the comparison by superimposition between T0 and T1 showed an increase of the distance between bilateral malar prominence (Pr – Pl) and a slight forward movement (87,65 ± 1,55 to 97,60 ± 5,91); p-value 0.007. All of the patients had improvement of α angle, ranging from 36,30 ± 1,70 to 38,45 ± 0,55, p-value 0,04 (αr) and 36,75 ± 1,58 to 38,45 ± 0,35; p-value 0,04 (αl). The distance S increased from 78,05 ± 2,48 to 84,2 ± 1,20; p-value 0,04 (Sr) and 78,65 ± 2,16 to 82,60 ± 0,90 (Sl); p-value 0,03. Considering the soft tissue, the comparison by superimposition between T0 and T1 showed an antero-lateral movement (p-value 0.008 NVL; p-value 0.001 NVR) of the malar bone projection together with an increase in width measurements (p-value 0,05 VL; p-value 0,01 VR). Angular measurement confirmed the pattern of the bony changes (p-value 0.034 αL; p-value 0,05 αR). Conclusion: The malar valgization osteotomy in conjunction with orthognatic surgery is effective in improving zygomatic projection contributing to a balanced facial correction in midface hypoplasia.3D geometrical based volume and surface analysis demonstrate an increase in transversal and forward direction. The osteotomy can be safely performed in conjunction with orthognatic procedures.

Keywords: Malar augmentation , Orthognatic surgery , Zygomatic osteotomy