Image Processing & Applications

This paper presents a novel instant 3D whole body scanner for healthcare applications. It is based on photogrammetry, a digital technology which allows to reconstruct the surface of objects starting from multiple pictures. The motivation behind this work is the development of minimally invasive procedures for instant data acquisitions of anatomical structure. The scanner provides several features of interests in 3D body scanning technologies for the healthcare domains: (i) instant capture of human body models; (ii) magnitude of accuracy in the order of 1 mm; (iii) simplicity of use; (iv) possibility to scan using different settings; (v) possibility to reconstruct the texture. The system is built upon a modular and distributed architecture. In this paper we highlight its key concepts and the methodology which has led to the current product. We illustrate its potential through one of the most promising 3D scanning healthcare applications: the data acquisition and processing of human body models for the digital manufacturing process of prostheses and orthoses. We validate the overall system in terms of conformity with the the initial requirements.

Keywords: 3D reconstruction , Body scanning , Healthcare , Human body measurements , Human body visualization , Photogrammetry , Proshetics and orthotics

The combination of mirrors and lenses, which defines a catadioptric sensor, is widely used in the computer vision field. The definition of a catadioptric sensors is based on three main features: hardware setup, projection modelling and calibration process. In this paper, a complete description of these aspects is given for an omnidirectional sensor based on a spherical mirror. The projection model of a catadioptric system can be described by the forward projection task (FP, from 3D scene point to 2D pixel coordinates) and backward projection task (BP, from 2D coordinates to 3D direction of the incident light). The forward projection of non-central catadioptric vision systems, typically obtained by using curved mirrors, is usually modelled by using a central approximation and/or by adopting iterative approaches. In this paper, an analytical closed-form solution to compute both forward and backward projection for a non-central catadioptric system with a spherical mirror is presented. In particular, the forward projection is reduced to a 4th order polynomial by determining the reflection point on the mirror surface through the intersection between a sphere and an ellipse. A matrix format of the implemented models, suitable for fast point clouds handling, is also described. A robust calibration procedure is also proposed and applied to calibrate a catadioptric sensor by determining the mirror radius and center with respect to the camera.

Keywords: Backward projection model , Catadioptric sensor , Computer vision , Forward projection model , Spherical mirror

A 3D full-field optical system for high frequency vibration measurement is proposed. The system is composed of a single low-frame-rate camera and two planar mirrors. This compact optical setup overcomes the typical drawback of capturing synchronous acquisitions in the case of a camera pair. Moreover, planar mirrors allow for the use of the classical pinhole model and, thus, conventional stereo-calibration techniques. The use of a low-frame-rate camera provides on the one hand a high-resolution sensor with a relatively low-cost hardware but imposes, on the other, the adoption of a down-sampling approach, which is applicable only when a single (known) sinusoidal load is applied to the structure. The effectiveness of the proposed setup has been verified by the 3D vibration measurement of two different targets up to a frequency of 1 kHz, corresponding to a displacement amplitude of 0.01 mm.

Keywords: digital image correlation , down-sampling approach , Reverse engineering , single low-speed camera

3D virtual reconstruction of human body parts is nowadays a common practice in many research fields such as the medical one, the manufacturing of customized products or the creation of personal avatar for gaming purpose. The acquisition can be performed with the use of an active stereo system (i.e., laser scanner, structured light sensors) or with the use of a passive image-based approach. While the former represents a consolidated approach in human modeling, the second is still an active research field. Usually, the reconstruction of a body part through a scanning system is expensive and requests to project light on the patient's body. On the other hand, the image-based approach could use multi-photo technique to reconstruct a real scene and provides some advantages: low equipment costs (only one camera) and rapid acquisition process of the photo set. In this work, the use of the photogrammetry approach for the reconstruction of humans' face has been investigated as an alternative to active scanning systems. Two different photogrammetric approaches have been tested to verify their potentiality and their sensitivity to configuration parameters. An initial comparison among them has been performed, considering the overall number of points detected (sparse point cloud reconstruction, dense point cloud reconstruction). Besides, to evaluate the accuracy of the reconstruction, a set of measures used in the design of wearable head-related products has been assessed.

In the computer vision field, the reconstruction of target surfaces is usually achieved by using 3D optical scanners assembled integrating digital cameras and light emitters. However, these solutions are limited by the low field of view, which requires multiple acquisition from different views to reconstruct complex free-form geometries. The combination of mirrors and lenses (catadioptric systems) can be adopted to overcome this issue. In this work, a stereo catadioptric optical scanner has been developed by assembling two digital cameras, a spherical mirror and a multimedia white light projector. The adopted configuration defines a non-single viewpoint system, thus a non-central catadioptric camera model has been developed. An analytical solution to compute the projection of a scene point onto the image plane (forward projection) and vice-versa (backward projection) is presented. The proposed optical setup allows omnidirectional stereo vision thus allowing the reconstruction of target surfaces with a single acquisition. Preliminary results, obtained measuring a hollow specimen, demonstrated the effectiveness of the described approach.

Keywords: 3D acquisition , catadioptric stereo vision system , internal geometries acquisition , spherical mirror projection model , structured light scanning

Background and objective: The purpose of the present paper is to pave the road to the systematic optimization of complex craniofacial surgical intervention and to validate a design methodology for the virtual surgery and the fabrication of cranium vault custom plates. Recent advances in the field of medical imaging, image processing and additive manufacturing (AM) have led to new insights in several medical applications. The engineered combination of medical actions and 3D processing steps, foster the optimization of the intervention in terms of operative time and number of sessions needed. Complex craniofacial surgical intervention, such as for instance severe hypertelorism accompanied by skull holes, traditionally requires a first surgery to correctly “resize” the patient cranium and a second surgical session to implant a customized 3D printed prosthesis. Between the two surgical interventions, medical imaging needs to be carried out to aid the design the skull plate. Instead, this paper proposes a CAD/AM-based one-in-all design methodology allowing the surgeons to perform, in a single surgical intervention, both skull correction and implantation. Methods: A strategy envisaging a virtual/mock surgery on a CAD/AM model of the patient cranium so as to plan the surgery and to design the final shape of the cranium plaque is proposed. The procedure relies on patient imaging, 3D geometry reconstruction of the defective skull, virtual planning and mock surgery to determine the hypothetical anatomic 3D model and, finally, to skull plate design and 3D printing. Results: The methodology has been tested on a complex case study. Results demonstrate the feasibility of the proposed approach and a consistent reduction of time and overall cost of the surgery, not to mention the huge benefits on the patient that is subjected to a single surgical operation. Conclusions: Despite a number of AM-based methodologies have been proposed for designing cranial implants or to correct orbital hypertelorism, to the best of the authors' knowledge, the present work is the first to simultaneously treat osteotomy and titanium cranium plaque.

Keywords: Additive manufacturing , CAD , Cranium surgery , Image processing

With the aim of retrieving 3D surfaces starting from single shaded images, i.e. for solving the widely known shape from shading problem, an important class of methods is based on minimisation techniques where the expected surface to be retrieved is supposed to be coincident with the one that minimise a properly developed functional, consisting of several contributions. Despite several different contributes that can be explored to define a functional, the so called 'smoothness constraint' is a cornerstone since it is the most relevant contribute to guide the convergence of the minimisation process towards a more accurate solution. Unfortunately, in case input shaded image is characterised by areas where actual brightness changes rapidly, such a constraint introduces an undesired over-smoothing effect for the retrieved surface. The present work proposes an original strategy for avoiding such a typical over-smoothing effect, with regard to the image regions in which this is particularly undesired such as, for instance, zones where surface details are to be preserved in the reconstruction. The proposed strategy is tested against a set of case studies and compared with other traditional SFS-based methods to prove its effectiveness.

Keywords: 3D model , Minimisation , SFS , shape from shading , Smoothing , Smoothness constraint , variational approach

Visual localization across seasons and under varying weather and lighting conditions is a challenging task in robotics. In this paper, we present a new sequence-based approach to visual localization using the Partial Order Kernel (POKer), a convolution kernel for string comparison, that is able to handle appearance changes and is robust to speed variations. We use multiple sequence alignment to construct directed acyclic graph representations of the database image sequences, where sequences of images of the same place acquired at different times are represented as alternative paths in a graph. We then use the POKer to compute the pairwise similarities between these graphs and the query image sequences obtained in a subsequent traversal of the environment, and match the corresponding locations. We evaluated our approach on a dataset which features extreme appearance variations due to seasonal changes. The results demonstrate the effectiveness of our approach, where it achieves higher precision and recall than two state-of-the-art baseline methods.

Keywords: Kernel methods , Partial order graphs , Visual localization

The ever increasing availability of digital data from the Arts and cultural heritage calls for efficient methods to organise, categorise, and retrieve such information in an effective and reliable way. In this context, painting classification has attracted significant research interest in recent years. In this work we address the problem of style classification, which involves determining the school, period and art movement to which a painting belongs. Notably, this job is peculiarly different from other machine vision applications - such as material, object and scene recognition - since the concept of 'similarity' is much more difficult to define in this case. For this specific task we evaluate, in this study, the effectiveness of an array of hand-designed visual descriptors against a set of feature extractors based on last-generation convolutional neural networks. We also investigate the effect of pre-processing methods such as image split and pyramidal decomposition. The experiments are based on the open-access Pandora dataset. The results show that pre-trained models can significantly outperform hand-designed descriptors with overall accuracy surpassing 67%. This represents an improvement on the state-of-the-art by ≈12 percentage points.

Background/Aim. We retrospectively investigated the prognostic potential (correlation with overall survival) of 9 shape and 21 textural features from non-contrast-enhanced computed tomography (CT) in patients with non-small-cell lung cancer. Materials and Methods. We considered a public dataset of 203 individuals with inoperable, histologically- or cytologically-confirmed NSCLC. Three-dimensional shape and textural features from CT were computed using proprietary code and their prognostic potential evaluated through four different statistical protocols. Results. Volume and grey-level run length matrix (GLRLM) run length non-uniformity were the only two features to pass all four protocols. Both features correlated negatively with overall survival. The results also showed a strong dependence on the evaluation protocol used. Conclusion: Tumour volume and GLRLM run-length non-uniformity from CT were the best predictor of survival in patients with non-small-cell lung cancer. We did not find enough evidence to claim a relationship with survival for the other features.

Keywords: Computed tomography , Non-small-cell lung cancer , Radiomics. , Shape , Texture

Texture classification plays a major role in many computer vision applications. Local binary patterns (LBP) encoding schemes have largely been proven to be very effective for this task. Improved LBP (ILBP) are conceptually simple, easy to implement, and highly effective LBP variants based on a point-to-average thresholding scheme instead of a point-to-point one. We propose the use of this encoding scheme for extracting intra- and interchannel features for color texture classification. We experimentally evaluated the resulting improved opponent color LBP alone and in concatenation with the ILBP of the local color contrast map on a set of image classification tasks over 9 datasets of generic color textures and 11 datasets of biomedical textures. The proposed approach outperformed other grayscale and color LBP variants in nearly all the datasets considered and proved competitive even against image features from last generation convolutional neural networks, particularly for the classification of biomedical images.

Keywords: Color texture , Convolutional neural networks , Image classification , Local binary patterns

Convolutional Neural Networks have proved extremely successful in object classification applications; however, their suitability for texture analysis largely remains to be established. We investigate the use of pre-trained CNNs as texture descriptors by tapping the output of the last fully connected layer, an approach that has proved its effectiveness in other domains. Comparison with classical descriptors based on signal processing or statistics over a range of standard databases suggests that CNNs may be more effective where the intra-class variability is large. Conversely, classical approaches may be preferable where classes are well defined and homogeneous.

Keywords: Convolutional Neural Networks , Image classification , Local Binary Patterns , Texture

Features from pre-trained Convolutional Neural Newtorks (CNN) have proved to be effective for many tasks such as object, scene and face recognition. Compared with traditional, hand-designed image descriptors, CNN-based features produce higher-dimensional feature vectors. In specific applications where the number of samples may be limited – as in the case of histopatological images – high dimensionality could potentially cause overfitting and redundancy in the information to be processed and stored. To overcome these potential problems feature reduction methods can be applied, at the cost of a moderate reduction in the discrimination accuracy. In this paper we investigate dimensionality reduction schemes for CNN-based features applied to computer-assisted classification of histopathological images. The purpose of this study is to find the best trade-off between accuracy and dimensionality. Specifically, we test two well-known techniques (i.e.: Principal Component Analysis and Gaussian Random Projection) and propose a novel reduction strategy based on the cross-correlation between the components of the feature vector. The results show that it is possible to reduce CNN-based features by a high ratio with a moderate decrease in accuracy with respect to the original values.

Keywords: Convolutional Neural Networks , Feature reduction , Histopathological images , Image classification

In this paper, we aim at providing results concerning the application of desktop systems for rapid prototyping of medical replicas that involve complex shapes, as, for example, folds of a colon. Medical replicas may assist preoperative planning or tutoring in surgery to better understand the interaction among pathology and organs. Major goals of the paper concern with guiding the digital design workflow of the replicas and understanding their final performance, according to the requirements asked by the medics (shape accuracy, capability of seeing both inner and outer details, and support and possible interfacing with other organs). In particular, after the analysis of these requirements, we apply digital design for colon replicas, adopting two desktop systems. The experimental results confirm that the proposed preprocessing strategy is able to conduct to the manufacturing of colon replicas divided in self-supporting segments, minimizing the supports during printing. This allows also to reach an acceptable level of final quality, according to the request of having a 3D presurgery overview of the problems. These replicas are compared through reverse engineering acquisitions made by a structured-light system, to assess the achieved shape and dimensional accuracy. Final results demonstrate that low-cost desktop systems, coupled with proper strategy of preprocessing, may have shape deviation in the range of ±1 mm, good for physical manipulations during medical diagnosis and explanation.

Finite Element Analysis (FEA) has gained an extensive application in the medical field, such as soft tissues simulations. In particular, colorectal simulations can be used to understand the interaction with the surrounding tissues, or with instruments used in surgical procedures. Although several works have been introduced considering small displacements, as a result of the forces exerted on adjacent tissues, FEA applied to colorectal surgical scenarios is still a challenge. Therefore, this work aims to provide a sensitivity analysis on three geometric models, taking in mind different bioengineering tasks. In this way, a set of simulations has been performed using three mechanical models named Linear Elastic, Hyper-Elastic with a Mooney-Rivlin material model, and Hyper-Elastic with a YEOH material model.

Keywords: Computer assisted surgical planning , Finite element analysis , Soft tissues simulation , Surface modeling

A machine learning technique was used to monitor the state of damage of steel specimens subjected to cyclic contact fatigue tests and to make a continuous measurement without coming into contact with the specimens examined. Six different sets of specimens were considered and the torque and vibration data obtained from their respective tests were analyzed. Different statistical and spectral characteristics were obtained from each dataset, to represent their information content. The correlation existing between the characteristics was studied to select only the essential ones. The global data volume was reduced by a principal component analysis. The data thus obtained were subsequently processed, and classified, developing a K-means algorithm. The results were formulated in probabilistic terms with the aid of membership charts, confusion matrices and probability curves. The results obtained from this first analysis are highly encouraging. The algorithm is able to assess the damage status of the specimens, as well as detect the operating conditions of the machine. A substantial advantage of this method is the ability to estimate quantitatively the damage by calculating the distance of the current data from specific references. The performances of this approach were numerically estimated in terms of sensitivity, specificity and accuracy.

Twin-disk tests are an effective method to characterize the material response to rolling sliding contact, to reproduce the damage phenomena of real components at a laboratory scale in controlled working conditions. Usually the monitoring is performed by means of “gross” parameters, such as weight loss, coefficient of friction and Barkhausen noise, and micrographs of the sample sections, at the end of the test. Visual inspection of the sample contact surface at the macro-scale yields further information about the process under analysis. In twin-disk tests, the samples are visually inspected at predetermined steps, typically by acquiring the image of their surface when the samples are stationary. The availability of a system able to capture the images of the samples while they roll during the tests is of interest to better monitor the damage evolution. In this paper, we present the results of the experiments carried out to extract quantitative information from the images captured on railway wheel samples during rolling contact tests: suitable image processing has been designed with the objective of finding meaningful, synthetic indices for the monitoring and the interpretation of the wear process, also in relation to prior knowledge about the process. The experimental work was focused on the definition of the indices, on the analysis of their behavior on different steels, and on their usefulness to predict uneven wear during the tests on the test bench.

Keywords: Damage monitoring , Image acquisition , Image processing , Twin-disk tests , Wear

Rolling Contact Fatigue (RCF) tests are a common effective method to study the behavior of wheel- and rail-steels. The measurements usually performed are discrete and destructive: they can only be performed at each intermediate stop of a test and they result in the alteration or destruction of the examined specimens. This work aims to assess the damage level steel samples during RCF tests, making continuous, non-destructive, and contactless measurements. A machine-learning technique based on vibration and torque measurements, together with 2D image was applied to RCF-dry tests carried out on different railway wheel steels tested according to the same operating parameters. The proposed algorithm was able to quantitatively estimate the damage level of the samples by calculating the current data distance from specific references, e.g. a defined final damaged state. The used approach ensures a good degree of reliability both in terms of specificity and sensitivity.

A vision-based experimental methodology was developed for monitoring the surface state evolution of specimens during bi-disc rolling contact tests, aimed at providing information for identifying the damage phenomena. The system is based on a high-speed camera and three laser pointers for illuminating the specimen surface. Images of the specimen surface are acquired and processed, allowing the definition of synthetic surface state indexes, as well as the 3D reconstruction of the damaged surface topology. The vision system was applied to alternated dry-wet rollingsliding contact tests on railway wheel steel specimens, highlighting its effectiveness in the damage evaluation.

Keywords: Damage assessment. , Image processing , Online monitoring , Rolling contact fatigue , Wear

Given a to-be-watermarked 3D model, a transformed domain analysis is needed to guarantee a robust embedding without compromising the visual quality of the result. A multiresolution remeshing of the model allows to represent the 3D surface in a transformed domain suitable for embedding a robust and imperceptible watermark signal. Simplification of polygonal meshes is the basic step for a multiresolution remeshing of a 3D model; this step is needed to obtain the model approximation (coarse version) from which a refinement framework (i.e. 3D wavelet analysis, spectral analysis, …) able to represent the model at multiple resolution levels, can be performed. The simplification algorithm should satisfy some requirements to be used in a watermarking system: the repeatability of the simplification, and the robustness of it to noise or, more generally, to slight modifications of the full resolution mesh. The performance of a number of software packages for mesh simplification, including both commercial and academic offerings, are compared in this survey. We defined a benchmark for testing the different software in the watermarking scenario and reported a comprehensive analysis of the software performances based on the geometric distortions measurement of the simplified versions.

Keywords: 3D watermarking , Mesh comparison , Mesh simplification , Wavelets 3D