Combined multi-kernel support vector machine and wavelet analysis for hyperspectral remote sensing image classification

Many remote sensing image classifiers are limited in their ability to combine spectral features with spatial features.Multi-kernel classifiers,however,are capable of integrating spectral features with spatial or structural features using multiple kernels and summing them for final outputs.Using a support vector machine(SVM) as classifier,different multi-kernel classifiers are constructed and tested using 64-band Operational Modular Imaging Spectrometer II hyperspectral image of Changping Area,Beijing City.Results show that by integrating spectral and wavelet texture information,multi-kernel SVM classifiers can obtain more accurate classification results than sole-kernel SVM classifiers and cross-information SVM kernel classifiers.Moreover,when the multi-kernel SVM classifier is used,the combination of the first four principal components from principal component analysis and wavelet texture provides the highest accuracy(97.06%).Multi-kernel SVM is therefore an effective approach to improve the accuracy of hyperspectral image classification and to expand possibilities for remote sensing image interpretation and application.     

Improving the Retrieval of Arabic Web Search Results Using Enhanced k-Means Clustering Algorithm

Traditional information retrieval systems return a ranked list of results to a user’s query. This list is often long, and the user cannot explore all the results retrieved. It is also ineffective for a highly ambiguous language such as Arabic. The modern writing style of Arabic excludes the diacritical marking, without which Arabic words become ambiguous. For a search query, the user has to skim over the document to infer if the word has the same meaning they are after, which is a time-consuming task. It is hoped that clustering the retrieved documents will collate documents into clear and meaningful groups. In this paper, we use an enhanced k-means clustering algorithm, which yields a faster clustering time than the regular k-means. The algorithm uses the distance calculated from previous iterations to minimize the number of distance calculations. We propose a system to cluster Arabic search results using the enhanced k-means algorithm, labeling each cluster with the most frequent word in the cluster. This system will help Arabic web users identify each cluster’s topic and go directly to the required cluster. Experimentally, the enhanced k-means algorithm reduced the execution time by 60% for the stemmed dataset and 47% for the non-stemmed dataset when compared to the regular k-means, while slightly improving the purity.     

Wavelet SVM in Reproducing Kernel Hilbert Space for hyperspectral remote sensing image classification

Combining Support Vector Machine (SVM) with wavelet analysis, we constructed wavelet SVM (WSVM) classifier based on wavelet kernel functions in Reproducing Kernel Hilbert Space (RKHS). In conventional kernel theory, SVM is faced with the bottleneck of kernel parameter selection which further results in time-consuming and low classification accuracy. The wavelet kernel in RKHS is a kind of multidimensional wavelet function that can approximate arbitrary nonlinear functions. Implications on semiparametric estimation are proposed in this paper. Airborne Operational Modular Imaging Spectrometer II (OMIS II) hyperspectral remote sensing image with 64 bands and Reflective Optics System Imaging Spectrometer (ROSIS) data with 115 bands were used to experiment the performance and accuracy of the proposed WSVM classifier. The experimental results indicate that the WSVM classifier can obtain the highest accuracy when using the Coiflet Kernel function in wavelet transform. In contrast with some traditional classifiers, including Spectral Angle Mapping (SAM) and Minimum Distance Classification (MDC), and SVM classifier using Radial Basis Function kernel, the proposed wavelet SVM classifier using the wavelet kernel function in Reproducing Kernel Hilbert Space is capable of improving classification accuracy obviously.     

Classification of degraded cartilage through multiparametric MRI analysis

MRI analysis of cartilage matrix may play an important role in early detection and development of therapeutic protocols for degenerative joint disease. Correlations between MRI parameters and matrix integrity have been established in many studies, but the substantial overlap in values observed for normal and for degraded cartilage greatly limits the specificity of these analyses. We implemented established multiparametric analysis methods to define data clusters corresponding to control and degraded bovine nasal cartilage in two-, three-, and four-dimensional parameter spaces, and applied these results to discriminant analysis of a validation data set. Analyses were performed using the parameters (T₁, T₂, k_m, ADC), where k_m is the magnetization transfer rate and ADC is the apparent diffusion coefficient. Results were compared to univariate analyses. Multiparametric k-means clustering led to no improvement over univariate analyses, with a maximum sensitivity and specificity in the range of 60–70% for the detection of degradation using T₁, and in the range of 80% sensitivity but only 36% specificity using the parameter pair (T₁, k_m). In contrast, model-based analysis using more general Gaussian clusters resulted in markedly improved classification, with sensitivity and specificity reaching levels of 80–90% using the pair (T₁, k_m). Finally, a fuzzy clustering technique was implemented which may be still more appropriate to the continuum of degradation seen in degenerative cartilage disease. In view of its success in identifying mild cartilage degradation, the formal multiparametric approach implemented here may be applicable to the nondestructive evaluation of other biomaterials using MRI.     

A framework for classification with single feature kernel matrix

In this paper, we propose a novel classification framework using single feature kernel matrix. Different from the traditional kernel matrices which make use of the whole features of samples to build the kernel matrix, this research uses features of the same dimension of any two samples to build a sub-kernel matrix and sums up all the sub-kernel matrices to get the single feature kernel matrix. We also use single feature kernel matrix to build a new SVM classifier, and adapt SMO (Sequential Minimal Optimization) algorithm to solve the problem of SVM classifier. The results of the experiments on several artificial datasets and some challenging public cancer datasets display the classification performance of the algorithm. The comparisons between our algorithm and L₂-norm SVM on the cancer datasets demonstrate that the accuracy of our algorithm is higher, and the number of support vectors selected is fewer, indicating that our proposed framework is a more practical approach.     

k -Nearest Neighbors for automated classification of celestial objects

The nearest neighbors (NNs) classifiers, especially the k-Nearest Neighbors (kNNs) algorithm, are among the simplest and yet most efficient classification rules and widely used in practice. It is a nonparametric method of pattern recognition. In this paper, k-Nearest Neighbors, one of the most commonly used machine learning methods, work in automatic classification of multi-wavelength astronomical objects. Through the experiment, we conclude that the running speed of the kNN classier is rather fast and the classification accuracy is up to 97.73%. As a result, it is efficient and applicable to discriminate active objects from stars and normal galaxies with this method. The classifiers trained by the kNN method can be used to solve the automated classification problem faced by astronomy and the virtual observatory (VO). Supported by the National Natural Science Foundation of China (Grant Nos. 10473013, 10778724 and 90412016)     

Classifier ensembles for fMRI data analysis: an experiment

Functional magnetic resonance imaging (fMRI) is becoming a forefront brain–computer interface tool. To decipher brain patterns, fast, accurate and reliable classifier methods are needed. The support vector machine (SVM) classifier has been traditionally used. Here we argue that state-of-the-art methods from pattern recognition and machine learning, such as classifier ensembles, offer more accurate classification. This study compares 18 classification methods on a publicly available real data set due to Haxby et al. [Science 293 (2001) 2425–2430]. The data comes from a single-subject experiment, organized in 10 runs where eight classes of stimuli were presented in each run. The comparisons were carried out on voxel subsets of different sizes, selected through seven popular voxel selection methods. We found that, while SVM was robust, accurate and scalable, some classifier ensemble methods demonstrated significantly better performance. The best classifiers were found to be the random subspace ensemble of SVM classifiers, rotation forest and ensembles with random linear and random spherical oracle.     

Diagnosis of response and non-response to dry eye treatment using infrared thermography images

The dry eye treatment outcome depends on the assessment of clinical relevance of the treatment effect. The potential approach to assess the clinical relevance of the treatment is to identify the symptoms responders and non-responders to the given treatments using the responder analysis. In our work, we have performed the responder analysis to assess the clinical relevance effect of the dry eye treatments namely, hot towel, EyeGiene®, and Blephasteam® twice daily and 12 min session of Lipiflow®. Thermography is performed at week 0 (baseline), at weeks 4 and 12 after treatment. The clinical parameters such as, change in the clinical irritations scores, tear break up time (TBUT), corneal staining and Schirmer’s symptoms tests values are used to obtain the responders and non-responders groups. We have obtained the infrared thermography images of dry eye symptoms responders and non-responders to the three types of warming treatments. The energy, kurtosis, skewness, mean, standard deviation, and various entropies namely Shannon, Renyi and Kapoor are extracted from responders and non-responders thermograms. The extracted features are ranked based on t-values. These ranked features are fed to the various classifiers to get the highest performance using minimum features. We have used decision tree (DT), K nearest neighbour (KNN), Naves Bayesian (NB) and support vector machine (SVM) to classify the features into responder and non-responder classes. We have obtained an average accuracy of 99.88%, sensitivity of 99.7% and specificity of 100% using KNN classifier using ten-fold cross validation.     

近红外光谱和多分类器融合的葡萄酒品种判别研究

将多种单分类器模型融合,并用融合后的模型对不同品种干红葡萄酒进行判别分析。用BRUKER MPA傅里叶变换型近红外光谱仪采集170个干红葡萄酒样品的近红外透射光谱,选取PLS-DA,SVM,Fisher和AdaBoost作为单分类器建模方法,分别建立葡萄酒品种判别模型,通过差异性度量值对单分类器进行筛选,得到差异性较大的四个单分类器作为基分类器,其中基分类器对测试集葡萄酒品种判别准确率最高为88.24%,最低为81.18%。然后通过加权投票机制对基分类器进行融合,融合后的模型对测试集葡萄酒品种判别准确率提高至92.94%,误判样品个数由单分类器最少的9个降为6个。实验结果表明多分类器融合所建立的模型优于传统近红外光谱定性分析一般采用单分类器模型结果,提高了葡萄酒品种判别的准确性,采用基于近红外光谱的多分类融合方法对葡萄酒种类判定具有可行性。     

Acoustic classification of Australian anurans based on hybrid spectral-entropy approach

A new hybrid method for automated frog sound identification, using spectral centroid, Shannon entropy and Rényi entropy is proposed. The advantage of using entropy based information theoretic approach for analyzing complexity of bioacoustics signals in animal vocalization is discussed. Sound samples from nine species of Microhylidae frogs are first segmented into syllables. Fourier spectral centroid, Shannon entropy and Rényi entropy of the syllables are then determined. Finally, nonparametric k-th nearest neighbour (k-NN) classifier is used to recognize the frog species based on these three extracted features. Result shows that the k-NN classifier based on these selected features is capable to identify the species of the frogs with an average accuracy of 98%. It is found that the accuracy reduces significantly only when the noise levels higher than −20 dB.     

Raman spectroscopy utilizing Fisher‐based feature selection combined with Support Vector Machines for the characterization of breast cell lines

Raman spectroscopy has the potential to significantly aid in the research and diagnosis of cancer. The information dense, complex spectra generate massive datasets in which subtle correlations may provide critical clues for biological analysis and pathological classification. Therefore, implementing advanced data mining techniques is imperative for complete, rapid and accurate spectral processing. Numerous recent studies have employed various data methods to Raman spectra for classification and biochemical analysis. Although, as Raman datasets from biological specimens are often characterized by high dimensionality and low sample numbers, many of these classification models are subject to overfitting. Furthermore, attempts to reduce dimensionality result in transformed feature spaces making the biological evaluation of significant and discriminative spectral features problematic. We have developed a novel data mining framework optimized for Raman datasets, called Fisher‐based Feature Selection Support Vector Machines (FFS‐SVM). This framework provides simultaneous supervised classification and user‐defined Fisher criterion‐based feature selection, reducing overfitting and directly yielding significant wavenumbers from the original feature space. Herein, we investigate five cancerous and non‐cancerous breast cell lines using Raman microspectroscopy and our unique FFS‐SVM framework. Our framework classification performance is then compared to several other frequently employed classification methods on four classification tasks. The four tasks were constructed by an unsupervised clustering method yielding the four different categories of cell line groupings (e.g. cancer vs non‐cancer) studied. FFS‐SVM achieves both high classification accuracies and the extraction of biologically significant features. The top ten most discriminative features are discussed in terms of cell‐type specific biological relevance. Our framework provides comprehensive cellular level characterization and could potentially lead to the discovery of cancer biomarker‐type information, which we have informally termed ‘Raman‐based spectral biomarkers’. The FFS‐SVM framework along with Raman spectroscopy will be used in future studies to investigate in‐situ dynamic biological phenomena. Copyright © 2013 John Wiley & Sons, Ltd.     

Automatic segmentation of brain white matter and white matter lesions in normal aging: comparison of five multispectral techniques

White matter loss, ventricular enlargement and white matter lesions are common findings on brain scans of older subjects. Accurate assessment of these different features is therefore essential for normal aging research. Recently, we developed a novel unsupervised classification method, named ‘Multispectral Coloring Modulation and Variance Identification’ (MCMxxxVI), that fuses two different structural magnetic resonance imaging (MRI) sequences in red/green color space and uses Minimum Variance Quantization (MVQ) as the clustering technique to segment different tissue types. Here we investigate how this method performs compared with several commonly used supervised image classifiers in segmenting normal-appearing white matter, white matter lesions and cerebrospinal fluid in the brains of 20 older subjects with a wide range of white matter lesion load and brain atrophy. The three tissue classes were segmented from T₁-, T₂-, T₂?- and fluid attenuation inversion recovery (FLAIR)-weighted structural MRI data using MCMxxxVI and the four supervised multispectral classifiers available in the Analyze package, namely, Back-Propagated Neural Networks, Gaussian classifier, Nearest Neighbor and Parzen Windows. Bland–Altman analysis and Jaccard index values indicated that, in general, MCMxxxVI performed better than the supervised multispectral classifiers in identifying the three tissue classes, although final manual editing was still required to deliver radiologically acceptable results. These analyses show that MVQ, as implemented in MCMxxxVI, has the potential to provide quick and accurate white matter segmentations in the aging brain, although further methodological developments are still required to automate fully this technique.     

基于图像局部区域梯度特征描述的红外人体识别算法

提出了一种新的红外图像中人体目标识别方案并进行了算法实现。通过直方图聚类分析对红外图像进行分割,根据二值化图像团块的特点,确定图像中的候选目标图像区域。将候选目标图像按比例划分为多个区域,使用梯度位置朝向直方图(GLOH,Gradient location-orientation histogram)对候选目标图像进行描述。与其它红外图像中人体识别算法相比,不需要多种特征提取算法组合进行分步骤识别,仅使用单个SVM分类器即可达到满意的识别率,避免了分类器的级联,算法简单有效。     

Blast noise classification with common sound level meter metrics

A common set of signal features measurable by a basic sound level meter are analyzed, and the quality of information carried in subsets of these features are examined for their ability to discriminate military blast and non-blast sounds. The analysis is based on over 120?000 human classified signals compiled from seven different datasets. The study implements linear and Gaussian radial basis function (RBF) support vector machines (SVM) to classify blast sounds. Using the orthogonal centroid dimension reduction technique, intuition is developed about the distribution of blast and non-blast feature vectors in high dimensional space. Recursive feature elimination (SVM-RFE) is then used to eliminate features containing redundant information and rank features according to their ability to separate blasts from non-blasts. Finally, the accuracy of the linear and RBF SVM classifiers is listed for each of the experiments in the dataset, and the weights are given for the linear SVM classifier.     

Cluster-Based Improved Isolation Forest

Outlier detection is an important research direction in the field of data mining. Aiming at the problem of unstable detection results and low efficiency caused by randomly dividing features of the data set in the Isolation Forest algorithm in outlier detection, an algorithm CIIF (Cluster-based Improved Isolation Forest) that combines clustering and Isolation Forest is proposed. CIIF first uses the k-means method to cluster the data set, selects a specific cluster to construct a selection matrix based on the results of the clustering, and implements the selection mechanism of the algorithm through the selection matrix; then builds multiple isolation trees. Finally, the outliers are calculated according to the average search length of each sample in different isolation trees, and the Top-n objects with the highest outlier scores are regarded as outliers. Through comparative experiments with six algorithms in eleven real data sets, the results show that the CIIF algorithm has better performance. Compared to the Isolation Forest algorithm, the average AUC (Area under the Curve of ROC) value of our proposed CIIF algorithm is improved by 7%.     

Invariant subspaces of clustering operators

A class of clustering operators is defined which is a generalization of a transfer matrix of a Gibbs lattice field with an exponential decay of correlations. It is proved that for small values of the clustering operator has invariant subspaces which are similar tok-particle subspaces of the Fock space. The restriction of the clustering operator onto these subspaces resembles the operator exp(-H _k, whereH _k is thek- particle Schrödinger Hamiltonian in nonrelativistic quantum mechanics. The spectrum of eachH _k,k1, is contained in the interval (C ₁^k,C ₂^k). These intervals do not intersect with each other.     

基于可见光谱和支持向量机的黄瓜叶部病害识别方法研究

以黄瓜叶部病害作为研究对象,基于可见光谱反射率差异识别黄瓜叶部病害,研究基于SVM的黄瓜叶部病害识别预测模型。采用小波变换进行数据预处理;选取Otsu、边缘分割法和K均值聚类三类分割方法进行病斑分割,比较错分率和运行时间,K均值聚类方法更适合黄瓜叶部病斑分割;提取纹理、颜色和形状特征参数,共15个特征参数;通过交叉验证选择最优参数c和g,对核函数参数进行优化处理,并通过比较线性核、多项式核、RBF核等不同核函数情况下SVM的正确识别率,确定RBF核SVM模式识别方法能够更精准地识别黄瓜叶部病害。并将基于SVM与另外两种常见的黄瓜叶部病害识别方法,BP神经网络和模糊聚类进行比较,结果表明,基于SVM的识别模型对霜霉病的正确识别率为95%,白粉病和褐斑病的正确识别率均为90%,平均诊断正确率为92%;该模式识别方法识别效果最佳,运行时间最短,为基于可见光谱的黄瓜病害识别模型提供参考。