LTC: A latent tree approach to classification

Latent tree models were proposed as a class of models for unsupervised learning, and have been applied to various problems such as clustering and density estimation. In this paper, we study the usefulness of latent tree models in another paradigm, namely supervised learning. We propose a novel generative classifier called latent tree classifier (LTC). An LTC represents each class-conditional distribution of attributes using a latent tree model, and uses Bayes rule to make prediction. Latent tree models can capture complex relationship among attributes. Therefore, LTC is able to approximate the true distribution behind data well and thus achieves good classification accuracy. We present an algorithm for learning LTC and empirically evaluate it on an extensive collection of UCI data. The results show that LTC compares favorably to the state-of-the-art in terms of classification accuracy. We also demonstrate that LTC can reveal underlying concepts and discover interesting subgroups within each class.     

Incremental Model-Based Clustering for Large Datasets With Small Clusters

Clustering is often useful for analyzing and summarizing information within large datasets. Model-based clustering methods have been found to be effective for determining the number of clusters, dealing with outliers, and selecting the best clustering method in datasets that are small to moderate in size. For large datasets, current model-based clustering methods tend to be limited by memory and time requirements and the increasing difficulty of maximum likelihood estimation. They may fit too many clusters in some portions of the data and/or miss clusters containing relatively few observations. We propose an incremental approach for data that can be processed as a whole in memory, which is relatively efficient computationally and has the ability to find small clusters in large datasets. The method starts by drawing a random sample of the data, selecting and fitting a clustering model to the sample, and extending the model to the full dataset by additional EM iterations. New clusters are then added incrementally, initialized with the observations that are poorly fit by the current model. We demonstrate the effectiveness of this method by applying it to simulated data, and to image data where its performance can be assessed visually.     

Impacts of random noise and specification on estimates of capacity derived from data envelopment analysis

Data envelopment analysis (DEA) is widely used to estimate the efficiency of firms and has also been proposed as a tool to measure technical capacity and capacity utilization (CU). Random variation in output data can lead to downward bias in DEA estimates of efficiency and, consequently, upward bias in estimates of technical capacity. This can be particularly problematic for industries such as agriculture, aquaculture and fisheries where the production process is inherently stochastic due to environmental influences. This research uses Monte Carlo simulations to investigate possible biases in DEA estimates of technically efficient output and capacity output attributable to noisy data and investigates the impact of using a model specification that allows for variable returns to scale (VRS). We demonstrate a simple method of reducing noise induced bias when panel data is available. We find that DEA capacity estimates are highly sensitive to noise and model specification. Analogous conclusions can be drawn regarding DEA estimates of average efficiency.     

һ����������Ķ�̬�����

??Dynamic complex network has become a popular topic in the many fields, such as population ecology, social ecology, biology and Internet. Meanwhile cluster analysis is a common tool to extract network structure. Previous articles on network clustering mostly supposed that observations are conditionally independent. However, we construct novel model which combines the stochastic block model, the hidden structure in Markov process and the autoregressive model to relax this assumption. We also propose relative statistical inference and VEM algorithm. Finally, the Monte Carlo simulations are performed well, which shows the consistency and robustness of the work.     

Dynamic Cluster Analysis of Dependent Networks

Dynamic complex network has become a popular topic in the many fields, such as population ecology, social ecology, biology and Internet. Meanwhile cluster analysis is a common tool to extract network structure. Previous articles on network clustering mostly supposed that observations are conditionally independent. However, we construct novel model which combines the stochastic block model, the hidden structure in Markov process and the autoregressive model to relax this assumption. We also propose relative statistical inference and VEM algorithm. Finally, the Monte Carlo simulations are performed well, which shows the consistency and robustness of the work.     

Identification of input random field samples causing extreme responses

Consider a physical system modeled by a differential equation that depends on a coefficient random field. The objective of this work is to identify samples of this random field which yield extreme response as a means to: study the law of the input conditioned on rare events and predict if a random field sample causes such an event. This differs from reliability engineering which focuses on computation of failure probabilities. We investigate two classification schemes that identify these samples of interest: physics-based indicators which are functionals of the input random field and surrogate models which approximate the response. As an alternative to these approaches, we propose a general framework consisting of two stages that combines the use of a physics-based surrogate model and a machine learning classifier. In the first stage, a multifidelity surrogate that requires infrequent evaluations of the full model is designed. This surrogate is then used to generate a sufficient number of samples of random fields that yield extreme events to train a machine learning classifier in the second stage. We study the analytical properties required of the surrogate model and demonstrate through numerical examples the synergy of the proposed approach.     

TRANSFORM-ANN for online optimization of complex industrial processes: Casting process as case study

Artificial Neural Networks (ANNs) are well known for their credible ability to capture non-linear trends in scientific data. However, the heuristic nature of estimation of parameters associated with ANNs has prevented their evolution into efficient surrogate models. Further, the dearth of optimal training size estimation algorithms for the data greedy ANNs resulted in their overfitting. Therefore, through this work, we aim to contribute a novel ANN building algorithm called TRANSFORM aimed at simultaneous and optimal estimation of ANN architecture, training size and transfer function. TRANSFORM is integrated with three standalone Sobol sampling based training size determination algorithms which incorporate the concepts of hypercube sampling and optimal space filling. TRANSFORM was used to construct ANN surrogates for a highly non-linear industrially validated continuous casting model from steel plant. Multiobjective optimization of casting model to ensure maximum productivity, maximum energy saving and minimum operational cost was performed by ANN assisted Non-dominated Sorting Genetic Algorithms (NSGA-II). The surrogate assisted optimization was found to be 13 times faster than conventional optimization, leading to its online implementation. Simple operator's rules were deciphered from the optimal solutions using Pareto front characterization and K-means clustering for optimal functioning of casting plant. Comprehensive studies on (a) computational time comparisons between proposed training size estimation algorithms and (b) predictability comparisons between constructed ANNs and state of art statistical models, Kriging Interpolators adds to the other highlights of this work. TRANSFORM takes physics based model as the only input and provides parsimonious ANNs as outputs, making it generic across all scientific domains.     

Unsupervised quadratic surface support vector machine with application to credit risk assessment

Unsupervised classification is a highly important task of machine learning methods. Although achieving great success in supervised classification, support vector machine (SVM) is much less utilized to classify unlabeled data points, which also induces many drawbacks including sensitive to nonlinear kernels and random initializations, high computational cost, unsuitable for imbalanced datasets. In this paper, to utilize the advantages of SVM and overcome the drawbacks of SVM-based clustering methods, we propose a completely new two-stage unsupervised classification method with no initialization: a new unsupervised kernel-free quadratic surface SVM (QSSVM) model is proposed to avoid selecting kernels and related kernel parameters, then a golden-section algorithm is designed to generate the appropriate classifier for balanced and imbalanced data. By studying certain properties of proposed model, a convergent decomposition algorithm is developed to implement this non-covex QSSVM model effectively and efficiently (in terms of computational cost). Numerical tests on artificial and public benchmark data indicate that the proposed unsupervised QSSVM method outperforms well-known clustering methods (including SVM-based and other state-of-the-art methods), particularly in terms of classification accuracy. Moreover, we extend and apply the proposed method to credit risk assessment by incorporating the T-test based feature weights. The promising numerical results on benchmark personal credit data and real-world corporate credit data strongly demonstrate the effectiveness, efficiency and interpretability of proposed method, as well as indicate its significant potential in certain real-world applications.     

Bias analysis for misclassification in a multicategorical exposure in a logistic regression model

We derive the expressions of asymptotic biases when ignoring the misclassification in a multicategory exposure. For a model with a misclassified exposure variable only, we provide a general conclusion on the direction of the biases under nondifferential misclassification assumption. To better understand the bias formulas, we use a numerical example.     

Model-based clustering of time series in group-specific functional subspaces

This work develops a general procedure for clustering functional data which adapts the clustering method high dimensional data clustering (HDDC), originally proposed in the multivariate context. The resulting clustering method, called funHDDC, is based on a functional latent mixture model which fits the functional data in group-specific functional subspaces. By constraining model parameters within and between groups, a family of parsimonious models is exhibited which allow to fit onto various situations. An estimation procedure based on the EM algorithm is proposed for determining both the model parameters and the group-specific functional subspaces. Experiments on real-world datasets show that the proposed approach performs better or similarly than classical two-step clustering methods while providing useful interpretations of the groups and avoiding the uneasy choice of the discretization technique. In particular, funHDDC appears to always outperform HDDC applied on spline coefficients.     

Optimal estimation of sensor biases for asynchronous multi-sensor data fusion

An important step in a multi-sensor surveillance system is to estimate sensor biases from their noisy asynchronous measurements. This estimation problem is computationally challenging due to the highly nonlinear transformation between the global and local coordinate systems as well as the measurement asynchrony from different sensors. In this paper, we propose a novel nonlinear least squares formulation for the problem by assuming the existence of a reference target moving with an (unknown) constant velocity. We also propose an efficient block coordinate decent (BCD) optimization algorithm, with a judicious initialization, to solve the problem. The proposed BCD algorithm alternately updates the range and azimuth bias estimates by solving linear least squares problems and semidefinite programs. In the absence of measurement noise, the proposed algorithm is guaranteed to find the global solution of the problem and the true biases. Simulation results show that the proposed algorithm significantly outperforms the existing approaches in terms of the root mean square error.     

Facial expression classification: An approach based on the fusion of facial deformations using the transferable belief model

A method for the classification of facial expressions from the analysis of facial deformations is presented. This classification process is based on the transferable belief model (TBM) framework. Facial expressions are related to the six universal emotions, namely Joy, Surprise, Disgust, Sadness, Anger, Fear, as well as Neutral. The proposed classifier relies on data coming from a contour segmentation technique, which extracts an expression skeleton of facial features (mouth, eyes and eyebrows) and derives simple distance coefficients from every face image of a video sequence. The characteristic distances are fed to a rule-based decision system that relies on the TBM and data fusion in order to assign a facial expression to every face image. In the proposed work, we first demonstrate the feasibility of facial expression classification with simple data (only five facial distances are considered). We also demonstrate the efficiency of TBM for the purpose of emotion classification. The TBM based classifier was compared with a Bayesian classifier working on the same data. Both classifiers were tested on three different databases.     

Some theoretical properties of Silverman's method for Smoothed functional principal component analysis

Principal component analysis (PCA) is one of the key techniques in functional data analysis. One important feature of functional PCA is that there is a need for smoothing or regularizing of the estimated principal component curves. Silverman’s method for smoothed functional principal component analysis is an important approach in a situation where the sample curves are fully observed due to its theoretical and practical advantages. However, lack of knowledge about the theoretical properties of this method makes it difficult to generalize it to the situation where the sample curves are only observed at discrete time points. In this paper, we first establish the existence of the solutions of the successive optimization problems in this method. We then provide upper bounds for the bias parts of the estimation errors for both eigenvalues and eigenfunctions. We also prove functional central limit theorems for the variation parts of the estimation errors. As a corollary, we give the convergence rates of the estimations for eigenvalues and eigenfunctions, where these rates depend on both the sample size and the smoothing parameters. Under some conditions on the convergence rates of the smoothing parameters, we can prove the asymptotic normalities of the estimations.     

A study of biases of DNA copy number estimation based on PICR model

Affymetrix single-nucleotide polymorphism (SNP) arrays have been widely used for SNP genotype calling and copy number variation (CNV) studies, both of which are dependent on accurate DNA copy number estimation significantly. However, the methods for copy number estimation may suffer from kinds of difficulties: probe dependent binding affinity, crosshybridization of probes, and the whole genome amplification (WGA) of DNA sequences. The probe intensity composite representation (PICR) model, one former established approach, can cope with most complexities and achieve high accuracy in SNP genotyping. Nevertheless, the copy numbers estimated by PICR model still show array and site dependent biases for CNV studies. In this paper, we propose a procedure to adjust the biases and then make CNV inference based on both PICR model and our method. The comparison indicates that our correction of copy numbers is necessary for CNV studies.     

Thermal image colorization using Markov decision processes

Displaying night-vision thermal images with day-time colors is paramount for scene interpretation and target tracking. In this paper, we employ object recognition methods for colorization, which amounts to segmenting thermal images into plants, buildings, sky, water, roads and others, then calculating colors to each class. The main thrust of our work is the introduction of Markov decision processes (MDP) to deal with the computational complexity of the colorization problem. MDP provides us with the approaches of neighborhood analysis and probabilistic classification which we exploit to efficiently solve chromatic estimation. We initially label the segments with a classifier, paving the way for the neighborhood analysis. We then update classification confidences of each class by MDP under the consideration of neighboring consistency and scenery layout. Finally we calculate the colors for every segment by blending the characteristic colors of each class it belongs to in a probabilistic way. Experimental results show that the colorized appearance of our algorithm is satisfactory and harmonious; the computational speed is quite fast as well.     

Biased versus unbiased estimation

Statisticians have begun to realize that certain deliberately induced biases can dramatically improve estimation properties when there are several parameters to be estimated. This represents a radical departure from the tradition of unbiased estimation which has dominated statistical thinking since the work of Gauss. We briefly describe the new methods and give three examples of their practical application.     

Analysing and exploiting the Mantin biases in RC4

We explore the use of the Mantin biases (Mantin, Eurocrypt 2005) to recover plaintexts from RC4-encrypted traffic. We provide a more fine-grained analysis of these biases than in Mantin’s original work. We show that, in fact, the original analysis was incorrect in certain cases: the Mantin biases are sometimes non-existent, and sometimes stronger than originally predicted. We then show how to use these biases in a plaintext recovery attack. Our attack targets two unknown bytes of plaintext that are located close to sequences of known plaintext bytes, a situation that arises in practice when RC4 is used in, for example, TLS. We provide a statistical framework that enables us to make predictions about the performance of this attack and its variants. We then extend the attack using standard dynamic programming techniques to tackle the problem of recovering longer plaintexts, a setting of practical interest in recovering HTTP session cookies and user passwords that are protected by RC4 in TLS. We perform experiments showing that we can successfully recover 16-byte plaintexts with 80% success rate using \(2^{31}\) ciphertexts, an improvement over previous attacks.     

A skin detection approach based on the Dempster–Shafer theory of evidence

Skin detection is an important step for a wide range of research related to computer vision and image processing and several methods have already been proposed to solve this problem. However, most of these methods suffer from accuracy and reliability problems when they are applied to a variety of images obtained under different conditions. Performance degrades further when fewer training data are available. Besides these issues, some methods require long training times and a significant amount of parameter tuning. Furthermore, most state-of-the-art methods incorporate one or more thresholds, and it is difficult to determine accurate threshold settings to obtain desirable performance. These problems arise mostly because the available training data for skin detection are imprecise and incomplete, which leads to uncertainty in classification. This requires a robust fusion framework to combine available information sources with some degree of certainty. This paper addresses these issues by proposing a fusion-based method termed Dempster–Shafer-based Skin Detection (DSSD). This method uses six prominent skin detection criteria as sources of information (SoI), quantifies their reliabilities (confidences), and then combines their confidences based on the Dempster–Shafer Theory (DST) of evidence. We use the DST as it offers a powerful and flexible framework for representing and handling uncertainties in available information and thus helps to overcome the limitations of the current state-of-the-art methods. We have verified this method on a large dataset containing a variety of images, and achieved a 90.17% correct detection rate (CDR). We also demonstrate how DSSD can be used when very little training data are available, achieving a CDR as high as 87.47% while the best result achieved by a Bayesian classifier is only 68.81% on the same dataset. Finally, a generalized DSSD (GDSSD) is proposed achieving 91.12% CDR.     

Evaluation of the evolution of relationships between topics over time

Topics that attract public attention can originate from current events or developments, might be influenced by situations in the past, and often continue to be of interest in the future. When respective information is made available textually, one possibility of detecting such topics of public importance consists in scrutinizing, e.g., appropriate press articles using—given the continual growth of information—text processing techniques enriched by computer routines which examine present-day textual material, check historical publications, find newly emerging topics, and are able to track topic trends over time. Information clustering based on content-(dis)similarity of the underlying textual material and graph-theoretical considerations to deal with the network of relationships between content-similar topics are described and combined in a new approach. Explanatory examples of topic detection and tracking in online news articles illustrate the usefulness of the approach in different situations.     

The Internet, evolutionary variational inequalities, and the time-dependent Braess paradox

In this paper, we develop an evolutionary variational inequality model of the Internet with multiple classes of traffic and demonstrate its utility through the formulation and solution of a time-dependent Braess paradox. The model can handle time-dependent changes in demand as a consequence of developing news stories, following, for example, natural disasters or catastrophes or major media events. The model can also capture the time-varying demand for Internet resources during a regular weekday with its more regular rhythm of work and breaks. In addition, the model includes time-varying capacities on the route flows due to, for example, government interventions or network-type failures.