A penalized criterion for variable selection in classification

In this paper, the problem of variable selection in classification is considered. On the basis of recent developments in model selection theory, we provide a criterion based on penalized empirical risk, where the penalization explicitly takes into account the number of variables of the considered models. Moreover, we give an oracle-type inequality that non-asymptotically guarantees the performance of the resulting classification rule. We discuss the optimality of the proposed criterion and present an application of the main result to backward and forward selection procedures.     

Convergence proof of matrix dynamics for online linear discriminant analysis

In this paper, we analyze matrix dynamics for online linear discriminant analysis (online LDA). Convergence of the dynamics have been studied for nonsingular cases; our main contribution is an analysis of singular cases, that is a key for efficient calculation without full-size square matrices. All fixed points of the dynamics are identified and their stability is examined.     

Linear discrimination with equicorrelated training vectors

Fisher's linear discrimination rule requires uncorrelated training vectors. In this paper a linear discrimination method is developed to be used when the training vectors are equicorrelated. Also, maximum likelihood ratio tests are proposed to decide whether the training samples are uncorrelated or equicorrelated.     

Learning rates of gradient descent algorithm for classification

In this paper, a stochastic gradient descent algorithm is proposed for the binary classification problems based on general convex loss functions. It has computational superiority over the existing algorithms when the sample size is large. Under some reasonable assumptions on the hypothesis space and the underlying distribution, the learning rate of the algorithm has been established, which is faster than that of closely related algorithms.     

Minimum distance classification rules for high dimensional data

In this article, the problem of classifying a new observation vector into one of the two known groups Π_i,i=1,2, distributed as multivariate normal with common covariance matrix is considered. The total number of observation vectors from the two groups is, however, less than the dimension of the observation vectors. A sample-squared distance between the two groups, using Moore-Penrose inverse, is introduced. A classification rule based on the minimum distance is proposed to classify an observation vector into two or several groups. An expression for the error of misclassification when there are only two groups is derived for large p and n=O(p^δ),0<δ<1.     

Clustering and classification based on the L1 data depth

Clustering and classification are important tasks for the analysis of microarray gene expression data. Classification of tissue samples can be a valuable diagnostic tool for diseases such as cancer. Clustering samples or experiments may lead to the discovery of subclasses of diseases. Clustering genes can help identify groups of genes that respond similarly to a set of experimental conditions. We also need validation tools for clustering and classification. Here, we focus on the identification of outliers—units that may have been misallocated, or mislabeled, or are not representative of the classes or clusters.We present two new methods: DDclust and DDclass, for clustering and classification. These non-parametric methods are based on the intuitively simple concept of data depth. We apply the methods to several gene expression and simulated data sets. We also discuss a convenient visualization and validation tool—the relative data depth plot.     

An iterative method for solving nonlinear equations

An iterative method for finding a solution of the equation f(x)=0$f(x)=0$ is presented. The method is based on some specially derived quadrature rules. It is shown that the method can give better results than the Newton method.     

Ontology concepts and tools for statistical genomics

In computer science, an ontology is any formally structured vocabulary covering a conceptual domain. Gene Ontology (GO) is a structured collection of terms defining biological processes, cellular components, or molecular functions for the purpose of characterizing gene products and functions. The structure of GO is a directed acyclic graph (DAG) with typed edges. We describe a simple formalism for working with ontologies for statistical purposes, and define object-ontology complexes, which encode the usage of the vocabulary to label objects under analysis. Recently developed concepts of information content and semantic similarity are evaluated and used to explore the association between LocusLink loci and GO. We investigate relations between GO DAG structure, association evidence codes and term information content, illustrate computation of semantic similarities of genes within and between clusters discovered in a microarray, and describe a more general ontology and its use in inference on genetic network structure.     

An extension of the factorization theorem to the non-positive case

This paper presents a method of determining joint distributions by known conditional distributions. A generalization of the Factorization Theorem is proposed. The generalized theorem is proved under the assumption that the support of unknown joint distribution may be divided into a countable number of sets, which all satisfy the relative weak positivity condition. This condition is defined in the paper and it generalizes the positivity condition introduced by Hammersley and Clifford. The theorem is illustrated with three examples. In the first example we determine a joint density in the case when the support of an unknown density is a continuous nonproduct set from Euclidean space . In the second example we seek the joint probability for the number of trials and the number of successes in Bernoulli's scheme. We also examine a simple example given by Kaiser and Cressie (J. Multivariate Anal. 73 (2000) 199).     

Asymptotic properties of computationally efficient alternative estimators for a class of multivariate normal models

Parameters of Gaussian multivariate models are often estimated using the maximum likelihood approach. In spite of its merits, this methodology is not practical when the sample size is very large, as, for example, in the case of massive georeferenced data sets. In this paper, we study the asymptotic properties of the estimators that minimize three alternatives to the likelihood function, designed to increase the computational efficiency. This is achieved by applying the information sandwich technique to expansions of the pseudo-likelihood functions as quadratic forms of independent normal random variables. Theoretical calculations are given for a first-order autoregressive time series and then extended to a two-dimensional autoregressive process on a lattice. We compare the efficiency of the three estimators to that of the maximum likelihood estimator as well as among themselves, using numerical calculations of the theoretical results and simulations.     

An adjusted maximum likelihood method for solving small area estimation problems

For the well-known Fay-Herriot small area model, standard variance component estimation methods frequently produce zero estimates of the strictly positive model variance. As a consequence, an empirical best linear unbiased predictor of a small area mean, commonly used in small area estimation, could reduce to a simple regression estimator, which typically has an overshrinking problem. We propose an adjusted maximum likelihood estimator of the model variance that maximizes an adjusted likelihood defined as a product of the model variance and a standard likelihood (e.g., a profile or residual likelihood) function. The adjustment factor was suggested earlier by Carl Morris in the context of approximating a hierarchical Bayes solution where the hyperparameters, including the model variance, are assumed to follow a prior distribution. Interestingly, the proposed adjustment does not affect the mean squared error property of the model variance estimator or the corresponding empirical best linear unbiased predictors of the small area means in a higher order asymptotic sense. However, as demonstrated in our simulation study, the proposed adjustment has a considerable advantage in small sample inference, especially in estimating the shrinkage parameters and in constructing the parametric bootstrap prediction intervals of the small area means, which require the use of a strictly positive consistent model variance estimate.     

On nonparametric classification with missing covariates

General procedures are proposed for nonparametric classification in the presence of missing covariates. Both kernel-based imputation as well as Horvitz-Thompson-type inverse weighting approaches are employed to handle the presence of missing covariates. In the case of imputation, it is a certain regression function which is being imputed (and not the missing values). Using the theory of empirical processes, the performance of the resulting classifiers is assessed by obtaining exponential bounds on the deviations of their conditional errors from that of the Bayes classifier. These bounds, in conjunction with the Borel-Cantelli lemma, immediately provide various strong consistency results.     

Stein's Lemma for elliptical random vectors

For the family of multivariate normal distribution functions, Stein's Lemma presents a useful tool for calculating covariances between functions of the component random variables. Motivated by applications to corporate finance, we prove a generalization of Stein's Lemma to the family of elliptical distributions.     

Multivariate stress-strength reliability model and its evaluation for coherent structures

Consider a system which has n independent components (or subsystems) each consisting of m dependent elements. Let , i=1,2,…,n denote the random strength vector of the ith component, where denotes the random strength of the jth element of the ith component. The elements of the components are subjected to a common random stress over time. In this paper, we setup a multivariate stress-strength model based on the conditional ordering between s and and evaluate the reliability of coherent structures in this setup.     

Posterior contraction rates for the Bayesian approach to linear ill-posed inverse problems

We consider a Bayesian nonparametric approach to a family of linear inverse problems in a separable Hilbert space setting with Gaussian noise. We assume Gaussian priors, which are conjugate to the model, and present a method of identifying the posterior using its precision operator. Working with the unbounded precision operator enables us to use partial differential equations (PDE) methodology to obtain rates of contraction of the posterior distribution to a Dirac measure centered on the true solution. Our methods assume a relatively weak relation between the prior covariance, noise covariance and forward operator, allowing for a wide range of applications.     

Conditional independence models for seemingly unrelated regressions with incomplete data

We consider normal ≡ Gaussian seemingly unrelated regressions (SUR) with incomplete data (ID). Imposing a natural minimal set of conditional independence constraints, we find a restricted SUR/ID model whose likelihood function and parameter space factor into the product of the likelihood functions and the parameter spaces of standard complete data multivariate analysis of variance models. Hence, the restricted model has a unimodal likelihood and permits explicit likelihood inference. In the development of our methodology, we review and extend existing results for complete data SUR models and the multivariate ID problem.     

A decomposition for a stochastic matrix with an application to MANOVA

The aim of this paper is to propose a simple method in order to evaluate the (approximate) distribution of matrix quadratic forms when Wishartness conditions do not hold. The method is based upon a factorization of a general Gaussian stochastic matrix as a special linear combination of nonstochastic matrices with the standard Gaussian matrix. An application of previous result is proposed for matrix quadratic forms arising in MANOVA for a multivariate split-plot design with circular dependence structure.     

Statistical estimation of a mixture of Gaussian distributions

The paper is devoted to the problem of statistical estimation of a multivariate distribution density, which is a discrete mixture of Gaussian distributions. A heuristic approach is considered, based on the use of the EM algorithm and nonparametric density estimation with a sequential increase in the number of components of the mixture. Criteria for testing of model adequacy are discussed.