Spline-based sieve maximum likelihood estimation in the partly linear model under monotonicity constraints

We study a spline-based likelihood method for the partly linear model with monotonicity constraints. We use monotone B-splines to approximate the monotone nonparametric function and apply the generalized Rosen algorithm to compute the estimators jointly. We show that the spline estimator of the nonparametric component achieves the possible optimal rate of convergence under the smooth assumption and that the estimator of the regression parameter is asymptotically normal and efficient. Moreover, a spline-based semiparametric likelihood ratio test is established to make inference of the regression parameter. Also an observed profile information method to consistently estimate the standard error of the spline estimator of the regression parameter is proposed. A simulation study is conducted to evaluate the finite sample performance of the proposed method. The method is illustrated by an air pollution study.     

Difference based estimation for partially linear regression models with measurement errors

This paper is concerned with the estimating problem of the partially linear regression models where the linear covariates are measured with additive errors. A difference based estimation is proposed to estimate the parametric component. We show that the resulting estimator is asymptotically unbiased and achieves the semiparametric efficiency bound if the order of the difference tends to infinity. The asymptotic normality of the resulting estimator is established as well. Compared with the corrected profile least squares estimation, the proposed procedure avoids the bandwidth selection. In addition, the difference based estimation of the error variance is also considered. For the nonparametric component, the local polynomial technique is implemented. The finite sample properties of the developed methodology is investigated through simulation studies. An example of application is also illustrated.     

Marginal parameterizations of discrete models defined by a set of conditional independencies

It is well-known that a conditional independence statement for discrete variables is equivalent to constraining to zero a suitable set of log–linear interactions. In this paper we show that this is also equivalent to zero constraints on suitable sets of marginal log–linear interactions, that can be formulated within a class of smooth marginal log–linear models. This result allows much more flexibility than known until now in combining several conditional independencies into a smooth marginal model. This result is the basis for a procedure that can search for such a marginal parameterization, so that, if one exists, the model is smooth.     

A new test for sphericity of the covariance matrix for high dimensional data

In this paper we propose a new test procedure for sphericity of the covariance matrix when the dimensionality, p, exceeds that of the sample size, N=n+1. Under the assumptions that (A) as p→∞ for i=1,…,16 and (B) p/n→c<∞ known as the concentration, a new statistic is developed utilizing the ratio of the fourth and second arithmetic means of the eigenvalues of the sample covariance matrix. The newly defined test has many desirable general asymptotic properties, such as normality and consistency when (n,p)→∞. Our simulation results show that the new test is comparable to, and in some cases more powerful than, the tests for sphericity in the current literature.     

Concordance measures for multivariate non-continuous random vectors

A notion of multivariate concordance suitable for non-continuous random variables is defined and many of its properties are established. This allows the definition of multivariate, non-continuous versions of Kendall’s tau, Spearman’s rho and Spearman’s footrule, which are concordance measures. Since the maximum values of these association measures are not +1 in general, a special attention is given to the computation of upper bounds. The latter turn out to be multivariate generalizations of earlier findings made by Nešlehová (2007) [9] and Denuit and Lambert (2005) [2]. They are easy to compute and can be estimated from a data set of (possibly) discontinuous random vectors. Corrected versions are considered as well.     

Multivariate logistic regression with incomplete covariate and auxiliary information

In this article, we propose and explore a multivariate logistic regression model for analyzing multiple binary outcomes with incomplete covariate data where auxiliary information is available. The auxiliary data are extraneous to the regression model of interest but predictive of the covariate with missing data. Horton and Laird [N.J. Horton, N.M. Laird, Maximum likelihood analysis of logistic regression models with incomplete covariate data and auxiliary information, Biometrics 57 (2001) 34–42] describe how the auxiliary information can be incorporated into a regression model for a single binary outcome with missing covariates, and hence the efficiency of the regression estimators can be improved. We consider extending the method of [9] to the case of a multivariate logistic regression model for multiple correlated outcomes, and with missing covariates and completely observed auxiliary information. We demonstrate that in the case of moderate to strong associations among the multiple outcomes, one can achieve considerable gains in efficiency from estimators in a multivariate model as compared to the marginal estimators of the same parameters.     

The multiple hybrid bootstrap — Resampling multivariate linear processes

The paper reconsiders the autoregressive aided periodogram bootstrap (AAPB) which has been suggested in Kreiss and Paparoditis (2003) [18]. Their idea was to combine a time domain parametric and a frequency domain nonparametric bootstrap to mimic not only a part but as much as possible the complete covariance structure of the underlying time series. We extend the AAPB in two directions. Our procedure explicitly leads to bootstrap observations in the time domain and it is applicable to multivariate linear processes, but agrees exactly with the AAPB in the univariate case, when applied to functionals of the periodogram. The asymptotic theory developed shows validity of the multiple hybrid bootstrap procedure for the sample mean, kernel spectral density estimates and, with less generality, for autocovariances.     

Bayesian inference for dependent elliptical measurement error models

In this article we provide a Bayesian analysis for dependent elliptical measurement error models considering nondifferential and differential errors. In both cases we compute posterior distributions for structural parameters by using squared radial prior distributions for the precision parameters. The main result is that the posterior distribution of location parameters, for specific priors, is invariant with respect to changes in the generator function, in agreement with previous results obtained in the literature under different assumptions. Finally, although the results obtained are valid for any elliptical distribution for the error term, we illustrate those results by using the student-t distribution and a real data set.     

On the layered nearest neighbour estimate, the bagged nearest neighbour estimate and the random forest method in regression and classification

Let be identically distributed random vectors in R^d, independently drawn according to some probability density. An observation is said to be a layered nearest neighbour (LNN) of a point if the hyperrectangle defined by and contains no other data points. We first establish consistency results on , the number of LNN of . Then, given a sample of independent identically distributed random vectors from R^d×R, one may estimate the regression function by the LNN estimate , defined as an average over the Y_i’s corresponding to those which are LNN of . Under mild conditions on r, we establish the consistency of towards 0 as n→∞, for almost all and all p≥1, and discuss the links between r_n and the random forest estimates of Breiman (2001) [8]. We finally show the universal consistency of the bagged (bootstrap-aggregated) nearest neighbour method for regression and classification.     

Tail order and intermediate tail dependence of multivariate copulas

In order to study copula families that have tail patterns and tail asymmetry different from multivariate Gaussian and t copulas, we introduce the concepts of tail order and tail order functions. These provide an integrated way to study both tail dependence and intermediate tail dependence. Some fundamental properties of tail order and tail order functions are obtained. For the multivariate Archimedean copula, we relate the tail heaviness of a positive random variable to the tail behavior of the Archimedean copula constructed from the Laplace transform of the random variable, and extend the results of Charpentier and Segers [7] [A. Charpentier, J. Segers, Tails of multivariate Archimedean copulas, Journal of Multivariate Analysis 100 (7) (2009) 1521–1537] for upper tails of Archimedean copulas. In addition, a new one-parameter Archimedean copula family based on the Laplace transform of the inverse Gamma distribution is proposed; it possesses patterns of upper and lower tails not seen in commonly used copula families. Finally, tail orders are studied for copulas constructed from mixtures of max-infinitely divisible copulas.     

Robustness of Stein-type estimators under a non-scalar error covariance structure

The Stein-rule (SR) and positive-part Stein-rule (PSR) estimators are two popular shrinkage techniques used in linear regression, yet very little is known about the robustness of these estimators to the disturbances’ deviation from the white noise assumption. Recent studies have shown that the OLS estimator is quite robust, but whether this is so for the SR and PSR estimators is less clear as these estimators also depend on the F statistic which is highly susceptible to covariance misspecification. This study attempts to evaluate the effects of misspecifying the disturbances as white noise on the SR and PSR estimators by a sensitivity analysis. Sensitivity statistics of the SR and PSR estimators are derived and their properties are analyzed. We find that the sensitivity statistics of these estimators exhibit very similar properties and both estimators are extremely robust to MA(1) disturbances and reasonably robust to AR(1) disturbances except for the cases of severe autocorrelation. The results are useful in light of the rising interest of the SR and PSR techniques in the applied literature.     

Empirical Bayes predictive densities for high-dimensional normal models

This paper addresses the problem of estimating the density of a future outcome from a multivariate normal model. We propose a class of empirical Bayes predictive densities and evaluate their performances under the Kullback–Leibler (KL) divergence. We show that these empirical Bayes predictive densities dominate the Bayesian predictive density under the uniform prior and thus are minimax under some general conditions. We also establish the asymptotic optimality of these empirical Bayes predictive densities in infinite-dimensional parameter spaces through an oracle inequality.     

Variable selection in model-based discriminant analysis

A general methodology for selecting predictors for Gaussian generative classification models is presented. The problem is regarded as a model selection problem. Three different roles for each possible predictor are considered: a variable can be a relevant classification predictor or not, and the irrelevant classification variables can be linearly dependent on a part of the relevant predictors or independent variables. This variable selection model was inspired by a previous work on variable selection in model-based clustering. A BIC-like model selection criterion is proposed. It is optimized through two embedded forward stepwise variable selection algorithms for classification and linear regression. The model identifiability and the consistency of the variable selection criterion are proved. Numerical experiments on simulated and real data sets illustrate the interest of this variable selection methodology. In particular, it is shown that this well ground variable selection model can be of great interest to improve the classification performance of the quadratic discriminant analysis in a high dimension context.     

Classification of a screened data into one of two normal populations perturbed by a screening scheme

In normal classification analysis, there may be cases where the population distributions are perturbed by a screening scheme. This paper considers a new classification method for screened data that is obtained from the perturbed normal distributions. Properties of each population distribution is considered and the best region for classifying the screened data is obtained. These developments yield yet another optimal rule for the classification. The rule is studied from several aspects such as a linear approximation, error rates, and estimation of the rule using the EM algorithm. Relationships among these aspects as well as investigation of the rule’s performance are also considered. The screened classification ideas are illustrated in detail using numerical examples.     

Subsampling tests for variance changes in the presence of autoregressive parameter shifts

In this paper, we consider the problem of testing for variance changes in the linear autoregressive processes including AR(p) processes when there are autoregressive parameter shifts. In performing a test, we employ the conventional residual CUSUM of squares test (RCUSQ) statistic. The RCUSQ test is based on the subsampling method introduced by Jach and Kokoszka (2004) [16] to eliminate the influence caused by autoregressive parameter shifts. It is shown that under regularity conditions, the test statistic behaves asymptotically the function of a standard Brownian bridge. We establish the asymptotic validity of this method and assess its performance both theoretically and numerically.     

A characterization of multivariate normality through univariate projections

This paper introduces a new characterization of multivariate normality of a random vector based on univariate normality of linear combinations of its components.     

Depth notions for orthogonal regression

Global depth, tangent depth and simplicial depths for classical and orthogonal regression are compared in examples, and properties that are useful for calculations are derived. The robustness of the maximum simplicial depth estimates is shown in examples. Algorithms for the calculation of depths for orthogonal regression are proposed, and tests for multiple regression are transferred to orthogonal regression. These tests are distribution free in the case of bivariate observations. For a particular test problem, the powers of tests that are based on simplicial depth and tangent depth are compared by simulations.     

Consistent multiple testing for change points

The change point problem for independent normal means is considered as a multiple testing problem. Two stepwise methods are considered. Namely, the binary segmentation method of Vostrikova (1981) [7] and the maximum residual down method of Cohen et al. (2009) [5]. Both of these methods are shown to be consistent. Consistent here means that as sample sizes tend to infinity, the probability of making an error (false rejection or false acceptance) tends to zero.     

On the limiting spectral distribution of the covariance matrices of time-lagged processes

We consider two continuous-time Gaussian processes, one being partially correlated to a time-lagged version of the other. We first give the limiting spectral distribution for the covariance matrices of the increments of the processes when the span between two observations tends to zero. Then, we derive the limiting distribution of the eigenvalues of the sample covariance matrices. This result is obtained when the number of paths of the processes is asymptotically proportional to the number of observations for each single path. As an application, we use the second moment of this distribution together with auxiliary volatility and correlation estimates to construct an adaptive estimator of the time lag between the two processes. Finally, we provide an asymptotic theory for our estimation procedure.