在线Logistic回归

针对连续数据流分类问题,基于在线学习理论,提出一种在线logistic回归算法.研究带有正则项的在线logistic回归,提出了在线logistic-l2回归模型,并给出了理论界估计.最终实验结果表明,随着在线迭代次数的增加,提出的模型与算法能够达到离线预测的分类结果.本文工作为处理海量流数据分类问题提供了一种新的有效方法.     

Multinomial logistic regression algorithm

The lower bound principle (introduced in Böhning and Lindsay 1988, Ann. Inst. Statist. Math., 40, 641–663), Böhning (1989, Biometrika, 76, 375–383) consists of replacing the second derivative matrix by a global lower bound in the Loewner ordering. This bound is used in the Newton-Raphson iteration instead of the Hessian matrix leading to a monotonically converging sequence of iterates. Here, we apply this principle to the multinomial logistic regression model, where it becomes specifically attractive.Supplement to Monotonicity of quadratic-approximation algorithms by Böhning and Lindsay (1988). Ann. Inst. Statist. Math., 40, 641–663.This research was supported by the German Research Foundation.     

Improved ridge regression estimators for the logistic regression model

The estimation of the regression parameters for the ill-conditioned logistic regression model is considered in this paper. We proposed five ridge regression (RR) estimators, namely, unrestricted RR, restricted ridge regression, preliminary test RR, shrinkage ridge regression and positive rule RR estimators for estimating the parameters $(\beta )$ when it is suspected that the parameter $\beta $ may belong to a linear subspace defined by $H\beta =h$ . Asymptotic properties of the estimators are studied with respect to quadratic risks. The performances of the proposed estimators are compared based on the quadratic bias and risk functions under both null and alternative hypotheses, which specify certain restrictions on the regression parameters. The conditions of superiority of the proposed estimators for departure and ridge parameters are given. Some graphical representations and efficiency analysis have been presented which support the findings of the paper.     

Kernel logistic regression using truncated Newton method

Kernel logistic regression (KLR) is a powerful nonlinear classifier. The combination of KLR and the truncated-regularized iteratively re-weighted least-squares (TR-IRLS) algorithm, has led to a powerful classification method using small-to-medium size data sets. This method (algorithm), is called truncated-regularized kernel logistic regression (TR-KLR). Compared to support vector machines (SVM) and TR-IRLS on twelve benchmark publicly available data sets, the proposed TR-KLR algorithm is as accurate as, and much faster than, SVM and more accurate than TR-IRLS. The TR-KLR algorithm also has the advantage of providing direct prediction probabilities.     

An improved model averaging scheme for logistic regression

Recently, penalized regression methods have attracted much attention in the statistical literature. In this article, we argue that such methods can be improved for the purposes of prediction by utilizing model averaging ideas. We propose a new algorithm that combines penalized regression with model averaging for improved prediction. We also discuss the issue of model selection versus model averaging and propose a diagnostic based on the notion of generalized degrees of freedom. The proposed methods are studied using both simulated and real data.     

Modified restricted Liu estimator in logistic regression model

?iray et al. (Commun Stat Simul Comput, 2014) proposed a restricted Liu estimator in logistic regression model with linear restrictions. However, this estimator did not satisfy the linear restrictions. In this paper, we introduce a modified restricted Liu estimator in logistic regression model with linear restrictions. Our results show that the new estimator satisfies the linear restrictions. We also discuss the properties of the new estimator under the matrix mean squared error criterion. Finally, a Monte Carlo study and a numerical example are given to show the performances of the new estimator.     

Prospective and retrospective analyses under logistic regression models

In logistic case-control studies, Prentice and Pyke (Biometrika 66 (1979) 403-411) showed that valid point estimators of the odds-ratio parameters and their standard errors may be obtained by fitting the prospective logistic regression model to case-control data. Wang and Carroll (Biometrika 80 (1993) 237-241; J. Statist. Plann. Inference 43 (1995) 331-340) generalized Prentice and Pyke's (Biometrika 66 (1979) 403-411) results to robust logistic case-control studies. In this paper, we extend the results of Prentice and Pyke (Biometrika 66 (1979) 403-411) and Wang and Carroll (Biometrika 80 (1993) 237-241; J. Statist. Plann. Inference 43 (1995) 331-340) to a class of statistics and a class of unbiased estimating equations. We present some results on simulation and on the analysis of two real datasets.     

Robust logistic zero-sum regression for microbiome compositional data

Advances in Data Analysis and Classification - We introduce the Robust Logistic Zero-Sum Regression (RobLZS) estimator, which can be used for a two-class problem with high-dimensional compositional...     

A dynamic logistic regression for network link prediction

In social network analysis, link prediction is a problem of fundamental importance. How to conduct a comprehensive and principled link prediction, by taking various network structure information into consideration, is of great interest. To this end, we propose here a dynamic logistic regression method. Specifically, we assume that one has observed a time series of network structure. Then the proposed model dynamically predicts future links by studying the network structure in the past. To estimate the model, we find that the standard maximum likelihood estimation (MLE) is computationally forbidden. To solve the problem, we introduce a novel conditional maximum likelihood estimation (CMLE) method, which is computationally feasible for large-scale networks. We demonstrate the performance of the proposed method by extensive numerical studies.     

Multivariate logistic regression with incomplete covariate and auxiliary information

In this article, we propose and explore a multivariate logistic regression model for analyzing multiple binary outcomes with incomplete covariate data where auxiliary information is available. The auxiliary data are extraneous to the regression model of interest but predictive of the covariate with missing data. Horton and Laird [N.J. Horton, N.M. Laird, Maximum likelihood analysis of logistic regression models with incomplete covariate data and auxiliary information, Biometrics 57 (2001) 34–42] describe how the auxiliary information can be incorporated into a regression model for a single binary outcome with missing covariates, and hence the efficiency of the regression estimators can be improved. We consider extending the method of [9] to the case of a multivariate logistic regression model for multiple correlated outcomes, and with missing covariates and completely observed auxiliary information. We demonstrate that in the case of moderate to strong associations among the multiple outcomes, one can achieve considerable gains in efficiency from estimators in a multivariate model as compared to the marginal estimators of the same parameters.     

Bias-corrected AIC for selecting variables in multinomial logistic regression models

In this paper, we consider the bias correction of Akaike’s information criterion (AIC) for selecting variables in multinomial logistic regression models. For simplifying a formula of the bias-corrected AIC, we calculate the bias of the AIC to a risk function through the expectations of partial derivatives of the negative log-likelihood function. As a result, we can express the bias correction term of the bias-corrected AIC with only three matrices consisting of the second, third, and fourth derivatives of the negative log-likelihood function. By conducting numerical studies, we verify that the proposed bias-corrected AIC performs better than the crude AIC.     

Two advanced methods for adjusting the main coefficient in logistic regression

A binary disease outcome is commonly modeled with continuous covariates (e.g., biochemical concentration) in medical research, and the corresponding exploration may employ a normal discrimination approach. The covariate relationship affects the estimated association between binary outcome and the interesting covariate. The method of value deviated from a fitted value (fractional polynomial), which is abbreviated as VDFV, may reduce the estimation bias especially when the relationship between the covariates is nonlinear. However, when the extraneous variable relates to the outcome, the pooled data (cases and controls) are replaced by the control data only for the purpose of fitting values. Based on two association patterns, the extraneous variable unrelated to the outcome (I) and that related to the outcome (II), the simulation study reveals that VDFV-p (using pooled data) is reliable, with less bias and a smaller mean square error (MSE) in pattern (I) and that VDFV-c (using control data) shows less bias in pattern (II). The conventional covariate adjustment performs worse in (I) but fairly well in (II). Note that a huge MSE is never observed in VDFV-p or VDFV-c, while this is a common issue related to small sample size or sparse data in logistic regression. Two fetal studies are illustrated—one for pattern (I) and one for pattern (II).     

Bayesian networks with a logistic regression model for the conditional probabilities

Logistic regression techniques can be used to restrict the conditional probabilities of a Bayesian network for discrete variables. More specifically, each variable of the network can be modeled through a logistic regression model, in which the parents of the variable define the covariates. When all main effects and interactions between the parent variables are incorporated as covariates, the conditional probabilities are estimated without restrictions, as in a traditional Bayesian network. By incorporating interaction terms up to a specific order only, the number of parameters can be drastically reduced. Furthermore, ordered logistic regression can be used when the categories of a variable are ordered, resulting in even more parsimonious models. Parameters are estimated by a modified junction tree algorithm. The approach is illustrated with the Alarm network.     

Robust semiparametric inference for polytomous logistic regression with complex survey design

Advances in Data Analysis and Classification - Analyzing polytomous response from a complex survey scheme, like stratified or cluster sampling is very crucial in several socio-economics...     

On connectivity of fibers with positive marginals in multiple logistic regression

In this paper we consider exact tests of a multiple logistic regression with categorical covariates via Markov bases. In many applications of multiple logistic regression, the sample size is positive for each combination of levels of the covariates. In this case we do not need a whole Markov basis, which guarantees connectivity of all fibers. We first give an explicit Markov basis for multiple Poisson regression. By the Lawrence lifting of this basis, in the case of bivariate logistic regression, we show a simple subset of the Markov basis which connects all fibers with a positive sample size for each combination of levels of covariates.     

基于经验似然的Logistic回归模型的变点检验

基于经验似然对Logistic回归模型进行变点检验及估计.通过建立变点模型,构造经验对数似然比统计量,在大样本下,证明了经验对数似然比统计量与经典参数对数似然比统计量具有相同的极值分布,同时得到变点的估计及估计的相合性,并通过数值模拟及实例说明经验似然方法检验变点的可行性.     

Random effects logistic regression model for ranking efficiency in data envelopment analysis

Ranking efficiency based on data envelopment analysis (DEA) results can be used for grouping decision-making units (DMUs). The resulting group membership can be partly related to the environmental characteristics of DMU, which are not used either as input or output. Utilizing the expert knowledge on super efficiency DEA results, we propose a multinomial Dirichlet regression model, which can be used for the purpose of selection of new projects. A case study is presented in the context of ranking analysis of new information technology commercialization projects. It is expected that our proposed approach can complement the DEA ranking results with environmental factors and at the same time it facilitates the prediction of efficiency of new DMUs with only given environmental characteristics.     

A variable selection method based on Tabu search for logistic regression models

A Tabu search method is proposed and analysed for selecting variables that are subsequently used in Logistic Regression Models. The aim is to find from among a set of m variables a smaller subset which enables the efficient classification of cases. Reducing dimensionality has some very well-known advantages that are summarized in literature. The specific problem consists in finding, for a small integer value of p, a subset of size p of the original set of variables that yields the greatest percentage of hits in Logistic Regression. The proposed Tabu search method performs a deep search in the solution space that alternates between a basic phase (that uses simple moves) and a diversification phase (to explore regions not previously visited). Testing shows that it obtains significantly better results than the Stepwise, Backward or Forward methods used by classic statistical packages. Some results of applying these methods are presented.