Minimum f-divergence estimators and quasi-likelihood functions

Maximum quasi-likelihood estimators have several nice asymptotic properties. We show that, in many situations, a family of estimators, called the minimum f-divergence estimators, can be defined such that each estimator has the same asymptotic properties as the maximum quasi-likelihood estimator. The family of minimum f-divergence estimators include the maximum quasi-likelihood estimators as a special case. When a quasi-likelihood is the log likelihood from some exponential family, Amari's dual geometries can be used to study the maximum likelihood estimator. A dual geometric structure can also be defined for more general quasi-likelihood functions as well as for the larger family of minimum f-divergence estimators. The relationship between the f-divergence and the quasi-likelihood function and the relationship between the f-divergence and the power divergence is discussed.This work was supported by National Science Foundation grant DMS 88-03584.     

Boosting local search with Lagrangian relaxation

Local search algorithms play an essential role in solving large-scale combinatorial optimization problems. Traditionally, the local search procedure is guided mainly by the objective function of the problem. Hence, the greedy improvement paradigm poses the potential threat of prematurely getting trapped in low quality attraction basins. In this study, we intend to utilize the information extracted from the relaxed problem, to enhance the performance of the local search process. Considering the Lin-Kernighan-based local search (LK-search) for the p-median problem as a case study, we propose the Lagrangian relaxation Assisted Neighborhood Search (LANS). In the proposed algorithm, two new mechanisms, namely the neighborhood reduction and the redundancy detection, are developed. The two mechanisms exploit the information gathered from the relaxed problem, to avoid the search from prematurely targeting low quality directions, and to cut off the non-promising searching procedure, respectively. Extensive numerical results over the benchmark instances demonstrate that LANS performs favorably to LK-search, which is among the state-of-the-art local search algorithms for the p-median problem. Furthermore, by embedding LANS into other heuristics, the best known upper bounds over several benchmark instances could be updated. Besides, run-time distribution analysis is also employed to investigate the reason why LANS works. The findings of this study confirm that the idea of improving local search by leveraging the information induced from relaxed problem is feasible and practical, and might be generalized to a broad class of combinatorial optimization problems.     

Rate of strong consistency of the maximum quasi-likelihood estimator in quasi-likelihood nonlinear models

Quasi-likelihood nonlinear models （QLNM） include generalized linear models as a special case. Under some regularity conditions, the rate of the strong consistency of the maximum quasi-likelihood estimation （MQLE） is obtained in QLNM. In an important case, this rate is O（n-^1/2（loglogn）^1/2）, which is just the rate of LIL of partial sums for i.i.d variables, and thus cannot be improved anymore.     

The reliability of local error estimators for convection-diffusion equations

We assess the reliability of a simple a posteriori error estimatorfor steady-state convection–diffusion equations in caseswhere convection dominates. Our estimator is computed by solvinga local Poisson problem with Neumann boundary conditions. Itgives global upper and local lower bounds on the error measuredin the H¹ semi-norm. However, the error may be overestimatedlocally within boundary layers if these are not resolved bythe mesh, that is, when the local mesh Péclet numberis significantly greater than unity. We discuss the implicationsof this overestimation in a practical context where the estimatoris used as a local error indicator within a self-adaptive meshrefinement process. Received 18 June 1999. Accepted 7 March 2000.     

The quasi-likelihood estimation in regression

I propose a simply method to estimate the regression parameters in quasi-likelihood model My main approach utilizes the dimension reduction technique to first reduce the dimension of the regressor X to one dimension before solving the quasi-likelihood equations. In addition, the real advantage of using dimension reduction technique is that it provides a good initial estimate for one-step estimator of the regression parameters. Under certain design conditions, the estimators are asymptotically multivariate normal and consistent. Moreover, a Monte Carlo simulation is used to study the practical performance of the procedures, and I also assess the cost of CPU time for computing the estimates.This research partially supported by the National Science Council, R.O.C. (Plan No. NSC 82-0208-M-032-023-T).     

Consistency and asymptotic normality of the maximum quasi-likelihood estimator in quasi-likelihood nonlinear models with random regressors

This paper proposes some regularity conditions, which result in the existence, strong consistency and asymptotic normality of maximum quasi-likelihood estimator （MQLE） in quasi-likelihood nonlinear models （QLNM） with random regressors. The asymptotic results of generalized linear models （GLM） with random regressors are generalized to QLNM with random regressors.     

Fuzzy clusterwise quasi-likelihood generalized linear models

The quasi-likelihood method has emerged as a useful approach to the parameter estimation of generalized linear models (GLM) in circumstances where there is insufficient distributional information to construct a likelihood function. Despite its flexibility, the quasi-likelihood approach to GLM is currently designed for an aggregate-sample analysis based on the assumption that the entire sample of observations is taken from a single homogenous population. Thus, this approach may not be suitable when heterogeneous subgroups exist in the population, which involve qualitatively distinct effects of covariates on the response variable. In this paper, the quasi-likelihood GLM approach is generalized to a fuzzy clustering framework which explicitly accounts for such cluster-level heterogeneity. A simple iterative estimation algorithm is presented to optimize the regularized fuzzy clustering criterion of the proposed method. The performance of the proposed method in recovering parameters is investigated based on a Monte Carlo analysis involving synthetic data. Finally, the empirical usefulness of the proposed method is illustrated through an application to actual data on the coupon usage behaviour of a sample of consumers.     

Change point estimators by local polynomial fits under a dependence assumption

We study a random design regression model generated by dependent observations, when the regression function itself (or its ν-th derivative) may have a change or discontinuity point. A method based on the local polynomial fits with one-sided kernels to estimate the location and the jump size of the change point is applied in this paper. When the jump location is known, a central limit theorem for the estimator of the jump size is established; when the jump location is unknown, we first obtain a functional limit theorem for a local dilated-rescaled version estimator of the jump size and then give the asymptotic distributions for the estimators of the location and the jump size of the change point. The asymptotic results obtained in this paper can be viewed as extensions of corresponding results for independent observations. Furthermore, a simulated example is given to show that our theory and method perform well in practice.     

Adaptive quasi-likelihood estimate in generalized linear models

This paper gives a thorough theoretical treatment on the adaptive quasi-likelihood estimate of the parameters in the generalized linear models. The unknown covariance matrix of the response variable is estimated by the sample. It is shown that the adaptive estimator defined in this paper is asymptotically most efficient in the sense that it is asymptotic normal, and the covariance matrix of the limit distribution coincides with the one for the quasi-likelihood estimator for the case that the covariance matrix of the response variable is completely known.     

Quasi-likelihood or extended quasi-likelihood? An information-geometric approach

When the variance is a known function of the mean, as in quasi-likelihood applications, the sample variance also contains information about the mean and extensions of quasi-likelihood functions have been suggested that incorporate this additional information. In order to be sure these extensions are an improvement, further assumptions are made typically on the higher moments of the data so that there is a trade-off between the greater robustness of the quasi-likelihood estimates and the potentially improved estimates based on the extended quasi-likelihood functions. Improvement is often measured by relative efficiency but more insight can be gained by considering optimality of estimating functions, information loss, and sufficiency. All these measures can be described using the dual geometries of the quasi- and extended quasi-likelihood estimators. For a substantial range of models, the extended estimates offer little improvement when the coefficient of variation is small.     

Data driven time scale in Gaussian quasi-likelihood inference

We study parametric estimation of ergodic diffusions observed at high frequency. Different from the previous studies, we suppose that sampling stepsize is unknown, thereby making the conventional Gaussian quasi-likelihood not directly applicable. In this situation, we construct estimators of both model parameters and sampling stepsize in a fully explicit way, and prove that they are jointly asymptotically normally distributed. High order uniform integrability of the obtained estimator is also derived. Further, we propose the Schwarz (BIC) type statistics for model selection and show its model-selection consistency. We conducted some numerical experiments and found that the observed finite-sample performance well supports our theoretical findings.

     

Strong consistency of maximum quasi-likelihood estimates in generalized linear models

In a generalized linear model with q×1 responses, bounded and fixed p×q regressors zi and general link function, under the most general assumption on the minimum eigenvalue of ∑in=1 ZiZi', the moment condition on responses as weak as possible and other mild regular conditions, we prove that with probability one, the quasi-likelihood equation has a solution βn for all large sample size n, which converges to the true regression parameter β0. This result is an essential improvement over the relevant results in literature.     

Error estimators for advection–reaction–diffusion equations based on the solution of local problems

This paper deals with a posteriori error estimates for advection–reaction–diffusion equations. In particular, error estimators based on the solution of local problems are derived for a stabilized finite element method. These estimators are proved to be equivalent to the error, with equivalence constants eventually depending on the physical parameters. Numerical experiments illustrating the performance of this approach are reported.     

Conjugate Direction Boosting

Boosting in the context of linear regression has become more attractive with the invention of least angle regression (LARS), where the connection between the lasso and forward stagewise fitting (boosting) has been established. Earlier it has been found that boosting is a functional gradient optimization. Instead of the gradient, we propose a conjugate direction method (CDBoost). As a result, we obtain a fast forward stepwise variable selection algorithm. The conjugate direction of CDBoost is analogous to the constrained gradient in boosting. Using this analogy, we generalize CDBoost to: (1) include small step sizes (shrinkage) which often improves prediction accuracy; and (2) the nonparametric setting with fitting methods such as trees or splines, where least angle regression and the lasso seem to be unfeasible. The step size in CDBoost has a tendency to govern the degree between L₀- and L₁-penalization. This makes CDBoost surprisingly flexible. We compare the different methods on simulated and real datasets. CDBoost achieves the best predictions mainly in complicated settings with correlated covariates, where it is difficult to determine the contribution of a given covariate to the response. The gain of CDBoost over boosting is especially high in sparse cases with high signal to noise ratio and few effective covariates.     

On the second order local comparison between perturbed maximum likelihood estimators and Rao's statistic as test statistics

A bias-adjusted maximum likelihood estimator (mle), which has been shown to possess certain optimality criteria as an estimate of θ (see Ghosh, J. K., Sinha, B. K., and Joshi, S. N. (1982). In Statistical Decision Theory and Related Topics III (S. S. Gupta and J. O. Berger, Eds.), Vol. 1, pp. 403–456. Academic Press, Orlando, FL) is compared with Rao's statistic as a test statistic for the standard two-sided testing problem. It is shown that Rao's statistic is locally superior to any bias-adjusted mle in the sense of Chandra and Joshi (1983, Sankhy Ser. A 45 226–246). A second interpretation of a conjecture of C. R. Rao is proposed and Rao's statistic is shown to be superior as a test statistic according to the new interpretation as well. The last fact provides an interesting supplement to the results of Chandra and Joshi (1983, Sankhy Ser. A 45 226–246). Furthermore, a partial reason for the inferiority of the likelihood ratio and Wald's statistic to Rao's statistic is supplied and certain regularity assumptions of the last paper are eliminated. Finally, the local powers of certain modified versions of Rao's and Wald's statistics (see Skovgaard, I. M. (1985). Ann. Statist. 13 534–551) are studied.     

拟似然非线性模型的某些渐近推断:几何方法

Abstract. A modified Bates and Watts geometric framework is proposed for quasi-likelihoodnonlinear models in Euclidean inner product space. Based on the modified geometric framework,some asymptotic inference in terms of curvatures for quasi-likelihood nonlinear models is stud-ied. Several previous results for nonlinear regression models and exponential family nonlinearmodels etc. are extended to quasi-likelihood nonlinear models.     

Boosting the feasibility pump

The feasibility pump (FP) has proved to be an effective method for finding feasible solutions to mixed integer programming problems. FP iterates between a rounding procedure and a projection procedure, which together provide a sequence of points alternating between LP feasible but fractional solutions, and integer but LP infeasible solutions. The process attempts to minimize the distance between consecutive iterates, producing an integer feasible solution when closing the distance between them. We investigate the benefits of enhancing the rounding procedure with a clever integer line search that efficiently explores a large set of integer points. An extensive computational study on benchmark instances demonstrates the efficacy of the proposed approach.