A solution to anisotropic suboptimal filtering problem by convex optimization

This paper considers a robust filtering problem for a linear discrete time invariant system with measured and estimated outputs. The system is exposed to random disturbances with imprecisely known distributions generated by an unknown stable shaping filter from the Gaussian white noise. The stochastic uncertainty of the input disturbance is measured by the mean anisotropy functional. The estimation error is quantified by the anisotropic norm which is a stochastic analogue of the H _∞ norm. A sufficient condition for an estimator to exist and ensure that the error is less than a given threshold value is derived in form of a convex inequality on the determinant of a positive definite matrix and two linear matrix inequalities. The suboptimal problem setting results to a set of the estimators ensuring the anisotropic norm of the error to be strictly bounded thereby providing some additional degree of freedom to impose some additional constraints on the estimator performance specification.     

Empirical Bayes Estimation in Regression Model

This paper considers the empirical Bayes （EB） estimation problem for the parameter β of the linear regression model y = Xβ＋ ε with ε- N（0, σ^2I） given β. Based on Pitman closeness （PC） criterion and mean square error matrix （MSEM） criterion, we prove the superiority of the EB estimator over the ordinary least square estimator （OLSE）.     

Efficient eigenvalue computation for quasiseparable Hermitian matrices under low rank perturbations

In this paper we address the problem of efficiently computing all the eigenvalues of a large N×N Hermitian matrix modified by a possibly non Hermitian perturbation of low rank. Previously proposed fast adaptations of the QR algorithm are considerably simplified by performing a preliminary transformation of the matrix by similarity into an upper Hessenberg form. The transformed matrix can be specified by a small set of parameters which are easily updated during the QR process. The resulting structured QR iteration can be carried out in linear time using linear memory storage. Moreover, it is proved to be backward stable. Numerical experiments show that the novel algorithm outperforms available implementations of the Hessenberg QR algorithm already for small values of N.      

Optimum calibration points estimating distribution functions

The calibration method has been widely discussed in the recent literature on survey sampling, and calibration estimators are routinely computed by many survey organizations. The calibration technique was introduced in [12] to estimate linear parameters as mean or total. Recently, some authors have applied the calibration technique to estimate the finite distribution function and the quantiles. The computationally simpler method in [14] is built by means of constraints that require the use of a fixed value t₀. The precision of the resulting calibration estimator changes with the selected point t₀. In the present paper, we study the problem of determining the optimal value t₀ that gives the best estimation under simple random sampling without replacement. A limited simulation study shows that the improvement of this optimal calibrated estimator over possible alternatives can be substantial.     

Robust generalized <Emphasis Type="Italic">H</Emphasis><Subscript>∞</Subscript>-suboptimal control

We consider an optimal perturbation damping problem taking into account not only an external perturbation with bounded L ₂-norm and an initial perturbation caused by unknown initial conditions in the system but also unknown bounded parametric perturbations. We synthesize a robust generalized H _∞-suboptimal control minimizing the upper bound, expressed in terms of solutions of linear matrix inequalities, for the perturbation damping level under uncertainty in the closed system.     

Quadratic mixed integer programming and support vectors for deleting outliers in robust regression

We consider the problem of deleting bad influential observations (outliers) in linear regression models. The problem is formulated as a Quadratic Mixed Integer Programming (QMIP) problem, where penalty costs for discarding outliers are used into the objective function. The optimum solution defines a robust regression estimator called penalized trimmed squares (PTS). Due to the high computational complexity of the resulting QMIP problem, the proposed robust procedure is computationally suitable for small sample data. The computational performance and the effectiveness of the new procedure are improved significantly by using the idea of ε-Insensitive loss function from support vectors machine regression. Small errors are ignored, and the mathematical formula gains the sparseness property. The good performance of the ε-Insensitive PTS (IPTS) estimator allows identification of multiple outliers avoiding masking or swamping effects. The computational effectiveness and successful outlier detection of the proposed method is demonstrated via simulated experiments. This research has been partially funded by the Greek Ministry of Education under the program Pythagoras II.     

Sampling designs for regression coefficient estimation with correlated errors

The problem of estimating regression coefficients from observations at a finite number of properly designed sampling points is considered when the error process has correlated values and no quadratic mean derivative. Sacks and Ylvisaker (1966,Ann. Math. Statist.,39, 66–89) found an asymptotically optimal design for the best linear unbiased estimator (BLUE). Here, the goal is to find an asymptotically optimal design for a simpler estimator. This is achieved by properly adjusting the median sampling design and the simpler estimator introduced by Schoenfelder (1978, Institute of Statistics Mimeo Series No. 1201, University of North Carolina, Chapel Hill). Examples with stationary (Gauss-Markov) and nonstationary (Wiener) error processes and with linear and nonlinear regression functions are considered both analytically and numerically.Research supported by the Air Force Office of Scientific Research Contract No. 91-0030.     

Robust dimension reduction,fusion frames,and Grassmannian packings

We consider estimating a random vector from its measurements in a fusion frame, in presence of noise and subspace erasures. A fusion frame is a collection of subspaces, for which the sum of the projection operators onto the subspaces is bounded below and above by constant multiples of the identity operator. We first consider the linear minimum mean-squared error (LMMSE) estimation of the random vector of interest from its fusion frame measurements in the presence of additive white noise. Each fusion frame measurement is a vector whose elements are inner products of an orthogonal basis for a fusion frame subspace and the random vector of interest. We derive bounds on the mean-squared error (MSE) and show that the MSE will achieve its lower bound if the fusion frame is tight. We then analyze the robustness of the constructed LMMSE estimator to erasures of the fusion frame subspaces. We limit our erasure analysis to the class of tight fusion frames and assume that all erasures are equally important. Under these assumptions, we prove that tight fusion frames consisting of equi-dimensional subspaces have maximum robustness (in the MSE sense) with respect to erasures of one subspace among all tight fusion frames, and that the optimal subspace dimension depends on signal-to-noise ratio (SNR). We also prove that tight fusion frames consisting of equi-dimensional subspaces with equal pairwise chordal distances are most robust with respect to two and more subspace erasures, among the class of equi-dimensional tight fusion frames. We call such fusion frames equi-distance tight fusion frames. We prove that the squared chordal distance between the subspaces in such fusion frames meets the so-called simplex bound, and thereby establish connections between equi-distance tight fusion frames and optimal Grassmannian packings. Finally, we present several examples for the construction of equi-distance tight fusion frames.     

Minimax estimates of a linear parameter function in a regression model under restrictions on the parameters and variance-covariance matrix

Minimax nonhomogeneous linear estimators of scalar linear parameter functions are studied in the paper under restrictions on the parameters and variance-covariance matrix. The variance-covariance matrix of the linear model under consideration is assumed to be unknown but from a specific set R of nonnegativedefinite matrices. It is shown under this assumption that, without any restriction on the parameters, minimax estimators correspond to the least-squares estimators of the parameter functions for the “worst” variance-covariance matrix. Then the minimax mean-square error of the estimator is derived using the Bayes approach, and finally the exact formulas are derived for the calculation of minimax estimators under elliptical restrictions on the parameter space and for two special classes of possible variance-covariance matrices R. For example, it is shown that a special choice of a constant q ₀ and a matrixW ₀ defining one of the above classes R leads to the well known Kuks—Olman admissible estimator (see [16]) with a known variance-covariance matrixW ₀. Bibliography:32 titles. Translated fromObchyslyuval'na ta Prykladna Matematyka, No. 81, 1997, pp. 79–92.     

Synthesis of controllers for damping oscillations in dynamical systems under uncertainty

We review the modern approaches to the synthesis of robust H _∞ controllers that ensure optimal damping of oscillations in dynamical systems under uncertainty. In the synthesis method based on Riccati equations, these many-parameter equations can be solved only when the parameters are contained in a bounded parallelepiped with given boundaries. The synthesis of a robust H _∞ output control for systems with unknown bounded parameters is reducible to the solution of an optimization problem constrained by a system of linear matrix inequalities. The proposed controller synthesis algorithms are implemented using standard MATLAB procedures. The efficiency of the proposed methods and algorithms is demonstrated in application to optimal damping of oscillations in a parametrically excited pendulum. __________ Translated from Nelineinaya Dinamika i Upravlenie, No. 4, pp. 87–104, 2004.     

Estimation of a linear functional and extrapolation of a random field

We consider minimax estimation of a linear functional of a homogeneous random field. A linear optimal estimator of the functional is derived and the field achieving the minimax error of the functional estimate is determined.Translated from Vychislitel'naya i Prikladnaya Matematika, No. 59, pp. 105–113, 1986.     

On Adaptive Combination of Regression Estimators

Consider the problem of choosing between two estimators of the regression function, where one estimator is based on stronger assumptions than the other and thus the rates of convergence are different. We propose a linear combination of the estimators where the weights are estimated by Mallows' C _L . The adaptive estimator retains the optimal rates of convergence and is an extension of Stein-type estimators considered by Li and Hwang (1984, Ann. Statist., 12, 887-897) and related to an estimator in Burman and Chaudhuri (1999, Ann. Inst. Statist. Math. (to appear)).     

Iterative Weighted Semiparametric Least Squares Estimation in Repeated Measurement Partially Linear Regression Models

Consider a repeated measurement partially linear regression model with an unknown vector parameter β, an unknown function g(.), and unknown heteroscedastic error variances. In order to improve the semiparametric generalized least squares estimator (SGLSE) of β, we propose an iterative weighted semiparametric least squares estimator (IWSLSE) and show that it improves upon the SGLSE in terms of asymptotic covariance matrix. An adaptive procedure is given to determine the number of iterations. We also show that when the number of replicates is less than or equal to two, the IWSLSE can not improve upon the SGLSE. These results are generalizations of those in [2] to the case of semiparametric regressions.     

Maximal vectors and multi-objective optimization

Maximal vector andweak-maximal vector are the two basic notions underlying the various broader definitions (like efficiency, admissibility, vector maximum, noninferiority, Pareto's optimum, etc.) for optimal solutions of multi-objective optimization problems. Moreover, the understanding and characterization of maximal and weak-maximal vectors on the space of index vectors (vectors of values of the multiple objective functions) is fundamental and useful to the understanding and characterization of Pareto-optimal and weak-optimal solutions on the space of solutions.This paper is concerned with various characterizations of maximal and weak-maximal vectors in a general subset of the EuclideanN-space, and with necessary conditions for Pareto-optimal and weak-optimal solutions to a generalN-objective optimization problem having inequality, equality, and open-set constraints on then-space. A geometric method is described; the validity of scalarization by linear combination is studied, and weak conditioning by directional convexity is considered; local properties and a fundamental necessary condition are given. A necessary and sufficient condition for maximal vectors in a simplex or a polyhedral cone is derived. Necessary conditions for Pareto-optimal and weak-optimal solutions are given in terms of Lagrange multipliers, linearly independent gradients, Jacobian and Gramian matrices, and Jacobian determinants.Several advantages in approaching the multi-objective optimization problem in two steps (investigate optimal index vectors on the space of index vectors first, and study optimal solutions on the specific space of solutions next) are demonstrated in this paper.This work was supported by the National Science Foundation under Grant No. GK-32701.     

Delete-group Jackknife Estimate in Partially Linear Regression Models with Heteroscedasticity

Abstract Consider a partially linear regression model with an unknown vector parameter β,an unknownfunction g(.),and unknown heteroscedastic error variances.Chen,You proposed a semiparametric generalizedleast squares estimator(SGLSE)for β,which takes the heteroscedasticity into account to increase efficiency.Forinference based on this SGLSE,it is necessary to construct a consistent estimator for its asymptotic covariancematrix.However,when there exists within-group correlation, the traditional delta method and the delete-1jackknife estimation fail to offer such a consistent estimator.In this paper, by deleting grouped partial residualsa delete-group jackknife method is examined.It is shown that the delete-group jackknife method indeed canprovide a consistent estimator for the asymptotic covariance matrix in the presence of within-group correlations.This result is an extension of that in[21].     

A maximum a posteriori estimator for trajectories of diffusion processes

Let x _t be a diffusion process observed via a noisy sensor, whose output is y_t We consider the problem of evaluating the maximum a posteriori trajectory {x_s0≤ s ≤ t Based on results of Stratonovich [1] and Ikeda-Watanabe [2], we show that this estimator is given by the solution of an appropriate variational problem which is a slight modification of the "minimum energy" estimator. We compare our results to the non-linear filtering theory and show that for problems which possess a finite dimensional solution, our approach yields also explicit filters. For linear diffusions observed via linear sensors, these filters are identical to the Kalman-filter     

Characterization of the inverse of a particular circulant matrix by means of a continued fraction

The matrix A of the constraining system of a particular discrete optimization problem, which has been posed in [3] for modeling a particular scheduling program and for which an optimal solution has been found in [1], is considered and an explicit form of its inverse is given by means of a continued fraction. The knowledge of the inverse enables one to obtain an explicit form of the set of optimal solutions of the problem.Lastly, the connection between A^-1 and the definition of equi-assignment of binary vectors [5] is analysed.     

Robust estimation in the linear model with asymmetric error distributions

In the linear model X^{n × 1} = C^{n × p}θ^{p × 1} + E^{n × 1}, Huber's theory of robust estimation of the regression vector θ^{p × 1} is adapted for two models for the partially specified common distribution F of the i.i.d. components of the error vector E^{n × 1}. In the first model considered, the restriction of F to a set [−a₀, b₀] is a standard normal distribution contaminated, with probability , by an unknown distribution symmetric about 0. In the second model, the restriction of F to [−a₀, b₀] is completely specified (and perhaps asymmetrical). In both models, the distribution of F outside the set [−a₀, b₀] is completely unspecified. For both models, consistent and asymptotically normal M-estimators of θ^{p × 1} are constructed, under mild regularity conditions on the sequence of design matrices {C^{n × p}}. Also, in both models, M-estimators are found which minimize the maximal mean-squared error. The optimal M-estimators have influence curves which vanish off compact sets.     

Empirical Bayes estimation in a multiple linear regression model

Summary Estimation of the vector β of the regression coefficients in a multiple linear regressionY=Xβ+ε is considered when β has a completely unknown and unspecified distribution and the error-vector ε has a multivariate standard normal distribution. The optimal estimator for β, which minimizes the overall mean squared error, cannot be constructed for use in practice. UsingX, Y and the information contained in the observation-vectors obtained fromn independent past experiences of the problem, (empirical Bayes) estimators for β are exhibited. These estimators are compared with the optimal estimator and are shown to be asymptotically optimal. Estimators asymptotically optimal with rates nearO(n ⁻¹) are constructed. Supported in part by a Natural Sciences and Engineering Research Council of Canada grant.