Simultaneous estimation and variable selection in median regression using Lasso-type penalty

We consider the median regression with a LASSO-type penalty term for variable selection. With the fixed number of variables in regression model, a two-stage method is proposed for simultaneous estimation and variable selection where the degree of penalty is adaptively chosen. A Bayesian information criterion type approach is proposed and used to obtain a data-driven procedure which is proved to automatically select asymptotically optimal tuning parameters. It is shown that the resultant estimator achieves the so-called oracle property. The combination of the median regression and LASSO penalty is computationally easy to implement via the standard linear programming. A random perturbation scheme can be made use of to get simple estimator of the standard error. Simulation studies are conducted to assess the finite-sample performance of the proposed method. We illustrate the methodology with a real example.     

Variable selection in multivariate methods using global score estimation

A variable selection method using global score estimation is proposed, which is applicable as a selection criterion in any multivariate method without external variables such as principal component analysis, factor analysis and correspondence analysis. This method selects a subset of variables by which we approximate the original global scores as much as possible in the context of least squares, where the global scores, e.g. principal component scores, factor scores and individual scores, are computed based on the selected variables. Global scores are usually orthogonal. Therefore, the estimated global scores should be restricted to being mutually orthogonal. According to how to satisfy that restriction, we propose three computational steps to estimate the scores. Example data is analyzed to demonstrate the performance and usefulness of the proposed method, in which the proposed algorithm is evaluated and the results obtained using four cost-saving selection procedures are compared. This example shows that combining these steps and procedures yields more accurate results quickly.     

Minimization of transformed $$L_1$$ penalty: theory,difference of convex function algorithm,and robust application in compressed sensing

We study the minimization problem of a non-convex sparsity promoting penalty function, the transformed \(l_1\) (TL1), and its application in compressed sensing (CS). The TL1 penalty interpolates \(l_0\) and \(l_1\) norms through a nonnegative parameter \(a \in (0,+\infty )\), similar to \(l_p\) with \(p \in (0,1]\), and is known to satisfy unbiasedness, sparsity and Lipschitz continuity properties. We first consider the constrained minimization problem, and discuss the exact recovery of \(l_0\) norm minimal solution based on the null space property (NSP). We then prove the stable recovery of \(l_0\) norm minimal solution if the sensing matrix A satisfies a restricted isometry property (RIP). We formulated a normalized problem to overcome the lack of scaling property of the TL1 penalty function. For a general sensing matrix A, we show that the support set of a local minimizer corresponds to linearly independent columns of A. Next, we present difference of convex algorithms for TL1 (DCATL1) in computing TL1-regularized constrained and unconstrained problems in CS. The DCATL1 algorithm involves outer and inner loops of iterations, one time matrix inversion, repeated shrinkage operations and matrix-vector multiplications. The inner loop concerns an \(l_1\) minimization problem on which we employ the Alternating Direction Method of Multipliers. For the unconstrained problem, we prove convergence of DCATL1 to a stationary point satisfying the first order optimality condition. In numerical experiments, we identify the optimal value \(a=1\), and compare DCATL1 with other CS algorithms on two classes of sensing matrices: Gaussian random matrices and over-sampled discrete cosine transform matrices (DCT). Among existing algorithms, the iterated reweighted least squares method based on \(l_{1/2}\) norm is the best in sparse recovery for Gaussian matrices, and the DCA algorithm based on \(l_1\) minus \(l_2\) penalty is the best for over-sampled DCT matrices. We find that for both classes of sensing matrices, the performance of DCATL1 algorithm (initiated with \(l_1\) minimization) always ranks near the top (if not the top), and is the most robust choice insensitive to the conditioning of the sensing matrix A. DCATL1 is also competitive in comparison with DCA on other non-convex penalty functions commonly used in statistics with two hyperparameters.     

Continuity of $$L_{p}$$ Balls and an Application to Input-Output Systems

Mathematical Notes - In this paper, the continuity of the set-valued map $$p\rightarrow B_{\Omega,\mathcal{X},p}(r)$$ , $$p\in (1,+\infty)$$ , is proved where $$B_{\Omega,\mathcal{X},p}(r)$$ is the...     

Finite element approximation of the Dirichlet problem using the boundary penalty method

Numerische Mathematik - This paper considers a finite element approximation of the Dirichlet problem for a second order self-adjoint elliptic equation,Au=f, in a region Ω ⊂ ℝn (n=2...     

Total flux estimates for a finite element approximation of the Dirichlet problem using the boundary penalty method

Summary This paper considers a fully practical piecewise linear finite element approximation of the Dirichlet problem for a second order self-adjoint elliptic equation,Au=f, in a smooth region< ⁿ (n=2 or 3) by the boundary penalty method. Using an unfitted mesh; that is ^h , an approximation of with dist (, ^h )Ch ² is not in general a union of elements; and assuminguH ⁴ () we show that one can recover the total flux across a segment of the boundary of with an error ofO(h ²⁾. We use these results to study a fully practical piecewise linear finite element approximation of an elliptic equation by the boundary penalty method when the prescribed data on part of the boundary is the total flux.Supported by a SERC research studentship     

On the boundedness of threshold operators in $$L_1[0,1]$$ with respect to the Haar basis

We prove a near-unconditionality property for the normalized Haar basis of \(L_1[0,1]\).     

Application of the penalty method to nonstationary approximation of an optimization problem

We solve a general optimization problem, where only approximation sequences are known instead of exact values of the goal function and feasible set. Under these conditions we suggest to utilize a penalty function method. We show that its convergence is attained for rather arbitrary means of approximation via suitable coercivity type conditions.     

A continuous approximation to the generalized Schur decomposition

We consider the problem of approximating the generalized Schur decomposition of a matrix pencil A − λB by a family of differentiable orthogonal transformations. It is shown that when B is nonsingular this approach is feasible and can be fully expressed as an autonomous differential system. When B is singular, we show that the location of zero diagonal entries of B affects the feasibility of such an approach, and hence we conclude that at least the QZ algorithm cannot be extended continuously.     

Variable selection and coefficient estimation via composite quantile regression with randomly censored data

Composite quantile regression with randomly censored data is studied. Moreover, adaptive LASSO methods for composite quantile regression with randomly censored data are proposed. The consistency, asymptotic normality and oracle property of the proposed estimators are established. The proposals are illustrated via simulation studies and the Australian AIDS dataset.     

The uniqueness of the element of best mean approximation to a continuous function using splines with fixed nodes

Suppose that on the Interval [a, b] the nodes $$a = x_0< x_1< \ldots< x_m< x_{m + 1} = b$$ are given and the functions u₀(t)=ω₀(t) $$u_i (t) = \omega _0 (t)\smallint _0^t \omega _1 (\varepsilon _1 )d\varepsilon _1 \ldots \smallint _a^{\varepsilon _{\iota - 1} } \omega _1 (\varepsilon _1 )d\varepsilon _\iota ,\varepsilon _0 = t(i = 1,2, \ldots ,n)$$ where the functions ω_i(t)> 0 have continuous (n?i)-th derivatives (i=0, 1, ..., n). S_n,m will designate the subspace of functions that have continuous (n?1)-st derivatives on [a, b] and coincide on each of the intervals [x_j, x_j+1] (j=0, 1, ..., m) with some polynomial from the system {u_i(t)} _i=0 ⁿ .THEOREM. For every continuous function on [a, b] there exists in S_n,m a unique element of best mean approximation.     

Uniform harmonic approximation with continuous extension to the boundary

Let Ω be an open set in ℝ ⁿ andE be a relatively closed subset of Ω. Further, letC _e(E) be the collection of real-valued continuous functions onE which extend continuously to the closure ofE in ℝ ⁿ . We characterize those pairs (Ω,E) which have the following property: every function inC _e(E) which is harmonic onE ⁰ can be uniformly approximated onE by functions which are harmonic on Ω and whose restrictions toE belong toC _e(E).     

Survival schedules and the estimation of the basic reproduction number ($${{\varvec{R}}}_{0}$$) without the assumption of extreme cases

Closed form expressions for the estimation of \(\hbox {R}_{0}\) in age structured populations have been derived by making assumptions about the mortality of host populations. In general, these mortality assumptions tend to be unrealistic when compared with the survival schedules of most natural populations. Here, I review important results for the estimation of \(\hbox {R}_{0}\) when the force of infection is constant and age independent in age structured host populations. I also present the details of a simple method for \(\hbox {R}_{0}\) estimation that can use data on the age structure of a host population derived from cross-sectional epidemiological studies, provided a few but clearly stated assumptions are met. I illustrate the method using data from a cross-sectional study about cutaneous leishmaniasis exposure in dogs from an endemic rural village in Panamá and compare \(\hbox {R}_{0}\) estimates based on closed form expressions and using a smoothed survival schedule. Finally, the use of the smoothed survival schedule provided an R\(_{0}\) estimate bounded by those obtained using closed form expressions that make extreme assumptions about mortality.     

A penalty method for nonparametric estimation of the intensity function of a counting process

Nonparametric estimators are proposed for the logarithm of the intensity function of some univariate counting processes. An Aalen multiplicative intensity model is specified for our counting process and the estimators are derived by a penalized maximum likelihood method similar to the method introduced by Silverman for probability density estimation. Asymptotic properties of the estimators, such as uniform consistency and normality, are investigated and some illustrative examples from survival theory are analyzed.This work was conducted while the author was visiting the Department of Mathematics, University of California at Irvine.     

On a new continuous selection and its applications

In this paper, we shall prove a new continuous selection theorem, and as applications, we shall prove a generalization of the Debreu—Gale—Nikaido theorem and a new intersection theorem     

A penalty method for nonparametric estimation of the logarithmic derivative of a density function

Summary Given a random sample of sizen from a densityf ₀ on the real line satisfying certain regularity conditions, we propose a nonparametric estimator forψ ₀=−f ₀ ^′ /f⁰. The estimate is the minimizer of a quadratic functional of the formλJ(ψ)+∫[ψ ²−2ψ′]dF_n where λ>0 is a smoothing parameter,J(·) is a roughness penalty, andF _n is the empirical c.d.f. of the sample. A characterization of the estimate (useful for computational purposes) is given which is related to spline functions. A more complete study of the caseJ(ψ)=∫[d ²ψ/dx²]² is given, since it has the desirable property of giving the maximum likelihood normal estimate in the infinite smoothness limit (λ→∞). Asymptotics under somewhat restrictive assumptions (periodicity) indicate that the estimator is asymptotically consistent and achieves the optimal rate of convergence. This type of estimator looks promising because the minimization problem is simple in comparison with the analogous penalized likelihood estimators. This research was supported by the Office of Naval Research under Grant Number N00014-82-C-0062.     

Fractional-order regularization and wavelet approximation to the inverse estimation problem for random fields

The least-squares linear inverse estimation problem for random fields is studied in a fractional generalized framework. First, the second-order regularity properties of the random fields involved in this problem are analysed in terms of the fractional Sobolev norms. Second, the incorporation of prior information in the form of a fractional stochastic model, with covariance operator bicontinuous with respect to a certain fractional Sobolev norm, leads to a regularization of this problem. Third, a multiresolution approximation to the class of linear inverse problems considered is obtained from a wavelet-based orthogonal expansion of the input and output random models. The least-squares linear estimate of the input random field is then computed using these orthogonal wavelet decompositions. The results are applied to solving two important cases of linear inverse problems defined in terms of fractional integral operators.