Full likelihood inferences in the Cox model: an empirical likelihood approach

For the regression parameter β ₀ in the Cox model, there have been several estimators constructed based on various types of approximated likelihood, but none of them has demonstrated small-sample advantage over Cox’s partial likelihood estimator. In this article, we derive the full likelihood function for (β ₀, F ₀), where F ₀ is the baseline distribution in the Cox model. Using the empirical likelihood parameterization, we explicitly profile out nuisance parameter F ₀ to obtain the full-profile likelihood function for β ₀ and the maximum likelihood estimator (MLE) for (β ₀, F ₀). The relation between the MLE and Cox’s partial likelihood estimator for β ₀ is made clear by showing that Taylor’s expansion gives Cox’s partial likelihood estimating function as the leading term of the full-profile likelihood estimating function. We show that the log full-likelihood ratio has an asymptotic chi-squared distribution, while the simulation studies indicate that for small or moderate sample sizes, the MLE performs favorably over Cox’s partial likelihood estimator. In a real dataset example, our full likelihood ratio test and Cox’s partial likelihood ratio test lead to statistically different conclusions.     

A penalty method for nonparametric estimation of the logarithmic derivative of a density function

Summary Given a random sample of sizen from a densityf ₀ on the real line satisfying certain regularity conditions, we propose a nonparametric estimator forψ ₀=−f ₀ ^′ /f⁰. The estimate is the minimizer of a quadratic functional of the formλJ(ψ)+∫[ψ ²−2ψ′]dF_n where λ>0 is a smoothing parameter,J(·) is a roughness penalty, andF _n is the empirical c.d.f. of the sample. A characterization of the estimate (useful for computational purposes) is given which is related to spline functions. A more complete study of the caseJ(ψ)=∫[d ²ψ/dx²]² is given, since it has the desirable property of giving the maximum likelihood normal estimate in the infinite smoothness limit (λ→∞). Asymptotics under somewhat restrictive assumptions (periodicity) indicate that the estimator is asymptotically consistent and achieves the optimal rate of convergence. This type of estimator looks promising because the minimization problem is simple in comparison with the analogous penalized likelihood estimators. This research was supported by the Office of Naval Research under Grant Number N00014-82-C-0062.     

Local Whittle likelihood estimators and tests for non-Gaussian stationary processes

In this paper, we propose a local Whittle likelihood estimator for spectral densities of non-Gaussian processes and a local Whittle likelihood ratio test statistic for the problem of testing whether the spectral density of a non-Gaussian stationary process belongs to a parametric family or not. Introducing a local Whittle likelihood of a spectral density f _θ (λ) around λ, we propose a local estimator [^(q)] = [^(q)] (l){\hat{\theta } = \hat{\theta } (\lambda ) } of θ which maximizes the local Whittle likelihood around λ, and use f_{[^(q)] (l)} (l){f_{\hat{\theta } (\lambda )} (\lambda )} as an estimator of the true spectral density. For the testing problem, we use a local Whittle likelihood ratio test statistic based on the local Whittle likelihood estimator. The asymptotics of these statistics are elucidated. It is shown that their asymptotic distributions do not depend on non-Gaussianity of the processes. Because our models include nonlinear stationary time series models, we can apply the results to stationary GARCH processes. Advantage of the proposed estimator is demonstrated by a few simulated numerical examples.     

Functions of ultraexponential and infralogarithm types and general solution of the Abel functional equation

We propose generalized forms of ultraexponential and infralogarithm functions introduced and studied earlier by the author and present two classes of special functions, namely, ultraexponential and infralogarithm f -type functions. As a result of this investigation, we obtain a general solution of the Abel equation α(f(x)) = α (x) + 1 under some conditions on a real function f and prove a new completely different uniqueness theorem for the Abel equation stating that an infralogarithm f -type function is its unique solution. We also show that an infralogarithm f -type function is an essentially unique solution of the Abel equation. Similar theorems are proved for ultraexponential f -type functions and their functional equation β(x) = f(β(x − 1)), which can be considered as dual to the Abel equation. We also solve a certain problem unsolved before and study some properties of two considered functional equations and some relations between them.     

On the Einstein relation for a mechanical system

We consider a mechanical model in the plane, consisting of a vertical rod, subject to a constant horizontal force f and to elastic collisions with the particles of a free gas which is “horizontally” in equilibrium at some inverse temperature β. In a previous paper we proved that, in the appropriate space and time scaling, the motion of the rod is described as a drift term plus a diffusion term. In this paper we prove that the drift d(f) and the diffusivity σ ² (f) are continuous functions of f, and moreover that the Einstein relation holds, i.e., lim _{f → 0}  d(f)f = β2 σ ² (0) . Received: 26 January 1996 / In revised form: 2 October 1996     

Risk bounds for model selection via penalization

Performance bounds for criteria for model selection are developed using recent theory for sieves. The model selection criteria are based on an empirical loss or contrast function with an added penalty term motivated by empirical process theory and roughly proportional to the number of parameters needed to describe the model divided by the number of observations. Most of our examples involve density or regression estimation settings and we focus on the problem of estimating the unknown density or regression function. We show that the quadratic risk of the minimum penalized empirical contrast estimator is bounded by an index of the accuracy of the sieve. This accuracy index quantifies the trade-off among the candidate models between the approximation error and parameter dimension relative to sample size. If we choose a list of models which exhibit good approximation properties with respect to different classes of smoothness, the estimator can be simultaneously minimax rate optimal in each of those classes. This is what is usually called adaptation. The type of classes of smoothness in which one gets adaptation depends heavily on the list of models. If too many models are involved in order to get accurate approximation of many wide classes of functions simultaneously, it may happen that the estimator is only approximately adaptive (typically up to a slowly varying function of the sample size). We shall provide various illustrations of our method such as penalized maximum likelihood, projection or least squares estimation. The models will involve commonly used finite dimensional expansions such as piecewise polynomials with fixed or variable knots, trigonometric polynomials, wavelets, neural nets and related nonlinear expansions defined by superposition of ridge functions. Received: 7 July 1995 / Revised version: 1 November 1997     

Nonexistence and Existence of Multiple Positive Solutions for Superlinear Three-point Boundary Value Problems via Index Theory

This paper discusses both the nonexistence of positive solutions for second-order three-point boundary value problems when the nonlinear term f（t, x, y） is superlinear in y at y = 0 and the existence of multiple positive solutions for second-order three-point boundary value problems when the nonlinear term f（t, x,y） is superlinear in x at ＋∞.     

On estimation of the L r norm of a regression function

f be observed with noise. In the present paper we study the problem of nonparametric estimation of certain nonsmooth functionals of f, specifically, L _r norms ||f|| _r of f. Known from the literature results on functional estimation deal mostly with two extreme cases: estimating a smooth (differentiable in L ₂ ) functional or estimating a singular functional like the value of f at certain point or the maximum of f. In the first case, the convergence rate typically is n ^−1/2, n being the number of observations. In the second case, the rate of convergence coincides with the one of estimating the function f itself in the corresponding norm. We show that the case of estimating ||f|| _r is in some sense intermediate between the above extremes. The optimal rate of convergence is worse than n ^−1/2 but is better than the rate of convergence of nonparametric estimates of f. The results depend on the value of r. For r even integer, the rate occurs to be n ^{−β/(2β+1−1/r)} where β is the degree of smoothness. If r is not an even integer, then the nonparametric rate n ^{−β/(2β+1)} can be improved, but only by a logarithmic in n factor. Received: 6 February 1996hinspaceairsp/Revised version: 10 June 1998     

Adaptive estimation of the intensity of inhomogeneous Poisson processes via concentration inequalities

 In this paper, we establish oracle inequalities for penalized projection estimators of the intensity of an inhomogeneous Poisson process. We study consequently the adaptive properties of penalized projection estimators. At first we provide lower bounds for the minimax risk over various sets of smoothness for the intensity and then we prove that our estimators achieve these lower bounds up to some constants. The crucial tools to obtain the oracle inequalities are new concentration inequalities for suprema of integral functionals of Poisson processes which are analogous to Talagrand's inequalities for empirical processes. Received: 24 April 2001 / Revised version: 9 October 2002 / Published online: 15 April 2003 Mathematics Subject Classification (2000): 60E15, 62G05, 62G07 Key words or phrases: Inhomogeneous Poisson process – Concentration inequalities – Model selection – Penalized projection estimator – Adaptive estimation     

Adaptive estimation in circular functional linear models

We consider the problem of estimating the slope parameter in circular functional linear regression, where scalar responses Y ₁, ..., Y _n are modeled in dependence of 1-periodic, second order stationary random functions X ₁, ...,X _n . We consider an orthogonal series estimator of the slope function β, by replacing the first m theoretical coefficients of its development in the trigonometric basis by adequate estimators. We propose a model selection procedure for m in a set of admissible values, by defining a contrast function minimized by our estimator and a theoretical penalty function; this first step assumes the degree of ill-posedness to be known. Then we generalize the procedure to a random set of admissible m’s and a random penalty function. The resulting estimator is completely data driven and reaches automatically what is known to be the optimal minimax rate of convergence, in terms of a general weighted L ²-risk. This means that we provide adaptive estimators of both β and its derivatives.     

Boundary-Crossing Identities for Diffusions Having the Time-Inversion Property

We review and study a one-parameter family of functional transformations, denoted by (S ^(β)) _β∈ℝ, which, in the case β<0, provides a path realization of bridges associated to the family of diffusion processes enjoying the time-inversion property. This family includes Brownian motions, Bessel processes with a positive dimension and their conservative h-transforms. By means of these transformations, we derive an explicit and simple expression which relates the law of the boundary-crossing times for these diffusions over a given function f to those over the image of f by the mapping S ^(β), for some fixed β∈ℝ. We give some new examples of boundary-crossing problems for the Brownian motion and the family of Bessel processes. We also provide, in the Brownian case, an interpretation of the results obtained by the standard method of images and establish connections between the exact asymptotics for large time of the densities corresponding to various curves of each family.     

Genetic algorithms for the selection of smoothing parameters in additive models

Summary  Additive models of the type y=f ₁(x ₁)+...+f _p(x _p)+ε where f _j , j=1,..,p, have unspecified functional form, are flexible statistical regression models which can be used to characterize nonlinear regression effects. One way of fitting additive models is the expansion in B-splines combined with penalization which prevents overfitting. The performance of this penalized B-spline (called P-spline) approach strongly depends on the choice of the amount of smoothing used for components f _j . In particular for higher dimensional settings this is a computationaly demanding task. In this paper we treat the problem of choosing the smoothing parameters for P-splines by genetic algorithms. In several simulation studies this approach is compared to various alternative methods of fitting additive models. In particular functions with different spatial variability are considered and the effect of constant respectively local adaptive smoothing parameters is evaluated.     

Monotone perturbations of the laplacian inL 1(R N )

The semilinear perturbation of Poisson’s equation (E): −Δu+β(u)∋f, where β is a maximal monotone graph inR, has been investigated by Ph. Bénilan, H. Brézis and M. Crandall forf∈L ¹(R ^N ),N≧1, under the assumptions 0∈β(0) ifN≧3 and 0∈β(0) ∩ Int β(R) ifN=1,2. We discuss in this paper the solvability and well-posedness of (E) in terms of any maximal monotone graph β. In particular, if β takes only positive values andN≧3 we prove that no solution exists; ifN=2 we give necessary and sufficient conditions on β andf for (E) to be solvable in a natural sense.     

On the error terms for representation numbers of quadratic forms

Let f be a primitive positive integral binary quadratic form of discriminant −D, and r _f (n) the number of representations of n by f up to automorphisms of f. We first improve the error term E(x) of $ \sum\limits_{n \leqq x} {r_f (n)^\beta } $ \sum\limits_{n \leqq x} {r_f (n)^\beta } for any positive integer β. Next, we give an estimate of ∫₁ ^T |E(x)|² x ^−3/2 dx when β = 1.     

UNIFORM CONVERGENCE OF CESARO MEANS OF NEGATIVE ORDER OF DOUBLE TRIGONOMETRIC FOURIER SERIES

In this paper we prove that iff ∈ C([-π,π]2) and the function f is bounded partial p-variation for some p ∈ [1, ∞), then the double trigonometric Fourier series of a function f is uniformly (C;-α,-β) summable (α β＜ 1/p,α,β＞ 0) in the sense of Pringsheim. If α β≥ 1/p, then there exists a continuous function f0 of bounded partial double trigonometric Fourier series of fo diverge over cubes.     

Iterative Weighted Semiparametric Least Squares Estimation in Repeated Measurement Partially Linear Regression Models

Consider a repeated measurement partially linear regression model with an unknown vector parameter β, an unknown function g(.), and unknown heteroscedastic error variances. In order to improve the semiparametric generalized least squares estimator (SGLSE) of β, we propose an iterative weighted semiparametric least squares estimator (IWSLSE) and show that it improves upon the SGLSE in terms of asymptotic covariance matrix. An adaptive procedure is given to determine the number of iterations. We also show that when the number of replicates is less than or equal to two, the IWSLSE can not improve upon the SGLSE. These results are generalizations of those in [2] to the case of semiparametric regressions.     

Approximation in Learning Theory

This paper addresses some problems of supervised learning in the setting formulated by Cucker and Smale. Supervised learning, or learning-from-examples, refers to a process that builds on the base of available data of inputs x_i and outputs y_i, i = 1,...,m, a function that best represents the relation between the inputs x ∈ X and the corresponding outputs y ∈ Y. The goal is to find an estimator f_z on the base of given data z := ((x₁,y₁),...,(x_m,y_m)) that approximates well the regression function f_ρ (or its projection) of an unknown Borel probability measure ρ defined on Z = X × Y. We assume that (x_i,y_i), i = 1,...,m, are independent and distributed according to ρ. We discuss the following two problems: I. the projection learning problem (improper function learning problem); II. universal (adaptive) estimators in the proper function learning problem. In the first problem we do not impose any restrictions on a Borel measure ρ except our standard assumption that |y|≤ M a.e. with respect to ρ. In this case we use the data z to estimate (approximate) the L₂(ρ_X) projection (f_ρ)_W of f_ρ onto a function class W of our choice. Here, ρ_X is the marginal probability measure. In [KT1,2] this problem has been studied for W satisfying the decay condition ε_n(W,B) ≤ Dn^-r of the entropy numbers ε_n(W,B) of W in a Banach space B in the case B = C(X) or B = L₂(\rho_X). In this paper we obtain the upper estimates in the case ε_n(W,L₁(ρ_X)) ≤ Dn^-r with an extra assumption that W is convex. In the second problem we assume that an unknown measure ρ satisfies some conditions. Following the standard way from nonparametric statistics we formulate these conditions of the form f_ρ ∈ Θ. Next, we assume that the only a priori information available is that f_ρ belongs to a class Θ (unknown) from a known collection {Θ} of classes. We want to build an estimator that provides approximation of f_ρ close to the optimal for the class Θ. Along with standard penalized least squares estimators we consider a new method of construction of universal estimators. This method is based on a combination of two powerful ideas in building universal estimators. The first one is the use of penalized least squares estimators. This idea works well in the case of general setting with rather abstract methods of approximation. The second one is the idea of thresholding that works very well when we use wavelets expansions as an approximation tool. A new estimator that we call the big jump estimator uses the least squares estimators and chooses a right model by a thresholding criteria instead of the penalization. In this paper we illustrate how ideas and methods of approximation theory can be used in learning theory both in formulating a problem and in solving it.     

Pointwise convergence of cone-like restricted two-dimensional Fejér means of Walsh-Fourier series

For the two-dimensional Walsh system, Gat and Weisz proved the a.e. convergence of Fejer means σnf of integrable functions, where the set of indices is inside a positive cone around the identical function, that is, β^-1≤n1/n2 ≤β is provided with some fixed parameter ~ 〉 1. In this paper we generalize the result of Gat and Weisz. We not only generalize this theorem, but give a necessary and sufficient condition for cone-like sets in order to preserve this convergence property.     

Efficiency of profile likelihood in semi-parametric models

Profile likelihood is a popular method of estimation in the presence of an infinite-dimensional nuisance parameter, as the method reduces the infinite-dimensional estimation problem to a finite-dimensional one. In this paper we investigate the efficiency of a semi-parametric maximum likelihood estimator based on the profile likelihood. By introducing a new parametrization, we improve on the seminal work of Murphy and van der Vaart (J Am Stat Assoc, 95: 449–485, 2000): our improvement establishes the efficiency of the estimator through the direct quadratic expansion of the profile likelihood, which requires fewer assumptions. To illustrate the method an application to two-phase outcome-dependent sampling design is given.     

Weak and strong forms of β-irresolute functions

A subset A of a topological space X is said to be β-open [1] if A ⊂ Cl (Int (Cl (A))). A function f : X → Y is said to be β-irresolute [4] if for every β-open set V of Y, f ^-1(V) is β-open in X. In this paper we introduce weak and strong forms of β-irresolute functions and obtain several basic properties of such functions. This revised version was published online in August 2006 with corrections to the Cover Date.