Nonlinear regression modeling via regularized wavelets and smoothing parameter selection

We introduce regularized wavelet-based methods for nonlinear regression modeling when design points are not equally spaced. A crucial issue in the model building process is a choice of tuning parameters that control the smoothness of a fitted curve. We derive model selection criteria from an information-theoretic and also Bayesian approaches. Monte Carlo simulations are conducted to examine the performance of the proposed wavelet-based modeling technique.     

The admissible parameter space for exponential smoothing models

We discuss the admissible parameter space for some state space models, including the models that underly exponential smoothing methods. We find that the usual parameter restrictions (requiring all smoothing parameters to lie between 0 and 1) do not always lead to stable models. We also find that all seasonal exponential smoothing methods are unstable as the underlying state space models are neither reachable nor observable. This instability does not affect the forecasts, but does corrupt the state estimates. The problem can be overcome with a simple normalizing procedure. Finally we show that the admissible parameter space of a seasonal exponential smoothing model is much larger than that for a basic structural model, leading to better forecasts from the exponential smoothing model when there is a rapidly changing seasonal pattern.     

Covariate selection for identifying the effects of a particular type of conditional plan using causal networks

Suppose that cause-effect relationships between variables can be described by a causal network with a linear structural equation model. Kuroki and Miyakawa proposed a graphical criterion for selecting covariates to identify the effect of a conditional plan with one control variable [J. Roy. Statist. Soc. Ser. B, 2003, 65: 209–222]. In this paper, we study a particular type of conditional plan with more than one control variable and propose a graphical criterion for selecting covariates to identify the effect of a conditional plan of the studied type.     

Genetic algorithms for the selection of smoothing parameters in additive models

Summary  Additive models of the type y=f ₁(x ₁)+...+f _p(x _p)+ε where f _j , j=1,..,p, have unspecified functional form, are flexible statistical regression models which can be used to characterize nonlinear regression effects. One way of fitting additive models is the expansion in B-splines combined with penalization which prevents overfitting. The performance of this penalized B-spline (called P-spline) approach strongly depends on the choice of the amount of smoothing used for components f _j . In particular for higher dimensional settings this is a computationaly demanding task. In this paper we treat the problem of choosing the smoothing parameters for P-splines by genetic algorithms. In several simulation studies this approach is compared to various alternative methods of fitting additive models. In particular functions with different spatial variability are considered and the effect of constant respectively local adaptive smoothing parameters is evaluated.     

On the accelerated iterative method for estimating the optimum overrelaxation parameter

This paper is concerned with the iterative method for estimating the optimum overrelaxation parameter. The improved power method (IP method) with the greatest rate of convergence is derived and compared with the Chebyshev polynomial iterative method (CP method) and the other iterative methods. Two algorithms (algorithms A and B) based on the IP method are presented. Some numerical results are shown.     

A three-step local smoothing approach for estimating the mean and covariance functions of spatio-temporal Data

Spatio-temporal data are common in practice. Existing methods for analyzing such data often employ parametric modelling with different sets of model assumptions. However, spatio-temporal data in practice often have complicated structures, including complex spatial and temporal data variation, latent spatio-temporal data correlation, and unknown data distribution. Because such data structures reflect the complicated impact of confounding variables, such as weather, demographic variables, life styles, and other cultural and environmental factors, they are usually too complicated to describe by parametric models. In this paper, we suggest a general modelling framework for estimating the mean and covariance functions of spatio-temporal data using a three-step local smoothing procedure. The suggested method can well accommodate the complicated structure of real spatio-temporal data. Under some regularity conditions, the consistency of the proposed estimators is established. Both simulation studies and a real-data application show that our proposed method could work well in practice.

     

Subset selection for parameter estimation in an HIV model

This paper discusses methodologies for subset selection for nonlinear least squares parameter estimation. In particular, we will present approaches for partitioning the parameter space into well-conditioned and ill-conditioned subsets. The algorithms are applied to a simplified mathematical model of the physiologic response of the human immunodeficiency virus (HIV) in humans. (© 2008 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim)     

Confidence intervals for the regression parameter based on weighted log-rank estimating functions

In this paper construction of a confidence interval for the regression parameter under the accelerated life regression model is discussed. The confidence interval is based on the distribution of the regression estimator, approximated by a resampling method. The procedures are incorporated with some weight functions which have mass at censored data points as well as non-censored data points. Numerical studies show that for some weight functions, the proposed confidence interval performs well. We illustrate the procedures in a real data example.     

Optimal parameter selection for nonlinear multistage systems with time-delays

In this paper, we consider a novel dynamic optimization problem for nonlinear multistage systems with time-delays. Such systems evolve over multiple stages, with the dynamics in each stage depending on both the current state of the system and the state at delayed times. The optimization problem involves choosing the values of the time-delays, as well as the values of additional parameters that influence the system dynamics, to minimize a given cost functional. We first show that the partial derivatives of the system state with respect to the time-delays and system parameters can be computed by solving a set of auxiliary dynamic systems in conjunction with the governing multistage system. On this basis, a gradient-based optimization algorithm is proposed to determine the optimal values of the delays and system parameters. Finally, two example problems, one of which involves parameter identification for a realistic fed-batch fermentation process, are solved to demonstrate the algorithm’s effectiveness.     

On the choice of the smoothing parameter in kernel density estimation

Algorithms for the automatic choice of the smoothing parameter in the kernel density estimation problems are proposed. These algorithms use the behavior of the total variation of the estimate.     

On the role of the shrinkage parameter in local linear smoothing

Summary. It has been shown that local linear smoothing possesses a variety of very attractive properties, not least being its mean square performance. However, such results typically refer only to asymptotic mean squared error, meaning the mean squared error of the asymptotic distribution, and in fact, the actual mean squared error is often infinite. See Seifert and Gasser (1996). This difficulty may be overcome by shrinking the local linear estimator towards another estimator with bounded mean square. However, that approach requires information about the size of the shrinkage parameter. From at least a theoretical viewpoint, very little is known about the effects of shrinkage. In particular, it is not clear how small the shrinkage parameter may be chosen without affecting first-order properties, or whether infinitely supported kernels such as the Gaussian require shrinkage in order to achieve first-order optimal performance. In the present paper we provide concise and definitive answers to such questions, in the context of general ridged and shrunken local linear estimators. We produce necessary and sufficient conditions on the size of the shrinkage parameter that ensure the traditional mean squared error formula. We show that a wide variety of infinitely-supported kernels, with tails even lighter than those of the Gaussian kernel, do not require any shrinkage at all in order to achieve traditional first-order optimal mean square performance. Received: 22 May 1995 / In revised form: 23 January 1997     

Local smoothing effects for the water-wave problem with surface tension

The water-wave problem with a one-dimensional free surface of infinite depth is considered, based on the formulation as a second-order nonlinear dispersive equation. The local smoothing effects are established under the influence of surface tension, stating that on average in time solutions acquire locally 1/4 derivative of smoothness as compared to the initial state. The analysis combines energy methods with techniques of Fourier integral operators. To cite this article: H. Christianson et al., C. R. Acad. Sci. Paris, Ser. I 347 (2009).     

A generalized linear model with smoothing effects for claims reserving

In this paper, we continue the development of the ideas introduced in England and Verrall (2001) by suggesting the use of a reparameterized version of the generalized linear model (GLM) which is frequently used in stochastic claims reserving. This model enables us to smooth the origin, development and calendar year parameters in a similar way as is often done in practice, but still keep the GLM structure. Specifically, we use this model structure in order to obtain reserve estimates and to systemize the model selection procedure that arises in the smoothing process. Moreover, we provide a bootstrap procedure to achieve a full predictive distribution.     

Relationship of causal effects in a causal chain and related inference

This paper discusses the relationship among the total causal effect and local causal effects in a causal chain and identifiability of causal effects. We show a transmission relationship of causal effects in a causal chain. According to the relationship, we give an approach to eliminating confounding bias through controlling for intermediate variables in a causal chain.     

Automated parameter selection for rational Arnoldi approximation of Markov functions

Rational Arnoldi is a powerful method for approximating functions of large sparse matrices times a vector. The selection of asymptotically optimal parameters for this method is crucial for its fast convergence. We present a heuristic for the automated pole selection when the function to be approximated is of Markov type, such as the matrix square root. The performance of this approach is demonstrated at several numerical examples. (© 2011 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim)     

Hazard regression with penalized spline: The smoothing parameter choice and asymptotics

In this article, we use penalized spline to estimate the hazard function from a set of censored failure time data. A new approach to estimate the amount of smoothing is provided. Under regularity conditions we establish the consistency and the asymptotic normality of the penalized likelihood estimators. Numerical studies and an example are conducted to evaluate the performances of the new procedure.     

Asymptotically optimal choice of the smoothing parameter in a nonparametric kernel estimator for a probability density

This paper presents a method of estimation of an “optimal” smoothing parameter (window width) in kernel estimators for a probability density. The obtained estimator is calculated directly from observations. By “optimal” smoothing parameters we mean those parameters which minimize the mean integral square error (MISE) or the integral square error (ISE) of approximation of an unknown density by the kernel estimator. It is shown that the asymptotic “optimality” properties of the proposed estimator correspond (with respect to the order) to those of the well-known cross-validation procedure [1, 2]. Translated fromStatisticheskie Metody Otsenivaniya i Proverki Gipotez, pp. 67–80, Perm, 1990.