Error analysis of Jacobi derivative estimators for noisy signals

Recent algebraic parametric estimation techniques (see Fliess and Sira-Ramírez, ESAIM Control Optim Calc Variat 9:151–168, 2003, 2008) led to point-wise derivative estimates by using only the iterated integral of a noisy observation signal (see Mboup et al. 2007, Numer Algorithms 50(4):439–467, 2009). In this paper, we extend such differentiation methods by providing a larger choice of parameters in these integrals: they can be reals. For this, the extension is done via a truncated Jacobi orthogonal series expansion. Then, the noise error contribution of these derivative estimations is investigated: after proving the existence of such integral with a stochastic process noise, their statistical properties (mean value, variance and covariance) are analyzed. In particular, the following important results are obtained:

Minimax kernels for density estimation with biased data

This paper considers the asymptotic properties of two kernel estimates % MathType!MTEF!2!1!+-% feaafeart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn% hiov2DGi1BTfMBaeXafv3ySLgzGmvETj2BSbqefm0B1jxALjhiov2D% aebbfv3ySLgzGueE0jxyaibaiGc9yrFr0xXdbba91rFfpec8Eeeu0x% Xdbba9frFj0-OqFfea0dXdd9vqaq-JfrVkFHe9pgea0dXdar-Jb9hs% 0dXdbPYxe9vr0-vr0-vqpWqaaeaabiGaciaacaqabeaadaqaaqGaaO% qaaGqaciqa-zgagaacamaaBaaaleaadaWgaaadbaGaa8NBaaqabaaa% leqaaaaa!3E82!\[\tilde f_{_n }\]and % MathType!MTEF!2!1!+-% feaafeart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn% hiov2DGi1BTfMBaeXafv3ySLgzGmvETj2BSbqefm0B1jxALjhiov2D% aebbfv3ySLgzGueE0jxyaibaiGc9yrFr0xXdbba91rFfpec8Eeeu0x% Xdbba9frFj0-OqFfea0dXdd9vqaq-JfrVkFHe9pgea0dXdar-Jb9hs% 0dXdbPYxe9vr0-vr0-vqpWqaaeaabiGaciaacaqabeaadaqaaqGaaO% qaaGqaciqa-zgagaqcamaaBaaaleaadaWgaaadbaGaa8NBaaqabaaa% leqaaaaa!3E83!\[\hat f_{_n }\], which have been proposed by Bhattacharyya et al. (1988, Comm. Statist. Theory Methods, A17, 3629–3644) and Jones (1991, Biometrika, 78, 511–519), respectively, for estimating the underlying density f at a point under a general selection biased model. The asymptotic optimality of % MathType!MTEF!2!1!+-% feaafeart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn% hiov2DGi1BTfMBaeXafv3ySLgzGmvETj2BSbqefm0B1jxALjhiov2D% aebbfv3ySLgzGueE0jxyaibaiGc9yrFr0xXdbba91rFfpec8Eeeu0x% Xdbba9frFj0-OqFfea0dXdd9vqaq-JfrVkFHe9pgea0dXdar-Jb9hs% 0dXdbPYxe9vr0-vr0-vqpWqaaeaabiGaciaacaqabeaadaqaaqGaaO% qaaGqaciqa-zgagaqcamaaBaaaleaadaWgaaadbaGaa8NBaaqabaaa% leqaaaaa!3E83!\[\hat f_{_n }\]and % MathType!MTEF!2!1!+-% feaafeart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn% hiov2DGi1BTfMBaeXafv3ySLgzGmvETj2BSbqefm0B1jxALjhiov2D% aebbfv3ySLgzGueE0jxyaibaiGc9yrFr0xXdbba91rFfpec8Eeeu0x% Xdbba9frFj0-OqFfea0dXdd9vqaq-JfrVkFHe9pgea0dXdar-Jb9hs% 0dXdbPYxe9vr0-vr0-vqpWqaaeaabiGaciaacaqabeaadaqaaqGaaO% qaaGqaciqa-zgagaacamaaBaaaleaadaWgaaadbaGaa8NBaaqabaaa% leqaaaaa!3E82!\[\tilde f_{_n }\]is measured by the corresponding asymptotic minimax mean squared errors under a compactly supported Lipschitz continuous family of the underlying densities. It is shown that, in general, % MathType!MTEF!2!1!+-% feaafeart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn% hiov2DGi1BTfMBaeXafv3ySLgzGmvETj2BSbqefm0B1jxALjhiov2D% aebbfv3ySLgzGueE0jxyaibaiGc9yrFr0xXdbba91rFfpec8Eeeu0x% Xdbba9frFj0-OqFfea0dXdd9vqaq-JfrVkFHe9pgea0dXdar-Jb9hs% 0dXdbPYxe9vr0-vr0-vqpWqaaeaabiGaciaacaqabeaadaqaaqGaaO% qaaGqaciqa-zgagaqcamaaBaaaleaadaWgaaadbaGaa8NBaaqabaaa% leqaaaaa!3E83!\[\hat f_{_n }\]is a superior local estimate than % MathType!MTEF!2!1!+-% feaafeart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn% hiov2DGi1BTfMBaeXafv3ySLgzGmvETj2BSbqefm0B1jxALjhiov2D% aebbfv3ySLgzGueE0jxyaibaiGc9yrFr0xXdbba91rFfpec8Eeeu0x% Xdbba9frFj0-OqFfea0dXdd9vqaq-JfrVkFHe9pgea0dXdar-Jb9hs% 0dXdbPYxe9vr0-vr0-vqpWqaaeaabiGaciaacaqabeaadaqaaqGaaO% qaaGqaciqa-zgagaacamaaBaaaleaadaWgaaadbaGaa8NBaaqabaaa% leqaaaaa!3E82!\[\tilde f_{_n }\]in the sense that the asymptotic minimax risk of % MathType!MTEF!2!1!+-% feaafeart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn% hiov2DGi1BTfMBaeXafv3ySLgzGmvETj2BSbqefm0B1jxALjhiov2D% aebbfv3ySLgzGueE0jxyaibaiGc9yrFr0xXdbba91rFfpec8Eeeu0x% Xdbba9frFj0-OqFfea0dXdd9vqaq-JfrVkFHe9pgea0dXdar-Jb9hs% 0dXdbPYxe9vr0-vr0-vqpWqaaeaabiGaciaacaqabeaadaqaaqGaaO% qaaGqaciqa-zgagaqcamaaBaaaleaadaWgaaadbaGaa8NBaaqabaaa% leqaaaaa!3E83!\[\hat f_{_n }\]is lower than that of % MathType!MTEF!2!1!+-% feaafeart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn% hiov2DGi1BTfMBaeXafv3ySLgzGmvETj2BSbqefm0B1jxALjhiov2D% aebbfv3ySLgzGueE0jxyaibaiGc9yrFr0xXdbba91rFfpec8Eeeu0x% Xdbba9frFj0-OqFfea0dXdd9vqaq-JfrVkFHe9pgea0dXdar-Jb9hs% 0dXdbPYxe9vr0-vr0-vqpWqaaeaabiGaciaacaqabeaadaqaaqGaaO% qaaGqaciqa-zgagaacamaaBaaaleaadaWgaaadbaGaa8NBaaqabaaa% leqaaaaa!3E82!\[\tilde f_{_n }\]. The minimax kernels and bandwidths of % MathType!MTEF!2!1!+-% feaafeart1ev1aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn% hiov2DGi1BTfMBaeXafv3ySLgzGmvETj2BSbqefm0B1jxALjhiov2D% aebbfv3ySLgzGueE0jxyaibaiGc9yrFr0xXdbba91rFfpec8Eeeu0x% Xdbba9frFj0-OqFfea0dXdd9vqaq-JfrVkFHe9pgea0dXdar-Jb9hs% 0dXdbPYxe9vr0-vr0-vqpWqaaeaabiGaciaacaqabeaadaqaaqGaaO% qaaGqaciqa-zgagaqcamaaBaaaleaadaWgaaadbaGaa8NBaaqabaaa% leqaaaaa!3E83!\[\hat f_{_n }\]are computed explicity and shown to have simple forms and depend on the weight functions of the model.     

Reliable estimation via simulation

Leta ands denote the inter arrival times and service times in aGI/GI/1 queue. Let a⁽ⁿ⁾, s⁽ⁿ⁾ be the r.v.s. with distributions as the estimated distributions ofa ands from iid samples ofa ands of sizesn. Letw be a r.v. with the stationary distribution of the waiting times of the queue with input(a,s). We consider the problem of estimatingE[w ], > 0 and via simulations when (a ⁽ⁿ⁾,s⁽ⁿ⁾) are used as input. Conditions for the accuracy of the asymptotic estimate, continuity of the asymptotic variance and uniformity in the rate of convergence to the estimate are obtained. We also obtain rates of convergence for sample moments, the empirical process and the quantile process for the regenerative processes. Robust estimates are also obtained when an outlier contaminated sample ofa ands is provided. In the process we obtain consistency, continuity and asymptotic normality of M-estimators for stationary sequences. Some robustness results for Markov processes are included.     

On parameter estimation for cusp-type signals

We consider the problem of parameter estimation by continuous time observations of a deterministic signal in white Gaussian noise. It is supposed that the signal has a cusp-type singularity. The properties of the maximum-likelihood and Bayesian estimators are described in the asymptotics of small noise. Special attention is paid to the problem of parameter estimation in the situation of misspecification in regularity, i.e., when the statistician supposes that the observed signal has this singularity, but the real signal is smooth. The rate and the asymptotic distribution of the maximum-likelihood estimator in this situation are described.     

Bipower-type estimation in a noisy diffusion setting

We consider a new class of estimators for volatility functionals in the setting of frequently observed Itō diffusions which are disturbed by i.i.d. noise. These statistics extend the approach of pre-averaging as a general method for the estimation of the integrated volatility in the presence of microstructure noise and are closely related to the original concept of bipower variation in the no-noise case. We show that this approach provides efficient estimators for a large class of integrated powers of volatility and prove the associated (stable) central limit theorems. In a more general Itō semimartingale framework this method can be used to define both estimators for the entire quadratic variation of the underlying process and jump-robust estimators which are consistent for various functionals of volatility. As a by-product we obtain a simple test for the presence of jumps in the underlying semimartingale.     

Enhancing transformations and parameter estimation for analytical signals

Translated from Metody Matematicheskogo Modelirovaniya i Vychislitel'noi Diagnostika, pp. 125–132, Izd. Moskovskogo Universiteta, Moscow, 1990.     

Reliable a posteriori error estimation for state-constrained optimal control

We derive a reliable a posteriori error estimator for a state-constrained elliptic optimal control problem taking into account both regularisation and discretisation. The estimator is applicable to finite element discretisations of the problem with both discretised and non-discretised control. The performance of our estimator is illustrated by several numerical examples for which we also introduce an adaptation strategy for the regularisation parameter.     

Reconstruction of noisy signals by minimization of non-convex functionals

Non-convex functionals have shown sharper results in signal reconstruction as compared to convex ones, although the existence of a minimum has not been established in general. This paper addresses the study of a general class of either convex or non-convex functionals for denoising signals which combines two general terms for fitting and smoothing purposes, respectively. The first one measures how close a signal is to the original noisy signal. The second term aims at removing noise while preserving some expected characteristics in the true signal such as edges and fine details. A theoretical proof of the existence of a minimum for functionals of this class is presented. The main merit of this result is to show the existence of minimizer for a large family of non-convex functionals.     

Multidimensional models for analysing frequency modulated signals

In radio frequency (RF) applications, slowly varying signals often modulate the amplitude and frequency of fast carrier waves. Thus a numerical simulation of the differential algebraic equations (DAEs) modelling the electric circuit becomes tedious. Alternative models are required to achieve efficient simulations. A multivariate formulation of signals yields a suitable representation via decoupling the widely separated time scales. Consequently, the circuit's DAEs change into warped multirate partial DAEs. On the other hand, the transient behaviour of the circuit can also be approximated by a parameter-dependent DAE model including a multivariate structure. The properties of this alternative strategy are investigated. In particular, the two multidimensional approaches are compared with respect to the simulation of RF signals.     

On frequency–amplitude dependences for surface and internal standing waves

The analytic approach proposed by Sekerzh-Zenkovich [On the theory of standing waves of finite amplitude, Dokl. Akad. Nauk USSR 58 (1947) 551–554] is developed in the present study of standing waves. Generalizing the solution method, a set of standing wave problems are solved, namely, the infinite- and finite-depth surface standing waves and the infinite- and finite-depth internal standing waves. Two-dimensional wave motion of an irrotational incompressible fluid in a rectangular domain is considered to study weakly nonlinear surface and internal standing waves. The Lagrangian formulation of the problems is used and the fifth-order perturbation solutions are determined. Since most of the approximate analytic solutions to these problems were obtained using the Eulerian formulation, the comparison of the results, as an example the analytic frequency–amplitude dependences, obtained in Lagrangian variables with the corresponding ones known in Eulerian variables has been carried out in the paper. The analytic frequency–amplitude dependences are in complete agreement with previous results known in the literature. Computer algebra procedures were written for the construction of asymptotic solutions. The application of the model constructed in Lagrangian formulation to a set of different problems shows the ability to correctly reproduce and predict a wide range of situations with different characteristics and some advantages of Lagrangian particle models (for example, the bigger radius of convergence of an expansion parameter than in Eulerian variables, simplification of the boundary conditions, parametrization of a free boundary).     

Non-parametric estimation of the diffusion coefficient from noisy data

We consider a diffusion process (X _t ) _t????0, with drift b(x) and diffusion coefficient ??(x). At discrete times t _k ?=?k ?? for k from 1 to M, we observe noisy data of the sample path, ${Y_{k\delta}=X_{k\delta}+\varepsilon_{k}}$ . The random variables ${\left(\varepsilon_{k}\right)}$ are i.i.d, centred and independent of (X _t ). The process (X _t ) _t????0 is assumed to be strictly stationary, ??-mixing and ergodic. In order to reduce the noise effect, we split data into groups of equal size p and build empirical means. The group size p is chosen such that ???=?p ?? is small whereas M ?? is large. Then, the diffusion coefficient ?? ² is estimated in a compact set A in a non-parametric way by a penalized least squares approach and the risk of the resulting adaptive estimator is bounded. We provide several examples of diffusions satisfying our assumptions and we carry out various simulations. Our simulation results illustrate the theoretical properties of our estimators.     

Spectral density estimation with amplitude modulation and outlier detection

This paper studies spectral density estimation based on amplitude modulation including missing data as a specific case. A generalized periodogram is introduced and smoothed to give a consistent estimator of the spectral density by running local linear regression smoother. We explore the asymptotic properties of the proposed estimator and its application to time series data with periodic missing. A simple data-driven local bandwidth selection rule is proposed and an algorithm for computing the spectral density estimate is presented. The effectiveness of the proposed method is demonstrated using simulations. The application to outlier detection based on leave-one-out diagnostic is also considered. An illustrative example shows that the proposed diagnostic procedure succeeds in revealing outliers in time series without masking and smearing effects. Supported by Chinese NSF Grants 10001004 and 39930160, and Fellowship of City University of Hong Kong.     

Bayes estimation of number of signals

Bayes estimation of the number of signals, q, based on a binomial prior distribution is studied. It is found that the Bayes estimate depends on the eigenvalues of the sample covariance matrix S for white-noise case and the eigenvalues of the matrix S ₂ (S ₁+A)^–1 for the colored-noise case, where S ₁ is the sample covariance matrix of observations consisting only noise, S ₂ the sample covariance matrix of observations consisting both noise and signals and A is some positive definite matrix. Posterior distributions for both the cases are derived by expanding zonal polynomial in terms of monomial symmetric functions and using some of the important formulae of James (1964, Ann. Math. Statist., 35, 475–501).     

Parameter estimation for noisy chaotic systems based on an improved particle swarm optimization algorithm

It is of great importance to estimate the unknown parameters and time delays of chaotic systems in control and synchronization. This paper is concerned with the uncertain parameters and time delays of chaotic systems corrupted with random noise. Parameters and time delays of such chaotic systems are estimated based on the improved particle swarm optimization algorithm for its global searching ability. Numerical simulations are given to show satisfactory results.     

Fixed-point algorithms for frequency estimation and structured low rank approximation

We develop fixed-point algorithms for the approximation of structured matrices with rank penalties. In particular we use these fixed-point algorithms for making approximations by sums of exponentials, i.e., frequency estimation. For the basic formulation of the fixed-point algorithm we show that it converges to the solution of a related minimization problem, namely the one obtained by replacing the original objective function with its convex envelope and keeping the structured matrix constraint unchanged.It often happens that this solution agrees with the solution to the original minimization problem, and we provide a simple criterion for when this is true. We also provide more general fixed-point algorithms that can be used to treat the problems of making weighted approximations by sums of exponentials given equally or unequally spaced sampling. We apply the method to the case of missing data, although the above mentioned convergence results do not hold in this case. However, it turns out that the method often gives perfect reconstruction (up to machine precision) in such cases. We also discuss multidimensional extensions, and illustrate how the proposed algorithms can be used to recover sums of exponentials in several variables, but when samples are available only along a curve.     

Tighter uncertainty principles for periodic signals in terms of frequency

In this study, we propose some new uncertainty principles for periodic signals with sharper lower bounds than those in the existing ones. The improved lower bounds, in particular, are related to the frequency of the signal. Three examples are employed to demonstrate sharpness of the new uncertainty principles. Copyright © 2014 John Wiley & Sons, Ltd.     

Time-varying coefficient estimation in differential equation models with noisy time-varying covariates

We study the problem of estimating time-varying coefficients in ordinary differential equations. Current theory only applies to the case when the associated state variables are observed without measurement errors as presented in Chen and Wu (2008) [4] and [5]. The difficulty arises from the quadratic functional of observations that one needs to deal with instead of the linear functional that appears when state variables contain no measurement errors. We derive the asymptotic bias and variance for the previously proposed two-step estimators using quadratic regression functional theory.     

Rapid parameter identification of linear time-delay system from noisy frequency domain data

The technique to identify the system parameters thereof has attracted extensive research interest, since knowing the parameters would enable effective system control strategy and accurate response prediction. In this paper, a novel approach is developed to identify the parameters of the linear time-delay differential system by analyzing the complex system response in the frequency domain. Firstly, the complex frequency response of the time-delay system is expressed as a function of physical parameters and time-delay parameters, forming a typical optimization problem. Subsequently, the sensitivities with respect to the unknown parameters are derived. A novel sensitivity-based algorithm is adopted in the identification procedure. Trust-region constraint is implemented and hence tackled by Tikhonov regularization, which effectively enhances the efficiency of the algorithm. The feasibility and robustness of the identification procedure are evaluated by identifying the parameters of two numerical time-delay systems and an experimental case.