A statistical analysis of power-level-difference-based dual-channel post-filter estimator

By deriving the explicit expression of the probability density function (p.d.f.), this paper presents a statistical analysis of the power-level-difference-based dual-channel post-filter (PLD-DCPF) estimator. The derivation is based on the joint p.d.f. of the auto-spectra of a two-dimensional stationary Gaussian process with mean zero, where the theoretical expression is verified by numerical simulations. Using this theoretical p.d.f. expression, this paper studies the impacts of the correlative parameters on the amount of noise reduction and speech distortion. According to both the theoretical analysis results and the simulation results, four schemes are proposed to improve the performance of the traditional PLD-DCPF estimator.     

Space–time fractional diffusion equations and asymptotic behaviors of a coupled continuous time random walk model

In this paper, we consider a type of continuous time random walk model where the jump length is correlated with the waiting time. The asymptotic behaviors of the coupled jump probability density function in the Fourier–Laplace domain are discussed. The corresponding fractional diffusion equations are derived from the given asymptotic behaviors. Corresponding to the asymptotic behaviors of the joint probability density function in the Fourier–Laplace space, the asymptotic behaviors of the waiting time probability density and the conditional probability density for jump length are also discussed.     

Asymptotic Normality for Plug-In Estimators of Generalized Shannon’s Entropy

Shannon’s entropy is one of the building blocks of information theory and an essential aspect of Machine Learning (ML) methods (e.g., Random Forests). Yet, it is only finitely defined for distributions with fast decaying tails on a countable alphabet. The unboundedness of Shannon’s entropy over the general class of all distributions on an alphabet prevents its potential utility from being fully realized. To fill the void in the foundation of information theory, Zhang (2020) proposed generalized Shannon’s entropy, which is finitely defined everywhere. The plug-in estimator, adopted in almost all entropy-based ML method packages, is one of the most popular approaches to estimating Shannon’s entropy. The asymptotic distribution for Shannon’s entropy’s plug-in estimator was well studied in the existing literature. This paper studies the asymptotic properties for the plug-in estimator of generalized Shannon’s entropy on countable alphabets. The developed asymptotic properties require no assumptions on the original distribution. The proposed asymptotic properties allow for interval estimation and statistical tests with generalized Shannon’s entropy.     

Pseudo-random number generator based on asymptotic deterministic randomness

A novel approach to generate the pseudorandom-bit sequence from the asymptotic deterministic randomness system is proposed in this Letter. We study the characteristic of multi-value correspondence of the asymptotic deterministic randomness constructed by the piecewise linear map and the noninvertible nonlinearity transform, and then give the discretized systems in the finite digitized state space. The statistic characteristics of the asymptotic deterministic randomness are investigated numerically, such as stationary probability density function and random-like behavior. Furthermore, we analyze the dynamics of the symbolic sequence. Both theoretical and experimental results show that the symbolic sequence of the asymptotic deterministic randomness possesses very good cryptographic properties, which improve the security of chaos based PRBGs and increase the resistance against entropy attacks and symbolic dynamics attacks.     

Extraction of physical laws from joint experimental data

The extraction of a physical law y=y_o(x) from joint experimental data about x and y is treated. The joint, the marginal and the conditional probability density functions (PDF) are expressed by given data over an estimator whose kernel is the instrument scattering function. As an optimal estimator of y_o(x) the conditional average is proposed. The analysis of its properties is based upon a new definition of prediction quality. The joint experimental information and the redundancy of joint measurements are expressed by the relative entropy. With the number of experiments the redundancy on average increases, while the experimental information converges to a certain limit value. The difference between this limit value and the experimental information at a finite number of data represents the discrepancy between the experimentally determined and the true properties of the phenomenon. The sum of the discrepancy measure and the redundancy is utilized as a cost function. By its minimum a reasonable number of data for the extraction of the law y_o(x) is specified. The mutual information is defined by the marginal and the conditional PDFs of the variables. The ratio between mutual information and marginal information is used to indicate which variable is the independent one. The properties of the introduced statistics are demonstrated on deterministically and randomly related variables.     

Robust Estimation for Bivariate Poisson INGARCH Models

In the integer-valued generalized autoregressive conditional heteroscedastic (INGARCH) models, parameter estimation is conventionally based on the conditional maximum likelihood estimator (CMLE). However, because the CMLE is sensitive to outliers, we consider a robust estimation method for bivariate Poisson INGARCH models while using the minimum density power divergence estimator. We demonstrate the proposed estimator is consistent and asymptotically normal under certain regularity conditions. Monte Carlo simulations are conducted to evaluate the performance of the estimator in the presence of outliers. Finally, a real data analysis using monthly count series of crimes in New South Wales and an artificial data example are provided as an illustration.     

Probability Distribution on Full Rooted Trees

The recursive and hierarchical structure of full rooted trees is applicable to statistical models in various fields, such as data compression, image processing, and machine learning. In most of these cases, the full rooted tree is not a random variable; as such, model selection to avoid overfitting is problematic. One method to solve this problem is to assume a prior distribution on the full rooted trees. This enables the optimal model selection based on Bayes decision theory. For example, by assigning a low prior probability to a complex model, the maximum a posteriori estimator prevents the selection of the complex one. Furthermore, we can average all the models weighted by their posteriors. In this paper, we propose a probability distribution on a set of full rooted trees. Its parametric representation is suitable for calculating the properties of our distribution using recursive functions, such as the mode, expectation, and posterior distribution. Although such distributions have been proposed in previous studies, they are only applicable to specific applications. Therefore, we extract their mathematically essential components and derive new generalized methods to calculate the expectation, posterior distribution, etc.     

A parameter estimator based on adaptive noise canceller

I.Introductionwith'thewendevelopmentofadaptivesignalprocessingtheoryandtheraPiddevelopmentofD.S.Pdevices,adaptivenoisecancellerl1'2]iswidelyusedininformationprocessingfieldssuchasradar,sonartspeechsignalprocessingandcommunication.Butadaptivecancellerisseldomusedasaparameterestimatortoestimateaprocessorparameterssuchasbearing,rangetvelocityandpositionofamaneuveringtarget,becausetheparameters(i.e.thesrycalledsignaltobeestimatied)tobeestimatedareusuallynonstationaryandhavethenonzeromean,thatwill…     

Robust Statistical Inference in Generalized Linear Models Based on Minimum Renyi’s Pseudodistance Estimators

Minimum Renyi’s pseudodistance estimators (MRPEs) enjoy good robustness properties without a significant loss of efficiency in general statistical models, and, in particular, for linear regression models (LRMs). In this line, Castilla et al. considered robust Wald-type test statistics in LRMs based on these MRPEs. In this paper, we extend the theory of MRPEs to Generalized Linear Models (GLMs) using independent and nonidentically distributed observations (INIDO). We derive asymptotic properties of the proposed estimators and analyze their influence function to asses their robustness properties. Additionally, we define robust Wald-type test statistics for testing linear hypothesis and theoretically study their asymptotic distribution, as well as their influence function. The performance of the proposed MRPEs and Wald-type test statistics are empirically examined for the Poisson Regression models through a simulation study, focusing on their robustness properties. We finally test the proposed methods in a real dataset related to the treatment of epilepsy, illustrating the superior performance of the robust MRPEs as well as Wald-type tests.     

Inferring a Property of a Large System from a Small Number of Samples

Inferring the value of a property of a large stochastic system is a difficult task when the number of samples is insufficient to reliably estimate the probability distribution. The Bayesian estimator of the property of interest requires the knowledge of the prior distribution, and in many situations, it is not clear which prior should be used. Several estimators have been developed so far in which the proposed prior us individually tailored for each property of interest; such is the case, for example, for the entropy, the amount of mutual information, or the correlation between pairs of variables. In this paper, we propose a general framework to select priors that is valid for arbitrary properties. We first demonstrate that only certain aspects of the prior distribution actually affect the inference process. We then expand the sought prior as a linear combination of a one-dimensional family of indexed priors, each of which is obtained through a maximum entropy approach with constrained mean values of the property under study. In many cases of interest, only one or very few components of the expansion turn out to contribute to the Bayesian estimator, so it is often valid to only keep a single component. The relevant component is selected by the data, so no handcrafted priors are required. We test the performance of this approximation with a few paradigmatic examples and show that it performs well in comparison to the ad-hoc methods previously proposed in the literature. Our method highlights the connection between Bayesian inference and equilibrium statistical mechanics, since the most relevant component of the expansion can be argued to be that with the right temperature.     

Equivalent a Posteriori Error Estimator of Spectral Approximation for Control Problems with Integral Control-State Constraints in One Dimension

In this paper, we investigate the Galerkin spectral approximation for elliptic control problems with integral control and state constraints. Firstly, an a posteriori error estimator is established, which can be acted as the equivalent indicator with explicit expression. Secondly, appropriate base functions of the discrete spaces make it probable to solve the discrete system. Numerical test indicates the reliability and efficiency of the estimator, and shows the proposed method is competitive for this class of control problems. These discussions can certainly be extended to two- and three-dimensional cases.     

Precision of degree of polarization estimation in the presence of additive Gaussian detector noise

We address the problem of degree of polarization (DOP) estimation in images limited by additive Gaussian detector noise. We derive and analyze the probability density function (PDF) of the pixelwise DOP estimate, which is shown to have significantly different statistical properties than when noise is Gamma distributed (speckle). We then determine the Cramer-Rao Lower Bound and the maximum likelihood estimator of the DOP. We deduce from this study practical solutions for characterizing and reducing the noise in these images.     

The asymptotic spectrum of the EWMA covariance estimator

The exponentially weighted moving average (EWMA) covariance estimator is a standard estimator for financial time series, and its spectrum can be used for so-called random matrix filtering. Random matrix filtering using the spectrum of the sample covariance matrix is an established tool in finance and signal detection and the EWMA spectrum can be used analogously. In this paper, the asymptotic spectrum of the EWMA covariance estimator is calculated using the Mar?enko-Pastur theorem. Equations for the spectrum and the boundaries of the support of the spectrum are obtained and solved numerically. The spectrum is compared with covariance estimates using simulated i.i.d. data and log-returns from a subset of stocks from the S&P 500. The behaviour of the EWMA estimator in this limited empirical study is similar to the results in previous studies of sample covariance matrices. Correlations in the data are found to only affect a small part of the EWMA spectrum, suggesting that a large part may be filtered out.     

Novel summability methods generalizing the Borel method

The paper deals with moment constant summability methods. A method is constructed which provides an analytic continuation of a function regular at the origin onto its Mittag-Leffler (principal) star. In this sense the method is optimal in contrast to the Borel one. In the next a one parameter family of such methods is introduced. The methods may be useful both in field theory and in statistical physics. Applications to the Nevanlinna theorem, the Rayleigh-Schrö-dinger perturbation theory and the dispersion-like integral are given. The proofs of theorems can be easily adapted to the study of the Mellin transform of some entire functions and a simpler proof of asymptotic properties of the gamma function can be obtained.I am indebted to J. Fuka for stimulating discussions and J. Fischer for continuous interest in my work and valuable comments. Financial support of the Czech Literatury Fund is also gratefully acknowledged.     

Robust Test Statistics Based on Restricted Minimum Rényi’s Pseudodistance Estimators

The Rao’s score, Wald and likelihood ratio tests are the most common procedures for testing hypotheses in parametric models. None of the three test statistics is uniformly superior to the other two in relation with the power function, and moreover, they are first-order equivalent and asymptotically optimal. Conversely, these three classical tests present serious robustness problems, as they are based on the maximum likelihood estimator, which is highly non-robust. To overcome this drawback, some test statistics have been introduced in the literature based on robust estimators, such as robust generalized Wald-type and Rao-type tests based on minimum divergence estimators. In this paper, restricted minimum Rényi’s pseudodistance estimators are defined, and their asymptotic distribution and influence function are derived. Further, robust Rao-type and divergence-based tests based on minimum Rényi’s pseudodistance and restricted minimum Rényi’s pseudodistance estimators are considered, and the asymptotic properties of the new families of tests statistics are obtained. Finally, the robustness of the proposed estimators and test statistics is empirically examined through a simulation study, and illustrative applications in real-life data are analyzed.     

Geometric Partition Entropy: Coarse-Graining a Continuous State Space

Entropy is re-examined as a quantification of ignorance in the predictability of a one dimensional continuous phenomenon. Although traditional estimators for entropy have been widely utilized in this context, we show that both the thermodynamic and Shannon’s theory of entropy are fundamentally discrete, and that the limiting process used to define differential entropy suffers from similar problems to those encountered in thermodynamics. In contrast, we consider a sampled data set to be observations of microstates (unmeasurable in thermodynamics and nonexistent in Shannon’s discrete theory), meaning, in this context, it is the macrostates of the underlying phenomenon that are unknown. To obtain a particular coarse-grained model we define macrostates using quantiles of the sample and define an ignorance density distribution based on the distances between quantiles. The geometric partition entropy is then just the Shannon entropy of this finite distribution. Our measure is more consistent and informative than histogram-binning, especially when applied to complex distributions and those with extreme outliers or under limited sampling. Its computational efficiency and avoidance of negative values can also make it preferable to geometric estimators such as k-nearest neighbors. We suggest applications that are unique to this estimator and illustrate its general utility through an application to time series in the approximation of an ergodic symbolic dynamics from limited observations.     

Kernel Estimation of Cumulative Residual Tsallis Entropy and Its Dynamic Version under ρ-Mixing Dependent Data

Tsallis introduced a non-logarithmic generalization of Shannon entropy, namely Tsallis entropy, which is non-extensive. Sati and Gupta proposed cumulative residual information based on this non-extensive entropy measure, namely cumulative residual Tsallis entropy (CRTE), and its dynamic version, namely dynamic cumulative residual Tsallis entropy (DCRTE). In the present paper, we propose non-parametric kernel type estimators for CRTE and DCRTE where the considered observations exhibit an ρ-mixing dependence condition. Asymptotic properties of the estimators were established under suitable regularity conditions. A numerical evaluation of the proposed estimator is exhibited and a Monte Carlo simulation study was carried out.     

The role of ergodicity and mixing in the central limit theorem for Casati-Prosen triangle map variables

In this Letter we analyse the behaviour of the probability density function of the sum of N deterministic variables generated from the triangle map of Casati-Prosen. For the case in which the map is both ergodic and mixing the resulting probability density function quickly concurs with the Normal distribution. However, when the map is weakly chaotic, and fuzzily not mixing, the resulting probability density functions are described by power-laws. Moreover, contrarily to what it would be expected, as the number of added variables N increases the distance to Gaussian distribution increases. This behaviour goes against standard central limit theorem. By extrapolation of our finite size results we preview that in the limit of N going to infinity the distribution has the same asymptotic decay as a Lorentzian (or a q=2-Gaussian).