Robust Test Statistics Based on Restricted Minimum Rényi’s Pseudodistance Estimators

The Rao’s score, Wald and likelihood ratio tests are the most common procedures for testing hypotheses in parametric models. None of the three test statistics is uniformly superior to the other two in relation with the power function, and moreover, they are first-order equivalent and asymptotically optimal. Conversely, these three classical tests present serious robustness problems, as they are based on the maximum likelihood estimator, which is highly non-robust. To overcome this drawback, some test statistics have been introduced in the literature based on robust estimators, such as robust generalized Wald-type and Rao-type tests based on minimum divergence estimators. In this paper, restricted minimum Rényi’s pseudodistance estimators are defined, and their asymptotic distribution and influence function are derived. Further, robust Rao-type and divergence-based tests based on minimum Rényi’s pseudodistance and restricted minimum Rényi’s pseudodistance estimators are considered, and the asymptotic properties of the new families of tests statistics are obtained. Finally, the robustness of the proposed estimators and test statistics is empirically examined through a simulation study, and illustrative applications in real-life data are analyzed.     

Entropy Based Student’s t-Process Dynamical Model

Volatility, which represents the magnitude of fluctuating asset prices or returns, is used in the problems of finance to design optimal asset allocations and to calculate the price of derivatives. Since volatility is unobservable, it is identified and estimated by latent variable models known as volatility fluctuation models. Almost all conventional volatility fluctuation models are linear time-series models and thus are difficult to capture nonlinear and/or non-Gaussian properties of volatility dynamics. In this study, we propose an entropy based Student’s t-process Dynamical model (ETPDM) as a volatility fluctuation model combined with both nonlinear dynamics and non-Gaussian noise. The ETPDM estimates its latent variables and intrinsic parameters by a robust particle filtering based on a generalized H-theorem for a relative entropy. To test the performance of the ETPDM, we implement numerical experiments for financial time-series and confirm the robustness for a small number of particles by comparing with the conventional particle filtering.     

Sharp Guarantees and Optimal Performance for Inference in Binary and Gaussian-Mixture Models

We study convex empirical risk minimization for high-dimensional inference in binary linear classification under both discriminative binary linear models, as well as generative Gaussian-mixture models. Our first result sharply predicts the statistical performance of such estimators in the proportional asymptotic regime under isotropic Gaussian features. Importantly, the predictions hold for a wide class of convex loss functions, which we exploit to prove bounds on the best achievable performance. Notably, we show that the proposed bounds are tight for popular binary models (such as signed and logistic) and for the Gaussian-mixture model by constructing appropriate loss functions that achieve it. Our numerical simulations suggest that the theory is accurate even for relatively small problem dimensions and that it enjoys a certain universality property.     

Sparse Estimation Strategies in Linear Mixed Effect Models for High-Dimensional Data Application

In a host of business applications, biomedical and epidemiological studies, the problem of multicollinearity among predictor variables is a frequent issue in longitudinal data analysis for linear mixed models (LMM). We consider an efficient estimation strategy for high-dimensional data application, where the dimensions of the parameters are larger than the number of observations. In this paper, we are interested in estimating the fixed effects parameters of the LMM when it is assumed that some prior information is available in the form of linear restrictions on the parameters. We propose the pretest and shrinkage estimation strategies using the ridge full model as the base estimator. We establish the asymptotic distributional bias and risks of the suggested estimators and investigate their relative performance with respect to the ridge full model estimator. Furthermore, we compare the numerical performance of the LASSO-type estimators with the pretest and shrinkage ridge estimators. The methodology is investigated using simulation studies and then demonstrated on an application exploring how effective brain connectivity in the default mode network (DMN) may be related to genetics within the context of Alzheimer’s disease.     

On Selection Criteria for the Tuning Parameter in Robust Divergence

Although robust divergence, such as density power divergence and γ-divergence, is helpful for robust statistical inference in the presence of outliers, the tuning parameter that controls the degree of robustness is chosen in a rule-of-thumb, which may lead to an inefficient inference. We here propose a selection criterion based on an asymptotic approximation of the Hyvarinen score applied to an unnormalized model defined by robust divergence. The proposed selection criterion only requires first and second-order partial derivatives of an assumed density function with respect to observations, which can be easily computed regardless of the number of parameters. We demonstrate the usefulness of the proposed method via numerical studies using normal distributions and regularized linear regression.     

Asymptotic Normality for Plug-In Estimators of Generalized Shannon’s Entropy

Shannon’s entropy is one of the building blocks of information theory and an essential aspect of Machine Learning (ML) methods (e.g., Random Forests). Yet, it is only finitely defined for distributions with fast decaying tails on a countable alphabet. The unboundedness of Shannon’s entropy over the general class of all distributions on an alphabet prevents its potential utility from being fully realized. To fill the void in the foundation of information theory, Zhang (2020) proposed generalized Shannon’s entropy, which is finitely defined everywhere. The plug-in estimator, adopted in almost all entropy-based ML method packages, is one of the most popular approaches to estimating Shannon’s entropy. The asymptotic distribution for Shannon’s entropy’s plug-in estimator was well studied in the existing literature. This paper studies the asymptotic properties for the plug-in estimator of generalized Shannon’s entropy on countable alphabets. The developed asymptotic properties require no assumptions on the original distribution. The proposed asymptotic properties allow for interval estimation and statistical tests with generalized Shannon’s entropy.     

用于噪声鲁棒性语音识别的子带能量规整感知线性预测系数

为了提高感知线性预测系数(PLP)在噪声环境下的识别性能,使用子带能量偏差减的方法,提出了一种基于子带能量规整的感知线性预测系数(SPNPLP)。PLP有效地集中了语音中的有用信息,在安静环境下自动语音识别系统使用PLP可以取得良好的识别率;但是在噪声环境中其识别性能急剧下降。通过使用能量偏差减的方法对PLP的子带能量进行规整,抑制背景噪声激励,提出了SPNPLP,增强自动语音识别系统在噪声环境下的鲁棒性。在一个语法大小为501的孤立词识别任务和一个大词表连续语音识别任务上做了测试,SPNPLP在这两个任务上,与PLP相比,汉字识别精度分别绝对提升了11.26%和9.2%。实验结果表明SPNPLP比PLP具有更好的噪声鲁棒性。      

On the Regression Model for Generalized Normal Distributions

The traditional linear regression model that assumes normal residuals is applied extensively in engineering and science. However, the normality assumption of the model residuals is often ineffective. This drawback can be overcome by using a generalized normal regression model that assumes a non-normal response. In this paper, we propose regression models based on generalizations of the normal distribution. The proposed regression models can be used effectively in modeling data with a highly skewed response. Furthermore, we study in some details the structural properties of the proposed generalizations of the normal distribution. The maximum likelihood method is used for estimating the parameters of the proposed method. The performance of the maximum likelihood estimators in estimating the distributional parameters is assessed through a small simulation study. Applications to two real datasets are given to illustrate the flexibility and the usefulness of the proposed distributions and their regression models.     

Estimation of the Reliability of a Stress–Strength System from Poisson Half Logistic Distribution

This paper discussed the estimation of stress-strength reliability parameter R=P(Y<X) based on complete samples when the stress-strength are two independent Poisson half logistic random variables (PHLD). We have addressed the estimation of R in the general case and when the scale parameter is common. The classical and Bayesian estimation (BE) techniques of R are studied. The maximum likelihood estimator (MLE) and its asymptotic distributions are obtained; an approximate asymptotic confidence interval of R is computed using the asymptotic distribution. The non-parametric percentile bootstrap and student’s bootstrap confidence interval of R are discussed. The Bayes estimators of R are computed using a gamma prior and discussed under various loss functions such as the square error loss function (SEL), absolute error loss function (AEL), linear exponential error loss function (LINEX), generalized entropy error loss function (GEL) and maximum a posteriori (MAP). The Metropolis–Hastings algorithm is used to estimate the posterior distributions of the estimators of R. The highest posterior density (HPD) credible interval is constructed based on the SEL. Monte Carlo simulations are used to numerically analyze the performance of the MLE and Bayes estimators, the results were quite satisfactory based on their mean square error (MSE) and confidence interval. Finally, we used two real data studies to demonstrate the performance of the proposed estimation techniques in practice and to illustrate how PHLD is a good candidate in reliability studies.     

Power-Modified Kies-Exponential Distribution: Properties,Classical and Bayesian Inference with an Application to Engineering Data

We introduce here a new distribution called the power-modified Kies-exponential (PMKE) distribution and derive some of its mathematical properties. Its hazard function can be bathtub-shaped, increasing, or decreasing. Its parameters are estimated by seven classical methods. Further, Bayesian estimation, under square error, general entropy, and Linex loss functions are adopted to estimate the parameters. Simulation results are provided to investigate the behavior of these estimators. The estimation methods are sorted, based on partial and overall ranks, to determine the best estimation approach for the model parameters. The proposed distribution can be used to model a real-life turbocharger dataset, as compared with 24 extensions of the exponential distribution.     

Robustness quantification method for network intrusion detection models

Deep learning techniques have been successfully applied to network intrusion detection tasks, but as in the case of autonomous driving and face recognition, the reliability of the system itself has become a pressing issue. Robustness is a key attribute to determine whether a deep learning system is secure and reliable, and we also choose to explore the security of intrusion detection models from a new perspective of robustness quantification. In this paper, we focus on the intrusion detection model based on long and short-term memory, and use a fine-grained linear approximation method to derive a more accurate robustness bound on the nonlinear activation function with tighter linear constraints. We can use this bound to quantitatively measure the robustness of the detection model and determine whether the model is susceptible to the influence of adversarial samples. In our experiments, we test networks with various structures on the MNIST dataset, and the results show that our proposed method can effectively deduce the robustness bounds of output elements, and has good scalability and applicability.     

Wigner-Matrix-Based Normality Test and Application to Weak Signal Detection in SISO/SIMO Systems

Based on the asymptotic spectral distribution of Wigner matrices,a new normality test method is proposed via reforming the white noise sequence.In this work,the asymptotic cumulative distribution function(CDF)of eigenvalues of the Wigner matrix is deduced.A numerical Kullback-Leibler divergence of the empirical spectral CDF based on test samples from the deduced asymptotic CDF is established,which is treated as the test statistic.For validating the superiority of our proposed normality test,we apply the method to weak 8PSK signal detection in the single-input single-output(SISO) system and the single-input multiple-output(SIMO)system.By comparing with other common normality tests and the existing signal detection methods,simulation results show that the proposed method is superior and robust.     

A robust implicit multigrid method for RANS equations with two-equation turbulence models

The design of a new, truly robust multigrid framework for the solution of steady-state Reynolds-Averaged Navier–Stokes (RANS) equations with two-equation turbulence models is presented. While the mean-flow equations and the turbulence model equations are advanced in time in a loosely-coupled manner, their multigrid cycling is strongly coupled (FC-MG). Thanks to the loosely-coupled approach, the unconditionally positive-convergent implicit time-integration scheme for two-equation turbulence models (UPC) is used. An improvement to the basic UPC scheme convergence characteristics is developed and its extension within the multigrid method is proposed. The resulting novel FC-MG-UPC algorithm is nearly free of artificial stabilizing techniques, leading to increased multigrid efficiency. To demonstrate the robustness of the proposed algorithm, it is applied to linear and non-linear two-equation turbulence models. Numerical experiments are conducted, simulating separated flow about the NACA4412 airfoil, transonic flow about the RAE2822 airfoil and internal flow through a plane asymmetric diffuser. Results obtained from numerical simulations demonstrate the strong consistency and case-independence of the method.     

Electrophysiological Properties from Computations at a Single Voltage: Testing Theory with Stochastic Simulations

We use stochastic simulations to investigate the performance of two recently developed methods for calculating the free energy profiles of ion channels and their electrophysiological properties, such as current–voltage dependence and reversal potential, from molecular dynamics simulations at a single applied voltage. These methods require neither knowledge of the diffusivity nor simulations at multiple voltages, which greatly reduces the computational effort required to probe the electrophysiological properties of ion channels. They can be used to determine the free energy profiles from either forward or backward one-sided properties of ions in the channel, such as ion fluxes, density profiles, committor probabilities, or from their two-sided combination. By generating large sets of stochastic trajectories, which are individually designed to mimic the molecular dynamics crossing statistics of models of channels of trichotoxin, p7 from hepatitis C and a bacterial homolog of the pentameric ligand-gated ion channel, GLIC, we find that the free energy profiles obtained from stochastic simulations corresponding to molecular dynamics simulations of even a modest length are burdened with statistical errors of only 0.3 kcal/mol. Even with many crossing events, applying two-sided formulas substantially reduces statistical errors compared to one-sided formulas. With a properly chosen reference voltage, the current–voltage curves can be reproduced with good accuracy from simulations at a single voltage in a range extending for over 200 mV. If possible, the reference voltages should be chosen not simply to drive a large current in one direction, but to observe crossing events in both directions.     

改进的噪声鲁棒语音稀疏线性预测算法

语音线性预测分析算法在噪声环境下性能会急剧恶化,针对这一问题,提出一种改进的噪声鲁棒稀疏线性预测算法。首先采用学生t分布对具有稀疏性的语音线性预测残差建模,并显式考虑加性噪声的影响以提高模型鲁棒性,从而构建完整的概率模型。然后采用变分贝叶斯方法推导模型参数的近似后验分布,最终实现噪声鲁棒的稀疏线性预测参数估计。实验结果表明,与传统算法以及近几年提出的基于l₁范数优化的稀疏线性预测算法相比,该算法在多项指标上具有优势,对环境噪声具有更好的鲁棒性,并且谱失真度更小,因而能够有效提高噪声环境下的语音质量。      

Spatial Warped Gaussian Processes: Estimation and Efficient Field Reconstruction

A class of models for non-Gaussian spatial random fields is explored for spatial field reconstruction in environmental and sensor network monitoring. The family of models explored utilises a class of transformation functions known as Tukey g-and-h transformations to create a family of warped spatial Gaussian process models which can support various desirable features such as flexible marginal distributions, which can be skewed, leptokurtic and/or heavy-tailed. The resulting model is widely applicable in a range of spatial field reconstruction applications. To utilise the model in applications in practice, it is important to carefully characterise the statistical properties of the Tukey g-and-h random fields. In this work, we study both the properties of the resulting warped Gaussian processes as well as using the characterising statistical properties of the warped processes to obtain flexible spatial field reconstructions. In this regard we derive five different estimators for various important quantities often considered in spatial field reconstruction problems. These include the multi-point Minimum Mean Squared Error (MMSE) estimators, the multi-point Maximum A-Posteriori (MAP) estimators, an efficient class of multi-point linear estimators based on the Spatial-Best Linear Unbiased (S-BLUE) estimators, and two multi-point threshold exceedance based estimators, namely the Spatial Regional and Level Exceedance estimators. Simulation results and real data examples show the benefits of using the Tukey g-and-h transformation as opposed to standard Gaussian spatial random fields in a real data application for environmental monitoring.     

Asymptotic Properties of Estimators for Seasonally Cointegrated State Space Models Obtained Using the CVA Subspace Method

This paper investigates the asymptotic properties of estimators obtained from the so called CVA (canonical variate analysis) subspace algorithm proposed by Larimore (1983) in the case when the data is generated using a minimal state space system containing unit roots at the seasonal frequencies such that the yearly difference is a stationary vector autoregressive moving average (VARMA) process. The empirically most important special cases of such data generating processes are the I(1) case as well as the case of seasonally integrated quarterly or monthly data. However, increasingly also datasets with a higher sampling rate such as hourly, daily or weekly observations are available, for example for electricity consumption. In these cases the vector error correction representation (VECM) of the vector autoregressive (VAR) model is not very helpful as it demands the parameterization of one matrix per seasonal unit root. Even for weekly series this amounts to 52 matrices using yearly periodicity, for hourly data this is prohibitive. For such processes estimation using quasi-maximum likelihood maximization is extremely hard since the Gaussian likelihood typically has many local maxima while the parameter space often is high-dimensional. Additionally estimating a large number of models to test hypotheses on the cointegrating rank at the various unit roots becomes practically impossible for weekly data, for example. This paper shows that in this setting CVA provides consistent estimators of the transfer function generating the data, making it a valuable initial estimator for subsequent quasi-likelihood maximization. Furthermore, the paper proposes new tests for the cointegrating rank at the seasonal frequencies, which are easy to compute and numerically robust, making the method suitable for automatic modeling. A simulation study demonstrates by example that for processes of moderate to large dimension the new tests may outperform traditional tests based on long VAR approximations in sample sizes typically found in quarterly macroeconomic data. Further simulations show that the unit root tests are robust with respect to different distributions for the innovations as well as with respect to GARCH-type conditional heteroskedasticity. Moreover, an application to Kaggle data on hourly electricity consumption by different American providers demonstrates the usefulness of the method for applications. Therefore the CVA algorithm provides a very useful initial guess for subsequent quasi maximum likelihood estimation and also delivers relevant information on the cointegrating ranks at the different unit root frequencies. It is thus a useful tool for example in (but not limited to) automatic modeling applications where a large number of time series involving a substantial number of variables need to be modelled in parallel.     

New Estimators of the Bayes Factor for Models with High-Dimensional Parameter and/or Latent Variable Spaces

Formal Bayesian comparison of two competing models, based on the posterior odds ratio, amounts to estimation of the Bayes factor, which is equal to the ratio of respective two marginal data density values. In models with a large number of parameters and/or latent variables, they are expressed by high-dimensional integrals, which are often computationally infeasible. Therefore, other methods of evaluation of the Bayes factor are needed. In this paper, a new method of estimation of the Bayes factor is proposed. Simulation examples confirm good performance of the proposed estimators. Finally, these new estimators are used to formally compare different hybrid Multivariate Stochastic Volatility–Multivariate Generalized Autoregressive Conditional Heteroskedasticity (MSV-MGARCH) models which have a large number of latent variables. The empirical results show, among other things, that the validity of reduction of the hybrid MSV-MGARCH model to the MGARCH specification depends on the analyzed data set as well as on prior assumptions about model parameters.     

Numerical Solutions of Variable Coefficient Higher-Order Partial Differential Equations Arising in Beam Models

In this work, an efficient and robust numerical scheme is proposed to solve the variable coefficients’ fourth-order partial differential equations (FOPDEs) that arise in Euler–Bernoulli beam models. When partial differential equations (PDEs) are of higher order and invoke variable coefficients, then the numerical solution is quite a tedious and challenging problem, which is our main concern in this paper. The current scheme is hybrid in nature in which the second-order finite difference is used for temporal discretization, while spatial derivatives and solutions are approximated via the Haar wavelet. Next, the integration and Haar matrices are used to convert partial differential equations (PDEs) to the system of linear equations, which can be handled easily. Besides this, we derive the theoretical result for stability via the Lax–Richtmyer criterion and verify it computationally. Moreover, we address the computational convergence rate, which is near order two. Several test problems are given to measure the accuracy of the suggested scheme. Computations validate that the present scheme works well for such problems. The calculated results are also compared with the earlier work and the exact solutions. The comparison shows that the outcomes are in good agreement with both the exact solutions and the available results in the literature.