Reliability Estimation of Classification Algorithms. II. Integral Estimates

This article concludes the examination of reliability estimates of classification algorithms. It reviews the statistical methods used for interval estimation of the reliability of classifiers in the frequency and Bayesian approaches. “Hybrid” estimates combining both approaches are also considered. These estimates are particularly important as they are applicable to the small-sample case.__________Translated from Prikladnaya Matematika i Informatika, No. 17, pp. 112 – 128, 2004.     

Oracle Inequalities for Efromovich–Pinsker Blockwise Estimates

Oracle inequality is a relatively new statistical tool for the analysis of nonparametric adaptive estimates. Oracle is a good pseudo-estimate that is based on both data and an underlying estimated curve. An oracle inequality shows how well an adaptive estimator mimics the oracle for a particular underlying curve. The most advanced oracle inequalities have been recently obtained by Cavalier and Tsybakov (2001) for Stein type blockwise estimates used in filtering a signal from a stationary white Gaussian process. The authors also conjecture that a similar result can be obtained for Efromovich–Pinsker (EP) type blockwise estimators where their approach, based on Stein's formula for risk calculation, does not work. This article proves the conjecture and extends it upon more general models which include not stationary and dependent processes. Other possible extensions, a discussion of practical implications and a numerical study are also presented.     

Self-intersection local time for Gaussian ′( )-processes: Existence, path continuity and examples

In this paper we develop a criterion for existence or non-existence of self-intersection local time (SILT) for a wide class of Gaussian ′( ^d)-valued processes, we show that quite generally the SILT process has continuous paths, and we give several examples which illustrate existence of SILT for different ranges of dimensions (e.g., d ≤ 3, d ≤ 7 and 5 ≤ d ≤ 11 in the Brownian case). Some of the examples involve branching and exhibit “dimension gaps”. Our results generalize the work of Adler and coauthors, who studied the special case of “density processes” and proved that SILT paths are cadlag in the Brownian case making use of a “particle picture” approximation (this technique is not available for our general formulation).     

Randomly censored partially linear single-index models

This paper proposes a method for estimation of a class of partially linear single-index models with randomly censored samples. The method provides a flexible way for modelling the association between a response and a set of predictor variables when the response variable is randomly censored. It presents a technique for “dimension reduction” in semiparametric censored regression models and generalizes the existing accelerated failure-time models for survival analysis. The estimation procedure involves three stages: first, transform the censored data into synthetic data or pseudo-responses unbiasedly; second, obtain quasi-likelihood estimates of the regression coefficients in both linear and single-index components by an iteratively algorithm; finally, estimate the unknown nonparametric regression function using techniques for univariate censored nonparametric regression. The estimators for the regression coefficients are shown to be jointly root-n consistent and asymptotically normal. In addition, the estimator for the unknown regression function is a local linear kernel regression estimator and can be estimated with the same efficiency as all the parameters are known. Monte Carlo simulations are conducted to illustrate the proposed methodology.     

Large Deviation Probabilities for Square-Gaussian Stochastic Processes

We investigate properties of square-Gaussian stochastic processes. These processes are formed by quadratic forms of Gaussian processes or by limits in the mean square of quadratic forms of Gaussian processes. Special classes of these processes are determined and investigated. For processes from these classes estimates of large deviation probability are obtained. These estimates we use to estimate the probability that Gaussian vector-valued process leave some region on some interval of time. We construct asymptotic confidence regions for estimates of covariance functions of vector-valued Gaussian processes. Criterion of hypothesis testing on covariance functions of these processes is constructed.     

Fuzzy Relation Equations for Compression/Decompression Processes of Colour Images in the RGB and YUV Colour Spaces

We use particular fuzzy relation equations for compression/decompression of colour images in the RGB and YUV spaces, by comparing the results of the reconstructed images obtained in both cases. Our tests are made over well known images of 256×256 pixels (8 bits per pixel in each band) extracted from Corel Gallery. After the decomposition of each image in the three bands of the RGB and YUV colour spaces, the compression is performed using fuzzy relation equations of “min - →_t” type, where “t” is the Lukasiewicz t-norm and “→_t” is its residuum. Any image is subdivided in blocks and each block is compressed by optimizing a parameter inserted in the Gaussian membership functions of the fuzzy sets, used as coders in the fuzzy equations. The decompression process is realized via a fuzzy relation equation of max-t type. In both RGB and YUV spaces we evaluate and compare the root means square error (RMSE) and the consequentpeak signal to noise ratio (PSNR) on the decompressed images with respect to the original image under several compression rates.     

Kernel estimates and -spectral independence of generators of -semigroups

After the appearance of W. Arendt's result that “Gaussian estimate of a semigroup implies the L^p-spectral independence of the generator,” various generalizations have been obtained. This paper shows that a certain kernel estimate of a semigroup implies the L^p-spectral independence of the generator, generalizing the case of upper Gaussian estimate and “Gaussian estimate of order α(0,1] [S. Miyajima, H. Shindoh, Gaussian estimates of order α and L^p-spectral independence of generators of C₀-semigroups, Positivity 11 (1) (2007) 15–39], Definition 3.1.” The proof uses S. Karrmann's result about the L^p-spectral independence and B.A. Barnes' theorem about the spectrum of integral operators. As an application, the L^p-spectral independence of −[(−Δ)^α+V] (α(0,1]) for a suitable V is proved with the help of a recent result by V. Liskevich, H. Vogt and J. Voigt [V. Liskevich, H. Vogt, J. Voigt, Gaussian bounds for propagators perturbed by potentials, J. Funct. Anal. 238 (2006) 245–277].     

Asymptotic Normality of Cross-correlogram Estimates of the Response Function

The problem of estimation of an unknown response function of a time-invariant continuous linear system is considered. Discrete-time sample input–output cross-correlograms are taken as estimates of the response function. The inputs are supposed to be zero-mean stationary Gaussian processes close, in some sense, to a white noise. Both asymptotic normality of finite-dimensional distributions of the estimates and their asymptotic normality in spaces of continuous functions are studied. Our basic tool is a new integral representation for cumulants of the estimate as a finite sum of integrals involving cyclic products of kernels. Some inequalities for these integrals are obtained and their asymptotic behaviour is studied.     

Diagnosability of regular systems

A novel approach aimed at evaluating the diagnosability of regular systems under the PMC model is introduced. The diagnosability is defined as the ability to provide a correct diagnosis, although possibly incomplete. This concept is somehow intermediate between one-step diagnosability and sequential diagnosability. A lower bound to diagnosability is determined by lower bounding the minimum of a “syndrome-dependent” bound t_σ over the set of all the admissible syndromes. In turn, t_σ is determined by evaluating the cardinality of the smallest consistent fault set containing an aggregate of maximum cardinality. The new approach, which applies to any regular system, relies on the “edge-isoperimetric inequalities” of connected components of units declaring each other non-faulty. This approach has been used to derive tight lower bounds to the diagnosability of toroidal grids and hypercubes, which improve the existing bounds for the same structures.     

Discrete time Galerkin approximations to the nonlinear filtering solution

This paper concerns discrete time Galerkin approximations to the solution of the filtering problem for diffusions. Two families of schemes approximating the unnormalized conditional density, respectively, in an “average” and in a “pathwise” sense, are presented. L² error estimates are derived and it is shown that the rate of convergence is linear in the time increment or linear in the modulus of continuity of the sample path.     

Selecting preferred solutions in the minimax approach to dynamic programming problems under flexible constraints

Dynamic Programming is a powerful approach to the optimization of sequential or multistage decision processes, e.g., in planning or in system control. In this paper, we consider both theoretical and algorithmic issues in sequential decision processes under flexible constraints. Such processes must attain a given goal within some tolerance. Tolerances or preferences also apply to the values the decision variables may take or on the action chosen at each step. Such problems boil down to maximin optimization. Unfortunately, this approach suffers from the so-called “drowning effect” (lack of discrimination) and the optimality principle of dynamic programming is not always verified. In this context, we introduce a general framework for refined minimax optimization procedures in order to compare and select preferred alternatives. This framework encompasses already introduced methods such as LexiMin and DiscriMin, but it allows their extension to the comparison of vectors of unequal lengths. We show that these refined comparisons restore compatibility with the optimality principle, and that classical algorithms can be adapted to compute such preferred solutions, by exploiting existing results on idempotent semirings.     

Statistical inference for misspecified ergodic Lévy driven stochastic differential equation models

We consider the estimation problem of misspecified ergodic Lévy driven stochastic differential equation models based on high-frequency samples. We utilize a widely applicable and tractable Gaussian quasi-likelihood approach which focuses on mean and variance structure. It is shown that the Gaussian quasi-likelihood estimators of the drift and scale parameters still satisfy polynomial type probability estimates and asymptotic normality at the same rate as the correctly specified case. In their derivation process, the theory of extended Poisson equation for time-homogeneous Feller Markov processes plays an important role. Our result confirms the reliability of the Gaussian quasi-likelihood approach for SDE models.     

Nearly monotone and nearly convex approximation by smooth splines in ,

Given a monotone or convex function on a finite interval we construct splines of arbitrarily high order having maximum smoothness which are “nearly monotone” or “nearly convex” and provide the rate of -approximation which can be estimated in terms of the third or fourth (classical or Ditzian–Totik) moduli of smoothness (for uniformly spaced or Chebyshev knots). It is known that these estimates are impossible in terms of higher moduli and are no longer true for “purely monotone” and “purely convex” spline approximation.     

Some Asymptotic Formulae for Gaussian Distributions

This paper considers asymptotic expansions of certain expectations which appear in the theory of large deviation for Gaussian random vectors with values in a separable real Hilbert space. A typical application is to calculation of the “tails” of distributions of smooth functionals,p(r)=P{Φ(r⁻¹ξ)0},r→∞, e.g., the probability that a centered Gaussian random vector hits the exterior of a large sphere surrounding the origin. The method provides asymptotic formulae for the probability itself and not for its logarithm in a situation, where it is natural to expect thatp(r)=c′r^D exp{−c″r²}. Calculations are based on a combination of the method of characteristic functionals with the Laplace method used to find asymptotics of integrals containing a fast decaying function with “small” support.     

Moderate deviation principle for autoregressive processes

A moderate deviation principle for autoregressive processes is established. As statistical applications we provide the moderate deviation estimates of the least square and the Yule–Walker estimators of the parameter of an autoregressive process. The main assumption on the autoregressive process is the Gaussian integrability condition for the noise, which is weaker than the assumption of Logarithmic Sobolev Inequality in [H. Djellout, A. Guillin, L. Wu, Moderate deviations of empirical periodogram and nonlinear functionals of moving average processes, Ann. I. H. Poincaré-PR 42 (2006) 393–416].     

Degradation modeling of 2 fatigue‐crack growth characteristics based on inverse Gaussian processes: A case study

Most modern products that are highly reliable are complex in their inner and outer structures. This situation indicates quality characterization by the interaction of multiple performance characteristics, which motivates the utilization of robust reliability models to obtain robust estimates. It is paramount to obtaining substantial information about a product's life cycle; therefore, when multiple performance characteristics are dependent, it is important to find models that address the joint distribution of performance degradation of such. In this paper, a reliability model for products with 2 fatigue‐crack growth characteristics related to 2 degradation processes is developed. The proposed model considers the dependence among degradation processes by using copula functions considering the marginal degradation processes as inverse Gaussian processes. The statistical inference is performed by using a Bayesian approach to estimate the parameters of the joint bivariate model. A time‐scale transformation is considered to assure monotone paths of the degradation trajectories. The comparison results of the reliability analysis, under both dependent and independent assumptions, are reported with the implementation of the proposed modeling in a case study, which consists of the crack propagation data of 2 terminals of an electronic device.     

Estimates of the supremum distribution for a certain class of random processes

Exponential estimates of the tails of supremum distributions are obtained for a certain class of pre-Gaussian random processes. The results obtained are applied to the quadratic forms of Gaussian processes and to processes representable as stochastic integrals of processes with independent increments.Translated from Ukrainskii Matematicheskii Zhurnal, Vol. 45, No. 5, pp. 596–608, May, 1993.     

Sensitivity analysis for product design selection with an implicit value function

In product design selection the decision maker (DM) often does not have enough information about the end users’ needs to state the “preferences” with precision, as is required by many of the existing selection methods. We present, for the case where the DM gives estimates of the preferences, a concept for calculating a “robustness index.” The concept can be used with any iterative selection method that chooses a trial design for each iteration, and uses the DM’s preference parameters at that trial design to eliminate some design options which have lower value than the trial design. Such methods, like our previously published method, are applicable to cases where the DM’s value function is implicit. Our robustness index is a metric of the allowed variation between the actual and estimated preferences for which the set of non-eliminated trial designs (which could be singleton) will not change. The DM, through experience, can use the robustness index and other information generated in calculating the index to determine what action to take: make a final selection from the present set of non-eliminated designs; improve the precision of the preference estimates; or otherwise cope with the imprecision. We present an algorithm for finding the robustness index, and demonstrate and verify the algorithm with an engineering example and a numerical example.     

Asymptotic Palm likelihood theory for stationary point processes

In the present paper, we propose a Palm likelihood approach as a general estimating principle for stationary point processes in $\mathbf{R}^d$ for which the density of the second-order factorial moment measure is available in closed form or in an integral representation. Examples of such point processes include the Neyman–Scott processes and the log Gaussian Cox processes. The computations involved in determining the Palm likelihood estimator are simple. Conditions are provided under which the Palm likelihood estimator is strongly consistent and asymptotically normally distributed.     

A test for a hypothesis on the correlation function of Gaussian random processes

A test is proposed for a hypothesis on the correlation function of general Gaussian random processes. The test is based on theorems on estimates of the distribution of the supremum of sample estimators of correlation functions of Gaussian processes. For a wide class of stationary processes formulas are given that allow the test to be used immediately. Bibliography: 4 titles. Translated fromObchyslyuval’na ta Prykladna Matematyka, No. 77, 1993, pp. 61–74.