A novel piecewise multivariate function approximation method via universal matrix representation

Multivariance in science and engineering causes problematic situations even for continous and discrete cases. One way to overcome this situation is to decrease the multivariance level of the problem by using a divide—and—conquer based method. In this sense, Enhanced Multivariance Product Representation (EMPR) plays a part in the considered scenario and acts successfully. This method brings up a finite expansion to represent a multivariate function in terms of less-variate functions with the assistance of univariate support functions. This work aims to propose a new EMPR based algorithm which has two new features that improves the determination process of each expansion component through Fluctuation Free Integration method, which is an efficient method in evaluating multiple integrals through a universal matrix representation, and increases the approximation quality through inserting a piecewise structure into the standard EMPR algorithm. This new method is called Fluctuation Free Integration based piecewise EMPR. Some numerical implementations are also given to examine the performance of this proposed method.     

Combined small scale high dimensional model representation

Nowadays the utilization of High Dimensional Model Representation (HDMR), which is an algorithm for approximating multivariate functions, is becoming more pervasive in the applications of approximation theory. This extensive usage motivates new works on HDMR, to get better solutions while approximating to the multivariate functions. One of them is recently developed “Combined Small Scale High Dimensional Model Representation (CSSHDMR)". This new scheme not only optimises HDMR results but also provides good approximation with less terms than HDMR does. This paper presents the theory and the numerical results of the new method and shows that it is possible to apply approximation to multivariate functions by keeping only constant term of HDMR. From this aspect CSSHDMR can be used in any scientific problem which includes multivariate functions, from chemistry to statistics.     

Hybrid HDMR method with an optimized hybridity parameter in multivariate function representation

High Dimensional Model Representation (HDMR) based methods are used to generate an approximation for a given multivariate function in terms of less variate functions. This paper focuses on Hybrid HDMR which is composed of Plain HDMR and Logarithmic HDMR. The Plain HDMR method works well for representing multivariate functions having additive nature. If the function under consideration has a multiplicative nature, then the Logarithmic HDMR method produces better approximation. Hybrid HDMR method aims to successfully represent a multivariate function having neither purely additive nor purely multiplicative nature under a hybridity parameter. The performance of the Hybrid HDMR method strongly depends on the value of this hybridity parameter because this parameter manages the contribution level of Plain and Logarithmic HDMR expansions. The main purpose of this work is to optimize the hybridity parameter to get the best approximations. Fluctuationlessness Approximation Theorem is used in this optimization process and in evaluating the multiple integrals appearing in HDMR based methods. A number of numerical implementations are given at the end of the paper to show the performance of our proposed method.     

Weighted tridiagonal matrix enhanced multivariance products representation (WTMEMPR) for decomposition of multiway arrays: applications on certain chemical system data sets

This work focuses on the utilization of a very recently developed decomposition method, weighted tridiagonal matrix enhanced multivariance products representation (WTMEMPR) which can be equivalently used on continuous functions, and, multiway arrays after appropriate unfoldings. This recursive method has been constructed on the Bivariate EMPR and the remainder term of each step therein has been expanded into EMPR from step to step until no remainder term appears in one of the consecutive steps. The resulting expansion can also be expressed in a three factor product representation whose core factor is a tridiagonal matrix. The basic difference and novelty here is the non-constant weight utilization and the applications on certain chemical system data sets to show the efficiency of the WTMEMPR truncation approximants.     

Constancy maximization based weight optimization in high dimensional model representation for multivariate functions

High Dimensional Model Representation (HDMR) method is a technique that represents a multivariate function in terms of less-variate functions. Even though the method has a finite expansion, to determine the components of this expansion is very expensive due to integration based natures of the components. Hence, the HDMR expansion is generally truncated at certain multivariance level and such approximations are produced to represent the given multivariate function approximately. The weight function selection becomes an important issue for the HDMR based applications when it is desired to give different importances to function values at different points. An appropriately chosen weight function may increase the quality of the approximation incredibly. This work aims at a multivariate weight function optimization to obtain high quality approximations through the HDMR method to represent multivariate functions. The proposed optimization considers constancy measurer maximization which produces a quadratic vector equation to be solved. Another contribution of this work is to use a recently developed method, fluctuation free integration, with HDMR, to solve this equation easily. This work is an extension of a previous work about weight optimization in HDMR for univariate functions.     

Multivariate data modelling through Piecewise generalized HDMR method

This work aims to develop a new High Dimensional Model Representation (HDMR) based method which can construct an analytical structure for a given multivariate data modelling problem. Modelling multivariate data through a divide-and-conquer method stands for multivariate data partitioning process in which we deal with a number of less variate data sets instead of a single N dimensional problem. Generalized HDMR is one of these methods used to model a multivariate data set which has a number of scattered nodes with associated function values. However, Generalized HDMR includes a linear equation system with huge number of unknowns and equations to be solved. This equation sometimes has linearly dependent equations in it and this is an undesirable situation. This work offers a new method named Piecewise Generalized HDMR method which bypasses this disadvantage as well as reducing the mathematical complexity and CPU time needed to complete the algorithm of the previous method. Our new method splits the given problem domain into subdomains, applies the Generalized HDMR philosophy to each subdomain and superpositions the information coming from these subdomains. The algorithm of this new method and a number of numerical implementations are given in this paper.     

A high dimensional model representation based numerical method for solving ordinary differential equations

A new numerical method for solving ordinary differential equations by using High Dimensional Model Representation (HDMR) has been developed in this work. Higher order ordinary differential equations can be reduced to a set of first order ODEs. Although HDMR is generally used for multivariate functions, univariate functions are taken into account throughout the work because of the ODEs’ natures. Not the numerical solution but its image under an appropriately chosen linear ordinary differential operator is expressed as a linear combination of the positive deviation powers of independent variable from its initial value. The linear combination of these image functions are expected to form a basis set under consideration. The unknown constants in the linear combination are found by maximizing the constancy measurer formed in terms of the HDMR components after they are evaluated. Results are compared with well-known step size based numerical methods. A semi qualitative error analysis of the proposed method is also established.     

Fluctuation free multivariate integration based logarithmic HDMR in multivariate function representation

This paper focuses on the Logarithmic High Dimensional Model Representation (Logarithmic HDMR) method which is a divide–and–conquer algorithm developed for multivariate function representation in terms of less-variate functions to reduce both the mathematical and the computational complexities. The main purpose of this work is to bypass the evaluation of N–tuple integrations appearing in Logarithmic HDMR by using the features of a new theorem named as Fluctuationlessness Approximation Theorem. This theorem can be used to evaluate the complicated integral structures of any scientific problem whose values can not be easily obtained analytically and it brings an approximation to the values of these integrals with the help of the matrix representation of functions. The Fluctuation Free Multivariate Integration Based Logarithmic HDMR method gives us the ability of reducing the complexity of the scientific problems of chemistry, physics, mathematics and engineering. A number of numerical implementations are also given at the end of the paper to show the performance of this new method.     

Analytical HDMR formulas for functions expressed as quadratic polynomials with a multivariate normal distribution

High Dimensional Model Representation (HDMR) is a general set of quantitative model assessment and analysis tools for systems with many variables. A general formulation for the HDMR component functions with independent and correlated variables was obtained previously. Since the HDMR component functions generally are coupled to one another and involve multi-dimensional integrals, explicit formulas for the component functions are not available for an arbitrary function with an arbitrary probability distribution amongst their variables. This paper presents analytical formulas for the HDMR component functions and the corresponding sensitivity indexes for the common case of a function expressed as a quadratic polynomial with a multivariate normal distribution over its variables. This advance is important for practical applications of HDMR with correlated variables.     

A Matrix Based Indexing HDMR method for multivariate data modelling

Modelling multivariate data of real life problems from engineering, chemistry, physics, mathematics or other related sciences, in which function values are known only at arbitrarily distributed points of the problem domain, is an important and complicated issue since there exist mathematical and computational complexities in the analytical structure construction process coming from the multivariance. The Plain High Dimensional Model Representation (HDMR) method expresses a multivariate problem in terms of less-variate problems. In this work, a Matrix Based Indexing HDMR method is developed to make the Plain HDMR philosophy employable for the multivariate data partitioning process. This new method will have the ability of dealing with less-variate data sets by partitioning the given data set into univariate, bivariate and trivariate data sets. Interpolating these partitioned data sets will construct an approximate analytical structure as the model of the given multivariate data modelling problem.     

Regularized random-sampling high dimensional model representation (RS-HDMR)

High Dimensional Model Representation (HDMR) is under active development as a set of quantitative model assessment and analysis tools for capturing high-dimensional input–output system behavior. HDMR is based on a hierarchy of component functions of increasing dimensions. The Random-Sampling High Dimensional Model Representation (RS-HDMR) is a practical approach to HDMR utilizing random sampling of the input variables. To reduce the sampling effort, the RS-HDMR component functions are approximated in terms of a suitable set of basis functions, for instance, orthonormal polynomials. Oscillation of the outcome from the resultant orthonormal polynomial expansion can occur producing interpolation error, especially on the input domain boundary, when the sample size is not large. To reduce this error, a regularization method is introduced. After regularization, the resultant RS-HDMR component functions are smoother and have better prediction accuracy, especially for small sample sizes (e.g., often few hundred). The ignition time of a homogeneous H₂/air combustion system within the range of initial temperature, 1000 < T ₀ < 1500 K, pressure, 0.1 < P < 100 atm and equivalence ratio of H₂/O₂, 0.2 < R < 10 is used for testing the regularized RS-HDMR.      

Differentially expressed genes selection via Laplacian regularized low-rank representation method

With the rapid development of DNA microarray technology and next-generation technology, a large number of genomic data were generated. So how to extract more differentially expressed genes from genomic data has become a matter of urgency. Because Low-Rank Representation (LRR) has the high performance in studying low-dimensional subspace structures, it has attracted a chunk of attention in recent years. However, it does not take into consideration the intrinsic geometric structures in data.In this paper, a new method named Laplacian regularized Low-Rank Representation (LLRR) has been proposed and applied on genomic data, which introduces graph regularization into LRR. By taking full advantages of the graph regularization, LLRR method can capture the intrinsic non-linear geometric information among the data. The LLRR method can decomposes the observation matrix of genomic data into a low rank matrix and a sparse matrix through solving an optimization problem. Because the significant genes can be considered as sparse signals, the differentially expressed genes are viewed as the sparse perturbation signals. Therefore, the differentially expressed genes can be selected according to the sparse matrix. Finally, we use the GO tool to analyze the selected genes and compare the P-values with other methods.The results on the simulation data and two real genomic data illustrate that this method outperforms some other methods: in differentially expressed gene selection.     

Correlation method for variance reduction of Monte Carlo integration in RS-HDMR

The High Dimensional Model Representation (HDMR) technique is a procedure for efficiently representing high-dimensional functions. A practical form of the technique, RS-HDMR, is based on randomly sampling the overall function and utilizing orthonormal polynomial expansions. The determination of expansion coefficients employs Monte Carlo integration, which controls the accuracy of RS-HDMR expansions. In this article, a correlation method is used to reduce the Monte Carlo integration error. The determination of the expansion coefficients becomes an iteration procedure, and the resultant RS-HDMR expansion has much better accuracy than that achieved by direct Monte Carlo integration. For an illustration in four dimensions a few hundred random samples are sufficient to construct an RS-HDMR expansion by the correlation method with an accuracy comparable to that obtained by direct Monte Carlo integration with thousands of samples.     

Laplacian regularized low-rank representation for cancer samples clustering

Cancer samples clustering based on biomolecular data has been becoming an important tool for cancer classification. The recognition of cancer types is of great importance for cancer treatment. In this paper, in order to improve the accuracy of cancer recognition, we propose to use Laplacian regularized Low-Rank Representation (LLRR) to cluster the cancer samples based on genomic data. In LLRR method, the high-dimensional genomic data are approximately treated as samples extracted from a combination of several low-rank subspaces. The purpose of LLRR method is to seek the lowest-rank representation matrix based on a dictionary. Because a Laplacian regularization based on manifold is introduced into LLRR, compared to the Low-Rank Representation (LRR) method, besides capturing the global geometric structure, LLRR can capture the intrinsic local structure of high-dimensional observation data well. And what is more, in LLRR, the original data themselves are selected as a dictionary, so the lowest-rank representation is actually a similar expression between the samples. Therefore, corresponding to the low-rank representation matrix, the samples with high similarity are considered to come from the same subspace and are grouped into a class. The experiment results on real genomic data illustrate that LLRR method, compared with LRR and MLLRR, is more robust to noise and has a better ability to learn the inherent subspace structure of data, and achieves remarkable performance in the clustering of cancer samples.     

Efficient Implementation of High Dimensional Model Representations

Physical models of various phenomena are often represented by a mathematical model where the output(s) of interest have a multivariate dependence on the inputs. Frequently, the underlying laws governing this dependence are not known and one has to interpolate the mathematical model from a finite number of output samples. Multivariate approximation is normally viewed as suffering from the curse of dimensionality as the number of sample points needed to learn the function to a sufficient accuracy increases exponentially with the dimensionality of the function. However, the outputs of most physical systems are mathematically well behaved and the scarcity of the data is usually compensated for by additional assumptions on the function (i.e., imposition of smoothness conditions or confinement to a specific function space). High dimensional model representations (HDMR) are a particular family of representations where each term in the representation reflects the individual or cooperative contributions of the inputs upon the output. The main assumption of this paper is that for most well defined physical systems the output can be approximated by the sum of these hierarchical functions whose dimensionality is much smaller than the dimensionality of the output. This ansatz can dramatically reduce the sampling effort in representing the multivariate function. HDMR has a variety of applications where an efficient representation of multivariate functions arise with scarce data. The formulation of HDMR in this paper assumes that the data is randomly scattered throughout the domain of the output. Under these conditions and the assumptions underlying the HDMR it is argued that the number of samples needed for representation to a given tolerance is invariant to the dimensionality of the function, thereby providing for a very efficient means to perform high dimensional interpolation. Selected applications of HDMR's are presented from sensitivity analysis and time-series analysis.     

Ratio control variate method for efficiently determining high-dimensional model representations

The High-Dimensional Model Representation (HDMR) technique is a family of approaches to efficiently interpolate high-dimensional functions. RS(Random Sampling)-HDMR is a practical form of HDMR based on randomly sampling the overall function, and utilizing orthonormal polynomial expansions to approximate the RS-HDMR component functions. The determination of the expansion coefficients for the component functions employs Monte Carlo integration, which controls the accuracy of the RS-HDMR interpolation. The control variate method is an established approach to improve the accuracy of Monte Carlo integration. However, this method is often not practical for an arbitrary function f(x) because there is no general way to find the analytical control variate function h(x), which needs to be very similar to f(x). In this article, we show that truncated RS-HDMR expansions can be used as h(x) for arbitrary f(x), and a more stable iterative ratio control variate method was developed for the determination of the expansion coefficients for the RS-HDMR component functions. As the asymptotic error (standard deviation) of the estimator given by the ratio control variate method is proportional to 1/N(sample size), it is more efficient than the direct Monte Carlo integration, whose error is proportional to 1/square root(N). In an illustration of a four-dimensional atmospheric model a few hundred random samples are sufficient to construct an RS-HDMR expansion by the ratio control variate method with an accuracy comparable to that obtained by direct Monte Carlo integration with thousands of samples.     

基于核函数的非线性分类相关分析及其在化学模式识别中的应用

与统计分析和神经网络相比,基于结构风险最小的支持向量机有更好的分类性能。它用于非线性分类时,先将样本映射到更高维的特征空间,往往会增加复共线性与冗余信息,将影响样本分布,降低线性支持向量机分类器(LSVC)的预测性能。本研究提出非线性分类相关分析算法(NLCCA),利用核函数技术,无需了解非线性映射的算式,从特征空间的样本映像中提取分类相关成分,以消除冗余信息,改善样本分布。由此构建的NLCCA-LSVC集成分类器具有优良的预测性能。经模拟数据的测试,并实际用于两个复杂的化学模式识别问题,均取得令人满意的效果,也印证了算法的有效性。     

Variable-weighted least-squares support vector machine for multivariate spectral analysis

Multivariate spectral analysis has been widely applied in chemistry and other fields. Spectral data consisting of measurements at hundreds and even thousands of analytical channels can now be obtained in a few seconds. It is widely accepted that before a multivariate regression model is built, a well-performed variable selection can be helpful to improve the predictive ability of the model. In this paper, the concept of traditional wavelength variable selection has been extended and the idea of variable weighting is incorporated into least-squares support vector machine (LS-SVM). A recently proposed global optimization method, particle swarm optimization (PSO) algorithm is used to search for the weights of variables and the hyper-parameters involved in LS-SVM optimizing the training of a calibration set and the prediction of an independent validation set. All the computation process of this method is automatic. Two real data sets are investigated and the results are compared those of PLS, uninformative variable elimination-PLS (UVE-PLS) and LS-SVM models to demonstrate the advantages of the proposed method.     

Axially symmetric potentials in the oscillator representation

The Wick-ordering method called the Oscillator Representation in the nonrelativistic Schrödinger equation is proposed to calculate the energy spectrum for axially symmetric potentials allowing the existence of a bound state. In particular, the method is applied to calculate the energy spectrum of (2s) states of a hydrogen atom in a uniform magnetic field of an arbitrary strength. In the perturbation (external field) approximation, the energy spectrum of the so-called quadratic and spherical quadratic Zeeman problem and the problem of a hydrogen atom in a generalized van der Waals potential is calculated analytically. The results of the zeroth approximation of oscillator representation are in good agreement with the exact values