Combined small scale high dimensional model representation

Nowadays the utilization of High Dimensional Model Representation (HDMR), which is an algorithm for approximating multivariate functions, is becoming more pervasive in the applications of approximation theory. This extensive usage motivates new works on HDMR, to get better solutions while approximating to the multivariate functions. One of them is recently developed “Combined Small Scale High Dimensional Model Representation (CSSHDMR)". This new scheme not only optimises HDMR results but also provides good approximation with less terms than HDMR does. This paper presents the theory and the numerical results of the new method and shows that it is possible to apply approximation to multivariate functions by keeping only constant term of HDMR. From this aspect CSSHDMR can be used in any scientific problem which includes multivariate functions, from chemistry to statistics.     

High dimensional model representation constructed by support vector regression. I. Independent variables with known probability distributions

High dimensional model representation (HDMR) is a general set of quantitative model assessment and analysis tools for capturing high dimensional input-output system behavior. In practice, the HDMR component functions are each approximated by an appropriate basis function expansion. This procedure often requires many input-output samples which can restrict the treatment of high dimensional systems. In order to address this problem we introduce svr-based HDMR to efficiently and effectively construct the HDMR expansion by support vector regression (SVR) for a function \(f(\mathbf{x})\). In this paper the results for independent variables sampled over known probability distributions are reported. The theoretical foundation of the new approach relies on the kernel used in SVR itself being an HDMR expansion (referred to as the HDMR kernel ), i.e., an ANOVA kernel whose component kernels are mutually orthogonal and all non-constant component kernels have zero expectation. Several HDMR kernels are constructed as illustrations. While preserving the characteristic properties of HDMR, the svr-based HDMR method enables efficient construction of high dimensional models with satisfactory prediction accuracy from a modest number of samples, which also permits accurate computation of the sensitivity indices. A genetic algorithm is employed to optimally determine all the parameters of the component HDMR kernels and in SVR. The svr-based HDMR introduces a new route to advance HDMR algorithms. Two examples are used to illustrate the capability of the method.     

High-dimensional model representations generated from low order terms--lp-RS-HDMR

High-dimensional model representation (HDMR) is a general set of quantitative model assessment and analysis tools for improving the efficiency of deducing high dimensional input-output system behavior. RS-HDMR is a particular form of HDMR based on random sampling (RS) of the input variables. The component functions in an HDMR expansion are optimal choices tailored to the n-variate function f(x) being represented over the desired domain of the n-dimensional vector x. The high-order terms (usually larger than second order, or equivalently beyond cooperativity between pairs of variables) in the expansion are often negligible. When it is necessary to go beyond the first and the second order RS-HDMR, this article introduces a modified low-order term product (lp)-RS-HDMR method to approximately represent the high-order RS-HDMR component functions as products of low-order functions. Using this method the high-order truncated RS-HDMR expansions may be constructed without directly computing the original high-order terms. The mathematical foundations of lp-RS-HDMR are presented along with an illustration of its utility in an atmospheric chemical kinetics model.     

On the fundamental conjecture of HDMR: a Fourier analysis approach

Although the HDMR decomposition has become an important tool for the understanding of high dimensional functions, the fundamental conjecture underlying its practical utility is still open for theoretical analysis. In this paper, we introduce the HDMR decomposition in conjunction with the Fourier-HDMR approximation leading to the following conclusions: (1) we suggest a type of Fourier-HDMR approximation for certain classes of differentiable functions; (2) utilizing the Fourier-HDMR method, we prove the fundamental conjecture about the dominance of low order terms in the HDMR expansion under relevant conditions, and we also obtain error estimates of the truncated HDMR expansion up to order u; (3) we prove the domain decomposition approximation theorem which shows that the global Fourier-HDMR approximation is not always optimal for a given accuracy order; (4) and finally, a piecewise Fourier-HDMR approach is discussed for high dimensional modeling. These results help to further understand how to efficiently represent the high dimensional functions.     

Efficient Implementation of High Dimensional Model Representations

Physical models of various phenomena are often represented by a mathematical model where the output(s) of interest have a multivariate dependence on the inputs. Frequently, the underlying laws governing this dependence are not known and one has to interpolate the mathematical model from a finite number of output samples. Multivariate approximation is normally viewed as suffering from the curse of dimensionality as the number of sample points needed to learn the function to a sufficient accuracy increases exponentially with the dimensionality of the function. However, the outputs of most physical systems are mathematically well behaved and the scarcity of the data is usually compensated for by additional assumptions on the function (i.e., imposition of smoothness conditions or confinement to a specific function space). High dimensional model representations (HDMR) are a particular family of representations where each term in the representation reflects the individual or cooperative contributions of the inputs upon the output. The main assumption of this paper is that for most well defined physical systems the output can be approximated by the sum of these hierarchical functions whose dimensionality is much smaller than the dimensionality of the output. This ansatz can dramatically reduce the sampling effort in representing the multivariate function. HDMR has a variety of applications where an efficient representation of multivariate functions arise with scarce data. The formulation of HDMR in this paper assumes that the data is randomly scattered throughout the domain of the output. Under these conditions and the assumptions underlying the HDMR it is argued that the number of samples needed for representation to a given tolerance is invariant to the dimensionality of the function, thereby providing for a very efficient means to perform high dimensional interpolation. Selected applications of HDMR's are presented from sensitivity analysis and time-series analysis.     

Constancy maximization based weight optimization in high dimensional model representation for multivariate functions

High Dimensional Model Representation (HDMR) method is a technique that represents a multivariate function in terms of less-variate functions. Even though the method has a finite expansion, to determine the components of this expansion is very expensive due to integration based natures of the components. Hence, the HDMR expansion is generally truncated at certain multivariance level and such approximations are produced to represent the given multivariate function approximately. The weight function selection becomes an important issue for the HDMR based applications when it is desired to give different importances to function values at different points. An appropriately chosen weight function may increase the quality of the approximation incredibly. This work aims at a multivariate weight function optimization to obtain high quality approximations through the HDMR method to represent multivariate functions. The proposed optimization considers constancy measurer maximization which produces a quadratic vector equation to be solved. Another contribution of this work is to use a recently developed method, fluctuation free integration, with HDMR, to solve this equation easily. This work is an extension of a previous work about weight optimization in HDMR for univariate functions.     

Multivariate data modelling through Piecewise generalized HDMR method

This work aims to develop a new High Dimensional Model Representation (HDMR) based method which can construct an analytical structure for a given multivariate data modelling problem. Modelling multivariate data through a divide-and-conquer method stands for multivariate data partitioning process in which we deal with a number of less variate data sets instead of a single N dimensional problem. Generalized HDMR is one of these methods used to model a multivariate data set which has a number of scattered nodes with associated function values. However, Generalized HDMR includes a linear equation system with huge number of unknowns and equations to be solved. This equation sometimes has linearly dependent equations in it and this is an undesirable situation. This work offers a new method named Piecewise Generalized HDMR method which bypasses this disadvantage as well as reducing the mathematical complexity and CPU time needed to complete the algorithm of the previous method. Our new method splits the given problem domain into subdomains, applies the Generalized HDMR philosophy to each subdomain and superpositions the information coming from these subdomains. The algorithm of this new method and a number of numerical implementations are given in this paper.     

Random sampling-high dimensional model representation (RS-HDMR) and orthogonality of its different order component functions

High dimensional model representation is under active development as a set of quantitative model assessment and analysis tools for capturing high-dimensional input-output system behavior based on a hierarchy of functions of increasing dimensions. The HDMR component functions are optimally constructed from zeroth order to higher orders step-by-step. This paper extends the definitions of HDMR component functions to systems whose input variables may not be independent. The orthogonality of the higher order terms with respect to the lower order ones guarantees the best improvement in accuracy for the higher order approximations. Therefore, the HDMR component functions are constructed to be mutually orthogonal. The RS-HDMR component functions are efficiently constructed from randomly sampled input-output data. The previous introduction of polynomial approximations for the component functions violates the strictly desirable orthogonality properties. In this paper, new orthonormal polynomial approximation formulas for the RS-HDMR component functions are presented that preserve the orthogonality property. An integrated exposure and dose model as well as ionospheric electron density determined from measured ionosonde data are used as test cases, which show that the new method has better accuracy than the prior one.     

Analytical HDMR formulas for functions expressed as quadratic polynomials with a multivariate normal distribution

High Dimensional Model Representation (HDMR) is a general set of quantitative model assessment and analysis tools for systems with many variables. A general formulation for the HDMR component functions with independent and correlated variables was obtained previously. Since the HDMR component functions generally are coupled to one another and involve multi-dimensional integrals, explicit formulas for the component functions are not available for an arbitrary function with an arbitrary probability distribution amongst their variables. This paper presents analytical formulas for the HDMR component functions and the corresponding sensitivity indexes for the common case of a function expressed as a quadratic polynomial with a multivariate normal distribution over its variables. This advance is important for practical applications of HDMR with correlated variables.     

High Dimensional Model Representations Generated from Low Dimensional Data Samples. I. mp-Cut-HDMR

High dimensional model representation (HDMR) is a general set of quantitative model assessment and analysis tools for improving the efficiency of deducing high dimensional input–output system behavior. For a high dimensional system, an output f(x) is commonly a function of many input variables x=|x ₁,x ₂,...,x _n } with n10² or larger. HDMR describes f(x) by a finite hierarchical correlated function expansion in terms of the input variables. Various forms of HDMR can be constructed for different purposes. Cut- and RS-HDMR are two particular HDMR expansions. Since the correlated functions in an HDMR expansion are optimal choices tailored to f(x) over the entire domain of x, the high order terms (usually larger than second order, or beyond pair cooperativity) in the expansion are often negligible. When the approximations given by the first and the second order Cut-HDMR correlated functions are not adequate, this paper presents a monomial based preconditioned HDMR method to represent the higher order terms of a Cut-HDMR expansion by expressions similar to the lower order ones with monomial multipliers. The accuracy of the Cut-HDMR expansion can be significantly improved using preconditioning with a minimal number of additional input–output samples without directly invoking the determination of higher order terms. The mathematical foundations of monomial based preconditioned Cut-HDMR is presented along with an illustration of its applicability to an atmospheric chemical kinetics model.     

Hybrid HDMR method with an optimized hybridity parameter in multivariate function representation

High Dimensional Model Representation (HDMR) based methods are used to generate an approximation for a given multivariate function in terms of less variate functions. This paper focuses on Hybrid HDMR which is composed of Plain HDMR and Logarithmic HDMR. The Plain HDMR method works well for representing multivariate functions having additive nature. If the function under consideration has a multiplicative nature, then the Logarithmic HDMR method produces better approximation. Hybrid HDMR method aims to successfully represent a multivariate function having neither purely additive nor purely multiplicative nature under a hybridity parameter. The performance of the Hybrid HDMR method strongly depends on the value of this hybridity parameter because this parameter manages the contribution level of Plain and Logarithmic HDMR expansions. The main purpose of this work is to optimize the hybridity parameter to get the best approximations. Fluctuationlessness Approximation Theorem is used in this optimization process and in evaluating the multiple integrals appearing in HDMR based methods. A number of numerical implementations are given at the end of the paper to show the performance of our proposed method.     

Parametrization of analytic interatomic potential functions using neural networks

A generalized method that permits the parameters of an arbitrary empirical potential to be efficiently and accurately fitted to a database is presented. The method permits the values of a subset of the potential parameters to be considered as general functions of the internal coordinates that define the instantaneous configuration of the system. The parameters in this subset are computed by a generalized neural network (NN) with one or more hidden layers and an input vector with at least 3n-6 elements, where n is the number of atoms in the system. The Levenberg-Marquardt algorithm is employed to efficiently affect the optimization of the weights and biases of the NN as well as all other potential parameters being treated as constants rather than as functions of the input coordinates. In order to effect this minimization, the usual Jacobian employed in NN operations is modified to include the Jacobian of the computed errors with respect to the parameters of the potential function. The total Jacobian employed in each epoch of minimization is the concatenation of two Jacobians, one containing derivatives of the errors with respect to the weights and biases of the network, and the other with respect to the constant parameters of the potential function. The method provides three principal advantages. First, it obviates the problem of selecting the form of the functional dependence of the parameters upon the system's coordinates by employing a NN. If this network contains a sufficient number of neurons, it will automatically find something close to the best functional form. This is the case since Hornik et al., [Neural Networks 2, 359 (1989)] have shown that two-layer NNs with sigmoid transfer functions in the first hidden layer and linear functions in the output layer are universal approximators for analytic functions. Second, the entire fitting procedure is automated so that excellent fits are obtained rapidly with little human effort. Third, the method provides a procedure to avoid local minima in the multidimensional parameter hyperspace. As an illustrative example, the general method has been applied to the specific case of fitting the ab initio energies of Si(5) clusters that are observed in a molecular dynamics (MD) simulation of the machining of a silicon workpiece. The energies of the Si(5) configurations obtained in the MD calculations are computed using the B3LYP procedure with a 6-31G(**) basis set. The final ab initio database, which comprises the density functional theory energies of 10 202 Si(5) clusters, is fitted to an empirical Tersoff potential containing nine adjustable parameters, two of which are allowed to be the functions of the Si(5) configuration. The fitting error averaged over all 10 202 points is 0.0148 eV (1.43 kJ mol(-1)). This result is comparable to the accuracy achieved by more general fitting methods that do not rely on an assumed functional form for the potential surface.     

A novel method for multivariate data modelling: Piecewise Generalized EMPR

A multivariate data modelling problem consists of a number of nodes with associated function values. Increase in multivariance urges us to use divide-and-conquer algorithms in modelling process of these problems. High dimensional model representation based methods can partition a given multivariate data set into less-variate data sets and have the ability of building a model through these partitioned data sets. Generalized HDMR (GHDMR) is one of these methods and it is known that it works well for dominantly and purely additive natures. Piecewise Generalized HDMR is an alternative method and was developed to increase the efficiency of GHDMR but the performance of the method for modelling multiplicative natures is still not sufficient and acceptable. This work aims to develop a new piecewise method based on enhanced multivariance product representation which works well for representing multiplicative natures.     

Fluctuation free multivariate integration based logarithmic HDMR in multivariate function representation

This paper focuses on the Logarithmic High Dimensional Model Representation (Logarithmic HDMR) method which is a divide–and–conquer algorithm developed for multivariate function representation in terms of less-variate functions to reduce both the mathematical and the computational complexities. The main purpose of this work is to bypass the evaluation of N–tuple integrations appearing in Logarithmic HDMR by using the features of a new theorem named as Fluctuationlessness Approximation Theorem. This theorem can be used to evaluate the complicated integral structures of any scientific problem whose values can not be easily obtained analytically and it brings an approximation to the values of these integrals with the help of the matrix representation of functions. The Fluctuation Free Multivariate Integration Based Logarithmic HDMR method gives us the ability of reducing the complexity of the scientific problems of chemistry, physics, mathematics and engineering. A number of numerical implementations are also given at the end of the paper to show the performance of this new method.     

Using redundant coordinates to represent potential energy surfaces with lower-dimensional functions

We propose a method for fitting potential energy surfaces with a sum of component functions of lower dimensionality. This form facilitates quantum dynamics calculations. We show that it is possible to reduce the dimensionality of the component functions by introducing new and redundant coordinates obtained with linear transformations. The transformations are obtained from a neural network. Different coordinates are used for different component functions and the new coordinates are determined as the potential is fitted. The quality of the fits and the generality of the method are illustrated by fitting reference potential surfaces of hydrogen peroxide and of the reaction OH+H(2)-->H(2)O+H.     

Cis-->trans, trans-->cis isomerizations and N-O bond dissociation of nitrous acid (HONO) on an ab initio potential surface obtained by novelty sampling and feed-forward neural network fitting

The isomerization and dissociation dynamics of HONO are investigated on an ab initio potential surface obtained by fitting the results of electronic structure calculations at 21 584 configurations by using previously described novelty sampling and feed-forward neural network (NN) methods. The electronic structure calculations are executed by using GAUSSIAN 98 with a 6-311G(d) basis set at the MP4(SDQ) level of accuracy. The average absolute error of the NN fits varies from 0.012 eV (1.22 kJ mol(-1)) to 0.017 eV (1.64 kJ mol(-1)). The average computation time for a HONO trajectory using a single NN surface is approximately 4.8 s. These computation times compare very favorably with those required by other methods primarily because the NN fitting needs to be executed only one time rather than at every integration point. If the average result obtained from a committee of NNs is employed at each point rather than a single NN, increased fitting accuracy can be achieved at the expense of increased computational requirements. In the present investigation, we find that a committee comprising five NN potentials reduces the average absolute interpolation error to 0.0111 eV (1.07 kJ mol(-1)). Cis-trans isomerization rates with total energy of 1.70 eV (including zero point energy) have been computed for a variety of different initial distributions of the internal energy. In contrast to results previously reported by using an empirical potential, where cis-->trans to trans-->cis rate coefficient ratios at 1.70 eV total energy were found to lie in the range of 2.0-12.9 depending on the vibration mode excited, these ratios on the ab initio NN potential lie in the range of 0.63-1.94. It is suggested that this result is a reflection of much larger intramode coupling terms present in the ab initio potential surface. A direct consequence of this increased coupling is a significant decrease in the mode specific rate enhancement when compared to results obtained by using empirical surfaces. All isomerizations are found to be first order in accordance with the results reported by using empirical potentials. The dissociation rate to NO+OH has been investigated at internal HONO energies of 3.10 and 3.30 eV for different distributions of this energy among the six vibrational modes of HONO. These dissociations are also found to be first order. The computed dissociation rate coefficients exhibit only modest mode specific rate enhancement that is significantly smaller than that obtained on an empirical surface because of the much larger mode couplings present on the ab initio surface.     

Estimation of molecular properties by high-dimensional model representation

Additivity models have been widely employed to approximate unknown molecular properties based on previously measured or calculated data for similar molecules. This paper proposes an improved formulation of additivity, which is based on high-dimensional model representation (HDMR). HDMR is a general function-mapping technique that expresses the output of a multivariate system in terms of a hierarchy of cooperative effects among its input variables. HDMR rests on the general observation that, for many physical systems, only relatively low-order input variable cooperativity is significant. A molecule is expressed as a multivariate system by defining binary-valued input variables corresponding to the presence or absence of a chemical bond, with the molecular property as the output. Conventional additivity decomposes a molecular property into contributions from nonoverlapping subcomponents of fixed size. On the other hand, HDMR decomposes a molecular property into the exact contributions from the full hierarchy of its variable-sized subcomponents and contains additivity as a special case. The complete hierarchical structure of HDMR can in many cases lead to a much more accurate estimate than conventional additivity. Also, when full group additivity is not possible, HDMR gives an expression for a lower-order approximation for the missing group additivity value, greatly expanding the scope of HDMR compared to additivity. The component terms in an HDMR approximation have well-defined physical significance. Moreover, HDMR gives an exact expression for the truncation error in any given HDMR approximation, also with a well-defined physical significance. The HDMR model is tested for the enthalpy of formation of a broad range of organic molecules, and its advantages over additivity are illustrated.     

General formulation of HDMR component functions with independent and correlated variables

The High Dimensional Model Representation (HDMR) technique decomposes an n-variate function f (x) into a finite hierarchical expansion of component functions in terms of the input variables x = (x ₁, x ₂, . . . , x _n ). The uniqueness of the HDMR component functions is crucial for performing global sensitivity analysis and other applications. When x ₁, x ₂, . . . , x _n are independent variables, the HDMR component functions are uniquely defined under a specific so called vanishing condition. A new formulation for the HDMR component functions is presented including cases when x contains correlated variables. Under a relaxed vanishing condition, a general formulation for the component functions is derived providing a unique HDMR decomposition of f (x) for independent and/or correlated variables. The component functions with independent variables are special limiting cases of the general formulation. A novel numerical method is developed to efficiently and accurately determine the component functions. Thus, a unified framework for the HDMR decomposition of an n-variate function f (x) with independent and/or correlated variables is established. A simple three variable model with a correlated normal distribution of the variables is used to illustrate this new treatment.     

Potential energy surface and MULTIMODE vibrational analysis of C2H3+

A full dimensional, ab initio-based semiglobal potential energy surface for C(2)H(3) (+) is reported. The ab initio electronic energies for this molecule are calculated using the spin-restricted, coupled cluster method restricted to single and double excitations with triples corrections [RCCSD(T)]. The RCCSD(T) method is used with the correlation-consistent polarized valence triple-zeta basis augmented with diffuse functions (aug-cc-pVTZ). The ab initio potential energy surface is represented by a many-body (cluster) expansion, each term of which uses functions that are fully invariant under permutations of like nuclei. The fitted potential energy surface is validated by comparing normal mode frequencies at the global minimum and secondary minimum with previous and new direct ab initio frequencies. The potential surface is used in vibrational analysis using the "single-reference" and "reaction-path" versions of the code MULTIMODE.     

Ratio control variate method for efficiently determining high-dimensional model representations

The High-Dimensional Model Representation (HDMR) technique is a family of approaches to efficiently interpolate high-dimensional functions. RS(Random Sampling)-HDMR is a practical form of HDMR based on randomly sampling the overall function, and utilizing orthonormal polynomial expansions to approximate the RS-HDMR component functions. The determination of the expansion coefficients for the component functions employs Monte Carlo integration, which controls the accuracy of the RS-HDMR interpolation. The control variate method is an established approach to improve the accuracy of Monte Carlo integration. However, this method is often not practical for an arbitrary function f(x) because there is no general way to find the analytical control variate function h(x), which needs to be very similar to f(x). In this article, we show that truncated RS-HDMR expansions can be used as h(x) for arbitrary f(x), and a more stable iterative ratio control variate method was developed for the determination of the expansion coefficients for the RS-HDMR component functions. As the asymptotic error (standard deviation) of the estimator given by the ratio control variate method is proportional to 1/N(sample size), it is more efficient than the direct Monte Carlo integration, whose error is proportional to 1/square root(N). In an illustration of a four-dimensional atmospheric model a few hundred random samples are sufficient to construct an RS-HDMR expansion by the ratio control variate method with an accuracy comparable to that obtained by direct Monte Carlo integration with thousands of samples.