Restructuring forward step of MARS algorithm using a new knot selection procedure based on a mapping approach

In high dimensional data modeling, Multivariate Adaptive Regression Splines (MARS) is a popular nonparametric regression technique used to define the nonlinear relationship between a response variable and the predictors with the help of splines. MARS uses piecewise linear functions for local fit and apply an adaptive procedure to select the number and location of breaking points (called knots). The function estimation is basically generated via a two-stepwise procedure: forward selection and backward elimination. In the first step, a large number of local fits is obtained by selecting large number of knots via a lack-of-fit criteria; and in the latter one, the least contributing local fits or knots are removed. In conventional adaptive spline procedure, knots are selected from a set of all distinct data points that makes the forward selection procedure computationally expensive and leads to high local variance. To avoid this drawback, it is possible to restrict the knot points to a subset of data points. In this context, a new method is proposed for knot selection which bases on a mapping approach like self organizing maps. By this method, less but more representative data points are become eligible to be used as knots for function estimation in forward step of MARS. The proposed method is applied to many simulated and real datasets, and the results show that it proposes a time efficient forward step for the knot selection and model estimation without degrading the model accuracy and prediction performance.     

A comparison of neural networks and linear scoring models in the credit union environment

The purpose of the present paper is to explore the ability of neural networks such as multilayer perceptrons and modular neural networks, and traditional techniques such as linear discriminant analysis and logistic regression, in building credit scoring models in the credit union environment. Also, since funding and small sample size often preclude the use of customized credit scoring models at small credit unions, we investigate the performance of generic models and compare them with customized models. Our results indicate that customized neural networks offer a very promising avenue if the measure of performance is percentage of bad loans correctly classified. However, if the measure of performance is percentage of good and bad loans correctly classified, logistic regression models are comparable to the neural networks approach. The performance of generic models was not as good as the customized models, particularly when it came to correctly classifying bad loans. Although we found significant differences in the results for the three credit unions, our modular neural network could not accommodate these differences, indicating that more innovative architectures might be necessary for building effective generic models.     

Linear Least Squares Estimation of Regression Models for Two-Dimensional Random Fields

We consider the problem of estimating regression models of two-dimensional random fields. Asymptotic properties of the least squares estimator of the linear regression coefficients are studied for the case where the disturbance is a homogeneous random field with an absolutely continuous spectral distribution and a positive and piecewise continuous spectral density. We obtain necessary and sufficient conditions on the regression sequences such that a linear estimator of the regression coefficients is asymptotically unbiased and mean square consistent. For such regression sequences the asymptotic covariance matrix of the linear least squares estimator of the regression coefficients is derived.     

Detecting and modeling nonlinearity in the gas furnace data

Using a modification of the Hinich, J Time Ser Anal 3(3):169–176, (1982) bispectrum test for nonlinearity and Gaussianity, the residuals of the Tiao and Box, J Am Stat Assoc 76:802–816, (1981) constrained and unconstrained VAR models for the gas furnace data reject the assumption of Gaussianity and linearity over a grid of bandwidths for estimating the bispectrum. These findings call into question the specification of the linear VAR and VARMA models assumed by Tiao and Box, J Am Stat Assoc 76:802–816, (1981). Utilizing the alternative Hinich J Nonparametr Stat 6:205–221, (1996) nonlinearity test, the residuals of the VAR model were shown to exhibit episodic nonlinearity. The sensitivity of the findings to outliers is investigated by estimating and testing the residuals of L1 and MINIMAX models from 1–6 lags. Building on the linear dynamic specification, a multivariate adaptive regression splines (MARS) model is estimated, using two software implementations, and shown to remove the nonlinearity in the residuals. Leverage plots were used to illustrate the “cost” of imposing a linearity assumption. Out-of-sample forecasting tests from 1–6 periods ahead found that using the sum-of-squared errors criteria, the MARS model out performed ACE, GAM and projection pursuit models.     

A Generalized Estimating Equation Approach to Multivariate Adaptive Regression Splines

Multivariate adaptive regression splines (MARS) is a popular nonparametric regression tool often used for prediction and for uncovering important data patterns between the response and predictor variables. The standard MARS algorithm assumes responses are normally distributed and independent, but in this article we relax both of these assumptions by extending MARS to generalized estimating equations. We refer to this MARS-for-GEEs algorithm as “MARGE.” Our algorithm makes use of fast forward selection techniques, such that in the univariate case, MARGE has similar computation speed to a standard MARS implementation. Through simulation we show that the proposed algorithm has improved predictive performance than the original MARS algorithm when using correlated and/or nonnormal response data. MARGE is also competitive with alternatives in the literature, especially for problems with multiple interacting predictors. We apply MARGE to various ecological examples with different data types. Supplementary material for this article is available online.     

Modified piecewise linear DEA model

Recently, Cook and Zhu have proposed the Piecewise Linear Data Envelopment Analysis (PL-DEA) model, a situation in which a generalization of the DEA methodology which incorporates piecewise linear functions of factors is considered. Standard DEA models provide an efficiency score and targets for an inefficient unit, but the PL-DEA model fails to produce acceptable targets. Thus, this issue has been considered in the piecewise linear CCR model, in which a non-increasing set of multipliers describe the weight function. Also, the piecewise linear CCR model has been enhanced by introducing two MIP models for a two-stage procedure in order to set targets precisely. Furthermore as it follows, the above-mentioned models are compared with each other and an example is provided for the sake of lucidity.     

Analytical solution for a class of learning by doing models with multiplicative uncertainty

We find the closed form optimal solution for a class of learning by doing models, where multiplicative uncertainty is introduced in a piecewise linear cost reduction function. Previous literature does not find the closed form optimal solution for these models. We consider a monopolist, facing a linear demand function. The optimal policy for the resulting problem is shown to be piecewise linear and continuous. The optimal output increases with unit cost for certain values of the latter. Numerical examples are provided.     

Solution of Complementarity Problems via Piecewise Linearization

My master thesis concerns the solution linear complementarity problems (LCP). The Lemke algorithm, the most commonly used algorithm for solving a LCP until this day, was compared with the piecewise Newton method (PLN algorithm). The piecewise Newton method is an algorithm to solve a piecewise linear system on the basis of damped Newton methods. The linear complementarity problem is formulated as a piecewise linear system for the applicability of the PLN algorithm. Then, different application examples will be presented, solved with the PLN algorithm. As a result of the findings (of my master thesis) it can be assumed that – under the condition of coherent orientation – the PLN-algorithm requires fewer iterations to solve a linear complementarity problem than the Lemke algorithm. The coherent orientation for piecewise linear problems corresponds for linear complementarity problems to the P-matrix-property. (© 2013 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim)     

On Lyapunov equivalence of linear differential systems with unbounded coefficients

We show that any linear homogeneous differential system can be reduced by some linear piecewise differentiable transformation whose matrix, together with its inverse, is bounded on the half-line to a system with piecewise constant coefficients of the same growth order, and any system with a uniformly small perturbation can be reduced by this linear transformation to the same system with a piecewise constant perturbation of the same smallness.     

Suppressing grazing chaos in impacting system by structural nonlinearity

In this note we investigate the influence of structural nonlinearity of a simple cantilever beam impacting system on its dynamic responses close to grazing incidence by a means of numerical simulation. To obtain a clear picture of this effect we considered two systems exhibiting impacting motion, where the primary stiffness is either linear (piecewise linear system) or nonlinear (piecewise nonlinear system). Two systems were studied by constructing bifurcation diagrams, basins of attractions, Lyapunov exponents and parameter plots. In our analysis we focused on the grazing transitions from no impact to impact motion. We observed that the dynamic responses of these two similar systems are qualitatively different around the grazing transitions. For the piecewise linear system, we identified on the parameter space a considerable region with chaotic behaviour, while for the piecewise nonlinear system we found just periodic attractors. We postulate that the structural nonlinearity of the cantilever impacting beam suppresses chaos near grazing.     

Synthesis of Terminal Control by Piecewise Continuous Systems Using the Final State Method

A method based on the theory of final state models is proposed for synthesis of terminal control by a linear piecewise continuous system with a quadratic quality criterion. An example of the runaway control of a dc electrical actuator is presented.     

An algorithm for the estimation of a regression function by continuous piecewise linear functions

The problem of the estimation of a regression function by continuous piecewise linear functions is formulated as a nonconvex, nonsmooth optimization problem. Estimates are defined by minimization of the empirical L ₂ risk over a class of functions, which are defined as maxima of minima of linear functions. An algorithm for finding continuous piecewise linear functions is presented. We observe that the objective function in the optimization problem is semismooth, quasidifferentiable and piecewise partially separable. The use of these properties allow us to design an efficient algorithm for approximation of subgradients of the objective function and to apply the discrete gradient method for its minimization. We present computational results with some simulated data and compare the new estimator with a number of existing ones.     

A new approach to multivariate adaptive regression splines by using Tikhonov regularization and continuous optimization

This paper introduces a model-based approach to the important data mining tool Multivariate adaptive regression splines (MARS), which has originally been organized in a more model-free way. Indeed, MARS denotes a modern methodology from statistical learning which is important in both classification and regression, with an increasing number of applications in many areas of science, economy and technology. It is very useful for high-dimensional problems and shows a great promise for fitting nonlinear multivariate functions. The MARS algorithm for estimating the model function consists of two algorithms, these are the forward and the backward stepwise algorithm. In our paper, we propose not to use the backward stepwise algorithm. Instead, we construct a penalized residual sum of squares for MARS as a Tikhonov regularization problem which is also known as ridge regression. We treat this problem using continuous optimization techniques which we consider to become an important complementary technology and model-based alternative to the concept of the backward stepwise algorithm. In particular, we apply the elegant framework of conic quadratic programming. This is an area of convex optimization which is very well-structured, herewith, resembling linear programming and, hence, permitting the use of powerful interior point methods. Based on these theoretical and algorithmical studies, this paper also contains an application to diabetes data. We evaluate and compare the performance of the established MARS and our new CMARS in classifying diabetic persons, where CMARS turns out to be very competitive and promising.     

Piecewise Linear Time Series Estimation with GRASP

This paper describes a heuristic to build piecewise linear statistical models with multivariate thresholds, based on a Greedy Randomized Adaptive Search Procedure (GRASP). GRASP is an iterative randomized sampling technique that has been shown to quickly produce good quality solutions for a wide variety of optimization problems. In this paper we describe a GRASP to sequentially split an n-dimensional space in order to build a piecewise linear time series model.     

Prediction of work-related accidents according to working conditions using support vector machines

Support vector machines (SVMs), which are a kind of statistical learning methods, were applied in this research work to predict occupational accidents with success. In the first place, semi-parametric principal component analysis (SPPCA) was used in order to perform a dimensional reduction, but no satisfactory results were obtained. Next, a dimensional reduction was carried out using an innovative and intelligent computing regression algorithm known as multivariate adaptive regression splines (MARS) model with good results. The variables selected as important by the previous MARS model were taken as input variables for a SVM model. This SVM technique was able to classify, according to their working conditions, those workers that have suffered a work-related accident in the last 12 months and those that have not. SVM technique does not over-fit the experimental data and gives place to a better performance than back-propagation neural network models. Finally, the results and conclusions of this study are presented.     

Life Expectancy of Transient Microchaotic Behaviour

The application of digital control may lead to so-called transient chaotic behaviour. In the present paper, we analyse a simple model of a digitally controlled mechanical system, which may create such vibrations. As a consequence of the digital effects, i.e., the sampling and the round-off error, the behaviour of this system can be described by a one-dimensional piecewise linear map. The lifetime of chaotic transients is usually characterized by the so-called escape rate. In the literature, the reciprocal of the escape rate is considered to be the expected duration of the transient chaotic phenomenon. We claim that this approach is not always fruitful, and present a different way of calculating the mean lifetime in the case of one-dimensional piecewise linear maps. Our method might also be used to solve diffusion problems in one-dimensional models of periodic arrays.     

离散系统Lyapunov第二方法的一些问题

本文讨论线性时不变离散系统Lyapunov方程解集的几何性质以及分段线性离散系统的稳定性,得出每个子系统都是稳定的分段线性离散系统渐近稳定的一些充分条件,并把这些结果应用于二阶分段线性系统.