Nonlinear regression modeling and detecting change points via the relevance vector machine

We consider the problem of constructing nonlinear regression models in the case that the structure of data has discontinuities at some unknown points. We propose two-stage procedure in which the change points are detected by relevance vector machine at the first stage, and the smooth curve are effectively estimated along with the technique of regularization method at the second. In order to select tuning parameters in the regularization method, we derive a model selection and evaluation criterion from information-theoretic viewpoints. Simulation results and real data analyses demonstrate that our methodology performs well in various situations.     

Variational schemes for vector eigenvalue problems

We construct and study exact and truncated self-adjoint three-point variational schemes of any degree of accuracy for self-adjoint eigenvalue problems for systems of second-order ordinary differential equations.     

Support vector machine classification with indefinite kernels

We propose a method for support vector machine classification using indefinite kernels. Instead of directly minimizing or stabilizing a nonconvex loss function, our algorithm simultaneously computes support vectors and a proxy kernel matrix used in forming the loss. This can be interpreted as a penalized kernel learning problem where indefinite kernel matrices are treated as noisy observations of a true Mercer kernel. Our formulation keeps the problem convex and relatively large problems can be solved efficiently using the projected gradient or analytic center cutting plane methods. We compare the performance of our technique with other methods on several standard data sets.     

Predictive modeling in a steelmaking process using optimized relevance vector regression and support vector regression

Annals of Operations Research - The existence of contaminants in metal alloys products is the main problem affecting the product quality, which is an important requirement for competitiveness in...     

TT-cross approximation for multidimensional arrays

As is well known, a rank-r matrix can be recovered from a cross of r linearly independent columns and rows, and an arbitrary matrix can be interpolated on the cross entries. Other entries by this cross or pseudo-skeleton approximation are given with errors depending on the closeness of the matrix to a rank-r matrix and as well on the choice of cross. In this paper we extend this construction to d-dimensional arrays (tensors) and suggest a new interpolation formula in which a d-dimensional array is interpolated on the entries of some TT-cross (tensor train-cross). The total number of entries and the complexity of our interpolation algorithm depend on d linearly, so the approach does not suffer from the curse of dimensionality.We also propose a TT-cross method for computation of d-dimensional integrals and apply it to some examples with dimensionality in the range from d=100 up to d=4000 and the relative accuracy of order 10^-10. In all constructions we capitalize on the new tensor decomposition in the form of tensor trains (TT-decomposition).     

Linear classification tikhonov regularization knowledge-based support vector machine for tornado forecasting

A knowledge-based linear Tihkonov regularization classification model for tornado discrimination is presented. Twenty-three attributes, based on the National Severe Storms Laboratory’s Mesoscale Detection Algorithm, are used as prior knowledge. Threshold values for these attributes are employed to discriminate the data into two classes (tornado, non-tornado). The Weather Surveillance Radar 1998 Doppler is used as a source of data streaming every 6 min. The combination of data and prior knowledge is used in the development of a least squares problem that can be solved using matrix or iterative methods. Advantages of this formulation include explicit expressions for the classification weights of the classifier and its ability to incorporate and handle prior knowledge directly to the classifiers. Comparison of the present approach to that of Fung et al. [in Proceedings neural information processing systems (NIPS 2002), Vancouver, BC, December 10–12, 2002], over a suite of forecast evaluation indices, demonstrates that the Tikhonov regularization model is superior for discriminating tornadic from non-tornadic storms.     

Estimation of the misclassification error for multicategory support vector machine classification

The purpose of this paper is to provide an error analysis for the multicategory support vector machine （MSVM） classificaton problems. We establish the uniform convergency approach for MSVMs and estimate the misclassification error. The main difficulty we overcome here is to bound the offset vector. As a result, we confirm that the MSVM classification algorithm with polynomial kernels is always efficient when the degree of the kernel polynomial is large enough. Finally the rate of convergence and examples are given to demonstrate the main results.     

Variational problems with lags

In this paper, we study some questions concerning the minima of the functional $$J\left( y \right) = \int_{x_1 }^{x_2 } {f\left( {x,y\left( x \right),y\left( {x - r} \right),\dot y\left( x \right),\dot y\left( {x - r} \right)} \right)dx} $$ In Section 2, we obtain an analogue to the Jacobi condition to add to the list of previously obtained necessary conditions. A transversality condition is developed in Section 3. In Section 4, we obtain an existence theorem. The techniques used are modifications of those used in the classical problems. In Section 5, we show that the theory of fields for the classical problem fails to work for our problem.     

Variational problems with constraints

Summary The problem considered is that of maximizing dt subject to x=G(x, y), x(0)=c, and0 ≤y≤x. An essentially new feature is determining in what regions y=x,0<y<x, and y=0.     

Variational problems with topological constraints

We introduce a decomposition that captures much of the topology of level sets for functions in certain Sobolev spaces, and allows the definition of an analog of the decreasing rearrangement of a function which respects the topology of level sets. In a variety of settings this decomposition is preserved under weak limits, and so is useful in establishing existence of minimizers of various variational problems with prescribed topological properties. These include variational problems in which ‘topological rearrangements’ are fixed, and ones in which the functional depends on derivatives of rearrangements. Received: December 16, 1998 / Accepted: July 16, 1999     

Variational problems with free interfaces

Summary A large class of problems arise in the material sciences involving free interfaces. To establish the existence and regularity (including the regularity of free interfaces) of solutions has been an important and interesting issue. Here we were able to do so in a model case accounted in optimal designs.The method developed in this paper is rather general and may be useful for many other related problems.     

Quantile regression with group lasso for classification

Applications of regression models for binary response are very common and models specific to these problems are widely used. Quantile regression for binary response data has recently attracted attention and regularized quantile regression methods have been proposed for high dimensional problems. When the predictors have a natural group structure, such as in the case of categorical predictors converted into dummy variables, then a group lasso penalty is used in regularized methods. In this paper, we present a Bayesian Gibbs sampling procedure to estimate the parameters of a quantile regression model under a group lasso penalty for classification problems with a binary response. Simulated and real data show a good performance of the proposed method in comparison to mean-based approaches and to quantile-based approaches which do not exploit the group structure of the predictors.     

Variational problems with obstacles and integral constraints

Various problems in mathematics and physics can be formulated in terms of a variational problem with obstacles and integral constraints, e.g. finding a surface of minimal area with prescribed volume in a bounded region.We are concerned with the regularity of solutions of variational problems: We show that the minima of a variational integral under all Sobolewfunctions with prescribed boundary values, lying between two obstacles, and fulfilling some integral constraints, are bounded and Hölder-continuous. We do not assume any differentiability or convexity of the integrand, but only a Caratheodory and a growth condition.This research has been supported by the Sonderforschungsbereich 72 of the Deutsche Forschungsgemeinschaft.     

A discrete particle swarm optimization method for feature selection in binary classification problems

This paper investigates the feature subset selection problem for the binary classification problem using logistic regression model. We developed a modified discrete particle swarm optimization (PSO) algorithm for the feature subset selection problem. This approach embodies an adaptive feature selection procedure which dynamically accounts for the relevance and dependence of the features included the feature subset. We compare the proposed methodology with the tabu search and scatter search algorithms using publicly available datasets. The results show that the proposed discrete PSO algorithm is competitive in terms of both classification accuracy and computational performance.     

Unsupervised extreme learning machine and support vector regression hybrid model for predicting energy commodity futures index

Support vector regression (SVR) has been successfully applied in various domains, including predicting the prices of different financial instruments like stocks, futures, options, and indices. Because of the wide variation in financial time-series data, instead of using only a single standard prediction technique like SVR, we propose a hybrid model called USELM-SVR. It is a combination of unsupervised extreme learning machine (US-ELM)-based clustering and SVR forecasting. We assessed the feasibility and effectiveness of this hybrid model using a case study, predicting the one-, two-, and three-day ahead closing values of the energy commodity futures index traded on the Multi Commodity Exchange in India. Our experimental results show that the USELM-SVR is viable and effective, and produces better forecasts than our benchmark models (standard SVR, a hybrid of SVR with self-organizing map (SOM) clustering, and a hybrid of SVR with k-means clustering). Moreover, the proposed USELM-SVR architecture is useful as an alternative model for prediction tasks when we require more accurate predictions.