Image compression with a dynamic autoassociative neural network

Image compression using neural networks has been attempted with some promise. Among the architectures, feedforward backpropagation networks (FFBPN) have been used in several attempts. Although it is demonstrated that using the mean quadratic error function is equivalent to applying the Karhunen-Loeve transformation method, promise still arises from directed learning possibilities, generalization abilities and performance of the network once trained. In this paper we propose an architecture and an improved training method to attempt to solve some of the shortcomings of traditional data compression systems based on feedforward neural networks trained with backpropagation—the dynamic autoassociation neural network (DANN).The successful application of neural networks to any task requires proper training of the network. In this research, this issue is taken as the main consideration in the design of DANN. We emphasize the convergence of the learning process proposed by DANN. This process provides an escape mechanism, by adding neurons in a random state, to avoid the local minima trapping seen in traditional PFBPN. Also, DANN's training algorithm constrains the error for every pattern to an allowed interval to balance the training for every pattern, thus improving the performance rates in recognition and generalization. The addition of these two mechanisms to DANN's training algorithm has the result of improving the final quality of the images processed by DANN.The results of several tasks presented to DANN-based compression are compared and contrasted with the performance of an FFBPN-based system applied to the same task. These results indicate that DANN is superior to FFBPN when applied to image compression.     

The use of random weights for the training of multilayer networks of neurons with Heaviside characteristics

Artificial neural networks have, in recent years, been very successfully applied in a wide range of areas. A major reason for this success has been the existence of a training algorithm called backpropagation. This algorithm relies upon the neural units in a network having input/output characteristics that are continuously differentiable. Such units are significantly less easy to implement in silicon than are neural units with Heaviside (step-function) characteristics. In this paper, we show how a training algorithm similar to backpropagation can be developed for 2-layer networks of Heaviside units by treating the network weights (i.e., interconnection strengths) as random variables. This is then used as a basis for the development of a training algorithm for networks with any number of layers by drawing upon the idea of internal representations. Some examples are given to illustrate the performance of these learning algorithms.     

Forecasting and recombining time-series components by using neural networks

Operations and other business decisions often depend on accurate time-series forecasts. These time series usually consist of trend-cycle, seasonal, and irregular components. Existing methodologies attempt to first identify and then extrapolate these components to produce forecasts. The proposed process partners this decomposition procedure with neural network methodologies to combine the strengths of economics, statistics, and machine learning research. Stacked generalization first uses transformations and decomposition to pre-process a time series. Then a time-delay neural network receives the resulting components as inputs. The outputs of this neural network are then input to a backpropagation algorithm that synthesizes the processed components into a single forecast. Genetic algorithms guide the architecture selection for both the time-delay and backpropagation neural networks. The empirical examples used in this study reveal that the combination of transformation, feature extraction, and neural networks through stacked generalization gives more accurate forecasts than classical decomposition or ARIMA models.?Scope and Purpose.?The research reported in this paper examines two concurrent issues. The first evaluates the performance of neural networks in forecasting time series. The second assesses the use of stacked generalization as a way of refining this process. The methodology is applied to four economic and business time series. Those studying time series and neural networks, particularly in terms of combining tools from the statistical community with neural network technology, will find this paper relevant.     

Estimating Missing Values Using Neural Networks

The problem of missing values is common in statistical analysis. One approach to deal with missing values is to delete the incomplete cases from the data set. This approach may disregard valuable information, especially in small samples. An alternative approach is to reconstruct the missing values using the information in the data set. The major purpose of this paper is to investigate how a neural network approach performs compared to statistical techniques for reconstructing missing values. The backpropagation algorithm is used as the learning method to reconstruct missing values. The results of back-propagation are compared with results from two methods, viz., (1) using averages, and (2) using iterative regression analysis, to compute missing values. Experimental results show that backpropagation consistently outperforms other methods in both the training and the test data sets, and suggest that the neural network approach is a useful tool for reconstructing missing values in multivariate analysis.     

Benchmarking state-of-the-art classification algorithms for credit scoring

In this paper, we study the performance of various state-of-the-art classification algorithms applied to eight real-life credit scoring data sets. Some of the data sets originate from major Benelux and UK financial institutions. Different types of classifiers are evaluated and compared. Besides the well-known classification algorithms (eg logistic regression, discriminant analysis, k-nearest neighbour, neural networks and decision trees), this study also investigates the suitability and performance of some recently proposed, advanced kernel-based classification algorithms such as support vector machines and least-squares support vector machines (LS-SVMs). The performance is assessed using the classification accuracy and the area under the receiver operating characteristic curve. Statistically significant performance differences are identified using the appropriate test statistics. It is found that both the LS-SVM and neural network classifiers yield a very good performance, but also simple classifiers such as logistic regression and linear discriminant analysis perform very well for credit scoring.     

An adaptive learning rate backpropagation‐type neural network for solving n × n systems on nonlinear algebraic equations

This paper presents an MLP‐type neural network with some fixed connections and a backpropagation‐type training algorithm that identifies the full set of solutions of a complete system of nonlinear algebraic equations with n equations and n unknowns. The proposed structure is based on a backpropagation‐type algorithm with bias units in output neurons layer. Its novelty and innovation with respect to similar structures is the use of the hyperbolic tangent output function associated with an interesting feature, the use of adaptive learning rate for the neurons of the second hidden layer, a feature that adds a high degree of flexibility and parameter tuning during the network training stage. The paper presents the theoretical aspects for this approach as well as a set of experimental results that justify the necessity of such an architecture and evaluate its performance. Copyright © 2015 John Wiley & Sons, Ltd.     

Artificial immune algorithm-based credit evaluation for mobile telephone customers

The arrearage problem is a critical concern for China’s mobile communication services industry. Analysis of customer credit evaluation provides this study with a potential viable solution to the arrearage problem in China. By employing an artificial immune algorithm (AIA), a measure of customer credit based on customer attributes is proposed. This method was applied to one China mobile communication services company with approximately 400?000 customers yielding satisfying results. Utilizing traditional predictive accuracy and alternative metrics, performance comparisons of the proposed AIA were made using the feed-forward back propagation artificial neural network and the logistic regression model. A decision tree analysis of anticipated benefits was performed and indicates workability of the proposed method based on customer credit evaluation.     

Solving regression problem with complementary neural networks and an adjusted averaging technique

This research deals with complementary neural networks (CMTNN) for the regression problem. Complementary neural networks consist of a pair of neural networks called truth neural network and falsity neural network, which are trained to predict truth and falsity outputs, respectively. In this paper, a novel adjusted averaging technique is proposed in order to enhance the result obtained from the basic CMTNN. We test our proposed technique based on the classical benchmark problems including housing, concrete compressive strength, and computer hardware data sets from the UCI machine learning repository. We also realize our technique to the porosity prediction problem based on well log data set obtained from practical field data in the oil and gas industry. We found that our proposed technique provides better performance when compared to the traditional CMTNN, backpropagation neural network, and support vector regression with linear, polynomial, and radial basis function kernels.     

Prediction of work-related accidents according to working conditions using support vector machines

Support vector machines (SVMs), which are a kind of statistical learning methods, were applied in this research work to predict occupational accidents with success. In the first place, semi-parametric principal component analysis (SPPCA) was used in order to perform a dimensional reduction, but no satisfactory results were obtained. Next, a dimensional reduction was carried out using an innovative and intelligent computing regression algorithm known as multivariate adaptive regression splines (MARS) model with good results. The variables selected as important by the previous MARS model were taken as input variables for a SVM model. This SVM technique was able to classify, according to their working conditions, those workers that have suffered a work-related accident in the last 12 months and those that have not. SVM technique does not over-fit the experimental data and gives place to a better performance than back-propagation neural network models. Finally, the results and conclusions of this study are presented.     

Nonlinear dynamic system identification using recurrent neural network with multi-segment piecewise-linear connection weight

This paper introduces a new concept of the connection weight to the standard recurrent neural networks—Elman and Jordan networks. The architecture of the modified networks is the same as that of the original recurrent neural networks. However, unlike the original recurrent neural networks whose connection weight is a single real number, in the modified networks the weight of each connection is multi-valued, depending on the value of the input data involved. The backpropagation learning algorithm is also modified to suit the proposed concept. The modified networks have been benchmarked against the feedforward neural network and the original recurrent neural networks. The experimental results on twelve benchmark problems show that the modified networks are clearly superior to the other three methods.     

基于主分量分析和BP神经网络的个人信用评估模型

运用基于主分量分析和神经网络(PCA-NN)的个人信用评估模型以期取得更好的预测分类能力.经实证分析及与SVM方法、线性判别分析、Logistic回归分析、最近邻估计、分类回归树及神经网络等方法的对比,结果表明,该方法有很好的预测效果.     

A comparison of neural networks and linear scoring models in the credit union environment

The purpose of the present paper is to explore the ability of neural networks such as multilayer perceptrons and modular neural networks, and traditional techniques such as linear discriminant analysis and logistic regression, in building credit scoring models in the credit union environment. Also, since funding and small sample size often preclude the use of customized credit scoring models at small credit unions, we investigate the performance of generic models and compare them with customized models. Our results indicate that customized neural networks offer a very promising avenue if the measure of performance is percentage of bad loans correctly classified. However, if the measure of performance is percentage of good and bad loans correctly classified, logistic regression models are comparable to the neural networks approach. The performance of generic models was not as good as the customized models, particularly when it came to correctly classifying bad loans. Although we found significant differences in the results for the three credit unions, our modular neural network could not accommodate these differences, indicating that more innovative architectures might be necessary for building effective generic models.     

A neural fuzzy control system with structure and parameter learning

A general connectionist model, called neural fuzzy control network (NFCN), is proposed for the realization of a fuzzy logic control system. The proposed NFCN is a feedforward multilayered network which integrates the basic elements and functions of a traditional fuzzy logic controller into a connectionist structure which has distributed learning abilities. The NFCN can be constructed from supervised training examples by machine learning techniques, and the connectionist structure can be trained to develop fuzzy logic rules and find membership functions. Associated with the NFCN is a two-phase hybrid learning algorithm which utilizes unsupervised learning schemes for structure learning and the backpropagation learning scheme for parameter learning. By combining both unsupervised and supervised learning schemes, the learning speed converges much faster than the original backpropagation algorithm. The two-phase hybrid learning algorithm requires exact supervised training data for learning. In some real-time applications, exact training data may be expensive or even impossible to obtain. To solve this problem, a reinforcement neural fuzzy control network (RNFCN) is further proposed. The RNFCN is constructed by integrating two NFCNs, one functioning as a fuzzy predictor and the other as a fuzzy controller. By combining a proposed on-line supervised structure-parameter learning technique, the temporal difference prediction method, and the stochastic exploratory algorithm, a reinforcement learning algorithm is proposed, which can construct a RNFCN automatically and dynamically through a reward-penalty signal (i.e., “good” or “bad” signal). Two examples are presented to illustrate the performance and applicability of the proposed models and learning algorithms.     

Applications of Hybrid Monte Carlo to Bayesian Generalized Linear Models: Quasicomplete Separation and Neural Networks

Abstract

The “leapfrog” hybrid Monte Carlo algorithm is a simple and effective MCMC method for fitting Bayesian generalized linear models with canonical link. The algorithm leads to large trajectories over the posterior and a rapidly mixing Markov chain, having superior performance over conventional methods in difficult problems like logistic regression with quasicomplete separation. This method offers a very attractive solution to this common problem, providing a method for identifying datasets that are quasicomplete separated, and for identifying the covariates that are at the root of the problem. The method is also quite successful in fitting generalized linear models in which the link function is extended to include a feedforward neural network. With a large number of hidden units, however, or when the dataset becomes large, the computations required in calculating the gradient in each trajectory can become very demanding. In this case, it is best to mix the algorithm with multivariate random walk Metropolis—Hastings. However, this entails very little additional programming work.     

我国专利申请量的支持向量机预测模型研究

运用支持向量机(support vector machine,SVM)和浮点遗传算法相结合的方法对我国专利申请量进行预测。数据仿真显示支持向量机预测方法比人工神经网络和逻辑回归方法有更高的预测精度,结果显示运用浮点遗传算法参数选取的支持向量机方法对我国专利申请量进行预测是可行和有效的。     

基于时间序列的混合神经网络数据融合算法

针对传统的数据融合算法对高噪声、大规模、数据结构复杂的时间序列数据融合性能较差的问题,该文提出了一种混合神经网络的数据融合算法(即SCLG算法).SCLG算法的思想如下:首先利用奇异谱分析算法对数据分解重构以达到去噪的目的;其次,通过深层卷积神经网络提取数据的空间特征和短期时间特征;然后,利用长短期记忆(LSTM)网络...     

上市公司财务危机预警分析——基于数据挖掘的研究

本文以我国上市公司为研究对象,选取了1999-2001年被ST的公司和正常公司各73家作为训练样本,2002年被ST的公司和正常公司各43家作为检验样本,分析了财务危机出现前2年内各年两类公司15个财务指标。在进行数据挖掘中,我们运用了三种独立的方法,分别为判别分析、Logistic回归和神经网络,结果发现神经网络预测的效果要优于其它两种方法。最后,结合了这些方法的优点,建立了一种混合模型,研究表明预测的正确性要高于每种单独方法,从而提高了模型的预警效果。     

Data mining feature selection for credit scoring models

The features used may have an important effect on the performance of credit scoring models. The process of choosing the best set of features for credit scoring models is usually unsystematic and dominated by somewhat arbitrary trial. This paper presents an empirical study of four machine learning feature selection methods. These methods provide an automatic data mining technique for reducing the feature space. The study illustrates how four feature selection methods—‘ReliefF’, ‘Correlation-based’, ‘Consistency-based’ and ‘Wrapper’ algorithms help to improve three aspects of the performance of scoring models: model simplicity, model speed and model accuracy. The experiments are conducted on real data sets using four classification algorithms—‘model tree (M5)’, ‘neural network (multi-layer perceptron with back-propagation)’, ‘logistic regression’, and ‘k-nearest-neighbours’.     

Qualitative company performance evaluation: Linear discriminant analysis and neural network models

In this paper, we present a classification model to evaluate the performance of companies on the basis of qualitative criteria, such as organizational and managerial variables. The classification model evaluates the eligibility of the company to receive state subsidies for the development of high tech products. We furthermore created a similar model using the backpropagation learning algorithm and compare its classification performance against the linear model. We also focus on the robustness of the two approaches with respect to uncertain information. This research shows that backpropagation neural networks are not superior to LDA-models (Linear Discriminant Analysis), except when they are given highly uncertain information.