Backfitting neural networks

Summary  Regression and classification problems can be viewed as special cases of the problem of function estimation. It is rather well known that a two-layer perceptron with sigmoidal transformation functions can approximate any continuous function on the compact subsets ofRP if there are sufficient number of hidden nodes. In this paper, we present an algorithm for fitting perceptron models, which is quite different from the usual backpropagation or Levenberg-Marquardt algorithm. This new algorithm based on backfitting ensures a better convergence than backpropagation. We have also used resampling techniques to select an ideal number of hidden nodes automatically using the training data itself. This resampling technique helps to avoid the problem of overfitting that one faces for the usual perceptron learning algorithms without any model selection scheme. Case studies and simulation results are presented to illustrate the performance of this proposed algorithm.     

A CS-AdaBoost-BP model for product quality inspection

Quality inspection is essential in preventing defective products from entering the market. Due to the typically low percentage of defective products, it is generally challenging to detect them using algorithms that aim for the overall classification accuracy. To help solve this problem, we propose an ensemble learning classification model, where we employ adaptive boosting (AdaBoost) to cascade multiple backpropagation (BP) neural networks. Furthermore, cost-sensitive (CS) learning is introduced to adjust the loss function of the basic classifier of the BP neural network. For clarity, this model is called a CS-AdaBoost-BP model. To empirically verify its effectiveness, we use data from home appliance production lines from Bosch. We carry out tenfold cross-validation to evaluate and compare the performance between the CS-AdaBoost-BP model and three existing models: BP neural network, BP neural network based on sampling, and AdaBoost-BP. The results show that our proposed model not only performs better than the other models but also significantly improves the ability to identify defective products. Furthermore, based on the mean value of the Youden index, our proposed model has the highest stability.

     

On the predictive ability of narrative disclosures in annual reports

We investigate whether narrative disclosures in 10-K and 10K-405 filings contain value-relevant information for predicting market performance. We apply text classification techniques from computer science to machine code text disclosures in a sample of 4280 filings by 1236 firms over five years. Our methodology develops a model using documents and actual performance for a training sample. This model, when applied to documents from a test set, leads to performance prediction. We find that a portfolio based on model predictions earns significantly positive size-adjusted returns, indicating that narrative disclosures contain value-relevant information. Supplementary analyses show that the text classification model captures information not contained in document-level features of clarity, tone and risk sentiment considered in prior research. However, we find that the narrative score is not providing information incremental to traditional predictors such as size, market-to-book and momentum, but rather affects investors’ use of price momentum as a factor that predicts excess returns.     

Criteria satisfaction under measure based uncertainty

A fundamental task in decision-making is the determination, in the face of uncertain information, of the satisfaction of some criteria in terms of a scalar value. Our objective here is to help support this task. We first discuss the process of selecting an uncertainty model for our knowledge, here we emphasize the tradeoff between functionality of the representation and its ability to model our knowledge, cointention. We next discuss the process of scalarization, determining a single value to represent some uncertain value. Some features required of operations used for scalarization are introduced. We look at the scalarization procedures used in probability theory, the expected value, and that used in possibility theory. We then turn to a more general framework for the representation of uncertain information based on a set measure.     

An interval set model for learning rules from incomplete information table

A novel interval set approach is proposed in this paper to induce classification rules from incomplete information table, in which an interval-set-based model to represent the uncertain concepts is presented. The extensions of the concepts in incomplete information table are represented by interval sets, which regulate the upper and lower bounds of the uncertain concepts. Interval set operations are discussed, and the connectives of concepts are represented by the operations on interval sets. Certain inclusion, possible inclusion, and weak inclusion relations between interval sets are presented, which are introduced to induce strong rules and weak rules from incomplete information table. The related properties of the inclusion relations are proved. It is concluded that the strong rules are always true whatever the missing values may be, while the weak rules may be true when missing values are replaced by some certain known values. Moreover, a confidence function is defined to evaluate the weak rule. The proposed approach presents a new view on rule induction from incomplete data based on interval set.     

Active classification using belief functions and information gain maximization

Obtaining reliable estimates of the parameters of a probabilistic classification model is often a challenging problem because the amount of available training data is limited. In this paper, we present a classification approach based on belief functions that makes the uncertainty resulting from limited amounts of training data explicit and thereby improves classification performance. In addition, we model classification as an active information acquisition problem where features are sequentially selected by maximizing the expected information gain with respect to the current belief distribution, thus reducing uncertainty as quickly as possible. For this, we consider different measures of uncertainty for belief functions and provide efficient algorithms for computing them. As a result, only a small subset of features need to be extracted without negatively impacting the recognition rate. We evaluate our approach on an object recognition task where we compare different evidential and Bayesian methods for obtaining likelihoods from training data and we investigate the influence of different uncertainty measures on the feature selection process.     

Identifying predictive hubs to condense the training set of k-nearest neighbour classifiers

The $k$ -Nearest Neighbour classifier is widely used and popular due to its inherent simplicity and the avoidance of model assumptions. Although the approach has been shown to yield a near-optimal classification performance for an infinite number of samples, a selection of the most decisive data points can improve the classification accuracy considerably in real settings with a limited number of samples. At the same time, a selection of a subset of representative training samples reduces the required amount of storage and computational resources. We devised a new approach that selects a representative training subset on the basis of an evolutionary optimization procedure. This method chooses those training samples that have a strong influence on the correct prediction of other training samples, in particular those that have uncertain labels. The performance of the algorithm is evaluated on different data sets. Additionally, we provide graphical examples of the selection procedure.     

Optimally-informative path planning for dynamic Bayesian classification

An agent, consisting of an unmanned aerial vehicle (UAV) carrying strapped-down sensors, is to examine a number of unidentified objects within a given search area, collect information, and utilize that information to classify the objects. The problem is challenging because the mission time is often limited, the agent is only provided with partial a priori information, and the amount of information that the sensor can measure is dependent on the relative position of the agent with respect to the object. Our technical approach is three-fold. First, we model the motion of the agent using a kinematic model with constant altitude. Second, we use a performance prediction model that gives the probability of target discrimination as a function of the range from the sensor to the object. Third, a linear classifier that utilizes Bayes?? theorem diagnoses the status of the objects of interest while an information-theoretic measure is used to quantify the uncertainty in classification. We pose an optimal control problem that minimizes the classification uncertainty while taking differential constraints and the time history of the agent??s steering decisions as the control input. We investigate whether maximizing information by choosing informative paths always minimizes the classification uncertainty.     

Classification systems based on rough sets under the belief function framework

In this paper, we present two classification approaches based on Rough Sets (RS) that are able to learn decision rules from uncertain data. We assume that the uncertainty exists only in the decision attribute values of the Decision Table (DT) and is represented by the belief functions. The first technique, named Belief Rough Set Classifier (BRSC), is based only on the basic concepts of the Rough Sets (RS). The second, called Belief Rough Set Classifier, is more sophisticated. It is based on Generalization Distribution Table (BRSC-GDT), which is a hybridization of the Generalization Distribution Table and the Rough Sets (GDT-RS). The two classifiers aim at simplifying the Uncertain Decision Table (UDT) in order to generate significant decision rules for classification process. Furthermore, to improve the time complexity of the construction procedure of the two classifiers, we apply a heuristic method of attribute selection based on rough sets. To evaluate the performance of each classification approach, we carry experiments on a number of standard real-world databases by artificially introducing uncertainty in the decision attribute values. In addition, we test our classifiers on a naturally uncertain web usage database. We compare our belief rough set classifiers with traditional classification methods only for the certain case. Besides, we compare the results relative to the uncertain case with those given by another similar classifier, called the Belief Decision Tree (BDT), which also deals with uncertain decision attribute values.     

Robust conjugate duality for convex optimization under uncertainty with application to data classification

In this paper we present a robust conjugate duality theory for convex programming problems in the face of data uncertainty within the framework of robust optimization, extending the powerful conjugate duality technique. We first establish robust strong duality between an uncertain primal parameterized convex programming model problem and its uncertain conjugate dual by proving strong duality between the deterministic robust counterpart of the primal model and the optimistic counterpart of its dual problem under a regularity condition. This regularity condition is not only sufficient for robust duality but also necessary for it whenever robust duality holds for every linear perturbation of the objective function of the primal model problem. More importantly, we show that robust strong duality always holds for partially finite convex programming problems under scenario data uncertainty and that the optimistic counterpart of the dual is a tractable finite dimensional problem. As an application, we also derive a robust conjugate duality theorem for support vector machines which are a class of important convex optimization models for classifying two labelled data sets. The support vector machine has emerged as a powerful modelling tool for machine learning problems of data classification that arise in many areas of application in information and computer sciences.     

Uncertain Switching Costs and Purchase Decisions in Electronic Markets

As advances in information technologies (IT) significantly reduce the time and cost of acquiring and processing product information, some buyers that traditionally work with one or few suppliers have switched to the market environment. The rapid growth of e-commerce has led researchers to believe that a uniform shift to the electronic markets is inevitable. This article examines the issue from the perspective of uncertain supplier’s performance, switching costs, and the value of information. We show that the presence of uncertainty and switching costs favors contractual relationships between buyers and suppliers. As IT makes the market more competitive, the marginal value of information diminishes. Meanwhile, the overall effect of IT on uncertainty and switching costs is fairly limited. As a result, buyers facing high uncertain supplier’s performance and switching costs may find working with a small number of suppliers a better choice.     

Robust optimization of uncertain multistage inventory systems with inexact data in decision rules

In production-inventory problems customer demand is often subject to uncertainty. Therefore, it is challenging to design production plans that satisfy both demand and a set of constraints on e.g. production capacity and required inventory levels. Adjustable robust optimization (ARO) is a technique to solve these dynamic (multistage) production-inventory problems. In ARO, the decision in each stage is a function of the data on the realizations of the uncertain demand gathered from the previous periods. These data, however, are often inaccurate; there is much evidence in the information management literature that data quality in inventory systems is often poor. Reliance on data “as is” may then lead to poor performance of “data-driven” methods such as ARO. In this paper, we remedy this weakness of ARO by introducing a model that treats past data itself as an uncertain model parameter. We show that computational tractability of the robust counterparts associated with this extension of ARO is still maintained. The benefits of the new model are demonstrated by a numerical test case of a well-studied production-inventory problem. Our approach is also applicable to other ARO models outside the realm of production-inventory planning.     

A dual-response forwarding approach for containerizing air cargoes under uncertainty,based on stochastic mixed 0-1 programming

This paper proposes a dual-response forwarding approach for renting air containers and simultaneously determining how cargoes are distributed into the containers under uncertain information. Containers have to be booked in advance to obtain a discount rental rate from airlines, as urgent requirement or cancellation of containers on the day of shipping will incur a heavy penalty. We firstly formulate a mixed 0-1 integer model to determine the booking types and quantities of containers for the deterministic problem under accurate information. We then formulate a stochastic mixed 0-1 model to structure a dual-response forwarding system for the uncertain problem where accurate information is not available when booking. The first-stage response is to determine the booking types and quantities of containers. The second-stage response is to prepare for different scenarios that might occur on the day of shipping, including the types and quantities of containers required or returned for each scenario, and also the corresponding cargo loading plan. Computational results show that the stochastic model can provide a cost-efficient, flexible and responsive cargo forwarding system.     

The principal-agent problem with evolutionary learning

We extend agency theory to incorporate bounded rationality of both principals and agents. In this study we define a simple version of the principal-agent game and examine it using object-oriented computer simulation. Player learning is simulated with a genetic algorithm model. Our results show that players of incentive games in highly uncertain environments may take on defensive strategies. These defensive strategies lead to equilibria which are inferior to Nash equilibria. If agents are risk averse, the principal may not be able to provide enough monetary compensation to encourage them to take risks. But principals may be able to improve system performance by identifying good performers and facilitating information exchange among agents.The authors would like to thank the anonymous referees for their helpful suggestions.     

Time-dependent optimization of a multi-item uncertain supply chain network: A hybrid approximation algorithm

We consider the uncertain least cost shipping problem. The input is a multi-item supply chain network with time-evolving uncertain costs and capacities. Exploiting the operational law of uncertainty theory, a mathematical model of the problem is established and the indeterminacy factors are tackled. We use the scaling idea together with transformation approach and uncertainty programming to develop a hybrid algorithm to optimize and obtain the uncertainty distribution of the total shipping cost. We analyze the practical performance of the algorithm and present an illustrative example.     

基于证据推理的动态联盟伙伴选择优化模型

讨论了定性和定量相结合选择动态联盟伙伴的问题，针对权系数信息不完全和指标值不确定提出了一种基于证据推理的优化模型。该模型首先通过证据推理算法将方案的指标值集结，然后将效用值集结，结合不完全信息的权系数建立非线性规划模型，设计了遗传算法来求解，从而选出最满意的伙伴。最后以实例表明该模型的有效性。     

联合订货下基于ø-散度的多产品库存鲁棒优化模型

针对多产品联合库存决策问题,在市场需求不确定条件下,建立了考虑联合订货成本的多产品库存鲁棒优化模型。针对不确定市场需求,采用一系列未知概率的离散情景进行描述,给出了基于最小最大准则的鲁棒对应模型,并证明了(s,S)库存策略的最优性。进一步,在仅知多产品市场需求历史数据基础上,采用基于ø-散度的数据驱动方法构建了满足一定置信度要求的关于未知需求概率分布的不确定集。在此基础上,为获得(s,S)库存策略的相关参数,运用拉格朗日对偶方法将所建模型等价转化为易于求解的数学规划问题。最后,通过数值计算分析了Kullback-Leibler散度和Cressie-Read散度以及不同的置信水平下的多产品库存绩效,并将其与真实分布下应用鲁棒库存策略得到的库存绩效进行对比。结果表明,需求分布信息的缺失虽然会导致一定的库存绩效损失,但损失值很小,表明基于文中方法得到的库存策略能够有效抑制需求不确定性扰动,具有良好的鲁棒性。     

Comparative Analysis of Artificial Neural Network Models: Application in Bankruptcy Prediction

This study compares the predictive performance of three neural network methods, namely the learning vector quantization, the radial basis function, and the feedforward network that uses the conjugate gradient optimization algorithm, with the performance of the logistic regression and the backpropagation algorithm. All these methods are applied to a dataset of 139 matched-pairs of bankrupt and non-bankrupt US firms for the period 1983–1994. The results of this study indicate that the contemporary neural network methods applied in the present study provide superior results to those obtained from the logistic regression method and the backpropagation algorithm.     

The reliability analysis of probabilistic and interval hybrid structural system

The aim of this paper is to evaluate the reliability of probabilistic and interval hybrid structural system. The hybrid structural system includes two kinds of uncertain parameters—probabilistic parameters and interval parameters. Based on the interval reliability model and probabilistic operation, a new probabilistic and interval hybrid reliability model is proposed. Firstly, we use the interval reliability model to analyze the performance function, and then sum up reliability of all regions divided by the failure plane. Based on the presented optimal criterion enumerating the main failure modes of hybrid structural system and the relationship of failure modes, the reliability of structure system can be obtained. By means of the numerical examples, the hybrid reliability model and the traditional probabilistic reliability model are critically contrasted. The results indicate the presented reliability model is more suitable for analysis and design of these structural systems and it can ensure the security of system well, and it only needs less uncertain information.     

基于Fuzzy理论的数据挖掘算法研究Ⅱ

"数据挖掘"是数据处理的一个新领域.支持向量机是数据挖掘的一种新方法,该技术在很多领域得到了成功的应用.但是,支持向量机目前还存在许多局限,当支持向量机的训练集中含有模糊信息时,支持向量机将无能为力.为解决一般情况下支持向量机中含有模糊信息(模糊参数)问题,研究了模糊机会约束规划、模糊分类中的模糊特征及其表示方法,建立了模糊支持向量分类机理论,给出了模糊线性可分的模糊支持向量分类机算法.