Data mining feature selection for credit scoring models

The features used may have an important effect on the performance of credit scoring models. The process of choosing the best set of features for credit scoring models is usually unsystematic and dominated by somewhat arbitrary trial. This paper presents an empirical study of four machine learning feature selection methods. These methods provide an automatic data mining technique for reducing the feature space. The study illustrates how four feature selection methods—‘ReliefF’, ‘Correlation-based’, ‘Consistency-based’ and ‘Wrapper’ algorithms help to improve three aspects of the performance of scoring models: model simplicity, model speed and model accuracy. The experiments are conducted on real data sets using four classification algorithms—‘model tree (M5)’, ‘neural network (multi-layer perceptron with back-propagation)’, ‘logistic regression’, and ‘k-nearest-neighbours’.     

European generic scoring models using survival analysis

Credit scoring discriminates between ‘good’ and ‘bad’ credit risks to assist credit-grantors in making lending decisions. Such discrimination may not be a good indicator of profit, while survival analysis allows profit to be modelled. The paper explores the application of parametric accelerated failure time and proportional hazards models and Cox non-parametric model to the data from the retail card (revolving credit) from three European countries. The predictive performance of three national models is tested for different timescales of default and then compared to that of a single generic model for a timescale of 25 months. It is found that survival analysis national and generic models produce predictive quality, which is very close to the current industry standard—logistic regression. Stratification is investigated as a way of extending Cox non-parametric proportional hazards model to tackle heterogeneous segments in the population.     

Credit scoring for profitability objectives

In consumer credit markets lending decisions are usually represented as a set of classification problems. The objective is to predict the likelihood of customers ending up in one of a finite number of states, such as good/bad payer, responder/non-responder and transactor/non-transactor. Decision rules are then applied on the basis of the resulting model estimates. However, this represents a misspecification of the true objectives of commercial lenders, which are better described in terms of continuous financial measures such as bad debt, revenue and profit contribution. In this paper, an empirical study is undertaken to compare predictive models of continuous financial behaviour with binary models of customer default. The results show models of continuous financial behaviour to outperform classification approaches. They also demonstrate that scoring functions developed to specifically optimize profit contribution, using genetic algorithms, outperform scoring functions derived from optimizing more general functions such as sum of squared error.     

Towards profitability: a utility approach to the credit scoring problem

Since credit scoring was first applied in the 1940s the standard methodology has been to treat consumer lending decisions as binary classification problems, where the goal has been to make the best possible ‘good/bad’ classification of accounts on the basis of their eventual delinquency status. However, the real goal of commercial lending organizations is to forecast continuous financial measures such as contribution to profit, but there has been little research in this area. In this paper, continuous models of customer worth are compared to binary models of customer repayment behaviour. Empirical results show that while models of customer worth do not perform well in terms of classifying accounts by their good/bad status, they significantly outperform standard classification methodologies when ranking accounts based on their financial worth to lenders.     

Developing a measure of risk adjusted revenue (RAR) in credit cards market: Implications for customer relationship management

Current models of customer lifetime value (CLV) consider the discounted value of profits that a customer generates over an expected lifetime of relationship with the firm. This practice can be misleading in the financial services markets because it ignores the risk posed by the customer (such as delinquency and default). Specifically, in the credit card market, the correlation between revenue and risk is positive. Therefore, firms need to adjust a customer’s profits for the associated risk before developing a measure of customer lifetime value. We propose a new measure, risk adjusted revenue (RAR), that can incorporate multiple sources of risk and demonstrate the usefulness of the proposed measure in correctly assessing the value of a customer in the credit card market. The model can be extended to compute risk adjusted lifetime value (RALTV). We use the RAR metric to understand the effectiveness of different modes of acquisition, and of retention strategies such as affinity cards and reward cards. We find that both reward- and affinity-cardholders generate higher RAR than non-reward and non-affinity cardholders respectively. The ordering of different modes of acquisition with respect to RAR (in decreasing order) is as follows: Internet, direct mail, telesales, and direct selling.     

Two-phase pricing and inventory management for deteriorating and fashion goods under trade credit

This study models a finite horizon inventory problem for deteriorating and fashion goods under trade credit and partial backlogging conditions. Demand may vary with price or time. The supplier can extend credit to the retailer. As a result, the retailer does not have to pay for goods immediately upon acquiring them, and can instead earn interest on the retail price of the goods between the time they are sold and the end of the credit period. The proposed model considers two-phase pricing and inventory decisions. In other words, it determines both the optimal prices and the lengths of the in-stock and stock-out period. This paper is the first to consider different price decisions for in-stock and stock-out periods under trade credit. We develop an algorithm to determine the optimal pricing and replenishment strategy while still maximizing the total profit. Further, this study shows that the proposed two-phase pricing strategy is superior to a one-phase pricing strategy in terms of profit maximization. Computational analysis illustrates the solution procedures and the impacts of the related parameters on decisions and profits. The results of this study can serve as references for business managers or administrators.     

The benefit to consumers from generic scoring models based on credit reports

Received on 1 July 1991. The benefit to consumers from the use of informative creditreports is demonstrated by showing the improvement in creditdecisions when generic scoring models based on credit reportsare implemented. If these models are highly predictive, thenthe truncation of credit reports will reduce the predictivepower of bureau-based generic scoring systems. As a result,more good credit risks will be denied credit, and more poorcredit risks will be granted credit. It is shown that, evenwhen applied to credit applications that had already been screenedand approved, the use of generic scoring models significantlyimproves credit grantors' ability to predict and eliminate bankruptcies,charge-offs, and delinquencies. As applied to existing accounts,bureau-based generic scores are shown to have predictive valuefor at least 3 months, while scores 12 months old may not bevery powerful. Even though bureau-based scores shift towardsthe high-risk end of the distribution during a recession, theycontinue to rank risk very well. When coupled with application-basedcredit-scoring models, scores based on credit-bureau data furtherimprove the predictive power of the model-the improvements beinggreater with more complete bureau information. We conclude thatgovernment-imposed limits on credit information are anti-consumerby fostering more errors in credit decisions.     

Adaptive credit scoring with kernel learning methods

Credit scoring is a method of modelling potential risk of credit applications. Traditionally, logistic regression and discriminant analysis are the most widely used approaches to create scoring models in the industry. However, these methods are associated with quite a few limitations, such as being instable with high-dimensional data and small sample size, intensive variable selection effort and incapability of efficiently handling non-linear features. Most importantly, based on these algorithms, it is difficult to automate the modelling process and when population changes occur, the static models usually fail to adapt and may need to be rebuilt from scratch. In the last few years, the kernel learning approach has been investigated to solve these problems. However, the existing applications of this type of methods (in particular the SVM) in credit scoring have all focused on the batch model and did not address the important problem of how to update the scoring model on-line. This paper presents a novel and practical adaptive scoring system based on an incremental kernel method. With this approach, the scoring model is adjusted according to an on-line update procedure that can always converge to the optimal solution without information loss or running into numerical difficulties. Non-linear features in the data are automatically included in the model through a kernel transformation. This approach does not require any variable reduction effort and is also robust for scoring data with a large number of attributes and highly unbalanced class distributions. Moreover, a new potential kernel function is introduced to further improve the predictive performance of the scoring model and a kernel attribute ranking technique is used that adds transparency in the final model. Experimental studies using real world data sets have demonstrated the effectiveness of the proposed method.     

Modelling the profitability of credit cards by Markov decision processes

This paper derives a Markov decision process model for the profitability of credit cards, which allows lenders to find an optimal dynamic credit limit policy. The states of the system are based on the borrower’s behavioural score and the decisions are what credit limit to give the borrower each period. In determining which Markov chain best describes the borrower’s performance, second order as well as first order Markov chains are considered and estimation procedures developed that deal with the low default levels that may exist in the data. A case study is given in which the optimal credit limit is derived and the results compared with the actual outcomes.     

PHAB scores: proportional hazards analysis behavioural scores

Credit scoring is one of the most widely used applications of quantitative analysis in business. Behavioural scoring is a type of credit scoring that is performed on existing customers to assist lenders in decisions like increasing the balance or promoting new products. This paper shows how using survival analysis tools from reliability and maintenance modelling, specifically Cox's proportional hazards regression, allows one to build behavioural scoring models. Their performance is compared with that of logistic regression. Also the advantages of using survival analysis techniques in building scorecards are illustrated by estimating the expected profit from personal loans. This cannot be done using the existing risk behavioural systems.     

A literature review on the application of evolutionary computing to credit scoring

The last years have seen the development of many credit scoring models for assessing the creditworthiness of loan applicants. Traditional credit scoring methodology has involved the use of statistical and mathematical programming techniques such as discriminant analysis, linear and logistic regression, linear and quadratic programming, or decision trees. However, the importance of credit grant decisions for financial institutions has caused growing interest in using a variety of computational intelligence techniques. This paper concentrates on evolutionary computing, which is viewed as one of the most promising paradigms of computational intelligence. Taking into account the synergistic relationship between the communities of Economics and Computer Science, the aim of this paper is to summarize the most recent developments in the application of evolutionary algorithms to credit scoring by means of a thorough review of scientific articles published during the period 2000–2012.     

Sample selection bias in credit scoring models

One of the aims of credit scoring models is to predict the probability of repayment of any applicant and yet such models are usually parameterised using a sample of accepted applicants only. This may lead to biased estimates of the parameters. In this paper we examine two issues. First, we compare the classification accuracy of a model based only on accepted applicants, relative to one based on a sample of all applicants. We find only a minimal difference, given the cutoff scores for the old model used by the data supplier. Using a simulated model we examine the predictive performance of models estimated from bands of applicants, ranked by predicted creditworthiness. We find that the lower the risk band of the training sample, the less accurate the predictions for all applicants. We also find that the lower the risk band of the training sample, the greater the overestimate of the true performance of the model, when tested on a sample of applicants within the same risk band — as a financial institution would do. The overestimation may be very large. Second, we examine the predictive accuracy of a bivariate probit model with selection (BVP). This parameterises the accept–reject model allowing for (unknown) omitted variables to be correlated with those of the original good–bad model. The BVP model may improve accuracy if the loan officer has overridden a scoring rule. We find that a small improvement when using the BVP model is sometimes possible.     

Efficient frontier cutoff policies in credit portfolios

Historically, account acquisition in scored retail credit and loan portfolios has focused on risk management in the sense of minimizing default losses. We believe that acquisition policies should focus on a broader set of business measures that explicitly recognize tradeoffs between conflicting objectives of losses, volume and profit. Typical business challenges are: ‘How do I maximize portfolio profit while keeping acceptance rate (volume, size) at acceptable levels?’ ‘How do I maximize profit without incurring default losses above a given level?’ ‘How do I minimize the risk of large loss exposures for a given market share?’ In this paper we are not concerned with which combination of objectives are appropriate, but rather focus on the cutoff policies that allow us to capture a number of different portfolio objectives. When there are conflicting objectives we show that optimal policies yield meaningful tradeoffs and efficient frontiers and that optimal shadow prices allow us to develop risk-adjusted tradeoffs between profit and market share. Some of the graphical solutions that we obtain are simple to derive and easy to understand without explicit mathematical formulations but even simple constraints may require formal use of non-linear programming techniques. We concentrate on models and insights that yield decision strategies and cutoff policies rather than the techniques for developing good predictors.     

An empirical comparison of conventional techniques,neural networks and the three stage hybrid Adaptive Neuro Fuzzy Inference System (ANFIS) model for credit scoring analysis: The case of Turkish credit card data

The number of Non-Performing Loans has increased in recent years, paralleling the current financial crisis, thus increasing the importance of credit scoring models. This study proposes a three stage hybrid Adaptive Neuro Fuzzy Inference System credit scoring model, which is based on statistical techniques and Neuro Fuzzy. The proposed model’s performance was compared with conventional and commonly utilized models. The credit scoring models are tested using a 10-fold cross-validation process with the credit card data of an international bank operating in Turkey. Results demonstrate that the proposed model consistently performs better than the Linear Discriminant Analysis, Logistic Regression Analysis, and Artificial Neural Network (ANN) approaches, in terms of average correct classification rate and estimated misclassification cost. As with ANN, the proposed model has learning ability; unlike ANN, the model does not stay in a black box. In the proposed model, the interpretation of independent variables may provide valuable information for bankers and consumers, especially in the explanation of why credit applications are rejected.     

A dynamic theory of the credit union

A topic of recent interest in the retail financial sector has been the growth of credit unions or “pure cooperatives”. Past credit union researchers built mathematical models of credit union operations. These models identified important operating characteristics but were modeled under assumptions of static operating environments. The model presented in this paper departs from the traditional static models and examines dynamic operation for a United States credit union. Its inter-temporal structure clarifies a number of issues—such as optimal equity retention and inter-temporal rate policy—not addressed by earlier studies. Given initial conditions, the model specifies equity retention and inter-temporal deposit and loan rate policies until an equilibrium state is reached.     

A bi‐level neural‐based fuzzy classification approach for credit scoring problems

The credit scoring is a risk evaluation task considered as a critical decision for financial institutions in order to avoid wrong decision that may result in huge amount of losses. Classification models are one of the most widely used groups of data mining approaches that greatly help decision makers and managers to reduce their credit risk of granting credits to customers instead of intuitive experience or portfolio management. Accuracy is one of the most important criteria in order to choose a credit‐scoring model; and hence, the researches directed at improving upon the effectiveness of credit scoring models have never been stopped. In this article, a hybrid binary classification model, namely FMLP, is proposed for credit scoring, based on the basic concepts of fuzzy logic and artificial neural networks (ANNs). In the proposed model, instead of crisp weights and biases, used in traditional multilayer perceptrons (MLPs), fuzzy numbers are used in order to better model of the uncertainties and complexities in financial data sets. Empirical results of three well‐known benchmark credit data sets indicate that hybrid proposed model outperforms its component and also other those classification models such as support vector machines (SVMs), K‐nearest neighbor (KNN), quadratic discriminant analysis (QDA), and linear discriminant analysis (LDA). Therefore, it can be concluded that the proposed model can be an appropriate alternative tool for financial binary classification problems, especially in high uncertainty conditions. © 2013 Wiley Periodicals, Inc. Complexity 18: 46–57, 2013     

The impact of sample bias on consumer credit scoring performance and profitability

This article seeks to gain insight into the influence of sample bias in a consumer credit scoring model. In earlier research, sample bias has been suggested to pose a sizeable threat to predictive performance and profitability due to its implications on either population drainage or biased estimates. Contrary to previous—mainly theoretical—research on sample bias, the unique features of the data set used in this study provide the opportunity to investigate the issue in an empirical setting. Based on the data of a mail-order company offering short-term consumer credit to their consumers, we show that (i) given a certain sample size, sample bias has a significant effect on consumer credit-scoring performance and profitability, (ii) its effect is composed of the inclusion of rejected orders in the scoring model, and—to a lesser extent—the inclusion of these orders into the variable-selection process, and (iii) the impact of the effect of sample bias on consumer credit-scoring performance and profitability is modest.     

A comparison of neural networks and linear scoring models in the credit union environment

The purpose of the present paper is to explore the ability of neural networks such as multilayer perceptrons and modular neural networks, and traditional techniques such as linear discriminant analysis and logistic regression, in building credit scoring models in the credit union environment. Also, since funding and small sample size often preclude the use of customized credit scoring models at small credit unions, we investigate the performance of generic models and compare them with customized models. Our results indicate that customized neural networks offer a very promising avenue if the measure of performance is percentage of bad loans correctly classified. However, if the measure of performance is percentage of good and bad loans correctly classified, logistic regression models are comparable to the neural networks approach. The performance of generic models was not as good as the customized models, particularly when it came to correctly classifying bad loans. Although we found significant differences in the results for the three credit unions, our modular neural network could not accommodate these differences, indicating that more innovative architectures might be necessary for building effective generic models.     

Subagging for credit scoring models

The logistic regression framework has been for long time the most used statistical method when assessing customer credit risk. Recently, a more pragmatic approach has been adopted, where the first issue is credit risk prediction, instead of explanation. In this context, several classification techniques have been shown to perform well on credit scoring, such as support vector machines among others. While the investigation of better classifiers is an important research topic, the specific methodology chosen in real world applications has to deal with the challenges arising from the real world data collected in the industry. Such data are often highly unbalanced, part of the information can be missing and some common hypotheses, such as the i.i.d. one, can be violated. In this paper we present a case study based on a sample of IBM Italian customers, which presents all the challenges mentioned above. The main objective is to build and validate robust models, able to handle missing information, class unbalancedness and non-iid data points. We define a missing data imputation method and propose the use of an ensemble classification technique, subagging, particularly suitable for highly unbalanced data, such as credit scoring data. Both the imputation and subagging steps are embedded in a customized cross-validation loop, which handles dependencies between different credit requests. The methodology has been applied using several classifiers (kernel support vector machines, nearest neighbors, decision trees, Adaboost) and their subagged versions. The use of subagging improves the performance of the base classifier and we will show that subagging decision trees achieve better performance, still keeping the model simple and reasonably interpretable.     

Time will tell: behavioural scoring and the dynamics of consumer credit assessment

This paper discusses the use of dynamic modelling in consumercredit risk assessment. It surveys the approaches and objectivesof behavioural scoring, customer scoring and profit scoring.It then investigates how Markov chain stochastic processes canbe used to model the dynamics of the delinquency status andbehavioural scores of consumers. It discusses the use of segmentation,mover–stayer models and the use of second- and third-ordermodels to improve the fit of such models. The alternative survivalanalysis proportional hazards approach to estimating when defaultoccurs is considered. Comparisons are made between the wayscredit risk is modelled in consumer lending and corporate lending.