An efficient modified boosting method for solving classification problems

Based on the Adaboost algorithm, a modified boosting method is proposed in this paper for solving classification problems. This method predicts the class label of an example as the weighted majority voting of an ensemble of classifiers. Each classifier is obtained by applying a given weak learner to a subsample (with size smaller than that of the original training set) which is drawn from the original training set according to the probability distribution maintained over the training set. A parameter is introduced into the reweighted scheme proposed in Adaboost to update the probabilities assigned to training examples so that the algorithm can be more accurate than Adaboost. The experimental results on synthetic and several real-world data sets available from the UCI repository show that the proposed method improves the prediction accuracy, the execution speed as well as the robustness to classification noise of Adaboost. Furthermore, the diversity–accuracy patterns of the ensemble classifiers are investigated by kappa–error diagrams.     

Semantic hashing

We show how to learn a deep graphical model of the word-count vectors obtained from a large set of documents. The values of the latent variables in the deepest layer are easy to infer and give a much better representation of each document than Latent Semantic Analysis. When the deepest layer is forced to use a small number of binary variables (e.g. 32), the graphical model performs “semantic hashing”: Documents are mapped to memory addresses in such a way that semantically similar documents are located at nearby addresses. Documents similar to a query document can then be found by simply accessing all the addresses that differ by only a few bits from the address of the query document. This way of extending the efficiency of hash-coding to approximate matching is much faster than locality sensitive hashing, which is the fastest current method. By using semantic hashing to filter the documents given to TF-IDF, we achieve higher accuracy than applying TF-IDF to the entire document set.     

Improved Generalization via Tolerant Training

Theoretical and computational justification is given for improved generalization when the training set is learned with less accuracy. The model used for this investigation is a simple linear one. It is shown that learning a training set with a tolerance improves generalization, over zero-tolerance training, for any testing set satisfying a certain closeness condition to the training set. These results, obtained via a mathematical programming formulation, are placed in the context of some well-known machine learning results. Computational confirmation of improved generalization is given for linear systems (including nine of the twelve real-world data sets tested), as well as for nonlinear systems such as neural networks for which no theoretical results are available at present. In particular, the tolerant training method improves generalization on noisy, sparse, and overparameterized problems.     

An analysis of convergence for a learning version of the subspace method

The learning subspace method of pattern recognition has been earlier introduced by Kohonen et al. in a speech recognition application, where the phonemes to be classified are given as spectral representations. In that method, the class subspaces are updated recursively using special rotation matrices, which depend on the training vectors entering one at a time. Here the learning algorithm based on these operators is represented in a general mathematical form, and almost sure convergence is shown to a given criterion that is a function of the statistics of the training set as well as of a set of nonrandom but free parameters. The proof employs current techniques in stochastic approximation theory. For illustration, the resulting classification criterion is then applied to a concrete pattern recognition situation with suitably chosen parameter values.     

On perfect hashing of numbers with sparse digit representation via multiplication by a constant

Consider the set of vectors over a field having non-zero coefficients only in a fixed sparse set and multiplication defined by convolution, or the set of integers having non-zero digits (in some base b) in a fixed sparse set. We show the existence of an optimal (or almost-optimal, in the latter case) ‘magic’ multiplier constant that provides a perfect hash function which transfers the information from the given sparse coefficients into consecutive digits. Studying the convolution case we also obtain a result of non-degeneracy for Schur functions as polynomials in the elementary symmetric functions in positive characteristic.     

Interleaving two-phased jobs on a single machine

In this paper, we consider a single machine that processes a set of jobs having two (ordered) phases. After processing the first phase of a job, this job must be removed from the machine for some exact amount of time, after which the machine must immediately begin processing its second phase. During this “dead time” between job phases, the machine may be used to process other similar jobs. We first prove that the problem of interleaving these jobs in order to minimize the makespan (or to process as many jobs as possible by a given deadline) is strongly NP-hard. Next, we compare the effectiveness of a mixed-integer programming formulation based on a continuous time domain to that of a discrete-time integer programming model for solving problems having different data characteristics. These comparisons are performed on a set of realistic synthetic problems based on different scenarios arising in radar pulsing applications.     

Neural network as a simulation metamodel in economic analysis of risky projects

An artificial neural network (ANN) model for economic analysis of risky projects is presented in this paper. Outputs of conventional simulation models are used as neural network training inputs. The neural network model is then used to predict the potential returns from an investment project having stochastic parameters. The nondeterministic aspects of the project include the initial investment, the magnitude of the rate of return, and the investment period. Backpropagation method is used in the neural network modeling. Sigmoid and hyperbolic tangent functions are used in the learning aspect of the system. Analysis of the outputs of the neural network model indicates that more predictive capability can be achieved by coupling conventional simulation with neural network approaches. The trained network was able to predict simulation output based on the input values with very good accuracy for conditions not in its training set. This allowed an analysis of the future performance of the investment project without having to run additional expensive and time-consuming simulation experiments.     

Incorporating performance measures with target levels in data envelopment analysis

Data envelopment analysis (DEA) is a technique for evaluating relative efficiencies of peer decision making units (DMUs) which have multiple performance measures. These performance measures have to be classified as either inputs or outputs in DEA. DEA assumes that higher output levels and/or lower input levels indicate better performance. This study is motivated by the fact that there are performance measures (or factors) that cannot be classified as an input or output, because they have target levels with which all DMUs strive to achieve in order to attain the best practice, and any deviations from the target levels are not desirable and may indicate inefficiency. We show how such performance measures with target levels can be incorporated in DEA. We formulate a new production possibility set by extending the standard DEA production possibility set under variable returns-to-scale assumption based on a set of axiomatic properties postulated to suit the case of targeted factors. We develop three efficiency measures by extending the standard radial, slacks-based, and Nerlove–Luenberger measures. We illustrate the proposed model and efficiency measures by applying them to the efficiency evaluation of 36 US universities.     

Logical analysis of data: classification with justification

Learning from examples is a frequently arising challenge, with a large number of algorithms proposed in the classification, data mining and machine learning literature. The evaluation of the quality of such algorithms is frequently carried out ex post, on an experimental basis: their performance is measured either by cross validation on benchmark data sets, or by clinical trials. Few of these approaches evaluate the learning process ex ante, on its own merits. In this paper, we discuss a property of rule-based classifiers which we call “justifiability”, and which focuses on the type of information extracted from the given training set in order to classify new observations. We investigate some interesting mathematical properties of justifiable classifiers. In particular, we establish the existence of justifiable classifiers, and we show that several well-known learning approaches, such as decision trees or nearest neighbor based methods, automatically provide justifiable classifiers. We also identify maximal subsets of observations which must be classified in the same way by every justifiable classifiers. Finally, we illustrate by a numerical example that using classifiers based on “most justifiable” rules does not seem to lead to overfitting, even though it involves an element of optimization.     

A primal-dual method for approximating tree cover with two weights

The tree cover (TC) problem is to compute a minimum weight connected edge set, given a connected and edge-weighted graph G, such that its vertex set forms a vertex cover for G. Unlike related problems of vertex cover or edge dominating set, weighted TC is not yet known to be approximable in polynomial time as well as the unweighted version is. Moreover, the best approximation algorithm known so far for weighted TC is far from practical in its efficiency. In this paper we consider a restricted version of weighted TC, as a first step towards better approximation of general TC, where only two edge weights differing by at least a factor of 2 are available. It will be shown that a factor 2 approximation can be attained efficiently (in the complexity of max flow) in this case by a primal-dual method. Even under the limited weights as such, the primal-dual arguments used will be seen to be quite involved, having a nontrivial style of dual assignments as an essential part, unlike the case of uniform weights.     

Quality inspection scheduling for multi-unit service enterprises

The traveling salesman problem is a classic NP-hard problem used to model many production and scheduling problems. The problem becomes even more difficult when additional salesmen are added to create a multiple traveling salesman problem (MTSP). We consider a variation of this problem where one salesman visits a given set of cities in a series of short trips. This variation is faced by numerous franchise companies that use quality control inspectors to ensure properties are maintaining acceptable facility and service levels. We model an actual franchised hotel chain using traveling quality inspectors to demonstrate the technique. The model is solved using a commercially available genetic algorithm (GA) tool as well as a custom GA program. The custom GA is proven to be an effective method of solving the proposed model.     

A branch and price solution approach for order acceptance and capacity planning in make-to-order operations

Make-to-order (MTO) operations have to effectively manage their capacity to make long-term sustainable profits. This objective can be met by selectively accepting available customer orders and simultaneously planning for capacity. We model a MTO operation of a job-shop with multiple resources having regular and non-regular capacity. The MTO firm has a set of customer orders at time zero with fixed due-dates. The process route, processing times, and sales price for each order are given. Since orders compete for limited resources, the firm can only accept some orders. In this paper a Mixed-Integer Linear Program (MILP) is proposed to aid an operational manager to decide which orders to accept and how to allocate resources such that the overall profit is maximized. A branch-and-price (B&P) algorithm is devised to solve the MILP effectively. The MILP is first decomposed into a master problem and several sub-problems using Dantzig-Wolfe decomposition. Each sub-problem is represented as a network flow problem and an exact procedure is proposed to solve the sub-problems efficiently. We also propose an approximate B&P scheme, Lagrangian bounds, and approximations to fathom nodes in the branch-and-bound tree. Computational analysis shows that the proposed B&P algorithm can solve large problem instances with relatively short time.     

An overview and framework for PD backtesting and benchmarking

In order to manage model risk, financial institutions need to set up validation processes so as to monitor the quality of the models on an ongoing basis. Validation can be considered from both a quantitative and qualitative point of view. Backtesting and benchmarking are key quantitative validation tools, and the focus of this paper. In backtesting, the predicted risk measurements (PD, LGD, EAD) will be contrasted with observed measurements using a workbench of available test statistics to evaluate the calibration, discrimination and stability of the model. A timely detection of reduced performance is crucial since it directly impacts profitability and risk management strategies. The aim of benchmarking is to compare internal risk measurements with external risk measurements so as to better gauge the quality of the internal rating system. This paper will focus on the quantitative PD validation process within a Basel II context. We will set forth a traffic light indicator approach that employs all relevant statistical tests to quantitatively validate the used PD model, and document this approach with a real-life case study. The set forth methodology and tests are the summary of the authors’ statistical expertise and experience of world-wide observed business practices.     

Optimal sequential inspection policies

We consider the problem of combining a given set of diagnostic tests into an inspection system to classify items of interest (cases) with maximum accuracy such that the cost of performing the tests does not exceed a given budget constraint. One motivating application is sequencing diagnostic tests for container inspection, where the diagnostic tests may correspond to radiation sensors, document checks, or imaging systems. We consider mixtures of decision trees as inspection systems following the work of Boros et al. (Nav. Res. Logist. 56:404?C420, 2009). We establish some properties of efficient inspection systems and characterize the optimal classification of cases, based on some of their test scores. The measure of performance is the fraction of all cases in a specific class of interest, which are classified correctly. We propose a dynamic programming algorithm that constructs more complex policies by iteratively prefixing devices to a subset of policies and thereby enumerating all of the efficient (i.e., undominated) inspection policies in the two dimensional cost-detection space. Our inspection policies may sequence an arbitrary number of tests and are not restricted in the branching factor. Our approach directly solves the bi-criterion optimization problem of maximizing detection and minimizing cost, and thus supports sensitivity analysis over a wide range of budget and detection requirements.