The Design and Analysis of Benchmark Experiments

The assessment of the performance of learners by means of benchmark experiments is an established exercise. In practice, benchmark studies are a tool to compare the performance of several competing algorithms for a certain learning problem. Cross-validation or resampling techniques are commonly used to derive point estimates of the performances which are compared to identify algorithms with good properties. For several benchmarking problems, test procedures taking the variability of those point estimates into account have been suggested. Most of the recently proposed inference procedures are based on special variance estimators for the cross-validated performance. We introduce a theoretical framework for inference problems in benchmark experiments and show that standard statistical test procedures can be used to test for differences in the performances. The theory is based on well-defined distributions of performance measures which can be compared with established tests. To demonstrate the usefulness in practice, the theoretical results are applied to regression and classification benchmark studies based on artificial and real world data.     

Evaluation of Commodity Trading Advisors using fixed and variable and benchmark models

This paper examines the performance of Commodity Trading Advisors (CTAs) using fixed and variable benchmarking models. In order to avoid the troublesome passive and active commodity and managed futures benchmarks (indices) when examining the performance of CTAs, we innovate by using data envelopment analysis (DEA). Because this alternative class has non-linear returns due to long/short positions, and derivatives (i.e., dynamic trading strategies), DEA can alleviate the problems usually associated with these indices. The effectiveness of using benchmarking models in a DEA setting will provide investors with an alternative technique in assessing the performance and identifying efficient CTAs.     

On the Effect of Misspecifying the Density Ratio Model

The density ratio model specifies that the log-likelihood ratio of two unknown densities is of known linear form which depends on some finite dimensional parameters. The model can be broadened to allow for m-samples in a quite natural way. Estimation of both parametric and nonparametric part of the model is carried out by the method of empirical likelihood. However the assumed linear form has an impact on the estimation and testing for the parametric part. The goal of this study is to quantify the effect of choosing an incorrect linear form and its impact to inference. The issue of misspecification is addressed by embedding the unknown linear form to some parametric transformation family which yields ultimately to its identification. Simulated examples and data analysis integrate the presentation.     

Benchmarking with quality-adjusted DEA (Q-DEA) to seek lower-cost high-quality service: Evidence from a U.S.bank application

Benchmarking is a widely cited method to identify and adopt best-practices as a means to improve performance. Data envelopment analysis (DEA) has been demonstrated to be a powerful benchmarking methodology for situations where multiple inputs and outputs need to be assessed to identify best-practices and improve productivity in organizations. Most DEA benchmarking studies have excluded quality, even in service-sector applications such as health care where quality is a key element of performance. This limits the practical value of DEA in organizations where maintaining and improving service quality is critical to achieving performance objectives. In this paper, alternative methods incorporating quality in DEA benchmarking are demonstrated and evaluated. It is shown that simply treating the quality measures as DEA outputs does not help in discriminating the performance. Thus, the current study presents a new, more sensitive, quality-adjusted DEA (Q-DEA), which effectively deals with quality measures in benchmarking. We report the results of applying Q-DEA to a U.S. bank's 200-branch network that required a method for benchmarking to help manage operating costs and service quality. Q-DEA findings helped the bank achieve cost savings and improved operations while preserving service quality, a dimension critical to its mission. New insights about ways to improve branch operations based on the best-practice (high-quality low-cost) benchmarks identified with Q-DEA are also described in the paper. This demonstrates the practical need and potential benefits of Q-DEA and its efficacy in one application, and also suggests the need for further research on measuring and incorporating quality into DEA benchmarking. The review process of this paper was handled by the Edit-in-Chief Peter Hammer.     

Performance analysis of single stage kanban controlled production systems using parametric decomposition

We present a new approach that permits efficient performance analysis of kanban systems with general demand processes, material arrival processes, and service times. The approach is based on parametric characterization of the traffic processes (arrival and departure) in the network and uses two-moment approximations to estimate performance measures at individual stations. We derive traffic flow constraints that are particular to closed queuing networks with synchronization stations and use these to establish relationships between the parameters characterizing arrival and departure processes at the stations in the network. The resultant set of non-linear equations is solved to estimate network performance measures. Numerical studies show that the approach is not only fast but also reasonably accurate when compared to simulation. These studies also provide insights with respect to the impact of different types of variability on the performance of a kanban system. This work also provides a fundamental building block that can be used in the analysis of multi-stage kanban systems. AMS Subject Classifications 68M20, 60K20, 90B05, 90B30     

Evaluating the financial performance of bank branches

We evaluate the financial performance of most of the branch offices of a large European savings bank for a recent accounting period. We employ a complementary pair of nonparametric techniques to evaluate their financial performance, in terms of their ability to conserve on the expenses they incur in building their customer bases and providing customer services. We find variation in the ability of branch offices to perform this task, and agreement on the identity of the laggard branches. We then employ parametric techniques to determine that the list of indicators on which their financial performance is evaluated can be reduced without statistically significant loss of information to bank management. Both findings suggest ways in which the bank can increase the profitability of its branch network. A previous version of this paper was presented at CORE, Université Catholique de Louvain, Louvain-la-Neuve, Belgium, where we received many helpful comments from the audience. We are grateful to three referees for their helpful comments as well.     

A discrete model for optimal operation of fossil-fuel generators of electricity

This paper presents a new discrete approach to the price-based dynamic economic dispatch (PBDED) problem of fossil-fuel generators of electricity. The objective is to find a sequence of generator temperatures that maximizes profit over a fixed-length time horizon. The generic optimization model presented in this paper can be applied to automatic operation of fossil-fuel generators or to prepare market bids, and it works with various price forecasts. The model’s practical applications are demonstrated by the results of simulation experiments involving 2009 NYISO electricity market data, branch-and-bound, and tabu-search optimization techniques.     

Application of generalized diagonal dominance in wireless sensor network optimization problems

The recent application of the diagonal dominance in the development of the optimization algorithms in the wireless sensor networks design, has been done by Yuan and Yu (2006) [14], extended in Yu et al. (2006) [9], and surveyed in Machado and Tekinay [11]. In this paper, we will use the concept of generalized diagonal dominance, to improve the obtained results regarding the power control game, in three directions. We also discuss the applicability of such improvements.     

格的凸子格格与幂格

研究格的凸子格的性质,建立格的非空凸子格组成的幂格与幂格的联系,证明在同构意义下,分配格的幂格都是它的非空凸子格组成的幂格的子格。     

Production scheduling with supply and delivery considerations to minimize the makespan

In this paper we study a scheduling model that simultaneously considers production scheduling, material supply, and product delivery. One vehicle with limited loading capacity transports unprocessed jobs from the supplier’s warehouse to the factory in a fixed travelling time. Another capacitated vehicle travels between the factory and the customer to deliver finished jobs to the customer. The objective is to minimize the arrival time of the last delivered job to the customer. We show that the problem is NP-hard in the strong sense, and propose an O(n) time heuristic with a tight performance bound of 2. We identify some polynomially solvable cases of the problem, and develop heuristics with better performance bounds for some special cases of the problem. Computational results show that all the heuristics are effective in producing optimal or near-optimal solutions quickly.