Application of the cross-entropy method to clustering and vector quantization

We apply the cross-entropy (CE) method to problems in clustering and vector quantization. The CE algorithm for clustering involves the following iterative steps: (a) generate random clusters according to a specified parametric probability distribution, (b) update the parameters of this distribution according to the Kullback–Leibler cross-entropy. Through various numerical experiments, we demonstrate the high accuracy of the CE algorithm and show that it can generate near-optimal clusters for fairly large data sets. We compare the CE method with well-known clustering and vector quantization methods such as K-means, fuzzy K-means and linear vector quantization, and apply each method to benchmark and image analysis data.     

Queueing models,block hessenberg matrices and the method of neuts

We introduce a numerical method to compute the stationary probability vector of queueing models whose infinitesimal generator is of block Hessenberg form. It is shown that the stationary probability vector is equal to the first column of the inverse of the coefficient matrix. Furthermore, it is shown that the first column of the inverse of an upper (or lower) Hessenberg matrix may be obtained in a relatively small number of operations. Together, these results allow us to define a powerful algorithm for solving certain queueing models. The efficiency of this algorithm is discussed and a comparison with the method of Neuts is undertaken. A relationship with the method of Gaussian elimination is established and used to develop some stability results.This work was supported in part by NSF Grant MCS-83-00438.     

区间判断的凸锥模型与排序方法

本文用凸锥模型对层次分析法中区间判断条件下的排序问题进行研究，给出了权重向量的合理算法．并对区间判断下权重向量的合理位置，产生逆序的可能性等重要问题进行了探讨和研究，给出了计算实例．     

<Emphasis Type="Italic">I</Emphasis>-projection onto isotonic cones and its applications to maximum likelihood estimation for log-linear models

A frequently occurring problem is to find a probability vector,p∈D, which minimizes theI-divergence between it and a given probability vector π. This is referred to as theI-projection of π ontoD. Darroch and Ratcliff (1972,Ann. Math. Statist.,43, 1470–1480) gave an algorithm whenD is defined by some linear equalities and in this paper, for simplicity of exposition, we propose an iterative procedure whenD is defined by some linear inequalities. We also discuss the relationship betweenI-projection and the maximum likelihood estimation for multinomial distribution. All of the results can be applied to isotonic cone.     

一个求解二阶锥规划的光滑牛顿算法

本文研究了二阶锥规划问题.利用新的最小值函数的光滑函数,给出一个求解二阶锥规划的光滑牛顿算法.算法可以从任意点出发,在每一步迭代只需求解一个线性方程组并进行一次线性搜索.在不需要满足严格互补假设条件下,证明了算法是全局收敛和局部二阶收敛的.数值试验表明算法是有效的.     

Componentwise bounds for nearly completely decomposable Markov chains using stochastic comparison and reordering

This paper presents an improved version of a componentwise bounding algorithm for the state probability vector of nearly completely decomposable Markov chains, and on an application it provides the first numerical results with the type of algorithm discussed. The given two-level algorithm uses aggregation and stochastic comparison with the strong stochastic (st) order. In order to improve accuracy, it employs reordering of states and a better componentwise probability bounding algorithm given st upper- and lower-bounding probability vectors. Results in sparse storage show that there are cases in which the given algorithm proves to be useful.     

Subset solving algorithm: A novel sensitivity-based method for damage detection of structures

We present an efficient method for structural damage detection using natural frequencies. The method, which is based on sensitivity analysis of the structure, consists of two main stages. In the first stage, the structural elements are ordered based on their damage probability into a vector referred to as elements damage probability ordering vector (EDPOV). In the second stage, a rather small subset of EDPOV elements are judiciously selected to form a nonlinear system of equations, which are subsequently solved to detect potential damages. In the second stage, the procedure of subset selecting and solving iterates until a feasible solution is achieved. In order to assess the merits of the proposed method some illustrative cases are studied. Numerical results demonstrate the high efficiency of the proposed algorithm compared to those found in the literature.     

M/M/c休假排队系统稳态分布的数值计算

从数值计算角度研究M/M/c休假排队系统稳定状态的概率分布.采用GMRES方法求解概率分布向量所满足的大型线性方程,构造了一个循环预处理算子加速GMRES方法的收敛.数值实例验证了该算法的优越性.     

Natural gradient algorithm for stochastic distribution systems with output feedback

In this paper, we use natural gradient algorithm to control the shape of the conditional output probability density function for the stochastic distribution systems from the viewpoint of information geometry. The considered system here is of multi-input and single output with an output feedback and a stochastic noise. Based on the assumption that the probability density function of the stochastic noise is known, we obtain the conditional output probability density function whose shape is only determined by the control input vector under the condition that the output feedback is known at any sample time. The set of all the conditional output probability density functions forms a statistical manifold (M), and the control input vector and the output feedback are considered as the coordinate system. The Kullback divergence acts as the distance between the conditional output probability density function and the target probability density function. Thus, an iterative formula for the control input vector is proposed in the sense of information geometry. Meanwhile, we consider the convergence of the presented algorithm. At last, an illustrative example is utilized to demonstrate the effectiveness of the algorithm.     

鲁棒半监督ν-支持向量分类机

支持向量机在近十年成为机器学习的主要学习技术,而且已经成功应用到有监督学习问题中。Fung和Mangasarian利用支持向量机对于既有已标类别样本又有未知类别样本的训练集进行训练,方法主要是利用少量已标明类别的样本进行训练得到一个分类器的同时对于未标明类别的样本进行分类,使得间隔最大化。此优化问题中假定样本是精确的,而在现实生活中,样本通常带有统计误差。因此,考虑样本带有扰动信息的半监督两类分类问题,给出鲁棒半监督v-支持向量分类算法。该算法的参数v易于选择,而数值试验也表明该算法具有良好的稳定性和较好的分类结果。     

Differential evolution with generalized differentials

In this paper, we study the mutation operation of the differential evolution (DE) algorithm. In particular, we propose the differential of scaled vectors, called the ‘generalized differential’, as opposed to the existing scaled differential vector in the mutation of DE. We derive the probability distribution of points generated by the mutation with ‘generalized differentials’. We incorporate a vector-projection-based exploratory method within the new mutation scheme. The vector projection is not mandatory and it is only invoked if trial points continue to be unsuccessful. An algorithm is then proposed which implements the mutation strategy based on the difference of the scaled vectors as well as the vector projection technique. A numerical study is carried out using a set of 50 test problems, many of which are inspired by practical applications. Numerical results suggest that the new algorithm is superior to DE.     

随机结构系统基于可靠性的优化设计

提出了以梁板（薄板）为基体的随机结构系统（即结构元件的面积、长度、弹性模量和强度等均为随机变量）在随机载荷作用下,基于可靠性的优化设计方法．给出了随机结构系统安全余量和系统可靠性指标的敏度表达式;给出最佳矢量型算法．首先是用改进的一次二阶矩和随机有限元法求出安全余量的可靠性指标的表达式,然后用概率网络估算(PNET)法求出系统失效概率的公式,对该式两边求导得出了系统可靠性指标的敏度表达式,进而用最佳矢量型算法进行优化设计．在优化迭代过程中,采用梯度步和最佳矢量步相结合的方法进行计算．最后给出了一个算例,说明该方法计算效率高,收敛稳定,适合工程应用．     

MC-based Algorithm for a telecommunication network under node and budget constraints

It is an important issue to design some performance indexes in order to measure the performance for a telecommunication network. Network analysis is an available approach to solve the performance problem for a real-life system. We construct a two-commodity stochastic-flow network with unreliable nodes (arcs and nodes all have several possible capacities and may fail) to model the telecommunication network. In which, all types of commodity are transmitted through the same network simultaneously and compete the capacities. This paper defines the system capacity as a 2-tuple vector, and then proposes a performance index, the probability that the upper bound of the system capacity equals a demand vector subject to the budget constraint. An upper boundary point is a vector representing the capacities of arcs and nodes, and is the maximal vector exactly meeting the demand vector. A simple algorithm based on minimal cuts (or named MC-based algorithm) is then presented to generate all upper boundary points in order to evaluate the performance index. The storage and computational time complexity of this algorithm are also analyzed. The performance evaluation for the multicommodity case can be extended easily.     

A Differential Free Point Generation Scheme in the Differential Evolution Algorithm

In this paper we derive the probability distribution of trial points in the differential evolution (de) algorithm, in particular the probability distribution of points generated by mutation. We propose a point generation scheme that uses an approximation to this distribution. The scheme can dispense with the differential vector used in the mutation of de. We propose a de algorithm that replaces the differential based mutation scheme with a probability distribution based point generation scheme. We also propose a de algorithm that uses a probabilistic combination of the point generation by the probability distribution and the point generation by mutation. A numerical study is carried out using a set of 50 test problems, many of which are inspired by practical applications. Numerical results suggest that the new algorithms are superior to the original version both in terms of the number of function evaluations and cpu times.     

The evaluation of two-sided orthant probabilities for a quadrivariate normal distribution

In this paper it is shown how a general two-sided orthant probability for a quadrivariate normal distribution can be evaluated by a one-dimensional numerical integral calculation. The quadrivariate normal distribution can have any covariance matrix and any mean vector. This affords a practical and efficient method for the calculation of these probabilities which is superior to simulation methods. The implementation of the algorithm is discussed, and some examples of its performance are provided.     

Convexity and optimization with copulæ structured probabilistic constraints

Probability constraints play a key role in optimization problems involving uncertainties. These constraints request that an inequality system depending on a random vector has to be satisfied with a high enough probability. In specific settings, copulæ can be used to model the probabilistic constraints with uncertainty on the left-hand side. In this paper, we provide eventual convexity results for the feasible set of decisions under local generalized concavity properties of the constraint mappings and involved copulæ. The results cover all Archimedean copulæ. We consider probabilistic constraints wherein the decision and random vector are separated, i.e. left/right-hand side uncertainty. In order to solve the underlying optimization problem, we propose and analyse convergence of a regularized supporting hyperplane method: a stabilized variant of generalized Benders decomposition. The algorithm is tested on a large set of instances involving several copulæ among which the Gaussian copula. A Numerical comparison with a (pure) supporting hyperplane algorithm and a general purpose solver for non-linear optimization is also presented.     

A Class of Cone Bounded Quasiconvex Mappings in Topological Vector Spaces

Within the context of cone-ordered topological vector spaces, this paper introduces the concepts of cone bounded point and cone bounded set for vector set. With their aid, a class of new cone quasiconvex mappings in topological vector spaces is defined, and their fundamental properties are presented. The relationships between the cone bounded quasiconvex mapping defined in this paper and cone convex mapping, and other known cone quasiconvex mapping are also discussed.     

Convergence of a transition probability tensor of a higher‐order Markov chain to the stationary probability vector

In this paper, first we introduce a new tensor product for a transition probability tensor originating from a higher‐order Markov chain. Subsequently, some properties of the new tensor product are explained, and its relationship with the stationary probability vector is studied. Also, similarity between results obtained by this new product and the first‐order case is shown. Furthermore, we prove the convergence of a transition probability tensor to the stationary probability vector. Finally, we show how to achieve a stationary probability vector with some numerical examples and make some comparison between the proposed method and another existing method for obtaining stationary probability vectors. Copyright © 2016 John Wiley & Sons, Ltd.     

A procedure for finding Nash equilibria in bi-matrix games

In this paper we consider the computation of Nash equilibria for noncooperative bi-matrix games. The standard method for finding a Nash equilibrium in such a game is the Lemke-Howson method. That method operates by solving a related linear complementarity problem (LCP). However, the method may fail to reach certain equilibria because it can only start from a limited number of strategy vectors. The method we propose here finds an equilibrium by solving a related stationary point problem (SPP). Contrary to the Lemke-Howson method it can start from almost any strategy vector. Besides, the path of vectors along which the equilibrium is reached has an appealing game-theoretic interpretation. An important feature of the algorithm is that it finds a perfect equilibrium when at the start all actions are played with positive probability. Furthermore, we can in principle find all Nash equilibria by repeated application of the algorithm starting from different strategy vectors.This author is financially supported by the Co-operation Centre Tilburg and Eindhoven Universities, The Netherlands.     

Minimizing loss probability bounds for portfolio selection

In this paper, we derive a portfolio optimization model by minimizing upper and lower bounds of loss probability. These bounds are obtained under a nonparametric assumption of underlying return distribution by modifying the so-called generalization error bounds for the support vector machine, which has been developed in the field of statistical learning. Based on the bounds, two fractional programs are derived for constructing portfolios, where the numerator of the ratio in the objective includes the value-at-risk (VaR) or conditional value-at-risk (CVaR) while the denominator is any norm of portfolio vector. Depending on the parameter values in the model, the derived formulations can result in a nonconvex constrained optimization, and an algorithm for dealing with such a case is proposed. Some computational experiments are conducted on real stock market data, demonstrating that the CVaR-based fractional programming model outperforms the empirical probability minimization.