基于投影方法的约束独立成分分析

独立成分分析是解决盲源分离问题的一种有效工具，但ICA具有伸(dilation)与排序(permutation)的不确定性的本质特征。本利用一些约束条件，采用Lagrange乘子法并结合简单的投影方法，可以以特定的形式来进行独立成分的排序，并且可以在信号分离过程中规范化解混矩阵(demixing matrix)，能够系统地减轻ICA对于伸缩与排序的不确定性。仿真结果证实了算法的有效性。     

Semi-blind source extraction algorithm for fetal electrocardiogram based on generalized autocorrelations and reference signals

Blind source extraction (BSE) has become one of the promising methods in the field of signal processing and analysis, which only desires to extract “interesting” source signals with specific stochastic property or features so as to save lots of computing time and resources. This paper addresses BSE problem, in which desired source signals have some available reference signals. Based on this prior information, we develop an objective function for extraction of temporally correlated sources. Maximizing this objective function, a semi-blind source extraction fixed-point algorithm is proposed. Simulations on artificial electrocardiograph (ECG) signals and the real-world ECG data demonstrate the better performance of the new algorithm. Moreover, comparisons with existing algorithms further indicate the validity of our new algorithm, and also show its robustness to the estimated error of time delay.     

A linear programming-based optimization algorithm for solving nonlinear programming problems

In this paper a linear programming-based optimization algorithm called the Sequential Cutting Plane algorithm is presented. The main features of the algorithm are described, convergence to a Karush–Kuhn–Tucker stationary point is proved and numerical experience on some well-known test sets is showed. The algorithm is based on an earlier version for convex inequality constrained problems, but here the algorithm is extended to general continuously differentiable nonlinear programming problems containing both nonlinear inequality and equality constraints. A comparison with some existing solvers shows that the algorithm is competitive with these solvers. Thus, this new method based on solving linear programming subproblems is a good alternative method for solving nonlinear programming problems efficiently. The algorithm has been used as a subsolver in a mixed integer nonlinear programming algorithm where the linear problems provide lower bounds on the optimal solutions of the nonlinear programming subproblems in the branch and bound tree for convex, inequality constrained problems.     

A fixed-point algorithm for blind source separation with nonlinear autocorrelation

This paper addresses blind source separation (BSS) problem when source signals have the temporal structure with nonlinear autocorrelation. Using the temporal characteristics of sources, we develop an objective function based on the nonlinear autocorrelation of sources. Maximizing the objective function, we propose a fixed-point source separation algorithm. Furthermore, we give some mathematical properties of the algorithm. Computer simulations for sources with square temporal autocorrelation and the real-world applications in the analysis of the magnetoencephalographic recordings (MEG) illustrate the efficiency of the proposed approach. Thus, the presented BSS algorithm, which is based on the nonlinear measure of temporal autocorrelation, provides a novel statistical property to perform BSS.     

Efficiency measurement using independent component analysis and data envelopment analysis

Efficiency measurement is an important issue for any firm or organization. Efficiency measurement allows organizations to compare their performance with their competitors’ and then develop corresponding plans to improve performance. Various efficiency measurement tools, such as conventional statistical methods and non-parametric methods, have been successfully developed in the literature. Among these tools, the data envelopment analysis (DEA) approach is one of the most widely discussed. However, problems of discrimination between efficient and inefficient decision-making units also exist in the DEA context (Adler and Yazhemsky, 2010). In this paper, a two-stage approach of integrating independent component analysis (ICA) and data envelopment analysis (DEA) is proposed to overcome this issue. We suggest using ICA first to extract the input variables for generating independent components, then selecting the ICs representing the independent sources of input variables, and finally, inputting the selected ICs as new variables in the DEA model. A simulated dataset and a hospital dataset provided by the Office of Statistics in Taiwan’s Department of Health are used to demonstrate the validity of the proposed two-stage approach. The results show that the proposed method can not only separate performance differences between the DMUs but also improve the discriminatory capability of the DEA’s efficiency measurement.     

Simulated annealing for complex portfolio selection problems

This paper describes the application of a simulated annealing approach to the solution of a complex portfolio selection model. The model is a mixed integer quadratic programming problem which arises when Markowitz’ classical mean–variance model is enriched with additional realistic constraints. Exact optimization algorithms run into difficulties in this framework and this motivates the investigation of heuristic techniques. Computational experiments indicate that the approach is promising for this class of problems.     

Post-processing of Markov chain Monte Carlo output in Bayesian latent variable models with application to multidimensional scaling

In Bayesian analysis of multidimensional scaling model with MCMC algorithm, we encounter the indeterminacy of rotation, reflection and translation of the parameter matrix of interest. This type of indeterminacy may be seen in other multivariate latent variable models as well. In this paper, we propose to address this indeterminacy problem with a novel, offline post-processing method that is easily implemented using easy-to-use Markov chain Monte Carlo (MCMC) software. Specifically, we propose a post-processing method based on the generalized extended Procrustes analysis to address this problem. The proposed method is compared with four existing methods to deal with indeterminacy thorough analyses of artificial as well as real datasets. The proposed method achieved at least as good a performance as the best existing method. The benefit of the offline processing approach in the era of easy-to-use MCMC software is discussed.     

A short and constructive proof of Tarski’s fixed-point theorem

I give short and constructive proofs of Tarski’s fixed-point theorem, and of Zhou’s extension of Tarski’s fixed-point theorem to set-valued maps.I thank Charles Blair, William Thomson, an associated editor and a referee for their helpful suggestions.I have taught Tarski’s Theorem with F continuous to Caltech undergraduates.     

Portfolio optimization with an envelope-based multi-objective evolutionary algorithm

The problem of portfolio selection is a standard problem in financial engineering and has received a lot of attention in recent decades. Classical mean–variance portfolio selection aims at simultaneously maximizing the expected return of the portfolio and minimizing portfolio variance. In the case of linear constraints, the problem can be solved efficiently by parametric quadratic programming (i.e., variants of Markowitz’ critical line algorithm). However, there are many real-world constraints that lead to a non-convex search space, e.g., cardinality constraints which limit the number of different assets in a portfolio, or minimum buy-in thresholds. As a consequence, the efficient approaches for the convex problem can no longer be applied, and new solutions are needed.In this paper, we propose to integrate an active set algorithm optimized for portfolio selection into a multi-objective evolutionary algorithm (MOEA). The idea is to let the MOEA come up with some convex subsets of the set of all feasible portfolios, solve a critical line algorithm for each subset, and then merge the partial solutions to form the solution of the original non-convex problem. We show that the resulting envelope-based MOEA significantly outperforms existing MOEAs.     

Optimal control synthesis for the constrained bilinear biquadratic regulator problem

Optimal process control with control constraints is a challenging task related to many real-life problems. In this paper, a single input continuous time constrained linear quadratic regulator problem, is defined and fully solved. The constraints include both bilinear inequality constraints and customary control force bounds. As a first step, the problem is reformulated as an equivalent constrained bilinear biquadratic optimal control problem. Next, Krotov’s method is used to solve it. To this end, a sequence of improving functions suitable to the problem’s new formulation is constructed and the corresponding successive algorithm is derived. The required computational steps are arranged as an algorithm and proof outlines for the convergence and optimality of the solution are given. The efficiency of the suggested method is demonstrated by numerical example.     

双模式网络停车限制交通需求管理模型与方法研究

在轨道网和公路网并存的双模式交通网络, 合理设计出行终点的停车容量可优化汽车出行需求, 改善路网交通环境。本文通过分析私家车与城市轨道两种交通模式的出行需求, 并考虑私家车模式的终点停车收费服务, 建立了一种带路段环境容量和终点停车需求容量共同约束的交通需求管理模型。模型中路网使用者的出行模式采用二元Logit模型来计算, 而私家车的路线选择行为服从Logit随机用户均衡, 因此该模型是一个带不动点约束的数学规划问题。针对模型求解困难, 文中采用灵敏度分析来获取各路段流量和需求量关于终点容量波动的梯度信息, 进而设计了一种新的灵敏度分析求解算法.最后通过数值仿真实验, 验证了算法的有效性, 同时分析了不同停车收费参数对模型各指标变化趋势的影响。     

Improved versions of the LEDE algorithm for constrained global optimization

Two modified versions of the authors’ recent differential evolution algorithm for constrained global optimization are proposed. They incorporate a filter set which results in more efficient implementations of the original algorithm. Numerical results are presented which suggest that the new algorithms are competitive.     

基于一维搜索的ICA自适应算法及其在股票分析中的应用

在固定步长的ICA极大似然估计自适应算法的基础上,通过一维搜索引入了步长修正方案,使新算法可在收敛速度和稳定状态时的失调误差这两个性能指标上达到最佳结合点,具有较好的时变系统跟踪能力。仿真结果证实了本文所提出的算法可以有效地提高ICA的自适应性,能够更准确地完成盲源分离。在此基础上将算法用在时变性很强的股票数据上,以验证该算法的有效性和可行性。     

New second-order and first-order algorithms for determining optimal control: A differential dynamic programming approach

In this paper, the notion of differential dynamic programming is used to develop new second-order and first-order successive-approximation methods for determining optimal control. The unconstrained, nonlinear control problem is first considered, and a second-order algorithm is developed which has wider application then existing second-variation and second-order algorithms. A new first-order algorithm emerges as a special case of the second-order one. Control inequality constraints are introduced into the problem and a second-order algorithm is devised which is able to solve this constrained problem. It is believed that control constraints have not been handled previously in this way. Again, a first-order algorithm emerges as a special case. The usefulness of the second-order algorithms is illustrated by the computer solution of three control problems. The methods presented in this paper have been extended by the author to solve problems with terminal constraints and implicitly given final time. Details of these procedures are not given in this paper, but the relevant references are cited.     

Implementation of a proximal algorithm for linearly constrained nonsmooth optimization problems and computational results

This paper describes an implementation of the so-calledproximal point algorithm for solving convex linearly constrained nonsmooth optimization problems. Contrary to other previous implementations of the same approach (which solve constrained nonsmooth problems as unconstrained problems via exact penalty function techniques), our implementation handles linear constraints explicitly (linear constraints being incorporated into the direction-finding subproblem). The relevance and efficiency of the approach is demonstrated through comparative computational experiments on many classical test problems from the literature, as well as on a series of large constrained dual transportation problems introduced and studied here for the first time.     

Constrained Global Optimization of Expensive Black Box Functions Using Radial Basis Functions

We present a new strategy for the constrained global optimization of expensive black box functions using response surface models. A response surface model is simply a multivariate approximation of a continuous black box function which is used as a surrogate model for optimization in situations where function evaluations are computationally expensive. Prior global optimization methods that utilize response surface models were limited to box-constrained problems, but the new method can easily incorporate general nonlinear constraints. In the proposed method, which we refer to as the Constrained Optimization using Response Surfaces (CORS) Method, the next point for costly function evaluation is chosen to be the one that minimizes the current response surface model subject to the given constraints and to additional constraints that the point be of some distance from previously evaluated points. The distance requirement is allowed to cycle, starting from a high value (global search) and ending with a low value (local search). The purpose of the constraint is to drive the method towards unexplored regions of the domain and to prevent the premature convergence of the method to some point which may not even be a local minimizer of the black box function. The new method can be shown to converge to the global minimizer of any continuous function on a compact set regardless of the response surface model that is used. Finally, we considered two particular implementations of the CORS method which utilize a radial basis function model (CORS-RBF) and applied it on the box-constrained Dixon–Szegö test functions and on a simple nonlinearly constrained test function. The results indicate that the CORS-RBF algorithms are competitive with existing global optimization algorithms for costly functions on the box-constrained test problems. The results also show that the CORS-RBF algorithms are better than other algorithms for constrained global optimization on the nonlinearly constrained test problem.     

Hybrid linear and nonlinear complexity pursuit for blind source separation

Blind source separation (BSS) is an increasingly popular data analysis technique with many applications. Several methods for BSS using the statistical properties of original sources have been proposed; for a famous case, non-Gaussianity, this leads to independent component analysis (ICA). In this paper, we propose a hybrid BSS method based on linear and nonlinear complexity pursuit, which combines three statistical properties of source signals: non-Gaussianity, linear predictability and nonlinear predictability. A gradient learning algorithm is presented by minimizing a loss function. Simulations verify the efficient implementation of the proposed method.     

A Bayesian approach to constrained single- and multi-objective optimization

This article addresses the problem of derivative-free (single- or multi-objective) optimization subject to multiple inequality constraints. Both the objective and constraint functions are assumed to be smooth, non-linear and expensive to evaluate. As a consequence, the number of evaluations that can be used to carry out the optimization is very limited, as in complex industrial design optimization problems. The method we propose to overcome this difficulty has its roots in both the Bayesian and the multi-objective optimization literatures. More specifically, an extended domination rule is used to handle objectives and constraints in a unified way, and a corresponding expected hyper-volume improvement sampling criterion is proposed. This new criterion is naturally adapted to the search of a feasible point when none is available, and reduces to existing Bayesian sampling criteria—the classical Expected Improvement (EI) criterion and some of its constrained/multi-objective extensions—as soon as at least one feasible point is available. The calculation and optimization of the criterion are performed using Sequential Monte Carlo techniques. In particular, an algorithm similar to the subset simulation method, which is well known in the field of structural reliability, is used to estimate the criterion. The method, which we call BMOO (for Bayesian Multi-Objective Optimization), is compared to state-of-the-art algorithms for single- and multi-objective constrained optimization.     

Separable nonlinear least squares fitting with linear bound constraints and its application in magnetic resonance spectroscopy data quantification

An application in magnetic resonance spectroscopy quantification models a signal as a linear combination of nonlinear functions. It leads to a separable nonlinear least squares fitting problem, with linear bound constraints on some variables. The variable projection (VARPRO) technique can be applied to this problem, but needs to be adapted in several respects. If only the nonlinear variables are subject to constraints, then the Levenberg–Marquardt minimization algorithm that is classically used by the VARPRO method should be replaced with a version that can incorporate those constraints. If some of the linear variables are also constrained, then they cannot be projected out via a closed-form expression as is the case for the classical VARPRO technique. We show how quadratic programming problems can be solved instead, and we provide details on efficient function and approximate Jacobian evaluations for the inequality constrained VARPRO method.     

Algorithms with high order convergence speed for blind source extraction

A rigorous convergence analysis for the fixed point ICA algorithm of Hyvärinen and Oja is provided and a generalization of it involving cumulants of an arbitrary order is presented. We consider a specific optimization problem OP(p), p>3, integer, arising from a Blind Source Extraction problem (BSE) and prove that every local maximum of OP(p) is a solution of (BSE) in sense that it extracts one source signal from a linear mixture of unknown statistically independent signals. An algorithm for solving OP(p) is constructed, which has a rate of convergence p?1.