A Dual Active-Set Algorithm for Regularized Monotonic Regression

Monotonic (isotonic) regression is a powerful tool used for solving a wide range of important applied problems. One of its features, which poses a limitation on its use in some areas, is that it produces a piecewise constant fitted response. For smoothing the fitted response, we introduce a regularization term in the monotonic regression, formulated as a least distance problem with monotonicity constraints. The resulting smoothed monotonic regression is a convex quadratic optimization problem. We focus on the case, where the set of observations is completely (linearly) ordered. Our smoothed pool-adjacent-violators algorithm is designed for solving the regularized problem. It belongs to the class of dual active-set algorithms. We prove that it converges to the optimal solution in a finite number of iterations that does not exceed the problem size. One of its advantages is that the active set is progressively enlarging by including one or, typically, more constraints per iteration. This resulted in solving large-scale test problems in a few iterations, whereas the size of that problems was prohibitively too large for the conventional quadratic optimization solvers. Although the complexity of our algorithm grows quadratically with the problem size, we found its running time to grow almost linearly in our computational experiments.     

Complementarity Active-Set Algorithm for Mathematical Programming Problems with Equilibrium Constraints

In this paper, an algorithm for solving a mathematical programming problem with complementarity (or equilibrium) constraints (MPEC) is introduced, which uses the active-set methodology while maintaining the complementarity restrictions throughout the procedure. Finite convergence of the algorithm to a strongly stationary point of the MPEC is established under reasonable hypotheses. The algorithm can be easily implemented by adopting any active-set code for nonlinear programming. Computational experience is included to highlight the efficacy of the proposed method in practice.     

Active-Set Projected Trust-Region Algorithm for Box-Constrained Nonsmooth Equations

In this paper, by means of an active-set strategy, we present a trust-region method for solving box-constrained nonsmooth equations. Nice properties of the proposed method include: (a) all iterates remain feasible; (b) the search direction, as adequate combination of the projected gradient direction and the trust-region direction, is an asymptotic Newton direction under mild conditions; (c) the subproblem of the proposed method, possessing the form of an unconstrained trust-region subproblem, can be solved by existing methods; (d) the subproblem of the proposed method is of reduced dimension, which is potentially cheaper when applied to solve large-scale problems. Under appropriate conditions, we establish global and local superlinear/quadratic convergence of the method. Preliminary numerical results are given.     

Large-Scale Active-Set Box-Constrained Optimization Method with Spectral Projected Gradients

A new active-set method for smooth box-constrained minimization is introduced. The algorithm combines an unconstrained method, including a new line-search which aims to add many constraints to the working set at a single iteration, with a recently introduced technique (spectral projected gradient) for dropping constraints from the working set. Global convergence is proved. A computer implementation is fully described and a numerical comparison assesses the reliability of the new algorithm.     

Two-Stage Nested Partitions Method for Stochastic Optimization

We address the problem of optimizing over a large but finite set when the objective function does not have an analytical expression and is evaluated using noisy estimation. Building on the recently proposed nested partitions method for stochastic optimization, we develop a new approach that combines this random search method and statistical selection for guiding the search. We prove asymptotic convergence and analyze the finite time behavior of the new approach. We also report extensive numerical results to illustrate the benefits of the new approach.     

Primal-Dual Active-Set Algorithm for Chemical Equilibrium Problems Related to the Modeling of Atmospheric Inorganic Aerosols

A general equilibrium model for multiphase multicomponent inorganic atmospheric aerosols is proposed. The thermodynamic equilibrium is given by the minimum of the Gibbs free energy for a system involving an aqueous phase, a gas phase, and solid salts. A primal-dual algorithm solving the Karush-Kuhn-Tucker conditions is detailed. An active set/Newton method permits to compute the minimum of the energy and tracks the presence or not of solid salts at the equilibrium. Numerical results show the efficiency of our algorithm for the prediction of multiphase multireaction chemical equilibria.Communicated by R. GlowinskiThis work has been partially supported by the United States Environmental Protection Agency through Cooperative Agreement X-83234201 to the University of Houston. The second author was supported by the Swiss National Science Foundation, Grant PBEL2-103152.     

Local Convergence Analysis of a Primal–Dual Method for Bound-Constrained Optimization Without SOSC

Journal of Optimization Theory and Applications - We propose a local convergence analysis of a primal–dual interior point algorithm for the solution of a bound-constrained optimization...     

A Trust-Region Algorithm for Global Optimization

We consider the global minimization of a bound-constrained function with a so-called funnel structure. We develop a two-phase procedure that uses sampling, local optimization, and Gaussian smoothing to construct a smooth model of the underlying funnel. The procedure is embedded in a trust-region framework that avoids the pitfalls of a fixed sampling radius. We present a numerical comparison to three popular methods and show that the new algorithm is robust and uses up to 20 times fewer local minimizations steps.     

多目标线性规划的一种交互式单纯形算法

本文基于分析有效极点解的有效变量的特点以及在有效点处各个目标函数的数值来得到改进的搜索方向的研究思想,提出了求解目标函数和约束均为线性的多目标线性规划问题的一种交互式算法。该方法可以保证每一步得到的解均为有效极点解,且根据决策者的偏好不断得到改进,直至最终得到满意的最终解。     

Two-Stage Optimization in a Transportation Problem

A study of the economic distribution of maize throughout South Africa is reported. Although the problem of minimizing total transportation costs in such a situation is a classical one, and its solution is well known, there was in this case a high degree of degeneracy in the system and thus the solution was not unique. Also, since a user is required to pay his own transportation costs, the various optimal solutions were not equivalent. A secondary problem thus arose, viz. that of selecting from these optimal solutions the one which would be fairest to all users. A heuristic and a goal programming method for solving this secondary problem are discussed.     

The Bound-Constrained Conjugate Gradient Method for Non-negative Matrices

Existing conjugate gradient (CG)-based methods for convex quadratic programs with bound constraints require many iterations for solving elastic contact problems. These algorithms are too cautious in expanding the active set and are hampered by frequent restarting of the CG iteration. We propose a new algorithm called the Bound-Constrained Conjugate Gradient method (BCCG). It combines the CG method with an active-set strategy, which truncates variables crossing their bounds and continues (using the Polak–Ribière formula) instead of restarting CG. We provide a case with n=3 that demonstrates that this method may fail on general cases, but we conjecture that it always works if the system matrix A is non-negative. Numerical results demonstrate the effectiveness of the method for large-scale elastic contact problems.     

A Stochastic Algorithm for Constrained Global Optimization

We present a stochastic algorithm to solve numerically the problem of finding the global minimizers of a real valued function subject to lower and upper bounds. This algorithm looks for the global minimizers following the paths of a suitable system of stochastic differential equations. Numerical experience on several test problems known in literature is shown.     

A Post-Optimality Analysis Algorithm for Multi-Objective Optimization

Algorithms for multi-objective optimization problems are designed to generate a single Pareto optimum (non-dominated solution) or a set of Pareto optima that reflect the preferences of the decision-maker. If a set of Pareto optima are generated, then it is useful for the decision-maker to be able to obtain a small set of preferred Pareto optima using an unbiased technique of filtering solutions. This suggests the need for an efficient selection procedure to identify such a preferred subset that reflects the preferences of the decision-maker with respect to the objective functions. Selection procedures typically use a value function or a scalarizing function to express preferences among objective functions. This paper introduces and analyzes the Greedy Reduction (GR) algorithm for obtaining subsets of Pareto optima from large solution sets in multi-objective optimization. Selection of these subsets is based on maximizing a scalarizing function of the vector of percentile ordinal rankings of the Pareto optima within the larger set. A proof of optimality of the GR algorithm that relies on the non-dominated property of the vector of percentile ordinal rankings is provided. The GR algorithm executes in linear time in the worst case. The GR algorithm is illustrated on sets of Pareto optima obtained from five interactive methods for multi-objective optimization and three non-linear multi-objective test problems. These results suggest that the GR algorithm provides an efficient way to identify subsets of preferred Pareto optima from larger sets.     

A New Algorithm for Box-Constrained Global Optimization

An important class of deterministic methods for global optimization is based on the theory of terminal attractors and repellers. Unfortunately, the utilization of scalar repellers is unsuitable, when the dimension n of the problem assumes values of operational interest. In previous papers the author et al. showed that BFSG-type methods, approximating the Hessian of twice continuously differentiable functions with a structured matrix, are very efficient to compute local minima, particularly in the secant case. On the other hand, the algorithms founded on the classical αBB technique are often ineffective for computational reasons. In order to increase the power of repellers in the tunneling phases, the utilization of repeller matrices with a proper structure is certainly promising and deserves investigation. In this work, it is shown that a BFGS-type method of low complexity, implemented in the local optimizations, can be effectively matched with proper repeller matrices in the tunneling phases. The novel algorithm FBαBB, which can be applied in the frame of the αBB computational scheme, is very efficient in terms of Number of Functions Generations (NFG), Success Rates (SR) in the evaluation of the global minimum and Number of Local Searches (NLS).     

A Modified BFGS Algorithm for Unconstrained Optimization

In this paper we present a modified BFGS algorithm for unconstrainedoptimization. The BFGS algorithm updates an approximate Hessianwhich satisfies the most recent quasi-Newton equation. The quasi-Newtoncondition can be interpreted as the interpolation conditionthat the gradient value of the local quadratic model matchesthat of the objective function at the previous iterate. Ourmodified algorithm requires that the function value is matched,instead of the gradient value, at the previous iterate. Themodified algorithm preserves the global and local superlinearconvergence properties of the BFGS algorithm. Numerical resultsare presented, which suggest that a slight improvement has beenachieved.     

A Novel Genetic Algorithm for Global Optimization

This paper presents a novel genetic algorithm for globally solving un-constraint optimization problem. In this algorithm, a new real coded crossover operator is proposed firstly. Furthermore, for improving the convergence speed and the searching ability of our algorithm, the good point set theory rather than random selection is used to generate the initial population, and the chaotic search operator is adopted in the best solution of the current iteration. The experimental results tested on nume...     

无约束最优化的一个修正的类BFGS算法

利用前一步得到的曲率信息代替xk到xk+1段二次模型的曲率给出一个具有和BFGS类似的收敛性质的类BFGS算法,并揭示新算法与自调比拟牛顿法的关系.从试验函数库CUTE中选择标准试验函数,对比标准BFGS算法及其它改进BFGS算法进行数值试验.试验结果表明这个新算法的表现有点象自调比拟牛顿算法.     

A Simple Multistart Algorithm for Global Optimization

1.IntroductionConsidertheunconstrainedoptimizationproblem:findx*suchthatf(x*)~caf(x),(1)wheref(x)isanonlinearfllnctiondefinedonW"andXCR".Ourobjectiveistofindtheglobalminimizeroff(x)inthefeasibleset.Withoutassuminganyconditionsonf(x)globaloptimizationproblemsareunsolvableinthefollowingsensefnoalgorithmcanbeguaranteedtofindaglobalminimizerofageneralnonlinearfunctionwithinfinitelymanyiterations.Supposethatanalgorithmappliedtoanonlinearfunctionf(x)producesiteratesxlandterminatesafterKiterations.…     

A Derivative-Free Algorithm for Bound Constrained Optimization

In this work, we propose a new globally convergent derivative-free algorithm for the minimization of a continuously differentiable function in the case that some of (or all) the variables are bounded. This algorithm investigates the local behaviour of the objective function on the feasible set by sampling it along the coordinate directions. Whenever a suitable descent feasible coordinate direction is detected a new point is produced by performing a linesearch along this direction. The information progressively obtained during the iterates of the algorithm can be used to build an approximation model of the objective function. The minimum of such a model is accepted if it produces an improvement of the objective function value. We also derive a bound for the limit accuracy of the algorithm in the minimization of noisy functions. Finally, we report the results of a preliminary numerical experience.