Linear Systems Associated with Numerical Methods for Constrained Optimization

Linear systems associated with numerical methods for constrained optimization are discussed in thia paper ,It is shown that the corresponding subproblems arise in most well-known methods,no matter line search methods or trust region methods for constrained optimization can be expressed as similar systems of linear equations.All these linear systems can be viewed as some kinds of approximation to the linear system derived by the Lagrange-Newton method .Some properties of these linear systems are analyzed.     

A Path Algorithm for Constrained Estimation

Many least-square problems involve affine equality and inequality constraints. Although there are a variety of methods for solving such problems, most statisticians find constrained estimation challenging. The current article proposes a new path-following algorithm for quadratic programming that replaces hard constraints by what are called exact penalties. Similar penalties arise in l ₁ regularization in model selection. In the regularization setting, penalties encapsulate prior knowledge, and penalized parameter estimates represent a trade-off between the observed data and the prior knowledge. Classical penalty methods of optimization, such as the quadratic penalty method, solve a sequence of unconstrained problems that put greater and greater stress on meeting the constraints. In the limit as the penalty constant tends to ∞, one recovers the constrained solution. In the exact penalty method, squared penalties are replaced by absolute value penalties, and the solution is recovered for a finite value of the penalty constant. The exact path-following method starts at the unconstrained solution and follows the solution path as the penalty constant increases. In the process, the solution path hits, slides along, and exits from the various constraints. Path following in Lasso penalized regression, in contrast, starts with a large value of the penalty constant and works its way downward. In both settings, inspection of the entire solution path is revealing. Just as with the Lasso and generalized Lasso, it is possible to plot the effective degrees of freedom along the solution path. For a strictly convex quadratic program, the exact penalty algorithm can be framed entirely in terms of the sweep operator of regression analysis. A few well-chosen examples illustrate the mechanics and potential of path following. This article has supplementary materials available online.     

Linear Relationships for Constrained Multiple-item Inventory Systems

Inventory control is a problem common to many businesses andmanufacturing systems. Management spends a fair amount of timetrying to determine appropriate inventory policies. The firstinventory model, which is still used widely today, is the economicorder quantity (EOQ) model of Harris. This model, however, doesnot consider constraints on the problem such as storage capacityor budget Limitations. It is the purpose of this paper to examinethe multiple-item deterministic inventory system with one linearconstraint, and establish linear relationships between the Lagrangemultiplier of the constrgint and artain system characteristics.It is then discussed how the insights from these relationshipscan be used to provide tight hunds to the optimal multiplier.Computational results for these bounds are also included.     

Direct Numerical Algorithm for Constrained Variational Problems

We propose a direct treatment for the numerical simulation of optimal solutions for vector, one-dimensional variational problems under pointwise constraints in the form of several inequalities. It is an iterative procedure to approximate the optimal solutions of such variational problems that rely on our ability to e?ciently approximate the optimal solutions of variational problems without restrictions, except possibly for end point constraints. One main advantage is that there is no need to control the free boundary, or the contact set, during the iterative process where constraints are active. In addition to proving some convergence results, the scheme is illustrated through several typical situations.     

Smooth and Semismooth Newton Methods for Constrained Approximation and Estimation

In the article, we show that the constrained L ₂ approximation problem, the positive polynomial interpolation, and the density estimation problems can all be reformulated as a system of smooth or semismooth equations by using Lagrange duality theory. The obtained equations contain integral functions of the same form. The differentiability or (strong) semismoothness of the integral functions and the Hölder continuity of the Jacobian of the integral function were investigated. Then a globalized Newton-type method for solving these problems was introduced. Global convergence and numerical tests for estimating probability density functions with wavelet basis were also given. The research in this article not only strengthened the theoretical results in literatures but also provided a possibility for solving the probability density function estimation problem by Newton-type method.     

Equations of Motion for Constrained Multibody Systems and their Control

This paper presents some recent advances in the dynamics and control of constrained multibody systems. The constraints considered need not satisfy the D’Alembert principle and therefore the results are of general applicability. They show that, in the presence of constraints, the constraint force acting on the multibody system can always be viewed as made up of the sum of two components whose explicit form is provided. The first of these components consists of the constraint force that would have existed were all the constraints ideal; the second is caused by the nonideal nature of the constraints, and though it needs specification by the mechanician who is modeling the specific system at hand, it has a specific form. The general equations of motion obtained herein provide new insights into the simplicity with which Nature seems to operate. They point toward the development of new and novel approaches for the exact control of complex multibody nonlinear systems. In honor of Bob kalaba, friend, colleague, and mentor.     

Empirical Balanced Truncation of Nonlinear Systems

Novel constructions of empirical controllability and observability gramians for nonlinear systems are proposed for subsequent use in a balanced truncation style of model reduction. The new gramians are based on a generalisation of the fundamental solution for a Linear Time-Varying system. Relationships between the given gramians for nonlinear systems and the standard gramians for both Linear Time-Invariant and Linear Time-Varying systems are established as well as relationships to prior constructions proposed for empirical gramians. Application of the new gramians is illustrated through a sample test-system.     

Structural Stability of Constrained Polynomial Systems

The structural stability of constrained polynomial differentialsystems of the form a(x, y)x'+b(x, y)y'=f(x, y), c(x, y)x'+d(x,y)y'=g(x, y), under small perturbations of the coefficientsof the polynomial functions a, b, c, d, f and g is studied.These systems differ from ordinary differential equations at‘impasse points’ defined by ad–bc=0. Extensionsto this case of results for smooth constrained differentialsystems [7] and for ordinary polynomial differential systems[5] are achieved here. 1991 Mathematics Subject Classification34C35, 34D30.     

Constrained Controllability of Fractional Dynamical Systems

The article deals with the controllability results for fractional dynamical systems with prescribed controls represented by the fractional integrodifferential equation in finite dimensional spaces. Sufficient conditions for the controllability results of nonlinear fractional dynamical systems are obtained using the contraction mapping principle. Examples are included to illustrate the theory.     

非线性模型中的拟似然估计和约束拟似然估计

该文证明了,在非线性回归模型中,若以均方误差或均方误差矩阵为标准,拟似然估计是正则广义拟似然估计类中的最优估计,并讨论了拟得分函数最优性与拟似然估计最优性的关系.为改进拟似然估计,该文提出了一种约束拟似然估计,并证明了约束拟似然估计比拟似然估计有较小的均方误差.     

约束力学系统的联络及其运动方程的测地性质

用现代整体微分几何方法研究非定常约束力学系统运动方程的测地性质，得到非定常力学系统的动力学流关于１＿射丛上的联络具有测地性质的充分必要条件·非定常情形下的动力学流关于无挠率的联络总具有测地性质，因此任何非定常约束力学系统在外力作用下的运动总可以表示为关于１＿射丛上无挠率的动力学联络的测地运动，这与定常力学的情形有所区别·     

Value Estimation Approach to the Iri-Imai Method for Constrained Convex Optimization

In this paper we propose an extension of the so-called Iri-Imai method to solve constrained convex programming problems. The original Iri-Imai method is designed for linear programs and assumes that the optimal objective value of the optimization problem is known in advance. Zhang (Ref. 9) extends the method for constrained convex optimization but the optimum value is still assumed to be known in advance. In our new extension this last requirement on the optimal value is relaxed; instead only a lower bound of the optimal value is needed. Our approach uses a multiplicative barrier function for the problem with a univariate parameter that represents an estimated optimum value of the original optimization problem. An optimal solution to the original problem can be traced down by minimizing the multiplicative barrier function. Due to the convexity of this barrier function the optimal objective value as well as the optimal solution of the original problem are sought iteratively by applying Newtons method to the multiplicative barrier function. A new formulation of the multiplicative barrier function is further developed to acquire computational tractability and efficiency. Numerical results are presented to show the efficiency of the new method.His research supported by Hong Kong RGC Earmarked Grant CUHK4233/01E.Communicated by Z. Q. Luo     

Constrained Stochastic Estimation Algorithms for a Class of Hybrid Stock Market Models

This paper is concerned with a class of hybrid stock market models, in which both the return rate and the volatility depend on a hidden, continuous-time Markov chain with a finite state space. One of the crucial issues is to estimate the generator of the underlying Markov chain. We develop a stochastic optimization procedure for this task, prove its convergence, and establish the rate of convergence. Numerical tests are carried out via simulation as well as using real market data. In addition, we demonstrate how to use the estimated generator in making stock liquidation decisions.     

一阶微分方程数值解的一种误差估计方法

针对于微分方程数值解,介绍了一种新的误差估计方法.方法证实了伪谱方法具有精度高速度快的优点,进而引出了修正的伪谱方法.     

Hölder Continuity of Adjoint States and Optimal Controls for State Constrained Problems

We investigate Hölder regularity of adjoint states and optimal controls for a Bolza problem under state constraints. We start by considering any optimal solution satisfying the constrained maximum principle in its normal form and we show that whenever the associated Hamiltonian function is smooth enough and has some monotonicity properties in the directions normal to the constraints, then both the adjoint state and optimal trajectory enjoy Hölder type regularity. More precisely, we prove that if the state constraints are smooth, then the adjoint state and the derivative of the optimal trajectory are Hölder continuous, while they have the two sided lower Hölder continuity property for less regular constraints. Finally, we provide sufficient conditions for Hölder type regularity of optimal controls.     

约束优化的一个改进的强次可行SQP算法及数值试验

本文对约束优化一个强次可行SQP算法进行改进,使之产生的迭代点在有限次迭代后全落入可行域;并对算法数值效果进行了大量的比较试验.     

Optimal Control of Constrained Piecewise Affine Discrete-Time Systems

This paper makes two contributions; firstly, it provides a characterization of the solution of the optimal control problem for piecewise affine discrete-time systems with a quadratic cost function (the generally preferred option) and, secondly, provides a simple method (reverse transformation) for solving this and the previously solved &ell problem. The characterization is useful for on-line implementation.