A Trajectory-Based Method for Constrained Nonlinear Optimization Problems

A trajectory-based method for solving constrained nonlinear optimization problems is proposed. The method is an extension of a trajectory-based method for unconstrained optimization. The optimization problem is transformed into a system of second-order differential equations with the aid of the augmented Lagrangian. Several novel contributions are made, including a new penalty parameter updating strategy, an adaptive step size routine for numerical integration and a scaling mechanism. A new criterion is suggested for the adjustment of the penalty parameter. Global convergence properties of the method are established.     

Pseudo expected improvement criterion for parallel EGO algorithm

The efficient global optimization (EGO) algorithm is famous for its high efficiency in solving computationally expensive optimization problems. However, the expected improvement (EI) criterion used for picking up candidate points in the EGO process produces only one design point per optimization cycle, which is time-wasting when parallel computing can be used. In this work, a new criterion called pseudo expected improvement (PEI) is proposed for developing parallel EGO algorithms. In each cycle, the first updating point is selected by the initial EI function. After that, the PEI function is built to approximate the real updated EI function by multiplying the initial EI function by an influence function of the updating point. The influence function is designed to simulate the impact that the updating point will have on the EI function, and is only corresponding to the position of the updating point (not the function value of the updating point). Therefore, the next updating point can be identified by maximizing the PEI function without evaluating the first updating point. As the sequential process goes on, a desired number of updating points can be selected by the PEI criterion within one optimization cycle. The efficiency of the proposed PEI criterion is validated by six benchmarks with dimension from 2 to 6. The results show that the proposed PEI algorithm performs significantly better than the standard EGO algorithm, and gains significant improvements over five of the six test problems compared against a state-of-the-art parallel EGO algorithm. Furthermore, additional experiments show that it affects the convergence of the proposed algorithm significantly when the global maximum of the PEI function is not found. It is recommended to use as much evaluations as one can afford to find the global maximum of the PEI function.     

一维自适应Kalman滤波的一种最优算法

1 引言 Kalman滤波是一种用于对含有随机摄动的动态系统的最优状态估值过程。更准确地讲,Kalman滤波器是一种从受噪声干扰的观测信号中,对被观测系统的状态进行统计估值的方法,这种估值是以线性、无偏、最小方差为准则的递推估值。它被广泛地应用于空间技术、雷达、导航、通信、工业自动化、气象和地震预报、生物医学工程等领域。 虽然Kalman滤波有许多成功的应用,但是从实用角度上看它仍有一些不足。众所周知,对于一个系统模型我们往往缺少对其真正特征的认识,即系统模型中常常含有未知的参数,而这一点将严重影响滤波器的工作。     

A new approach to filtering and adaptive control

A simple distribution-free method is proposed for directly estimating and updating a criterion function without recourse to prior state space specification, updated state probabilities, and Bayes' rule. Optimality properties and efficiency advantages of the method are illustrated in terms of a two-armed bandit problem. The relationship between direct criterion function estimation and Kalman-Bucy filtering is clarified.     

A filter interior-point algorithm with projected Hessian updating for nonlinear optimization

Filter approaches, initially presented by Fletcher and Leyffer in 2002, are attractive methods for nonlinear programming. In this paper, we propose an interior-point barrier projected Hessian updating algorithm with line search filter method for nonlinear optimization. The Lagrangian function value instead of the objective function value is used in the filter. The damped BFGS updating is employed to maintain the positive definiteness of the matrices in projected Hessian updating algorithm. The numerical experiments are reported to show the effectiveness of the proposed algorithm.     

Dynamic updates of the barrier parameter in primal-dual methods for nonlinear programming

We introduce a framework in which updating rules for the barrier parameter in primal-dual interior-point methods become dynamic. The original primal-dual system is augmented to incorporate explicitly an updating function. A Newton step for the augmented system gives a primal-dual Newton step and also a step in the barrier parameter. Based on local information and a line search, the decrease of the barrier parameter is automatically adjusted. We analyze local convergence properties, report numerical experiments on a standard collection of nonlinear problems and compare our results to a state-of-the-art interior-point implementation. In many instances, the adaptive algorithm reduces the number of iterations and of function evaluations. Its design guarantees a better fit between the magnitudes of the primal-dual residual and of the barrier parameter along the iterations.     

New results in Sridhar filtering theory: The discrete case

A nonlinear time-varying adaptive filter is introduced, and its derivation using optimal control concepts is given in detail. The filter, which is called the discrete Pontryagin filter, is basically an extension to Sridhar filtering theory. The proposed approach can easily replace the conventional methods of autoregressive (AR) and autoregressive moving average (ARMA) models in their many applications. Instead of using a large number of time-invariant parameters to describe the signal or the time series, a single time-varying function is enough. This function is estimated using optimization techniques. Many features are gained using this approach, such as simpler and compact filter equations and better overall accuracy. The statistical properties of the filter are given, and it is shown that the signal estimate will converge in thepth mean to the true value.     

Descent Direction Stochastic Approximation Algorithm with Adaptive Step Sizes

A stochastic approximation (SA) algorithm with new adaptive step sizes for solving unconstrained minimization problems in noisy environment is proposed. New adaptive step size scheme uses ordered statistics of fixed number of previous noisy function values as a criterion for accepting good and rejecting bad steps. The scheme allows the algorithm to move in bigger steps and avoid steps proportional to $1/k$ when it is expected that larger steps will improve the performance. An algorithm with the new adaptive scheme is defined for a general descent direction. The almost sure convergence is established. The performance of new algorithm is tested on a set of standard test problems and compared with relevant algorithms. Numerical results support theoretical expectations and verify efficiency of the algorithm regardless of chosen search direction and noise level. Numerical results on problems arising in machine learning are also presented. Linear regression problem is considered using real data set. The results suggest that the proposed algorithm shows promise.     

Characterizations of set order relations and constrained set optimization problems via oriented distance function

Set-valued optimization problems are important and fascinating field of optimization theory and widely applied to image processing, viability theory, optimal control and mathematical economics. There are two types of criteria of solutions for the set-valued optimization problems: the vector criterion and the set criterion. In this paper, we adopt the set criterion to study the optimality conditions of constrained set-valued optimization problems. We first present some characterizations of various set order relations using the classical oriented distance function without involving the nonempty interior assumption on the ordered cones. Then using the characterizations of set order relations, necessary and sufficient conditions are derived for four types of optimal solutions of constrained set optimization problem with respect to the set order relations. Finally, the image space analysis is employed to study the c-optimal solution of constrained set optimization problems, and then optimality conditions and an alternative result for the constrained set optimization problem are established by the classical oriented distance function.     

Tuning Strategy for the Proximity Parameter in Convex Minimization

We present a new bundle method in which the use of the proximal trajectory of the cutting plane function allows the automatic tuning of the proximity parameter. An updating criterion of the stability center based on the agreement between the objective function and the polyhedral model is presented. Convergence properties are provided together with some numerical results.This research has been partially supported by the Italian Ministero dell’Istruzione, dell’Università e della Ricerca Scientifica under FIRB Project RBNE01WBBB, Large-Scale Nonlinear Optimization.     

Ergodic control of multidimensional diffusions,II: Adaptive control

The self-tuning scheme for the adaptive control of a diffusion process is studied with long-run average cost criterion and maximum likelihood estimation of parameters. Asymptotic optimality under a suitable identifiability condition is established under two alternative sets of hypotheses—a Lyapunov-type stability criterion and a condition on cost which penalizes instability.     

Updating the regularization parameter in the adaptive cubic regularization algorithm

The adaptive cubic regularization method (Cartis et al. in Math. Program. Ser. A 127(2):245?C295, 2011; Math. Program. Ser. A. 130(2):295?C319, 2011) has been recently proposed for solving unconstrained minimization problems. At each iteration of this method, the objective function is replaced by a cubic approximation which comprises an adaptive regularization parameter whose role is related to the local Lipschitz constant of the objective??s Hessian. We present new updating strategies for this parameter based on interpolation techniques, which improve the overall numerical performance of the algorithm. Numerical experiments on large nonlinear least-squares problems are provided.     

Lanczos-Based Exponential Filtering for Discrete Ill-Posed Problems

We describe regularizing iterative methods for the solution of large ill-conditioned linear systems of equations that arise from the discretization of linear ill-posed problems. The regularization is specified by a filter function of Gaussian type. A parameter determines the amount of regularization applied. The iterative methods are based on a truncated Lanczos decomposition and the filter function is approximated by a linear combination of Lanczos polynomials. A suitable value of the regularization parameter is determined by an L-curve criterion. Computed examples that illustrate the performance of the methods are presented.     

A block Chebyshev-Davidson method for linear response eigenvalue problems

We present a Chebyshev-Davidson method to compute a few smallest positive eigenvalues and corresponding eigenvectors of linear response eigenvalue problems. The method is applicable to more general linear response eigenvalue problems where some purely imaginary eigenvalues may exist. For the Chebyshev filter, a tight upper bound is obtained by a computable bound estimator that is provably correct under a reasonable condition. When the condition fails, the estimated upper bound may not be a true one. To overcome that, we develop an adaptive strategy for updating the estimated upper bound to guarantee the effectiveness of our new Chebyshev-Davidson method. We also obtain an estimate of the rate of convergence for the Ritz values by our algorithm. Finally, we present numerical results to demonstrate the performance of the proposed Chebyshev-Davidson method.     

An adaptive truncation criterion,for linesearch-based truncated Newton methods in large scale nonconvex optimization

Starting from the paper by Nash and Sofer (1990), we propose a heuristic adaptive truncation criterion for the inner iterations within linesearch-based truncated Newton methods. Our aim is to possibly avoid “over-solving” of the Newton equation, based on a comparison between the predicted reduction of the objective function and the actual reduction obtained. A numerical experience on unconstrained optimization problems highlights a satisfactory effectiveness and robustness of the adaptive criterion proposed, when a residual-based truncation criterion is selected.     

Uniqueness and extinction properties of generalised Markov branching processes

This paper focuses on the basic problems regarding uniqueness and extinction properties for generalised Markov branching processes. The uniqueness criterion is firstly established and a differential-integral equation satisfied by the transition functions of such processes is derived. The extinction probability is then obtained. A closed form is presented for both the mean extinction time and the conditional mean extinction time. It turns out that these important quantities are closely related to the elementary gamma function.     

Impulsive synchronization of time-varying dynamical network

Synchronization of time-varying dynamical network is investigated via impulsive control. Based on the Lyapunov function method and stability theory of impulsive differential equation, a synchronization criterion with respect to the system parameters and the impulsive gains and intervals is analytically derived. Further, an adaptive strategy is introduced for designing unified impulsive controllers, with a corresponding synchronization criterion derived. In this proposed adaptive control scheme, the impulsive instants adjust themselves to the needed values as time goes on, and an algorithm for determining the impulsive instants is provided and evaluated. The derived theoretical results are illustrated to be effective by several numerical examples.     

求解带箱式约束全局优化问题的滤子填充函数方法

提出一个基于滤子技术的填充函数算法, 用于求解带箱式约束的非凸全局优化问题. 填充函数算法是求解全局优化问题的有效方法之一, 而滤子技术以其良好的数值效果广泛应用于局部优化算法中. 为优化填充函数方法, 应用滤子来监控迭代过程. 首先给出一个新的填充函数并讨论了其特性, 在此基础上提出了理论算法及算法性质. 最后列出数值实验结果以说明算法的有效性.     

Estimation of IIR Systems with Binary-Valued Observations*

Estimation and control problems with binary-valued observations exist widely in practical systems. However, most of the related works are devoted to finite impulse response (FIR for short) systems, and the theoretical problem of infinite impulse response (IIR for short) systems has been less explored. To study the estimation problems of IIR systems with binary-valued observations, the authors introduce a projected recursive estimation algorithm and analyse its global convergence properties, by using the stochastic Lyapunov function methods and the limit theory on double array martingales. It is shown that the estimation algorithm has similar convergence results as those for FIR systems under a weakest possible non-persistent excitation condition. Moreover, the upper bound for the accumulated regret of adaptive prediction is also established without resorting to any excitation condition.     

Robust adaptive modified function projective synchronization of different hyperchaotic systems subject to external disturbance

Robust adaptive modified function projective synchronization between two different hyperchaotic systems is investigated, where the external uncertainties are considered and the scale factors are different from each other. The synchronization criterion is presented, which can be realized by adaptive feedback controller with compensator to eliminate the influence of uncertainties effectively. The update laws of the unknown parameters are given and the sufficient conditions are deduced based on stability theory and adaptive control. And some mistakes in the previous works are pointed out and revised. Finally, the hyperchaotic Lü and new hyperchaotic Lorenz systems are taken for example and the numerical simulations are presented to verify the effectiveness and robustness of the proposed control scheme.