Improving an interior-point approach for large block-angular problems by hybrid preconditioners

The computational time required by interior-point methods is often dominated by the solution of linear systems of equations. An efficient specialized interior-point algorithm for primal block-angular problems has been used to solve these systems by combining Cholesky factorizations for the block constraints and a conjugate gradient based on a power series preconditioner for the linking constraints. In some problems this power series preconditioner resulted to be inefficient on the last interior-point iterations, when the systems became ill-conditioned. In this work this approach is combined with a splitting preconditioner based on LU factorization, which works well for the last interior-point iterations. Computational results are provided for three classes of problems: multicommodity flows (oriented and nonoriented), minimum-distance controlled tabular adjustment for statistical data protection, and the minimum congestion problem. The results show that, in most cases, the hybrid preconditioner improves the performance and robustness of the interior-point solver. In particular, for some block-angular problems the solution time is reduced by a factor of 10.     

A SELF-ADAPTIVE TECHNIQUE FOR A KIND OF NONLINEAR CONJUGATE GRADIENT METHODS

Conjugate gradient methods are a class of important methods for unconstrained optimization, especially when the dimension is large. In 2001, Dai and Liao have proposed a new conjugate condition, based on it two nonlinear conjugate gradient methods are constructed. With trust region idea, this paper gives a self-adaptive technique for the two methods. The numerical results show that this technique works well for the given nonlinear optimization test problems.     

A Proposal for Toeplitz Matrix Calculations

In contrast to the usual (and successful) direct methods for Toeplitz systems Ax = b, we propose an algorithm based on the conjugate gradient method. The preconditioner is a circulant, so that all matrices have constant diagonals and all matrix-vector multiplications use the Fast Fourier Transform. We also suggest a technique for the eigenvalue problem, where current methods are less satisfactory. If the first indications are supported by further experiment, this new approach may have useful applications—including nearly Toeplitz systems, and parallel computations.     

A descent hybrid conjugate gradient method based on the memoryless BFGS update

In this work, we present a new hybrid conjugate gradient method based on the approach of the convex hybridization of the conjugate gradient update parameters of DY and HS+, adapting a quasi-Newton philosophy. The computation of the hybrization parameter is obtained by minimizing the distance between the hybrid conjugate gradient direction and the self-scaling memoryless BFGS direction. Furthermore, a significant property of our proposed method is that it ensures sufficient descent independent of the accuracy of the line search. The global convergence of the proposed method is established provided that the line search satisfies the Wolfe conditions. Our numerical experiments on a set of unconstrained optimization test problems from the CUTEr collection indicate that our proposed method is preferable and in general superior to classic conjugate gradient methods in terms of efficiency and robustness.     

A class of improved conjugate gradient methods for nonconvex unconstrained optimization

In this paper, based on a new class of conjugate gradient methods which are proposed by Rivaie, Dai and Omer et al. we propose a class of improved conjugate gradient methods for nonconvex unconstrained optimization. Different from the above methods, our methods possess the following properties: (i) the search direction always satisfies the sufficient descent condition independent of any line search; (ii) these approaches are globally convergent with the standard Wolfe line search or standard Armijo line search without any convexity assumption. Moreover, our numerical results also demonstrated the efficiencies of the proposed methods.     

Symmetric Perry conjugate gradient method

A family of new conjugate gradient methods is proposed based on Perry’s idea, which satisfies the descent property or the sufficient descent property for any line search. In addition, based on the scaling technology and the restarting strategy, a family of scaling symmetric Perry conjugate gradient methods with restarting procedures is presented. The memoryless BFGS method and the SCALCG method are the special forms of the two families of new methods, respectively. Moreover, several concrete new algorithms are suggested. Under Wolfe line searches, the global convergence of the two families of the new methods is proven by the spectral analysis for uniformly convex functions and nonconvex functions. The preliminary numerical comparisons with CG_DESCENT and SCALCG algorithms show that these new algorithms are very effective algorithms for the large-scale unconstrained optimization problems. Finally, a remark for further research is suggested.     

Two new conjugate gradient methods based on modified secant equations

Following the approach proposed by Dai and Liao, we introduce two nonlinear conjugate gradient methods for unconstrained optimization problems. One of our proposed methods is based on a modified version of the secant equation proposed by Zhang, Deng and Chen, and Zhang and Xu, and the other is based on the modified BFGS update proposed by Yuan. An interesting feature of our methods is their account of both the gradient and function values. Under proper conditions, we show that one of the proposed methods is globally convergent for general functions and that the other is globally convergent for uniformly convex functions. To enhance the performance of the line search procedure, we also propose a new approach for computing the initial steplength to be used for initiating the procedure. We provide a comparison of implementations of our methods with the efficient conjugate gradient methods proposed by Dai and Liao, and Hestenes and Stiefel. Numerical test results show the efficiency of our proposed methods.     

约束优化问题的修正共轭梯度投影算法

对闭凸集约束的非线性规划问题构造了一个修正共轭梯度投影下降算法,在去掉迭代点列有界的条件下,分析了算法的全局收敛性.新算法与共轭梯度参数结合,给出了三类结合共轭梯度参数的修正共轭梯度投影算法.数值例子表明算法是有效的.     

Multi-step nonlinear conjugate gradient methods for unconstrained minimization

Conjugate gradient methods are appealing for large scale nonlinear optimization problems, because they avoid the storage of matrices. Recently, seeking fast convergence of these methods, Dai and Liao (Appl. Math. Optim. 43:87–101, 2001) proposed a conjugate gradient method based on the secant condition of quasi-Newton methods, and later Yabe and Takano (Comput. Optim. Appl. 28:203–225, 2004) proposed another conjugate gradient method based on the modified secant condition. In this paper, we make use of a multi-step secant condition given by Ford and Moghrabi (Optim. Methods Softw. 2:357–370, 1993; J. Comput. Appl. Math. 50:305–323, 1994) and propose two new conjugate gradient methods based on this condition. The methods are shown to be globally convergent under certain assumptions. Numerical results are reported.     

A new family of conjugate gradient methods

In this paper we develop a new class of conjugate gradient methods for unconstrained optimization problems. A new nonmonotone line search technique is proposed to guarantee the global convergence of these conjugate gradient methods under some mild conditions. In particular, Polak–Ribiére–Polyak and Liu–Storey conjugate gradient methods are special cases of the new class of conjugate gradient methods. By estimating the local Lipschitz constant of the derivative of objective functions, we can find an adequate step size and substantially decrease the function evaluations at each iteration. Numerical results show that these new conjugate gradient methods are effective in minimizing large-scale non-convex non-quadratic functions.     

Conjugate distributions in statistical problems with nuisance parameters

Developing the concept of distributions that are conjugate to a given likelihood function of an unknown parameter in statistical problems of the Bayesian kind, we introduce the notion of locally conjugate distribution families. We investigate some properties of conjugate and locally conjugate families, which are then applied to analytic elimination of the nuisance parameters. The potentialities of this approach are compared with those of the well-known approach related to naturally conjugate distributions. To illustrate the results obtained, we solve a series of known statistical problems. Proceedings of the Seminar on Stability Problems for Stochastic Models. Moscow, Russia, 1996, Part II.     

A new family of hybrid three-term conjugate gradient methods with applications in image restoration

In this paper, based on the hybrid conjugate gradient method and the convex combination technique, a new family of hybrid three-term conjugate gradient methods are proposed for solving unconstrained optimization. The conjugate parameter in the search direction is a hybrid of Dai-Yuan conjugate parameter and any one. The search direction then is the sum of the negative gradient direction and a convex combination in relation to the last search direction and the gradient at the previous iteration. Without choosing any specific conjugate parameters, we show that the search direction generated by the family always possesses the descent property independent of line search technique, and that it is globally convergent under usual assumptions and the weak Wolfe line search. To verify the effectiveness of the presented family, we further design a specific conjugate parameter, and perform medium-large-scale numerical experiments for smooth unconstrained optimization and image restoration problems. The numerical results show the encouraging efficiency and applicability of the proposed methods even compared with the state-of-the-art methods.

     

Numerical conformal mapping methods based on Faber series

Methods are presented for approximating the conformal map from the interior of various regions to the interior of simply-connected target regions with a smooth boundary. The methods for the disk due to Fornberg (1980) and the ellipse due to DeLillo and Elcrat (1993) are reformulated so that they may be extended to other new computational regions. The case of a cross-shaped region is introduced and developed. These methods are used to circumvent the severe ill-conditioning due to the crowding phenomenon suffered by conformal maps from the unit disk to target regions with elongated sections while preserving the fast Fourier methods available on the disk. The methods are based on expanding the mapping function in the Faber series for the regions. All of these methods proceed by approximating the boundary correspondence of the map with a Newton-like iteration. At each Newton step, a system of linear equations is solved using the conjugate gradient method. The matrix-vector multiplication in this inner iteration can be implemented with fast Fourier transforms at a cost of O(N log N). It is shown that the linear systems are discretizations of the identity plus a compact operator and so the conjugate gradient method converges superlinearly. Several computational examples are given along with a discussion of the accuracy of the methods.     

Stock market forecasting by using a hybrid model of exponential fuzzy time series

The initial aim of this study is to propose a hybrid method based on exponential fuzzy time series and learning automata based optimization for stock market forecasting. For doing so, a two-phase approach is introduced. In the first phase, the optimal lengths of intervals are obtained by applying a conventional fuzzy time series together with learning automata swarm intelligence algorithm to tune the length of intervals properly. Subsequently, the obtained optimal lengths are applied to generate a new fuzzy time series, proposed in this study, named exponential fuzzy time series. In this final phase, due to the nature of exponential fuzzy time series, another round of optimization is required to estimate certain method parameters. Finally, this model is used for future forecasts. In order to validate the proposed hybrid method, forty-six case studies from five stock index databases are employed and the findings are compared with well-known fuzzy time series models and classic methods for time series. The proposed model has outperformed its counterparts in terms of accuracy.     

初始点任意的解非线性不等式约束优化问题的结合共轭梯度参数的超记忆梯度广义投影算法

本文利用广义投影矩阵，对求解无约束规划的超记忆梯度算法中的参数给出一种新的取值范围以保证得到目标函数的超记忆梯度广义投影下降方向，并与处理任意初始点的方法技巧结合建立求解非线性不等式约束优化问题的一个初始点任意的超记忆梯度广义投影算法，在较弱条件下证明了算法的收敛性．同时给出结合FR，PR，HS共轭梯度参数的超记忆梯度广义投影算法，从而将经典的共轭梯度法推广用于求解约束规划问题．数值例子表明算法是有效的．     

解带线性或非线性约束最优化问题的三项记忆梯度Rosen投影算法

利用Rosen投影矩阵，建立求解带线性或非线性不等式约束优化问题的三项记忆梯度Rosen投影下降算法，并证明了算法的收敛性．同时给出了结合FR，PR，HS共轭梯度参数的三项记忆梯度Rosen投影算法，从而将经典的共轭梯度法推广用于求解约束规划问题．数值例子表明算法是有效的。     

On the absolute matrix summability of Fourier series and some associated series

The object of the paper is to study the absolute matrix summability problem of Fourier series, conjugate series and some associated series under a new set of conditions on matrix methods, generalising many known results in the literature.     

Goldstein线搜索下混合共轭梯度法的全局收敛性

本文结合FR算法和DY算法,给出了一类新的杂交共轭梯度算法,并结合Goldstein线搜索,在较弱的条件下证明了算法的收敛性．数值实验表明了新算法的有效性．     

Efficient generalized conjugate gradient algorithms,part 2: Implementation

In Part 1 of this paper (Ref. 1), a new, generalized conjugate gradient algorithm was proposed and its convergence investigated. In this second part, the new algorithm is compared numerically with other modified conjugate gradient methods and with limited-memory quasi-Newton methods.     

Two effective hybrid conjugate gradient algorithms based on modified BFGS updates

Based on two modified secant equations proposed by Yuan, and Li and Fukushima, we extend the approach proposed by Andrei, and introduce two hybrid conjugate gradient methods for unconstrained optimization problems. Our methods are hybridizations of Hestenes-Stiefel and Dai-Yuan conjugate gradient methods. Under proper conditions, we show that one of the proposed algorithms is globally convergent for uniformly convex functions and the other is globally convergent for general functions. To enhance the performance of the line search procedure, we propose a new approach for computing the initial value of the steplength for initiating the line search procedure. We give a comparison of the implementations of our algorithms with two efficiently representative hybrid conjugate gradient methods proposed by Andrei using unconstrained optimization test problems from the CUTEr collection. Numerical results show that, in the sense of the performance profile introduced by Dolan and Moré, the proposed hybrid algorithms are competitive, and in some cases more efficient.