On Lanczos Based Methods for the Regularization of Discrete Ill-Posed Problems

We study hybrid methods for the solution of linear ill-posed problems. Hybrid methods are based on he Lanczos process, which yields a sequence of small bidiagonal systems approximating the original ill-posed problem. In a second step, some additional regularization, typically the truncated SVD, is used to stabilize the iteration. We investigate two different hybrid methods and interpret these schemes as well-known projection methods, namely least-squares projection and the dual least-squares method. Numerical results are provided to illustrate the potential of these methods. This gives interesting insight in to the behavior of hybrid methods in practice.This revised version was published online in October 2005 with corrections to the Cover Date.     

BiCR variants of the hybrid BiCG methods for solving linear systems with nonsymmetric matrices

We propose Bi-Conjugate Residual (BiCR) variants of the hybrid Bi-Conjugate Gradient (BiCG) methods (referred to as the hybrid BiCR variants) for solving linear systems with nonsymmetric coefficient matrices. The recurrence formulas used to update an approximation and a residual vector are the same as those used in the corresponding hybrid BiCG method, but the recurrence coefficients are different; they are determined so as to compute the coefficients of the residual polynomial of BiCR. From our experience it appears that the hybrid BiCR variants often converge faster than their BiCG counterpart. Numerical experiments show that our proposed hybrid BiCR variants are more effective and less affected by rounding errors. The factor in the loss of convergence speed is analyzed to clarify the difference of the convergence between our proposed hybrid BiCR variants and the hybrid BiCG methods.     

Accelerated hybrid and shrinking projection methods for variational inequality problems

ABSTRACT

In this paper, we introduce several new extragradient-like approximation methods for solving variational inequalities in Hilbert spaces. Our algorithms are based on Tseng's extragradient method, subgradient extragradient method, inertial method, hybrid projection method and shrinking projection method. Strong convergence theorems are established under appropriate conditions. Our results extend and improve some related results in the literature. In addition, the efficiency of our algorithms is shown through numerical examples which are defined by the hybrid projection methods.     

Numerical Solution to Hybrid Stochastic Differential Systems

Abstract

In this article numerical methods for solving hybrid stochastic differential systems of Itô-type are developed by piecewise application of numerical methods for SDEs. We prove a convergence result if the corresponding method for SDEs is numerically stable with uniform convergence in the mean square sense. The Euler and Runge–Kutta methods for hybrid stochastic differential equations are specifically described and the order of the error is given for the Euler method. A numerical example is given to illustrate the theory.     

A unified study of continuous and discontinuous Galerkin methods

A unified study is presented in this paper for the design and analysis of different finite element methods(FEMs), including conforming and nonconforming FEMs, mixed FEMs, hybrid FEMs, discontinuous Galerkin(DG) methods, hybrid discontinuous Galerkin(HDG) methods and weak Galerkin(WG) methods.Both HDG and WG are shown to admit inf-sup conditions that hold uniformly with respect to both mesh and penalization parameters. In addition, by taking the limit of the stabilization parameters, a WG method is shown to converge to a mixed method whereas an HDG method is shown to converge to a primal method. Furthermore,a special class of DG methods, known as the mixed DG methods, is presented to fill a gap revealed in the unified framework.     

Block hybrid method using trigonometric basis for initial value problems with oscillating solutions

A continuous hybrid method using trigonometric basis (CHMTB) with one ‘off-step’ point is developed and used to produce two discrete hybrid methods which are simultaneously applied as numerical integrators by assembling them into a block hybrid method with trigonometric basis (BHMTB) for solving oscillatory initial value problems (IVPs). The stability property of the BHMTB is discussed and the performance of the method is demonstrated on some numerical examples to show accuracy and efficiency advantages.     

Direct and indirect methods for trajectory optimization

This paper gives a brief list of commonly used direct and indirect efficient methods for the numerical solution of optimal control problems. To improve the low accuracy of the direct methods and to increase the convergence areas of the indirect methods we suggest a hybrid approach. For this a special direct collocation method is presented. In a hybrid approach this direct method can be used in combination with multiple shooting. Numerical examples illustrate the direct method and the hybrid approach.     

Hybrid methods for large sparse nonlinear least squares

Hybrid methods are developed for improving the Gauss-Newton method in the case of large residual or ill-conditioned nonlinear least-square problems. These methods are used usually in a form suitable for dense problems. But some standard approaches are unsuitable, and some new possibilities appear in the sparse case. We propose efficient hybrid methods for various representations of the sparse problems. After describing the basic ideas that help deriving new hybrid methods, we are concerned with designing hybrid methods for sparse Jacobian and sparse Hessian representations of the least-square problems. The efficiency of hybrid methods is demonstrated by extensive numerical experiments.This work was supported by the Czech Republic Grant Agency, Grant 201/93/0129. The author is indebted to Jan Vlek for his comments on the first draft of this paper and to anonymous referees for many useful remarks.     

Digital contracts-driven method for pricing complex derivatives

In this paper, we propose a hybrid method of nonparametric and parametric methods, that is a digital contracts-driven (DCD) method, for pricing various complex options. Differing from general nonparametric data-driven methods, in which usually the observed data are used as training data directly, in the DCD method the European-style digital contracts of the underlying assets are used as basic inputs for a learning network. The digital contracts calculated from the observed data based upon the parametric method are used as hints in the learning process, and then enable the DCD method to have superior pricing accuracy to the common data-driven method in practical applications. Some Monte Carlo simulation experiments are performed and the results demonstrate that the proposed hybrid method not only has the advantages of generality and superior accuracy as the nonparametric method, but also the robust property to financial data with noise as the parametric method.     

Sufficient Descent Riemannian Conjugate Gradient Methods

This paper considers sufficient descent Riemannian conjugate gradient methods with line search algorithms. We propose two kinds of sufficient descent nonlinear conjugate gradient method and prove that these methods satisfy the sufficient descent condition on Riemannian manifolds. One is a hybrid method combining a Fletcher–Reeves-type method with a Polak–Ribière–Polyak-type method, and the other is a Hager–Zhang-type method, both of which are generalizations of those used in Euclidean space. Moreover, we prove that the hybrid method has a global convergence property under the strong Wolfe conditions and the Hager–Zhang-type method has the sufficient descent property regardless of whether a line search is used or not. Further, we review two kinds of line search algorithm on Riemannian manifolds and numerically compare our generalized methods by solving several Riemannian optimization problems. The results show that the performance of the proposed hybrid methods greatly depends on the type of line search used. Meanwhile, the Hager–Zhang-type method has the fast convergence property regardless of the type of line search used.

     

on hybrid preconditioning methods for large sparse saddle-point problems

Based on the block-triangular product approximation to a 2-by-2 block matrix, a class of hybrid preconditioning methods is designed for accelerating the MINRES method for solving saddle-point problems. The appropriate values for the parameters involved in the new preconditioners are estimated, so that the numerical conditioning and the spectral property of the saddle-point matrix of the linear system can be substantially improved. Several practical hybrid preconditioners and the corresponding preconditioning iterative methods are constructed and studied, too.     

Uncertainty propagation in stochastic fractional order processes using spectral methods: A hybrid approach

Stochastic spectral methods are widely used in uncertainty propagation thanks to its ability to obtain highly accurate solution with less computational demand. A novel hybrid spectral method is proposed here that combines generalized polynomial chaos (gPC) and operational matrix approaches. The hybrid method takes advantage of gPC’s efficient handling of large parameter uncertainties and overcomes its limited applicability to systems with relatively highly correlated inputs. The hybrid method’s use of operational matrices allows analyses of systems with low input correlations without suffering its restriction to small parameter uncertainties. The hybrid method is aimed to propagate uncertainties in fractional order systems with random parameters and random inputs with low correlation lengths. It is validated through several examples with different stochastic uncertainties. Comparison with Monte Carlo and gPC demonstrates the superior computational efficiency of the proposed method.     

A hybrid iterative method for symmetric indefinite linear systems

Hybrid iterative methods that combine a conjugate direction method with a simpler iteration scheme, such as Chebyshev or Richardson iteration, were first proposed in the 1950s. The ease with which Chebyshev and Richardson iteration can be implemented efficiently on a large variety of computer architectures has in recent years lead to renewed interest in iterative methods that use Chebyshev or Richardson iteration. This paper presents a new hybrid iterative method for the solution of linear systems of equations with a symmetric indefinite matrix. Our method combines the conjugate residual method with Richardson iteration. Special attention is paid to the determination of two real intervals, one on each side of the origin, that contain most of the eigenvalues of the matrix. These intervals are used to compute suitable iteration parameters for Richardson iteration. We also discuss when to switch between the methods. The hybrid scheme typically uses the Richardson method for most iterations, and this reduces the number of arithmetic vector operations significantly compared with the number of arithmetic vector operations required when only the conjugate residual method is used. Computed examples illustrate the competitiveness of the hybrid scheme.     

A Linear Hybridization of Dai-Yuan and Hestenes-Stiefel Conjugate Gradient Method for Unconstrained Optimization

Conjugate gradient methods are interesting iterative methods that solve large scale unconstrained optimization problems. A lot of recent research has thus focussed on developing a number of conjugate gradient methods that are more effective. In this paper, we propose another hybrid conjugate gradient method as a linear combination of Dai-Yuan (DY) method and the Hestenes-Stiefel (HS) method. The sufficient descent condition and the global convergence of this method are established using the generalized Wolfe line search conditions. Compared to the other conjugate gradient methods, the proposed method gives good numerical results and is effective.     

A CSP Versus a Zonotope-Based Method for Solving Guard Set Intersection in Nonlinear Hybrid Reachability

Computing the reachable set of hybrid dynamical systems in a reliable and verified way is an important step when addressing verification or synthesis tasks. This issue is still challenging for uncertain nonlinear hybrid dynamical systems. We show in this paper how to combine a method for computing continuous transitions via interval Taylor methods and a method for computing the geometrical intersection of a flowpipe with guard sets, to build an interval method for reachability computation that can be used with truly nonlinear hybrid systems. Our method for flowpipe guard set intersection has two variants. The first one relies on interval constraint propagation for solving a constraint satisfaction problem and applies in the general case. The second one computes the intersection of a zonotope and a hyperplane and applies only when the guard sets are linear. The performance of our method is illustrated on examples involving typical hybrid systems.     

混合杂交罚函数有限元方法及其应用

对一般非协调有限元,目前采用最多的两种方法是罚函数法和混合、杂交法.前一种方法总能保证收敛,但精度差,条件数和稀疏性不好;后一种方法则要满足“秩条件”才能保证收敛,故单元的构造受到很大的限制.本文提出把这两种方法结合一起的有限元方法——混合杂交罚函数法.从理论上严格证明了(在非常一般的条件下)这种新方法总是收敛的,并且其精度、条件数以及稀疏性等皆与协调元相同,也就是说都是最优的. 最后应用这一方法具体构造了一个新的九自由度任意三角形弯板单元(每个顶点给三个自由度——一个位移和两个转角),其单元刚度矩阵计算公式与旧的九自由度三角形弯板单元的计算公式相差不多.但它对任意几何形状的平板都收敛于真解,如果真解u∈H3的话,它的三个弯矩具有一阶精度,位移及两个转角均具有二阶精度.     

A Hybrid of the Newton-GMRES and Electromagnetic Meta-Heuristic Methods for Solving Systems of Nonlinear Equations

Solving systems of nonlinear equations is perhaps one of the most difficult problems in all numerical computation. Although numerous methods have been developed to attack this class of numerical problems, one of the simplest and oldest methods, Newton’s method is arguably the most commonly used. As is well known, the convergence and performance characteristics of Newton’s method can be highly sensitive to the initial guess of the solution supplied to the method. In this paper a hybrid scheme is proposed, in which the Electromagnetic Meta-Heuristic method (EM) is used to supply a good initial guess of the solution to the finite difference version of the Newton-GMRES method (NG) for solving a system of nonlinear equations. Numerical examples are given in order to compare the performance of the hybrid of the EM and NG methods. Empirical results show that the proposed method is an efficient approach for solving systems of nonlinear equations.     

Computing reachable sets for uncertain nonlinear hybrid systems using interval constraint-propagation techniques

We investigate solution techniques for numerical constraint-satisfaction problems and validated numerical set integration methods for computing reachable sets of nonlinear hybrid dynamical systems in the presence of uncertainty. To use interval simulation tools with higher-dimensional hybrid systems, while assuming large domains for either initial continuous state or model parameter vectors, we need to solve the problem of flow/sets intersection in an effective and reliable way. The main idea developed in this paper is first to derive an analytical expression for the boundaries of continuous flows, using interval Taylor methods and techniques for controlling the wrapping effect. Then, the event detection and localization problems underlying flow/sets intersection are expressed as numerical constraint-satisfaction problems, which are solved using global search methods based on branch-and-prune algorithms, interval analysis and consistency techniques. The method is illustrated with hybrid systems with uncertain nonlinear continuous dynamics and nonlinear invariants and guards.     

A new constraint test-case generator and the importance of hybrid optimizers

In the past decade, significant progress has been made in understanding problem complexity of discrete constraint problems. In contrast, little similar work has been done for constraint problems in the continuous domain. In this paper, we study the complexity of typical methods for non-linear constraint problems and present hybrid solvers with improved performance. To facilitate the empirical study, we propose a new test-case generator for generating non-linear constraint satisfaction problems (CSPs) and constrained optimization problems (COPs). The optimization methods tested include a sequential quadratic programming (SQP) method, a penalty method with a fixed penalty function, a penalty method with a sequence of penalty functions, and an augmented Lagrangian method. For hybrid solvers, we focus on the form that combines two or more optimization methods in sequence. In the experiments, we apply these methods to solve a series of continuous constraint problems with increasing constraint-to-variable ratios. The test problems include artificial benchmark problems from the test-case generator and problems derived from controlling a hyper-redundant modular manipulator. We obtain novel results on complexity phase transition phenomena of the various methods. Specifically, for constraint satisfaction problems, the SQP method is the best on weakly constrained problems, whereas the augmented Lagrangian method is the best on highly constrained ones. Although the static penalty method performs poorly by itself, by combining it with the SQP method, we show a hybrid solver that is significantly better than any of the individual methods on problems with moderate to large constraint-to-variable ratios. For constrained optimization problems, the hybrid solver obtains much better solutions than SQP, while spending comparable amount of time. In addition, the hybrid solver is flexible and can achieve good results on time-bounded applications by setting parameters according to the time limits.     

无记忆非拟 Newton 算法的导出和全局收敛性

In this paper, a new class of memoryless non-quasi-Newton method for solving unconstrained optimization problems is proposed, and the global convergence of this method with inexact line search is proved. Furthermore, we propose a hybrid method that mixes both the memoryless non-quasi-Newton method and the memoryless Perry-Shanno quasi-Newton method. The global convergence of this hybrid memoryless method is proved under mild assumptions. The initial results show that these new methods are efficient for the given test problems. Especially the memoryless non-quasi-Newton method requires little storage and computation, so it is able to efficiently solve large scale optimization problems.