Eigenvalue perturbation bounds for Hermitian block tridiagonal matrices

We derive new perturbation bounds for eigenvalues of Hermitian matrices with block tridiagonal structure. The main message of this paper is that an eigenvalue is insensitive to blockwise perturbation, if it is well-separated from the spectrum of the diagonal blocks nearby the perturbed blocks. Our bound is particularly effective when the matrix is block-diagonally dominant and graded. Our approach is to obtain eigenvalue bounds via bounding eigenvector components, which is based on the observation that an eigenvalue is insensitive to componentwise perturbation if the corresponding eigenvector components are small. We use the same idea to explain two well-known phenomena, one concerning aggressive early deflation used in the symmetric tridiagonal QR algorithm and the other concerning the extremal eigenvalues of Wilkinson matrices.     

Efficient numerical methods for pricing American options under stochastic volatility

Five numerical methods for pricing American put options under Heston's stochastic volatility model are described and compared. The option prices are obtained as the solution of a two‐dimensional parabolic partial differential inequality. A finite difference discretization on nonuniform grids leading to linear complementarity problems with M‐matrices is proposed. The projected SOR, a projected multigrid method, an operator splitting method, a penalty method, and a componentwise splitting method are considered. The last one is a direct method while all other methods are iterative. The resulting systems of linear equations in the operator splitting method and in the penalty method are solved using a multigrid method. The projected multigrid method and the componentwise splitting method lead to a sequence of linear complementarity problems with one‐dimensional differential operators that are solved using the Brennan and Schwartz algorithm. The numerical experiments compare the accuracy and speed of the considered methods. The accuracies of all methods appear to be similar. Thus, the additional approximations made in the operator splitting method, in the penalty method, and in the componentwise splitting method do not increase the error essentially. The componentwise splitting method is the fastest one. All multigrid‐based methods have similar rapid grid independent convergence rates. They are about two or three times slower that the componentwise splitting method. On the coarsest grid the speed of the projected SOR is comparable with the multigrid methods while on finer grids it is several times slower. ©John Wiley & Sons, Inc. © 2007 Wiley Periodicals, Inc. Numer Methods Partial Differential Eq, 2007     

Stochastic Relations of Random Variables and Processes

This paper generalizes the notion of stochastic order to a relation between probability measures over arbitrary measurable spaces. This generalization is motivated by the observation that for the stochastic ordering of two stationary Markov processes, it suffices that the generators of the processes preserve some, not necessarily reflexive or transitive, subrelation of the order relation. The main contributions of the paper are: a functional characterization of stochastic relations, necessary and sufficient conditions for the preservation of stochastic relations, and an algorithm for finding subrelations preserved by probability kernels. The theory is illustrated with applications to hidden Markov processes, population processes, and queueing systems.     

Convergence analysis of a global optimization algorithm using stochastic differential equations

We establish the convergence of a stochastic global optimization algorithm for general non-convex, smooth functions. The algorithm follows the trajectory of an appropriately defined stochastic differential equation (SDE). In order to achieve feasibility of the trajectory we introduce information from the Lagrange multipliers into the SDE. The analysis is performed in two steps. We first give a characterization of a probability measure (Π) that is defined on the set of global minima of the problem. We then study the transition density associated with the augmented diffusion process and show that its weak limit is given by Π.     

Estimation of an optimal solution of a LP problem with unknown objective function

We consider a linear programming problem with unknown objective function. Random observations related to the unknown objective function are sequentially available. We define a stochastic algorithm, based on the simplex method, that estimates an optimal solution of the linear programming problem. It is shown that this algorithm converges with probability one to the set of optimal solutions and that its failure probability is of order inversely proportional to the sample size. We also introduce stopping criteria for the algorithm. The asymptotic normality of some suitably defined residuals is also analyzed. The proposed estimation algorithm is motivated by the stochastic approximation algorithms but it introduces a generalization of these techniques when the linear programming problem has several optimal solutions. The proposed algorithm is also close to the stochastic quasi-gradient procedures, though their usual assumptions are weakened.Mathematics Subject Classification (2000): 90C05, 62L20, 90C15Acknowledgments. I would like to thank two unknown referees for their fruitful suggestions that have helped to improve the paper.     

Extreme behavior of bivariate elliptical distributions

This paper exploits a stochastic representation of bivariate elliptical distributions in order to obtain asymptotic results which are determined by the tail behavior of the generator. Under certain specified assumptions, we present the limiting distribution of componentwise maxima, the limiting upper copula, and a bivariate version of the classical peaks over threshold result.     

Algorithm for Predicting the Critical State of Dynamic Mass

A stochastic, weakly perturbed dynamic system with a singular diffusion matrix is investigated. An algorithm based on an action functional in Pontryagin form is proposed for estimating the probability of reaching the boundary of a given domain.     

Stochastic optimization on Bayesian nets

In this paper we are concerned with stochastic optimization problems in the case when the joint probability distribution, associated with random parameters, can be described by means of a Bayesian net. In such a case we suggest that the structured nature of the probability distribution can be exploited for designing efficient gradient estimation algorithm. Such gradient estimates can be used within the general framework of stochastic gradient (quasi-gradient) solution procedures in order to solve complex non-linear stochastic optimization problems. We describe a gradient estimation algorithm and present a case study related to the reliability of semiconductor manufacturing together with numerical experiments.     

Simulated annealing for discrete optimization with estimation

We extend the basic convergence results for the Simulated Annealing (SA) algorithm to a stochastic optimization problem where the objective function is stochastic and can be evaluated only through Monte Carlo simulation (hence, disturbed with random error). This extension is important when either the objective function cannot be evaluated exactly or such an evaluation is computationally expensive. We present a modified SA algorithm and show that under suitable conditions on the random error, the modified SA algorithm converges in probability to a global optimizer. Computational results and comparisons with other approaches are given to demonstrate the performance of the proposed modified SA algorithm.     

Practical sequential bounds for approximating two-terminal reliability

With increasing emphases on better and more reliable services, network systems have incorporated reliability analysis as an integral part in their planning, design and operation. This article first presents a simple exact decomposition algorithm for computing the NP-hard two-terminal reliability, which measures the probability of successful communication from specified source node to sink node in the network. Then a practical bounding algorithm, which is indispensable for large networks, is presented by modifying the exact algorithm for obtaining sequential lower and upper bounds on two-terminal reliability. Based on randomly generated large networks, computational experiments are conducted to compare the proposed algorithm to the well-known and widely used edge-packing approximation model and to explore the performance of the proposed bounding algorithm. Computational results reveal that the proposed bounding algorithm is superior to the edge-packing model, and the trade-off of accuracy for execution time ensures that an exact difference between upper and lower bounds on two-terminal reliability can be obtained within an acceptable time.     

Stochastic programming with fuzzy linear partial information on probability distribution

This paper deals with stochastic programming problems where the probability distribution is not explicitly known. We suppose that the probability distribution is defined by crisp or fuzzy inequalities on the probability of the different states of nature. We formulate the problem and present a solution strategy that uses the α-cut technique in order to transform our problem into a stochastic program with linear partial information on probability distribution (SPI). The obtained SPI problem is than solved using two approaches, namely, a chance constrained approach and a recourse approach. For the recourse approach, a modified L-shaped algorithm is designed and illustrated by an example.     

A stochastic extra-step quasi-Newton method for nonsmooth nonconvex optimization

In this paper, a novel stochastic extra-step quasi-Newton method is developed to solve a class of nonsmooth nonconvex composite optimization problems. We assume that the gradient of the smooth part of the objective function can only be approximated by stochastic oracles. The proposed method combines general stochastic higher order steps derived from an underlying proximal type fixed-point equation with additional stochastic proximal gradient steps to guarantee convergence. Based on suitable bounds on the step sizes, we establish global convergence to stationary points in expectation and an extension of the approach using variance reduction techniques is discussed. Motivated by large-scale and big data applications, we investigate a stochastic coordinate-type quasi-Newton scheme that allows to generate cheap and tractable stochastic higher order directions. Finally, numerical results on large-scale logistic regression and deep learning problems show that our proposed algorithm compares favorably with other state-of-the-art methods.

     

Tabu Search When Noise is Present: An Illustration in the Context of Cause and Effect Analysis

In the field of combinatorial optimization, it may be possible to more accurately represent reality through stochastic models rather than deterministic ones. When randomness is present in a problem, algorithm designers face new difficulties which complicate their task significantly. Finding a proper mathematical formulation and a fast evaluation of the objective function are two major issues. In this paper we propose a new tabu search algorithm based on sampling and statistical tests. The algorithm is shown to perform well in a stochastic environment where the quality of feasible solutions cannot be computed easily. This new search principle is illustrated in the field of cause and effect analysis where the true cause of an undesirable effect needs to be eliminated. A set of n potential causes is identified and each of them is assumed to be the true cause with a given probability. The time to investigate a cause is a random variable with a known probability distribution. Associated with each cause is the reward obtained if the cause is really the true cause. The decision problem is to sequence the n potential causes so as to maximize the expected reward realized before a specified time horizon.     

Perturbation analysis for mixed least squares–total least squares problems

In many linear parameter estimation problems, one can use the mixed least squares–total least squares (MTLS) approach to solve them. This paper is devoted to the perturbation analysis of the MTLS problem. Firstly, we present the normwise, mixed, and componentwise condition numbers of the MTLS problem, and find that the normwise, mixed, and componentwise condition numbers of the TLS problem and the LS problem are unified in the ones of the MTLS problem. In the analysis of the first‐order perturbation, we first provide an upper bound based on the normwise condition number. In order to overcome the problems encountered in calculating the normwise condition number, we give an upper bound for computing more effectively for the MTLS problem. As two estimation techniques for solving the linear parameter estimation problems, interesting connections between their solutions, their residuals for the MTLS problem, and the LS problem are compared. Finally, some numerical experiments are performed to illustrate our results.     

Stochastic ordering of Gini indexes for multivariate elliptical risks

In this paper, we show that the conjecture, made by Samanthi et al. (2016), on the ordering of Gini indexes of multivariate normal risks with respect to the strength of dependence, is not true. By using the positive semi-definite ordering of covariance matrices, we can obtain the usual stochastic order of the Gini indexes for multivariate normal risks. This can be generalized to multivariate elliptical risks. We also investigate the monotonicity of the Gini indexes in the usual stochastic order when the covariance (dispersion, resp.) matrices of multivariate normal (elliptical, resp) risks increase componentwise. In addition, we derive a large deviation result for the Gini indexes of multivariate normal risks.     

On solving discrete two-stage stochastic programs having mixed-integer first- and second-stage variables

In this paper, we propose a decomposition-based branch-and-bound (DBAB) algorithm for solving two-stage stochastic programs having mixed-integer first- and second-stage variables. A modified Benders' decomposition method is developed, where the Benders' subproblems define lower bounding second-stage value functions of the first-stage variables that are derived by constructing a certain partial convex hull representation of the two-stage solution space. This partial convex hull is sequentially generated using a convexification scheme such as the Reformulation-Linearization Technique (RLT) or lift-and-project process, which yields valid inequalities that are reusable in the subsequent subproblems by updating the values of the first-stage variables. A branch-and-bound algorithm is designed based on a hyperrectangular partitioning process, using the established property that any resulting lower bounding Benders' master problem defined over a hyperrectangle yields the same objective value as the original stochastic program over that region if the first-stage variable solution is an extreme point of the defining hyperrectangle or the second-stage solution satisfies the binary restrictions. We prove that this algorithm converges to a global optimal solution. Some numerical examples and computational results are presented to demonstrate the efficacy of this approach.     

Stabilization of Passive Nonlinear Stochastic Differential Systems by Bounded Feedback

Abstract

The purpose of this paper is to give a systematic method for global asymptotic stabilization in probability of nonlinear control stochastic differential systems the unforced dynamics of which are Lyapunov stable in probability. The approach developed in this paper is based on the concept of passivity for nonaffine stochastic differential systems together with the theory of Lyapunov stability in probability for stochastic differential equations. In particular, we prove that, as in the case of affine in the control stochastic differential systems, a nonlinear stochastic differential system is asymptotically stabilizable in probability provided its unforced dynamics are Lyapunov stable in probability and some rank conditions involving the affine part of the system coefficients are satisfied. Furthermore, for such systems, we show how a stabilizing smooth state feedback law can be designed explicitly. As an application of our analysis, we construct a dynamic state feedback compensator for a class of nonaffine stochastic differential systems.     

Stability analysis of stochastic programs with second order dominance constraints

In this paper we present a stability analysis of a stochastic optimization problem with stochastic second order dominance constraints. We consider a perturbation of the underlying probability measure in the space of regular measures equipped with pseudometric discrepancy distance (Römisch in Stochastic Programming. Elsevier, Amsterdam, pp 483–554, 2003). By exploiting a result on error bounds in semi-infinite programming due to Gugat (Math Program Ser B 88:255–275, 2000), we show under the Slater constraint qualification that the optimal value function is Lipschitz continuous and the optimal solution set mapping is upper semicontinuous with respect to the perturbation of the probability measure. In particular, we consider the case when the probability measure is approximated by an empirical probability measure and show an exponential rate of convergence of the sequence of optimal solutions obtained from solving the approximation problem. The analysis is extended to the stationary points.     

A Newton-type univariate optimization algorithm for locating the nearest extremum

This paper introduces an algorithm for univariate optimization using linear lower bounding functions (LLBF's). An LLBF over an interval is a linear function which lies below the given function over an interval and matches the given function at one end point of the interval. We first present an algorithm using LLBF's for finding the nearest root of a function in a search direction. When the root-finding method is applied to the derivative of an objective function, it is an optimization algorithm which guarantees to locate the nearest extremum along a search direction. For univariate optimization, we show that this approach is a Newton-type method, which is globally convergent with superlinear convergence rate. The applications of this algorithm to global optimization and other optimization problems are also discussed.     

加性二值噪声激励下Duffing系统的随机分岔

研究了Duffing系统在加性二值噪声作用下的随机分岔现象.首先,根据二值噪声的统计特性,推导得到二值噪声状态间的跃迁概率,据此对二值噪声进行了数值模拟.其次,利用四阶Runge-Kutta(龙格-库塔)数值算法得到该系统位移和速率的稳态联合概率密度及位移的稳态概率密度.然后,通过对位移稳态概率密度单双峰结构变化的研究,发现加性二值噪声的状态和强度能够诱导系统产生随机分岔现象.最后,观察到随着系统非对称参数的逐渐变化,系统同样产生了随机分岔现象.