Fixed-rank matrix factorizations and Riemannian low-rank optimization

Motivated by the problem of learning a linear regression model whose parameter is a large fixed-rank non-symmetric matrix, we consider the optimization of a smooth cost function defined on the set of fixed-rank matrices. We adopt the geometric framework of optimization on Riemannian quotient manifolds. We study the underlying geometries of several well-known fixed-rank matrix factorizations and then exploit the Riemannian quotient geometry of the search space in the design of a class of gradient descent and trust-region algorithms. The proposed algorithms generalize our previous results on fixed-rank symmetric positive semidefinite matrices, apply to a broad range of applications, scale to high-dimensional problems, and confer a geometric basis to recent contributions on the learning of fixed-rank non-symmetric matrices. We make connections with existing algorithms in the context of low-rank matrix completion and discuss the usefulness of the proposed framework. Numerical experiments suggest that the proposed algorithms compete with state-of-the-art algorithms and that manifold optimization offers an effective and versatile framework for the design of machine learning algorithms that learn a fixed-rank matrix.     

An optimal investment strategy for a stream of liabilities generated by a step process in a financial market driven by a Lévy process

In this paper we investigate an asset–liability management problem for a stream of liabilities written on liquid traded assets and non-traded sources of risk. We assume that the financial market consists of a risk-free asset and a risky asset which follows a geometric Lévy process. The non-tradeable factor (insurance risk or default risk) is driven by a step process with a stochastic intensity. Our framework allows us to consider financial risk, systematic and unsystematic insurance loss risk (including longevity risk), together with possible dependencies between them. An optimal investment strategy is derived by solving a quadratic optimization problem with a terminal objective and a running cost penalizing deviations of the insurer’s wealth from a specified profit-solvency target. Techniques of backward stochastic differential equations and the weak property of predictable representation are applied to obtain the optimal asset allocation.     

An optimal investment strategy for a stream of liabilities generated by a step process in a financial market driven by a Lévy process

In this paper we investigate an asset-liability management problem for a stream of liabilities written on liquid traded assets and non-traded sources of risk. We assume that the financial market consists of a risk-free asset and a risky asset which follows a geometric Lévy process. The non-tradeable factor (insurance risk or default risk) is driven by a step process with a stochastic intensity. Our framework allows us to consider financial risk, systematic and unsystematic insurance loss risk (including longevity risk), together with possible dependencies between them. An optimal investment strategy is derived by solving a quadratic optimization problem with a terminal objective and a running cost penalizing deviations of the insurer’s wealth from a specified profit-solvency target. Techniques of backward stochastic differential equations and the weak property of predictable representation are applied to obtain the optimal asset allocation.     

Estimating Stochastically Ordered Survival Functions via Geometric Programming

Many procedures have been proposed to compute nonparametric maximum likelihood estimators (NPMLEs) of survival functions under stochastic ordering constraints. However, each of them is only applicable to a specific type of stochastic ordering constraint and censoring, and is often hard to implement. In this article, we describe a general and flexible method based on geometric programming for computing the NPMLEs from right- or interval-censored data. To this end, we show that the monotonicity properties of the likelihood function and the stochastic ordering constraints considered in the literature allow us to reformulate the estimation problem as a geometric program (GP), a special type of mathematical optimization problem, which can be transformed to a convex optimization problem, and then solved globally and efficiently. We apply this GP-based method to real data examples to illustrate its generality in handling different types of ordering constraints and censoring. We also conduct simulation studies to examine its numerical performance for various sample sizes. Supplemental materials including technical details, computer code, and data files are available online.     

Evolutionary annealing: global optimization in measure spaces

Stochastic optimization methods such as evolutionary algorithms and Markov Chain Monte Carlo methods usually involve a Markov search of the optimization domain. Evolutionary annealing is an evolutionary algorithm that leverages all the information gathered by previous queries to the cost function. Evolutionary annealing can be viewed either as simulated annealing with improved sampling or as a non-Markovian selection mechanism for evolutionary algorithms. This article develops the basic algorithm and presents implementation details. Evolutionary annealing is a martingale-driven optimizer, where evaluation yields a source of increasingly refined information about the fitness function. A set of experiments with twelve standard global optimization benchmarks is performed to compare evolutionary annealing with six other stochastic optimization methods. Evolutionary annealing outperforms other methods on asymmetric, multimodal, non-separable benchmarks and exhibits strong performance on others. It is therefore a promising new approach to global optimization.     

An isogeometric boundary element approach for topology optimization using the level set method

Among the various types of structural optimization, topology has been occupying a prominent place over the last decades. It is considered the most versatile because it allows structural geometry to be determined taking into account only loading and fixing constraints. This technique is extremely useful in the design phase, which requires increasingly complex computational modeling. Modern geometric modeling techniques are increasingly focused on the use of NURBS basis functions. Consequently, it seems natural that topology optimization techniques also use this basis in order to improve computational performance. In this paper, we propose a way to integrate the isogeometric boundary techniques to topology optimization through the level set function. The proposed coupling occurs by describing the normal velocity field from the level set equation as a function of the normal shape sensitivity. This process is not well behaved in general, so some regularization technique needs to be specified. Limiting to plane linear elasticity cases, the numerical investigations proposed in this study indicate that this type of coupling allows to obtain results congruent with the current literature. Moreover, the additional computational costs are small compared to classical techniques, which makes their advantage for optimization purposes evident, particularly for boundary element method practitioners.     

Large deviations principles for stochastic scalar conservation laws

Large deviations principles for a family of scalar 1 + 1 dimensional conservative stochastic PDEs (viscous conservation laws) are investigated, in the limit of jointly vanishing noise and viscosity. A first large deviations principle is obtained in a space of Young measures. The associated rate functional vanishes on a wide set, the so-called set of measure-valued solutions to the limiting conservation law. A second order large deviations principle is therefore investigated, however, this can be only partially proved. The second order rate functional provides a generalization for non-convex fluxes of the functional introduced by Jensen and Varadhan in a stochastic particles system setting.     

梯度法简述

梯度法是一类求解优化问题的一阶方法。梯度法形式简单、计算开销小,在大规模问题的求解中得到了广泛应用。系统地介绍了光滑无约束问题梯度法的迭代格式、理论框架。梯度法中最重要的参数是步长,步长的选取直接决定了梯度法的收敛性质与收敛速度。从线搜索框架、近似技巧、随机技巧和交替重复步长四方面介绍了梯度步长的构造思想及相应梯度法的收敛性结果,还对非光滑及约束问题的梯度法、梯度法加速技巧和随机梯度法等扩展方向做了简要介绍。     

Risk-Averse Stochastic Programming and Distributionally Robust Optimization Via Operator Splitting

This work deals with a broad class of convex optimization problems under uncertainty. The approach is to pose the original problem as one of finding a zero of the sum of two appropriate monotone operators, which is solved by the celebrated Douglas-Rachford splitting method. The resulting algorithm, suitable for risk-averse stochastic programs and distributionally robust optimization with fixed support, separates the random cost mapping from the risk function composing the problem’s objective. Such a separation is exploited to compute iterates by alternating projections onto different convex sets. Scenario subproblems, free from the risk function and thus parallelizable, are projections onto the cost mappings’ epigraphs. The risk function is handled in an independent and dedicated step consisting of evaluating its proximal mapping that, in many important cases, amounts to projecting onto a certain ambiguity set. Variables get updated by straightforward projections on subspaces through independent computations for the various scenarios. The investigated approach enjoys significant flexibility and opens the way to handle, in a single algorithm, several classes of risk measures and ambiguity sets.

     

A Stochastic/Perturbation Global Optimization Algorithm for Distance Geometry Problems

We present a new global optimization approach for solving exactly or inexactly constrained distance geometry problems. Distance geometry problems are concerned with determining spatial structures from measurements of internal distances. They arise in the structural interpretation of nuclear magnetic resonance data and in the prediction of protein structure. These problems can be naturally formulated as global optimization problems which generally are large and difficult. The global optimization method that we present is related to our previous stochastic/perturbation global optimization methods for finding minimum energy configurations, but has several key differences that are important to its success. Our computational results show that the method readily solves a set of artificial problems introduced by Moré and Wu that have up to 343 atoms. On a set of considerably more difficult protein fragment problems introduced by Hendrickson, the method solves all the problems with up to 377 atoms exactly, and finds nearly exact solution for all the remaining problems which have up to 777 atoms. These preliminary results indicate that this approach has very good promise for helping to solve distance geometry problems.     

On the unimprovability of full-memory strategies in problems of guaranteed result optimization

A problem of guaranteed result optimization is considered for a control system described by an ordinary differential equation and for a performance functional that depends continuously on the trajectory of the system. The values of the control and of the disturbance satisfy compact geometric constraints. It is also assumed that the realization of the disturbance is subject to an unknown functional constraint from a given set of constraints that are compact in a Lebesgue space. It is shown that the optimal guaranteed result in the class of full-memory strategies in this problem coincides with the value of the optimal guaranteed result in the class of quasi-strategies.     

Multi-objective Geometry Optimization of a Gas Cyclone Using Triple-Fidelity Co-Kriging Surrogate Models

Cyclone separators are widely used in a variety of industrial applications. A low-mass loading gas cyclone is characterized by two performance parameters, namely the Euler and Stokes numbers. These parameters are highly sensitive to the geometrical design parameters defining the cyclone. Optimizing the cyclone geometry therefore is a complex problem. Testing a large number of cyclone geometries is impractical due to time constraints. Experimental data and even computational fluid dynamics simulations are time-consuming to perform, with a single simulation or experiment taking several weeks. Simpler analytical models are therefore often used to expedite the design process. However, this comes at the cost of model accuracy. Existing techniques used for cyclone shape optimization in literature do not take multiple fidelities into account. This work combines cheap-to-evaluate well-known mathematical models of cyclones, available data from computational fluid dynamics simulations and experimental data to build a triple-fidelity recursive co-Kriging model. This model can be used as a surrogate with a multi-objective optimization algorithm to identify a Pareto set of a finite number of solutions. The proposed scheme is applied to optimize the cyclone geometry, parametrized by seven design variables.     

On capacity expansion planning under strategic and operational uncertainties based on stochastic dominance risk averse management

A new scheme for dealing with uncertainty in scenario trees is presented for dynamic mixed 0–1 optimization problems with strategic and operational stochastic parameters. Let us generically name this type of problems as capacity expansion planning (CEP) in a given system, e.g., supply chain, production, rapid transit network, energy generation and transmission network, etc. The strategic scenario tree is usually a multistage one, and the replicas of the strategic nodes root structures in the form of either a special scenario graph or a two-stage scenario tree, depending on the type of operational activity in the system. Those operational scenario structures impact in the constraints of the model and, thus, in the decomposition methodology for solving usually large-scale problems. This work presents the modeling framework for some of the risk neutral and risk averse measures to consider for CEP problem solving. Two types of risk averse measures are considered. The first one is a time-inconsistent mixture of the chance-constrained and second-order stochastic dominance (SSD) functionals of the value of a given set of functions up to the strategic nodes in selected stages along the time horizon, The second type is a strategic node-based time-consistent SSD functional for the set of operational scenarios in the strategic nodes at selected stages. A specialization of the nested stochastic decomposition methodology for that problem solving is outlined. Its advantages and drawbacks as well as the framework for some schemes to, at least, partially avoid those drawbacks are also presented.     

Universal Method for Stochastic Composite Optimization Problems

A fast gradient method requiring only one projection is proposed for smooth convex optimization problems. The method has a visual geometric interpretation, so it is called the method of similar triangles (MST). Composite, adaptive, and universal versions of MST are suggested. Based on MST, a universal method is proposed for the first time for strongly convex problems (this method is continuous with respect to the strong convexity parameter of the smooth part of the functional). It is shown how the universal version of MST can be applied to stochastic optimization problems.     

A distributionally robust optimization model for batch nonlinear switched time-delay system considering uncertain output measurements

In this paper, we consider a nonlinear switched time-delay (NSTD) system with unknown switching times and unknown system parameters, where the output measurement is uncertain. This system is the underling dynamical system for the batch process of glycerol bioconversion to 1,3-propanediol induced by Klebsiella pneumoniae. The uncertain output measurement is regarded as a stochastic vector (whose components are stochastic variables) and the only information about its distribution is the first-order moment. The objective of this paper is to identify the unknown quantities of the NSTD system. For this, a distributionally robust optimization problem (a bi-level optimization problem) governed by the NSTD system is proposed, where the relative error under the environment of uncertain output measurements is involved in the cost functional. The bi-level optimization problem is transformed into a single-level optimization problem with non-smooth term through the application of duality theory in probability space. By applying the smoothing technique, the non-smooth term is approximated by a smooth term and the convergence of the approximation is established. Then, the gradients of the cost functional with respect to switching times and system parameters are derived. A hybrid optimization algorithm is developed to solve the transformed problem. Finally, we verify the obtained switching times and system parameters, as well as the effectiveness of the proposed algorithm, by solving this distributionally robust optimization problem.     

Backward Stochastic Riccati Equations and Infinite Horizon L-Q Optimal Control with Infinite Dimensional State Space and Random Coefficients

We study the Riccati equation arising in a class of quadratic optimal control problems with infinite dimensional stochastic differential state equation and infinite horizon cost functional. We allow the coefficients, both in the state equation and in the cost, to be random. In such a context backward stochastic Riccati equations are backward stochastic differential equations in the whole positive real axis that involve quadratic non-linearities and take values in a non-Hilbertian space. We prove existence of a minimal non-negative solution and, under additional assumptions, its uniqueness. We show that such a solution allows to perform the synthesis of the optimal control and investigate its attractivity properties. Finally the case where the coefficients are stationary is addressed and an example concerning a controlled wave equation in random media is proposed.     

Integrated process for structural–topological configuration design of weight-reduced vehicle components

Due to the speedy development trends of design stages, the material distributions and geometric configurations in view of the weight-reduction design of structures are the details taken into account from the stage of conceptual design with low cost, high performance and quality. In this point, a structural–topological configuration with a feasible design of structure is important. This paper presents the integrated process using three optimization techniques, in which the geometric boundaries and physical dimensions of the structure and material distribution of structure-configured elements are hierarchically optimized in the specifications of the maximum rigidity, plastic strain, residual deformations and lightweight.     

On large scale nonlinear Network optimization

Partial separability and partitioned quasi-Newton updating have been recently introduced and experimented with success in large scale nonlinear optimization, large nonlinear least squares calculations and in large systems of nonlinear equations. It is the purpose of this paper to apply this idea to large dimensional nonlinear network optimization problems. The method proposed thus uses these techniques for handling the cost function, while more classical tools as variable partitioning and specialized data structures are used in handling the network constraints. The performance of a code implementing this method, as well as more classical techniques, is analyzed on several numerical examples.     

Multicriteria optimization of a composite cylindrical shell subjected to thermal and dynamic actions

Multicriteria optimization of the structure and geometry of a laminated anisotropic composite shell subjected to thermal and dynamic actions is considered. From the known properties of monolayers and the given values of variable structural and geometric parameters, the thermoelastic properties of the layered anisotropic composite are determined. The criteria to be optimized—the natural frequency and the thermal stresses—depend on two variable design parameters, stochastic properties of the composite, and temperature. In the space of the optimization criteria, the domain of allowable solutions and the Pareto-optimal region are found.Translated from Mekhanika Kompozitnykh Materialov, Vol. 40, No. 6, pp. 753–760, November–December, 2004.     

Mean-variance portfolio selection for a non-life insurance company

We consider a collective insurance risk model with a compound Cox claim process, in which the evolution of a claim intensity is described by a stochastic differential equation driven by a Brownian motion. The insurer operates in a financial market consisting of a risk-free asset with a constant force of interest and a risky asset which price is driven by a Lévy noise. We investigate two optimization problems. The first one is the classical mean-variance portfolio selection. In this case the efficient frontier is derived. The second optimization problem, except the mean-variance terminal objective, includes also a running cost penalizing deviations of the insurer’s wealth from a specified profit-solvency target which is a random process. In order to find optimal strategies we apply techniques from the stochastic control theory.