Parameter identification for stochastic hybrid systems using randomized optimization: A case study on subtilin production by Bacillus subtilis

In this paper we study the parameter identification problem for a stochastic hybrid model of the production of the antibiotic subtilin by the bacterium B. subtilis. We pursue a simulation-based approach, in which the fit of candidate parameter values is evaluated by comparing simulated model trajectories with experimental data. Several score functions are considered to capture the goodness of the fit. Parameter estimation is accomplished via an evolutionary strategy that iteratively selects the best fitting parameters. Identifiability issues are discussed and are explored numerically by a Markov Chain Monte Carlo approach.     

An Unified Computational Framework for Parameter Identification of Material Models in Finite Inelasticity

Parameter identification processes concern the determination of parameters in a material model in order to fit experimental data. We provide a distinct, unified algorithmic setting of a generic class of material models and discuss the associated gradient–based optimization problem. Gradient–based optimization algorithms need derivatives of the objective function with respect to the material parameter vector κ . In order to obtain the necessary derivatives, an analytical sensitivity analysis is pointed out for the unified class of algorithmic material models. The quality of the parameter identification is demonstrated for a representative example.     

Extracting interpretable fuzzy models for nonlinear systems using gradient-based continuous ant colony optimization

This paper exploits the ability of a novel ant colony optimization algorithm called gradient-based continuous ant colony optimization, an evolutionary methodology, to extract interpretable first-order fuzzy Sugeno models for nonlinear system identification. The proposed method considers all objectives of system identification task, namely accuracy, interpretability, compactness and validity conditions. First, an initial structure of model is obtained by means of subtractive clustering. Then, an iterative two-step algorithm is employed to produce a simplified fuzzy model in terms of number of fuzzy sets and rules. In the first step, the parameters of the model are adjusted by utilizing the gradient-based continuous ant colony optimization. In the second step, the similar membership functions of an obtained model merge. The results obtained on three case studies illustrate the applicability of the proposed method to extract accurate and interpretable fuzzy models for nonlinear system identification.     

Solving the robust two-stage capacitated facility location problem with uncertain transportation costs

In this study, we start from a multi-source variant of the two-stage capacitated facility location problem (TSCFLP) and propose a robust optimization model of the problem that involves the uncertainty of transportation costs. Since large dimensions of the robust TSCFLP could not be solved to optimality, we design a memetic algorithm (MA), which represents a combination of an evolutionary algorithm (EA) and a modified simulated annealing heuristic (SA) that uses a short-term memory of undesirable moves from previous iterations. A set of computational experiments is conducted to examine the impact of different protection levels on the deviation of the objective function value. We also investigate the impact of variations of transportation costs that may occur on both transhipment stages on the total cost for a fixed protection level. The obtained results may help in identifying a sustainable and efficient strategy for designing a two stage capacitated transportation network with uncertain transportation costs, and may be applicable in the design and management of similar transportation networks.     

A survey on OR and mathematical methods applied on gene-environment networks

In this paper, we survey the recent advances and mathematical foundations of gene-environment networks. We explain their interdisciplinary implications with special regard to human and life sciences as well as financial sciences. Special attention is paid to applications in Operational Research and environmental protection. Originally developed in the context of modeling and prediction of gene-expression patterns, gene-environment networks have proved to provide a conceptual framework for the modeling of dynamical systems with respect to errors and uncertainty as well as the influence of certain environmental items. Given the noise-prone measurement data we extract nonlinear differential equations to describe and analyze the interactions and regulating effects between the data items of interest and the environmental items. In particular, these equations reflect data uncertainty by the use of interval arithmetics and comprise unknown parameters resulting in a wide variety of the model. For an identification of these parameters Chebychev approximation and generalized semi-infinite optimization are applied. In addition, the time-discrete counterparts of the nonlinear equations are introduced and their parametrical stability is investigated by a combinatorial algorithm which detects the region of parameter stability. Finally, we analyze the topological landscape of the gene-environment networks in terms of structural stability. We conclude with an application of our analysis and introduce the eco-finance networks.     

Optimal parameter selection for nonlinear multistage systems with time-delays

In this paper, we consider a novel dynamic optimization problem for nonlinear multistage systems with time-delays. Such systems evolve over multiple stages, with the dynamics in each stage depending on both the current state of the system and the state at delayed times. The optimization problem involves choosing the values of the time-delays, as well as the values of additional parameters that influence the system dynamics, to minimize a given cost functional. We first show that the partial derivatives of the system state with respect to the time-delays and system parameters can be computed by solving a set of auxiliary dynamic systems in conjunction with the governing multistage system. On this basis, a gradient-based optimization algorithm is proposed to determine the optimal values of the delays and system parameters. Finally, two example problems, one of which involves parameter identification for a realistic fed-batch fermentation process, are solved to demonstrate the algorithm’s effectiveness.     

Modeling mortality with a Bayesian vector autoregression

Parametric mortality models capture the cross section of mortality rates. These models fit the older ages better, because of the more complex cross section of mortality at younger and middle ages. Dynamic parametric mortality models fit a time series to the parameters, such as a Vector-auto-regression (VAR), in order to capture trends and uncertainty in mortality improvements. We consider the full age range using the Heligman and Pollard (1980) model, a cross-sectional mortality model with parameters that capture specific features of different age ranges. We make the Heligman–Pollard model dynamic using a Bayesian Vector Autoregressive (BVAR) model for the parameters and compare with more commonly used VAR models. We fit the models using Australian data, a country with similar mortality experience to many developed countries. We show how the Bayesian Vector Autoregressive (BVAR) models improve forecast accuracy compared to VAR models and quantify parameter risk which is shown to be significant.     

Application of particle swarm optimization in chaos synchronization in noisy environment in presence of unknown parameter uncertainty

In this paper, particle swarm optimization (PSO) is applied to synchronize chaotic systems in presence of parameter uncertainties and measurement noise. Particle swarm optimization is an evolutionary algorithm which is introduced by Kennedy and Eberhart. This algorithm is inspired by birds flocking. Optimization algorithms can be applied to control by defining an appropriate cost function that guarantees stability of system. In presence of environment noise and parameter uncertainty, robustness plays a crucial role in succeed of controller. Since PSO needs only rudimentary information about the system, it can be a suitable algorithm for this case. Simulation results confirm that the proposed controller can handle the uncertainty and environment noise without any extra information about them. A comparison with some earlier works is performed during simulations.     

Improving the accuracy while preserving the interpretability of fuzzy function approximators by means of multi-objective evolutionary algorithms

The identification of a model is one of the key issues in the field of fuzzy system modeling and function approximation theory. An important characteristic that distinguishes fuzzy systems from other techniques in this area is their transparency and interpretability. Especially in the construction of a fuzzy system from a set of given training examples, little attention has been paid to the analysis of the trade-off between complexity and accuracy maintaining the interpretability of the final fuzzy system. In this paper a multi-objective evolutionary approach is proposed to determine a Pareto-optimum set of fuzzy systems with different compromises between their accuracy and complexity. In particular, two fundamental and competing objectives concerning fuzzy system modeling are addressed: fuzzy rule parameter optimization and the identification of system structure (i.e. the number of membership functions and fuzzy rules), taking always in mind the transparency of the obtained system. Another key aspect of the algorithm presented in this work is the use of some new expert evolutionary operators, specifically designed for the problem of fuzzy function approximation, that try to avoid the generation of worse solutions in order to accelerate the convergence of the algorithm.     

A preprocessing method for parameter estimation in ordinary differential equations

Parameter estimation for nonlinear differential equations is notoriously difficult because of poor or even no convergence of the nonlinear fit algorithm due to the lack of appropriate initial parameter values. This paper presents a method to gather such initial values by a simple estimation procedure. The method first determines the tangent slope and coordinates for a given solution of the ordinary differential equation (ODE) at randomly selected points in time. With these values the ODE is transformed into a system of equations, which is linear for linear appearance of the parameters in the ODE. For numerically generated data of the Lorenz attractor good estimates are obtained even at large noise levels. The method can be generalized to nonlinear parameter dependency. This case is illustrated using numerical data for a biological example. The typical problems of the method as well as their possible mitigation are discussed. Since a rigorous failure criterion of the method is missing, its results must be checked with a nonlinear fit algorithm. Therefore the method may serve as a preprocessing algorithm for nonlinear parameter fit algorithms. It can improve the convergence of the fit by providing initial parameter estimates close to optimal ones.     

AR模型辨识的稀疏结构迭代算法

模型估计是机器学习领域一个重要的研究内容,动态数据的模型估计是系统辨识和系统控制的基础.针对AR时间序列模型辨识问题,证明了在给定阶数下AR模型参数的最小二乘估计本质上也是一种矩估计.根据结构风险最小化原理,通过对模型拟合度和模型复杂度的折衷,提出了基于稀疏结构迭代的AR序列模型估计算法,并讨论了基于广义岭估计的最优正则化参数选取规则.数值结果表明,方法能以节省参数的方式有效地实现AR模型的辨识,比矩估计法结果有明显改善.     

Adaptive Inverse Control Using a Gradient-Projection Optimization Technique

As an application of an optimization technique, a gradient-projection method is employed to derive an adaptive algorithm for updating the parameters of an inverse which is designed to cancel the effects of actuator uncertainties in a control system. The actuator uncertainty is parametrized by a set of unknown parameters which belong to a parameter region. A desirable inverse is implemented with adaptive estimates of the actuator parameters. Minimizing an estimation error, a gradient algorithm is used to update such parameter estimates. To ensure that the parameter estimates also belong to the parameter region, the adaptive update law is designed with parameter projection. With such an adaptive inverse, desired control system performance can be achieved despite the presence of the actuator uncertainties.     

On the analysis of a dynamic evolutionary algorithm

Evolutionary algorithms are applied as problem-independent optimization algorithms. They are quite efficient in many situations. However, it is difficult to analyze even the behavior of simple variants of evolutionary algorithms like the (1+1) EA on rather simple functions. Nevertheless, only the analysis of the expected run time and the success probability within a given number of steps can guide the choice of the free parameters of the algorithms. Here static (1+1) EAs with a fixed mutation probability are compared with dynamic (1+1) EAs with a simple schedule for the variation of the mutation probability. The dynamic variant is first analyzed for functions typically chosen as example-functions for evolutionary algorithms. Afterwards, it is shown that it can be essential to choose the suitable variant of the (1+1) EA. More precisely, functions are presented where each static (1+1) EA has exponential expected run time while the dynamic variant has polynomial expected run time. For other functions it is shown that the dynamic (1+1) EA has exponential expected run time while a static (1+1) EA with a good choice of the mutation probability has polynomial run time with overwhelming probability.     

Robust combinatorial optimization with variable cost uncertainty

We present in this paper a new model for robust combinatorial optimization with cost uncertainty that generalizes the classical budgeted uncertainty set. We suppose here that the budget of uncertainty is given by a function of the problem variables, yielding an uncertainty multifunction. The new model is less conservative than the classical model and approximates better Value-at-Risk objective functions, especially for vectors with few non-zero components. An example of budget function is constructed from the probabilistic bounds computed by Bertsimas and Sim. We provide an asymptotically tight bound for the cost reduction obtained with the new model. We turn then to the tractability of the resulting optimization problems. We show that when the budget function is affine, the resulting optimization problems can be solved by solving n+1$n+1$ deterministic problems. We propose combinatorial algorithms to handle problems with more general budget functions. We also adapt existing dynamic programming algorithms to solve faster the robust counterparts of optimization problems, which can be applied both to the traditional budgeted uncertainty model and to our new model. We evaluate numerically the reduction in the price of robustness obtained with the new model on the shortest path problem and on a survivable network design problem.     

The analysis of evolutionary algorithms on sorting and shortest paths problems

The analysis of evolutionary algorithms is up to now limited to special classes of functions and fitness landscapes. E.g., it is not possible to characterize the set of TSP instances (or another NP-hard combinatorial optimization problem) which are solved by a generic evolutionary algorithm (EA) in an expected time bounded by some given polynomial. As a first step from artificial functions to typical problems from combinatorial optimization, we analyze simple EAs on well-known problems, namely sorting and shortest paths. Although it cannot be expected that EAs outperform the well-known problem specific algorithms on these simple problems, it is interesting to analyze how EAs work on these problems. The following results are obtained:– Sorting is the maximization of sortedness which is measured by one of several well-known measures of presortedness. The different measures of presortedness lead to fitness functions of quite different difficulty for EAs.– Shortest paths problems are hard for all types of EA, if they are considered as single-objective optimization problems, whereas they are easy as multi-objective optimization problems.     

Parameter identification of Rossler’s chaotic system by an evolutionary algorithm

In this paper, a differential evolution (DE) algorithm is applied to parameter identification of Rossler’s chaotic system. The differential evolution has been shown to possess a powerful searching capability for finding the solutions for a given optimization problem, and it allows for parameter solution to appear directly in the form of floating point without further numerical coding or decoding. Three unknown parameters of Rossler’s Chaotic system are optimally estimated by using the DE algorithm. Finally, a numerical example is given to verify the effectiveness of the proposed method.     

Sampling Schemes for Bayesian Variable Selection in Generalized Linear Models

Bayesian approaches to prediction and the assessment of predictive uncertainty in generalized linear models are often based on averaging predictions over different models, and this requires methods for accounting for model uncertainty. When there are linear dependencies among potential predictor variables in a generalized linear model, existing Markov chain Monte Carlo algorithms for sampling from the posterior distribution on the model and parameter space in Bayesian variable selection problems may not work well. This article describes a sampling algorithm based on the Swendsen-Wang algorithm for the Ising model, and which works well when the predictors are far from orthogonality. In problems of variable selection for generalized linear models we can index different models by a binary parameter vector, where each binary variable indicates whether or not a given predictor variable is included in the model. The posterior distribution on the model is a distribution on this collection of binary strings, and by thinking of this posterior distribution as a binary spatial field we apply a sampling scheme inspired by the Swendsen-Wang algorithm for the Ising model in order to sample from the model posterior distribution. The algorithm we describe extends a similar algorithm for variable selection problems in linear models. The benefits of the algorithm are demonstrated for both real and simulated data.     

A two-phase tabu search based evolutionary algorithm for the maximum diversity problem

In this paper, we study the maximum diversity problem (MDP) which is equivalent to the quadratic unconstrained binary optimization (QUBO) problem with cardinality constraint. The MDP aims to select a subset of elements with given cardinality such that the sum of pairwise distances between any two elements in the selected subset is maximized. For solving this computationally challenging problem, we propose a two-phase tabu search based evolutionary algorithm (TPTS/EA), which integrates several distinguishing features to ensure the diversity and the quality of the evolution, such as a two-phase tabu search algorithm which consists of a dynamic candidate list (DCL) strategy-based traditional tabu search in the first phase and a solution-based tabu search procedure to refine the search in the second phase, and two path-relinking based recombination operators to generate new offspring solutions. Tested on three sets of totally 140 public instances in the literature, the study demonstrates the efficacy of the proposed TPTS/EA algorithm in terms of both solution quality and computational efficiency. Specifically, our proposed TPTS/EA algorithm is able to improve the previous best known results for 2 instances, while matching the previous best-known solutions for 130 instances. We also provide experimental evidences to highlight the beneficial effect of several important components in our TPTS/EA algorithm.     

Fast computing of some generalized linear mixed pseudo-models with temporal autocorrelation

This paper considers ways to increase computational speed in generalized linear mixed pseudo-models for the case of many repeated measurements on subjects. We obtain linearly increasing computing time with number of observations, as opposed to O(n ³) increasing computing time using numerical optimization. We also find a surprising result; that incomplete optimization for covariance parameters within the larger parameter estimation algorithm actually decreases time to convergence. After comparing various computing algorithms and choosing the best one, we fit a generalized linear mixed model to a binary time series data set with over 100 fixed effects, 50 random effects, and approximately 1.5 ×  10⁵ observations.     

Process planning in a fuzzy environment

Mixed-integer optimization models for chemical process planning typically assume that model parameters can be accurately predicted. As precise forecasts are difficult to obtain, process planning usually involves uncertainty and ambiguity in the data. This paper presents an application of fuzzy programming to process planning. The forecast parameters are assumed to be fuzzy with a linear or triangular membership function. The process planning problem is then formulated in terms of decision making in a fuzzy environment with fuzzy constraints and fuzzy net present value goals. The model is transformed to a deterministic mixed-integer linear program or mixed-integer nonlinear program depending on the type of uncertainty involved in the problem. For the nonlinear case, a global optimization algorithm is developed for its solution. This algorithm is applicable to general possibilistic programs and can be used as an alternative to the commonly used bisection method. Illustrative examples and computational results for a petrochemical complex with 38 processes and 24 products illustrate the applicability of the developed models and algorithms.