Robust consensus of nonlinear multi‐agent systems via reliable control with probabilistic time delay

In this paper, the consensus problem of uncertain nonlinear multi‐agent systems is investigated via reliable control in the presence of probabilistic time‐varying delay. First, the communication topology among the agents is assumed to be directed and fixed. Second, by introducing a stochastic variable which satisfies Bernoulli distribution, the information of probabilistic time‐varying delay is equivalently transformed into the deterministic time‐varying delay with stochastic parameters. Third, by using Laplacian matrix properties, the consensus problem is converted into the conventional stability problem of the closed‐loop system. The main objective of this paper is to design a state feedback reliable controller such that for all admissible uncertainties as well as actuator failure cases, the resulting closed‐loop system is robustly stable in the sense of mean‐square. For this purpose, through construction of a suitable Lyapunov–Krasovskii functional containing four integral terms and utilization of Kronecker product properties along with the matrix inequality techniques, a new set of delay‐dependent consensus stabilizability conditions for the closed‐loop system is obtained. Based on these conditions, the desired reliable controller is designed in terms of linear matrix inequalities which can be easily solved by using any of the effective optimization algorithms. Moreover, a numerical example and its simulations are included to demonstrate the feasibility and effectiveness of the proposed control design scheme. © 2016 Wiley Periodicals, Inc. Complexity 21: 138–150, 2016     

Mean–variance asset–liability management with asset correlation risk and insurance liabilities

Consider an insurer who invests in the financial market where correlations among risky asset returns are randomly changing over time. The insurer who faces the risk of paying stochastic insurance claims needs to manage her asset and liability by taking into account of the correlation risk. This paper investigates the impact of correlation risk to the optimal asset–liability management (ALM) of an insurer. We employ the Wishart process to model the stochastic covariance matrix of risky asset returns. The insurer aims to minimize the variance of the terminal wealth given an expected terminal wealth subject to the risk of paying out random liabilities of compound Poisson process. This ALM problem then becomes a linear–quadratic stochastic optimal control problem with stochastic volatilities, stochastic correlations and jumps. The recognition of an affine form in the solution process enables us to derive the explicit closed-form solution to the optimal ALM portfolio policy, obtain the efficient frontier, and identify the condition that the solution is well behaved.     

THE DYNAMICS OF AN ANIMAL—VEGETATION SYSTEM: SHEEP FARMING

The paper studies the economy and ecology of sheep farming at the farm level in a Nordic context, with a crucial distinction between the outdoors grazing season and the winter indoors feeding season, and where climate conditions fix the length of the grazing season. Two different categories of animals, ewes (adult females) and lambs, and one plant species are included in our ecological model. The farmer is assumed to maximize present‐value profit where the revenue is made up income from meat and wool production. We find that livestock cycles may represent an optimal management policy. We also show that in a possible steady state with a constant number of animals and constant vegetation quantity, slaughtering either only lambs or only ewes is optimal.     

Capital adequacy and risk management in banking industry

The present paper deals with the issue of bank capital adequacy and risk management within a stochastic dynamic setting. In particular, an explicit risk aggregation and capital expression is provided regarding the portfolio choice and capital requirements special context. Such a framework leads to a nonlinear stochastic optimal control problem whose solution may be determined by means of dynamic programming algorithm. The pertaining analysis relies heavily on the stochastic dynamic modeling of such balance sheet items as securities, loans, and regulatory capital with stochastic interest rates. In this respect, the special Kalman filter approach is used for the purpose of estimating the model parameters. The reached findings reveal well that the Tunisian bank, subject of study, generally exceeds the minimum requirements and is adequately capitalized to maintain the appropriate capital amount level commensurate with the aggregate risk. Besides, empirical evidence on the regulations' impact on driving bank capitalization and risk‐taking behavior has also been highlighted. Copyright © 2015 John Wiley & Sons, Ltd.     

Ergodic and adaptive control of hidden Markov models

A partially observed stochastic system is described by a discrete time pair of Markov processes. The observed state process has a transition probability that is controlled and depends on a hidden Markov process that also can be controlled. The hidden Markov process is completely observed in a closed set, which in particular can be the empty set and only observed through the other process in the complement of this closed set. An ergodic control problem is solved by a vanishing discount approach. In the case when the transition operators for the observed state process and the hidden Markov process depend on a parameter and the closed set, where the hidden Markov process is completely observed, is nonempty and recurrent an adaptive control is constructed based on this family of estimates that is almost optimal.     

Nonfragile H∞ output tracking control for uncertain singular Markovian jump delay systems with network‐induced delays and data packet dropouts

This article deals with the problem of nonfragile H_∞ output tracking control for a kind of singular Markovian jump systems with time‐varying delays, parameter uncertainties, network‐induced signal transmission delays, and data packet dropouts. The main objective is to design mode‐dependent state‐feedback controller under controller gain perturbations and bounded modes transition rates such that the output of the closed‐loop networked control system tracks the output of a given reference system with the required H_∞ output tracking performance. By constructing a more multiple stochastic Lyapunov–Krasovskii functional, the novel mode‐dependent and delay‐dependent conditions are obtained to guarantee the augmented output tracking closed‐loop system is not only stochastically admissible but also satisfies a prescribed H_∞‐norm level for all signal transmission delays, data packet dropouts, and admissible uncertainties. Then, the desired state‐feedback controller parameters are determined by solving a set of strict linear matrix inequalities. A simple production system example and two numerical examples are used to verify the effectiveness and usefulness of the proposed methods. © 2015 Wiley Periodicals, Inc. Complexity 21: 396–411, 2016     

On Stochastic Stabilization of Discrete‐Time Markovian Jump Systems with Delay in State

Abstract

In this paper, we investigate the stochastic stabilization problem for a class of linear discrete time‐delay systems with Markovian jump parameters. The jump parameters considered here is modeled by a discrete‐time Markov chain. Our attention is focused on the design of linear state feedback memoryless controller such that stochastic stability of the resulting closed‐loop system is guaranteed when the system under consideration is either with or without parameter uncertainties. Sufficient conditions are proposed to solve the above problems, which are in terms of a set of solutions of coupled matrix inequalities.     

Groundwater management: Waiting for a drought*

In this article we present a stylized model for optimal management of an unconfined groundwater resource when the threat of drought exists. The drought is modeled as a stochastic event that hits at an uncertain date and two benchmark management policies are investigated: (a) A policy of optimal dynamic management ignoring the threat of drought; and (b) an economically optimal policy that accounts for the threat of a drought. We show that the optimal predrought steady‐state equilibrium stock size of groundwater under policy b is larger than that under policy (a) Furthermore, we show that an increase in the probability of a drought gives rise to two counteracting effects: One in the direction of a larger predrought steady‐state equilibrium stock size (a recovery effect) and one in the direction of a lower predrought steady‐state equilibrium stock (an extinction effect). We find that the recovery effect dominates the extinction effect. Recommendations for Resource Managers: We analyze two groundwater extraction policies that can be used when a threat of drought exists: (a) Dynamic optimal management ignoring the threat of drought; and (b) dynamic optimal management taking the threat of drought into account. We show that the predrought steady‐state equilibrium stock size of water should be larger under the policy (b) than under policy (a). This conclusion has three implications for resource managers:

• Current groundwater management should take future extraction possibilities into account.

• A resource manager ought to take the threat of drought into account in groundwater management.

• A buffer stock of water should be built‐up before the drought to be drawn upon during the event.

     

A singular control model with application to the goodwill problem

We consider a stochastic system whose uncontrolled state dynamics are modelled by a general one-dimensional Itô diffusion. The control effort that can be applied to this system takes the form that is associated with the so-called monotone follower problem of singular stochastic control. The control problem that we address aims at maximising a performance criterion that rewards high values of the utility derived from the system’s controlled state but penalises any expenditure of control effort. This problem has been motivated by applications such as the so-called goodwill problem in which the system’s state is used to represent the image that a product has in a market, while control expenditure is associated with raising the product’s image, e.g., through advertising. We obtain the solution to the optimisation problem that we consider in a closed analytic form under rather general assumptions. Also, our analysis establishes a number of results that are concerned with analytic as well as probabilistic expressions for the first derivative of the solution to a second-order linear non-homogeneous ordinary differential equation. These results have independent interest and can potentially be of use to the solution of other one-dimensional stochastic control problems.     

A MIXED CONTROL PROBLEM OF THE MANAGEMENT OF NATURAL RESOURCES

In this paper, we study the optimal solutions of a model of natural resource management which allows for both impulse and continuous harvesting policies. This type of model is known in the literature as mixed optimal control problem. In the resource management context, each type of control represents a different harvesting technology, which has a different cost. In particular, we want to know when the following conjecture made by Clark is an optimal solution to this mixed optimal control problem: if the harvesting capacity is unlimited, it is optimal to jump immediately to the steady state of the continuous time problem and then to stay there. We show that under a particular relationship between the continuous and the impulse profit function, the conjecture made by Clark is true. In other cases, however, it is either better to use only continuous control variables or to jump to resource levels which are smaller than the steady state and then let the resource grow back to the steady state. These results emphasize the importance of the cost functions in the modeling of natural resource management.     

Constrained LQ Problem with a Random Jump and Application to Portfolio Selection

This paper deals with a constrained stochastic linear-quadratic(LQ for short)optimal control problem where the control is constrained in a closed cone. The state process is governed by a controlled SDE with random coefficients. Moreover, there is a random jump of the state process. In mathematical finance, the random jump often represents the default of a counter party. Thanks to the It-Tanaka formula, optimal control and optimal value can be obtained by solutions of a system of backward stochastic differential equations(BSDEs for short). The solvability of the BSDEs is obtained by solving a recursive system of BSDEs driven by the Brownian motions. The author also applies the result to the mean variance portfolio selection problem in which the stock price can be affected by the default of a counterparty.     

Stackelberg strategies in linear-quadratic stochastic differential games

This paper obtains the Stackelberg solution to a class of two-player stochastic differential games described by linear state dynamics and quadratic objective functionals. The information structure of the problem is such that the players make independent noisy measurements of the initial state and are permitted to utilize only this information in constructing their controls. Furthermore, by the very nature of the Stackelberg solution concept, one of the players is assumed to know, in advance, the strategy of the other player (the leader). For this class of problems, we first establish existence and uniqueness of the Stackelberg solution and then relate the derivation of the leader's Stackelberg solution to the optimal solution of a nonstandard stochastic control problem. This stochastic control problem is solved in a more general context, and its solution is utilized in constructing the Stackelberg strategy of the leader. For the special case Gaussian statistics, it is shown that this optimal strategy is affine in observation of the leader. The paper also discusses numerical aspects of the Stackelberg solution under general statistics and develops algorithms which converge to the unique Stackelberg solution.This work was performed while the second author was on sabbatical leave at the Department of Applied Mathematics, Twente University of Technology, Enschede, Holland.     

MODELING REGENERATION AND PEST CONTROL ALTERNATIVES FOR A FOREST SYSTEM IN THE PRESENCE OF FIRE RISK

This paper addresses a fire and pest protection forest management modeling problem by developing a flexible model which integrates the concepts of: 1) species diversity 2) infestation of susceptible species; 3) natural regeneration and planting; 4) conversion of susceptible to non-susceptible species by planting; 5) pest protection by spraying; 6) age-specific harvesting; 7) intertemporal harvest flow policies; and 8) catastrophic loss due to fire. A linear programming (LP) model economically evaluates alternative regimes for protection spraying of susceptible forest species against insect infestations and alternative harvesting strategies which include conversion of susceptible species to non-susceptible, by planting. These strategies are evaluated subject to catastrophic loss due to fire. An iterative simulation-LP approach tests how well the deterministic model holds in a simulated stochastic environment. This validation procedure involves solving the optimization problem deterministically using average values for the fire and infestation proportions and also at each time period updating the system state by simulating the state transition for the next time period using randomly generated updates and re-solving using the updated state as the new initial condition. An optimal wood supply trajectory in a simulated stochastic environment is therefore constructed. The results from the iterative stochastic solution provide a confidence measure for the deterministic solution.     

Solution of a class of stochastic linear-convex control problems using deterministic equivalents

In this paper, we identify a new class of stochastic linearconvex optimal control problems, whose solution can be obtained by solving appropriate equivalent deterministic optimal control problems. The term linear-convex is meant to imply that the dynamics is linear and the cost function is convex in the state variables, linear in the control variables, and separable. Moreover, some of the coefficients in the dynamics are allowed to be random and the expectations of the control variables are allowed to be constrained. For any stochastic linear-convex problem, the equivalent deterministic problem is obtained. Furthermore, it is shown that the optimal feedback policy of the stochastic problem is affine in its current state, where the affine transformation depends explicitly on the optimal solution of the equivalent deterministic problem in a simple way. The result is illustrated by its application to a simple stochastic inventory control problem.This research was supported in part by NSERC Grant A4617, by SSHRC Grant 410-83-0888, and by an INRIA Post-Doctoral Fellowship.     

Stochastic optimal control of DC pension funds

In this paper, we study the portfolio problem of a pension fund manager who wants to maximize the expected utility of the terminal wealth in a complete financial market with the stochastic interest rate. Using the method of stochastic optimal control, we derive a non-linear second-order partial differential equation for the value function. As it is difficult to find a closed form solution, we transform the primary problem into a dual one by applying a Legendre transform and dual theory, and try to find an explicit solution for the optimal investment strategy under the logarithm utility function. Finally, a numerical simulation is presented to characterize the dynamic behavior of the optimal portfolio strategy.     

An optimal investment strategy in bank management

This paper considers the application of stochastic optimization theory to asset and capital adequacy management in banking. The Basel II Capital Accord lays down regulations to control bank behaviour, and relies on regulatory ratios such as the capital adequacy ratio (CAR). In an attempt to address the problem of compliance to minimum CAR and under assumptions about retained earnings, loan‐loss reserves, the market and shareholder‐bank owner relationships, we construct a continuous‐time model of the Basel II CAR which is computed from the total risk‐weighted assets (TRWAs) and bank capital in a stochastic setting. In particular, we derive an optimal equity allocation strategy for the bank and monitor the performance of the Basel II CAR under the allocation strategy. Copyright © 2011 John Wiley & Sons, Ltd.     

Valuation of variable annuities with guaranteed minimum withdrawal and death benefits via stochastic control optimization

In this paper we present a numerical valuation of variable annuities with combined Guaranteed Minimum Withdrawal Benefit (GMWB) and Guaranteed Minimum Death Benefit (GMDB) under optimal policyholder behavior solved as an optimal stochastic control problem. This product simultaneously deals with financial risk, mortality risk and human behavior. We assume that market is complete in financial risk and mortality risk is completely diversified by selling enough policies and thus the annuity price can be expressed as appropriate expectation. The computing engine employed to solve the optimal stochastic control problem is based on a robust and efficient Gauss–Hermite quadrature method with cubic spline. We present results for three different types of death benefit and show that, under the optimal policyholder behavior, adding the premium for the death benefit on top of the GMWB can be problematic for contracts with long maturities if the continuous fee structure is kept, which is ordinarily assumed for a GMWB contract. In fact for some long maturities it can be shown that the fee cannot be charged as any proportion of the account value — there is no solution to match the initial premium with the fair annuity price. On the other hand, the extra fee due to adding the death benefit can be charged upfront or in periodic installment of fixed amount, and it is cheaper than buying a separate life insurance.     

Optimal Consumption and Portfolio with Ambiguity to Markovian Switching

This paper considers the problem of maximizing expected utility from consumption and terminal wealth under model uncertainty for a general semimartingale market, where the agent with an initial capital and a random endowment can invest. To find a solution to the investment problem we use the martingale method. We first prove that under appropriate assumptions a unique solution to the investment problem exists. Then we deduce that the value functions of primal problem and dual problem are convex conjugate functions. Furthermore we consider a diffusion-jump-model where the coefficients depend on the state of a Markov chain and the investor is ambiguity to the intensity of the underlying Poisson process. Finally, for an agent with the logarithmic utility function, we use the stochastic control method to derive the Hamilton-Jacobi-Bellmann (HJB) equation. And the solution to this HJB equation can be determined numerically. We also show how thereby the optimal investment strategy can be computed.     

Optimal partially reversible investment with entry decision and general production function

This paper studies the problem of a company that adjusts its stochastic production capacity in reversible investments with controls of expansion and contraction. The company may also decide on the activation time of its production. The profit production function is of a very general form satisfying minimal standard assumptions. The objective of the company is to find an optimal entry and production decision to maximize its expected total net profit over an infinite time horizon. The resulting dynamic programming principle is a two-step formulation of a singular stochastic control problem and an optimal stopping problem. The analysis of value functions relies on viscosity solutions of the associated Bellman variational inequations. We first state several general properties and in particular smoothness results on the value functions. We then provide a complete solution with explicit expressions of the value functions and the optimal controls: the company activates its production once a fixed entry-threshold of the capacity is reached, and invests in capital so as to maintain its capacity in a closed bounded interval. The boundaries of these regions can be computed explicitly and their behavior is studied in terms of the parameters of the model.     

条件平均场随机微分方程的最优控制问题

作者研究了一个条件平均场随机微分方程的最优控制问题.这种方程和某些部分信息下的随机最优控制问题有关,并且可以看做是平均场随机微分方程的推广.作者以庞特里雅金最大值原理的形式给出最优控制满足的必要和充分条件.此外,文中给出一个线性二次最优控制问题来说明理论结果的应用.