Asymptotic Analysis of Linear Feedback Nash Equilibria in Nonzero-Sum Linear-Quadratic Differential Games

In this paper, we discuss nonzero-sum linear-quadratic differential games. For this kind of games, the Nash equilibria for different kinds of information structures were first studied by Starr and Ho. Most of the literature on the topic of nonzero-sum linear-quadratic differential games is concerned with games of fixed, finite duration; i.e., games are studied over a finite time horizon t _f. In this paper, we study the behavior of feedback Nash equilibria for t _f.In the case of memoryless perfect-state information, we study the so-called feedback Nash equilibrium. Contrary to the open-loop case, we note that the coupled Riccati equations for the feedback Nash equilibrium are inherently nonlinear. Therefore, we limit the dynamic analysis to the scalar case. For the special case that all parameters are scalar, a detailed dynamical analysis is given for the quadratic system of coupled Riccati equations. We show that the asymptotic behavior of the solutions of the Riccati equations depends strongly on the specified terminal values. Finally, we show that, although the feedback Nash equilibrium over any fixed finite horizon is generically unique, there can exist several different feedback Nash equilibria in stationary strategies for the infinite-horizon problem, even when we restrict our attention to Nash equilibria that are stable in the dynamical sense.     

Magnus integrators for solving linear-quadratic differential games

We consider Magnus integrators to solve linear-quadratic N$N$-player differential games. These problems require to solve, backward in time, non-autonomous matrix Riccati differential equations which are coupled with the linear differential equations for the dynamic state of the game, to be integrated forward in time. We analyze different Magnus integrators which can provide either analytical or numerical approximations to the equations. They can be considered as time-averaging methods and frequently are used as exponential integrators. We show that they preserve some of the most relevant qualitative properties of the solution for the matrix Riccati differential equations as well as for the remaining equations. The analytical approximations allow us to study the problem in terms of the parameters involved. Some numerical examples are also considered which show that exponential methods are, in general, superior to standard methods.     

Linear–quadratic stochastic two-person nonzero-sum differential games: Open-loop and closed-loop Nash equilibria

In this paper, we consider a linear–quadratic stochastic two-person nonzero-sum differential game. Open-loop and closed-loop Nash equilibria are introduced. The existence of the former is characterized by the solvability of a system of forward–backward stochastic differential equations, and that of the latter is characterized by the solvability of a system of coupled symmetric Riccati differential equations. Sometimes, open-loop Nash equilibria admit a closed-loop representation, via the solution to a system of non-symmetric Riccati equations, which could be different from the outcome of the closed-loop Nash equilibria in general. However, it is found that for the case of zero-sum differential games, the Riccati equation system for the closed-loop representation of an open-loop saddle point coincides with that for the closed-loop saddle point, which leads to the conclusion that the closed-loop representation of an open-loop saddle point is the outcome of the corresponding closed-loop saddle point as long as both exist. In particular, for linear–quadratic optimal control problem, the closed-loop representation of an open-loop optimal control coincides with the outcome of the corresponding closed-loop optimal strategy, provided both exist.     

Stackelberg strategies in linear-quadratic stochastic differential games

This paper obtains the Stackelberg solution to a class of two-player stochastic differential games described by linear state dynamics and quadratic objective functionals. The information structure of the problem is such that the players make independent noisy measurements of the initial state and are permitted to utilize only this information in constructing their controls. Furthermore, by the very nature of the Stackelberg solution concept, one of the players is assumed to know, in advance, the strategy of the other player (the leader). For this class of problems, we first establish existence and uniqueness of the Stackelberg solution and then relate the derivation of the leader's Stackelberg solution to the optimal solution of a nonstandard stochastic control problem. This stochastic control problem is solved in a more general context, and its solution is utilized in constructing the Stackelberg strategy of the leader. For the special case Gaussian statistics, it is shown that this optimal strategy is affine in observation of the leader. The paper also discusses numerical aspects of the Stackelberg solution under general statistics and develops algorithms which converge to the unique Stackelberg solution.This work was performed while the second author was on sabbatical leave at the Department of Applied Mathematics, Twente University of Technology, Enschede, Holland.     

Systems of matrix rational differential equations arising in connection with linear stochastic systems with Markovian jumping

In this paper, a class of systems of matrix nonlinear differential equations containing as particular cases the systems of coupled Riccati differential equations arising in connection with control of some linear stochastic systems is considered.The system of differential equations considered in this paper are converted in a suitable nonlinear differential equation on a finite-dimensional Hilbert space adequately choosen.This allows us to use the positivity properties of the linear evolution operator defined by the linear differential equations of Lyapunov type.Our aim is to investigate properties of stabilizing and bounded solutions of the considered differential equations and to obtain some conditions ensuring the existence of such solutions.Conditions providing the existence of a maximal solution (minimal solution respectively) with respect to some classes of global solutions are presented. It is shown that if the coefficients of the equations are periodic functions all these special solutions (stabilizing, maximal, minimal) are periodic functions, too.Whenever possible the probabilistic arguments were avoided and so the results proved in the paper appear as results in the field of differential equations with interest in themselves.     

Differential games with mixed leadership: The open-loop solution

This paper introduces the notion of mixed leadership in nonzero-sum differential games, where there is no fixed hierarchy in decision making with respect to the players. Whether a particular player is leader or follower depends on the instrument variable s/he is controlling, and it is possible for a player to be both leader and follower, depending on the control variable. The paper studies two-player open-loop differential games in this framework, and obtains a complete set of equations (differential and algebraic) which yield the controls in the mixed-leadership Stackelberg solution. The underlying differential equations are coupled and have mixed-boundary conditions. The paper also discusses the special case of linear-quadratic differential games, in which case solutions to the coupled differential equations can be expressed in terms of solutions to coupled Riccati differential equations which are independent of the state trajectory.     

Existence and comparison theorems for partial differential equations of Riccati type

In this paper, we discuss the partial differential equation of Riccati type that describes the optimal filtering error covariance function for a linear distributed-parameter system with pointwise observations. Since this equation contains the Dirac delta function, it is impossible to apply directly the usual methods of functional analysis to prove existence and uniqueness of a bounded solution. By using properties of the fundamental solution and the classical technique of successive approximation, we prove the existence and uniqueness theorem. We then prove the comparison theorem for partial differential equations of Riccati type. Finally, we consider some applications of these theorems to the distributed-parameter optimal sensor location problem.     

On the existence of the stabilizing solution of generalized Riccati equations arising in zero-sum stochastic difference games: the time-varying case

In this paper, a large class of time-varying Riccati equations arising in stochastic dynamic games is considered. The problem of the existence and uniqueness of some globally defined solution, namely the bounded and stabilizing solution, is investigated. As an application of the obtained existence results, we address in a second step the problem of infinite-horizon zero-sum two players linear quadratic (LQ) dynamic game for a stochastic discrete-time dynamical system subject to both random switching of its coefficients and multiplicative noise. We show that in the solution of such an optimal control problem, a crucial role is played by the unique bounded and stabilizing solution of the considered class of generalized Riccati equations.     

Recursive solution of linear-quadratic Nash games for weakly interconnected systems

A recursive method is developed for the solution of coupled algebraic Riccati equations and corresponding linear Nash strategies of weakly interconnected systems. It is shown that the given algorithm converges to the exact solution with the rate of convergence ofO(²), where is a small coupling parameter. In addition, only low-order systems are involved in algebrdic computations; the amount of computations required does not grow per iteration and no analyticity assumption is imposed on the system coefficients.This work was supported by Rutgers University Research Council under Grant No. 2-02188.     

Global existence of solutions to a coupled coagulation system

We consider a model equations describing the coagulation process of a gas on a surface. The problem is modeled by two coupled equations. The first one is a nonlinear transport equation with bilinear coagulation operator while the second one is a nonlinear ordinary differential equation. The velocity and the boundary condition of the transport equation depend on the supersaturation function satisfying the nonlinear ode. We first prove global existence and uniqueness of solution to the nonlinear transport equation then, we consider the coupled problem and prove existence in the large of solutions to the full coagulation system.     

A Type of General Forward-Backward Stochastic Differential Equations and Applications

The authors discuss one type of general forward-backward stochastic differential equations (FBSDEs) with It?o’s stochastic delayed equations as the forward equations and anticipated backward stochastic differential equations as the backward equations. The existence and uniqueness results of the general FBSDEs are obtained. In the framework of the general FBSDEs in this paper, the explicit form of the optimal control for linearquadratic stochastic optimal control problem with delay and the Nash equilibrium point for nonzero sum differential games problem with delay are obtained.     

Stationary solutions of SPDEs and infinite horizon BDSDEs

In this paper we study the existence of stationary solutions for stochastic partial differential equations. We establish a new connection between valued solutions of backward doubly stochastic differential equations (BDSDEs) on infinite horizon and the stationary solutions of the SPDEs. Moreover, we prove the existence and uniqueness of the solutions of BDSDEs on both finite and infinite horizons, so obtain the solutions of initial value problems and the stationary solutions (independent of any initial value) of SPDEs. The connection of the weak solutions of SPDEs and BDSDEs has independent interests in the areas of both SPDEs and BSDEs.     

Two-tier equilibria for supply chains with a single manufacturer and multiple asymmetric retailers

We derive sufficient conditions for the existence and uniqueness of the Stackelberg–Nash–Cournot equilibria for a supply chain problem with a single manufacturer and multiple asymmetric retailers and characterize the first and second order derivatives of the total equilibrium quantities. The Stackelberg manufacturer is assumed to supply a homogeneous product to all retailers with the retail price determined by a general nonlinear inverse demand function. We provide several extensions of our previous results [G.J. Kyparisis, C. Koulamas, A note on equilibria for two-tier supply chains with a single manufacturer and multiple retailers, Operations Research Letters 39 (2011) 471–474] obtained for a similar supply chain with symmetric retailers.     

ANTI-PERIODIC BOUNDARY VALUE PROBLEM FOR FIRST ORDER DELAY DIFFERENTIAL EQUATIONS

The authors employ the method of upper and lower solutions coupled with the monotone iterative technique to obtain some results on the existence and uniqueness of the solution for anti-periodic boundary value problem of delay differential equations.     

Existence of solutions of two generalized riccati operator equations and their representation

The solutions of two generalized Riccati operator equations are discussed in terms of two critical parameter values, which are related to the application of optimal control under unknown disturbances. Explicit formulas for calculating these two critical parameters as well as the closedform solutions of these two generalized Riccati operator equations are given. The connection between these two parameters and a zero-sum differential game is also investigated.     

A BSDE approach to Nash equilibrium payoffs for stochastic differential games with nonlinear cost functionals

In this paper, we study Nash equilibrium payoffs for two-player nonzero-sum stochastic differential games via the theory of backward stochastic differential equations. We obtain an existence theorem and a characterization theorem of Nash equilibrium payoffs for two-player nonzero-sum stochastic differential games with nonlinear cost functionals defined with the help of doubly controlled backward stochastic differential equations. Our results extend former ones by Buckdahn et al. (2004) [3] and are based on a backward stochastic differential equation approach.     

一类高维Riccati方程的解

研究了一类具有指数型二分性的高维Riccati方程存在有界解、周期解的充分条件,得到一些结论。     

Sequential Stackelberg equilibria in two-person games

The concept of sequential Stackelberg equilibrium is introduced in the general framework of dynamic, two-person games defined in the Denardo contracting operator formalism. A relationship between this solution concept and the sequential Nash equilibrium for an associated extended game is established. This correspondence result, which can be related to previous results obtained by Baar and Haurie (1984), is then used for studying the existence of such solutions in a class of sequential games. For the zero-sum case, the sequential Stackelberg equilibrium corresponds to a sequential maxmin equilibrium. An algorithm is proposed for the computation of this particular case of equilibrium.This research was supported by SSHRC Grant No. 410-83-1012, NSERC Grant No. A4952, and FCAR Grants Nos. 86-CE-130 and EQ-0428.The authors thank T. R. Bielecki and J. A. Filar, who pointed out some mistakes and helped improving the paper.At the time of this research, this author was with GERMA, Ecole Mohammedia d'Ingénieurs, Rabat, Morocco.     

Deterministic Time-inconsistent Optimal Control Problems - an Essentially Cooperative Approach

A general deterministic time-inconsistent optimal control problem is formulated for ordinary differential equations. To find a time-consistent equilibrium value function and the corresponding time-consistent equilibrium control, a non-cooperative N-person differential game (but essentially cooperative in some sense) is introduced. Under certain conditions, it is proved that the open-loop Nash equilibrium value function of the N -person differential game converges to a time-consistent equilibrium value function of the original problem, which is the value function of a time-consistent optimal control problem. Moreover, it is proved that any optimal control of the time-consistent limit problem is a time-consistent equilibrium control of the original problem.     

Almost-periodic solutions for Riccati equations of stochastic control

We consider an average quadratic cost criteria for affine stochastic differential equations with almost-periodic coefficients. Under stabilizability and detectability conditions we show that the Riccati equation associated with the quadratic control problem has a unique almost-periodic solution. In the periodic case the corresponding result is proved in [4].