Exponential utility maximization for an insurer with time-inconsistent preferences

This paper studies the optimal consumption–investment–reinsurance problem for an insurer with a general discount function and exponential utility function in a non-Markovian model. The appreciation rate and volatility of the stock, the premium rate and volatility of the risk process of the insurer are assumed to be adapted stochastic processes, while the interest rate is assumed to be deterministic. The object is to maximize the utility of intertemporal consumption and terminal wealth. By the method of multi-person differential game, we show that the time-consistent equilibrium strategy and the corresponding equilibrium value function can be characterized by the unique solutions of a BSDE and an integral equation. Under appropriate conditions, we show that this integral equation admits a unique solution. Furthermore, we compare the time-consistent equilibrium strategies with the optimal strategy for exponential discount function, and with the strategies for naive insurers in two special cases.     

基于信用支付和现金折扣的变质物品库存模型

本文在供应商提供给零售商定期信用支付和现金折扣情况下,研究了零售商的变质物品最优库存问题。基于信用支付和现金折扣的两种支付条件下,分四种情况建立库存模型,并给出了寻求变质物品最优订购周期和最优付款时间的有效算法。最后,给出算例以及最优解,以说明本模型及求解过程。     

Optimal equilibrium barrier strategies for time-inconsistent dividend problems in discrete time

This paper studies a time-inconsistent dividend problem in discrete time with nonexponential discounting. Motivated by the decreasing impatience in behaviour economics, a general discount function is used and assumed to be log sub-additive. Using a game-theoretic approach equilibrium barrier strategies are considered. It is shown that in the case of multiple equilibria, there exists an optimal one that pointwisely dominates all the other equilibria. Case studies are conducted where there is no equilibrium, multiple equilibria, and a unique equilibrium.     

A two-person zero-sum Markov game with a stopped set

We consider a two-person zero-sum Markov game with continuous time up to the time that the game process goes into a fixed subset of a countable state space, this subset is called a stopped set of the game. We show that such a game with a discount factor has optimal value function and both players will have their optimal stationary strategies. The same result is proved for the case of a nondiscounted Markov game under some additional conditions, that is a reward rate function is nonnegative and the first time τ (entrance time) of the game process going to the stopped set is finite with probability one (i.e., p(τ < ∞) = 1). It is remarkable that in the case of a nondiscounted Markov game, if the expectation of the entrance time is bounded, and the reward rate function need not be nonnegative, then the same result holds.     

Asymptotic behavior of continuous stochastic games

Summary The aim of this paper is to analyze the asymptotic behavior of the value functions of a continuous stochastic game as the number of stages grows to infinity or the discount factor approaches 1. After the setup of the problem we prove that, in both cases, the extrema of the value functions converge to the same limits. The convergence of the value functions is then obtained from the unicity of the solution of a functional problem and it is thus possible to design hypotheses that assure the convergence to a constant. This allows to assign a value to an undiscounted infinite-stage stochastic game in several senses and to show that optimal strategies are available for both players. Furthermore the boundedness of the remainders of the value function after removing the principal terms is analyzed, with appropriate hypotheses, and related to the existence of solutions of a Howard's type functional equation. This allows to show that for an infinite-stage undiscounted stochastic game optimal stationary strategies exist at least if this functional equation has some solution.     

SOLVING DYNAMIC WILDLIFE RESOURCE OPTIMIZATION PROBLEMS USING REINFORCEMENT LEARNING

ABSTRACT. An important technical component of natural resource management, particularly in an adaptive management context, is optimization. This is used to select the most appropriate management strategy, given a model of the system and all relevant available information. For dynamic resource systems, dynamic programming has been the de facto standard for deriving optimal state‐specific management strategies. Though effective for small‐dimension problems, dynamic programming is incapable of providing solutions to larger problems, even with modern microcomputing technology. Reinforcement learning is an alternative, related procedure for deriving optimal management strategies, based on stochastic approximation. It is an iterative process that improves estimates of the value of state‐specific actions based in interactions with a system, or model thereof. Applications of reinforcement learning in the field of artificial intelligence have illustrated its ability to yield near‐optimal strategies for very complex model systems, highlighting the potential utility of this method for ecological and natural resource management problems, which tend to be of high dimension. I describe the concept of reinforcement learning and its approach of estimating optimal strategies by temporal difference learning. I then illustrate the application of this method using a simple, well‐known case study of Anderson [1975], and compare the reinforcement learning results with those of dynamic programming. Though a globally‐optimal strategy is not discovered, it performs very well relative to the dynamic programming strategy, based on simulated cumulative objective return. I suggest that reinforcement learning be applied to relatively complex problems where an approximate solution to a realistic model is preferable to an exact answer to an oversimplified model.     

Integrated inventory model with quantity discount and price-sensitive demand

Quantity discount has been a subject of study for a long time; however, little is known about its effect on integrated inventory models when price-sensitive demand is placed. The objective of this study is to find the optimal pricing and ordering strategies for an integrated inventory system when a quantity discount policy is applied. The pricing strategy discussed here is one in which the vendor offers a quantity discount to the buyer. Then, the buyer will adjust his retail price based on the purchasing cost, which will influence the customer demand as a result. Consequently, an integrated inventory model is established to find the optimal solutions for order quantity, retail price, and the number of shipments from vendor to buyer in one production run, so that the joint total profit incurred has the maximum value. Also, numerical examples and a sensitivity analysis are given to illustrate the results of the model.     

Optimal Continuous Policies for Repair and Replacement

This paper investigates the problem of finding optimal replacement policies for equipment subject to failures with randomly distributed repair costs, the degree of reliability of the equipment being considered as a state of a Markov process. Algorithms have been devised to find optimal combined policies both for preventive replacement and for replacement in case of failure by using repair-limit strategies.First a simple procedure to obtain an optimal discrete policy is described. Then an algorithm is formulated in order to calculate an optimal continuous policy: it is shown how the optimal repair limit is the solution to an ordinary differential equation, and how the value of the repair limit determines the optimal preventive replacement policy.     

Optimal consumption–investment strategy under the Vasicek model: HARA utility and Legendre transform

This paper studies the optimal consumption–investment strategy with multiple risky assets and stochastic interest rates, in which interest rate is supposed to be driven by the Vasicek model. The objective of the individuals is to seek an optimal consumption–investment strategy to maximize the expected discount utility of intermediate consumption and terminal wealth in the finite horizon. In the utility theory, Hyperbolic Absolute Risk Aversion (HARA) utility consists of CRRA utility, CARA utility and Logarithmic utility as special cases. In addition, HARA utility is seldom studied in continuous-time portfolio selection theory due to its sophisticated expression. In this paper, we choose HARA utility as the risky preference of the individuals. Due to the complexity of the structure of the solution to the original Hamilton–Jacobi–Bellman (HJB) equation, we use Legendre transform to change the original non-linear HJB equation into its linear dual one, whose solution is easy to conjecture in the case of HARA utility. By calculations and deductions, we obtain the closed-form solution to the optimal consumption–investment strategy in a complete market. Moreover, some special cases are also discussed in detail. Finally, a numerical example is given to illustrate our results.     

Asset allocation,sustainable withdrawal,longevity risk and non-exponential discounting

The present paper studies an optimal withdrawal and investment problem for a retiree who is interested in sustaining her retirement consumption above a pre-specified minimum consumption level. Apparently, the withdrawal and investment policy depends substantially on the retiree’s health condition and her time preferences (subjective discount factor). We assume that the health of the retiree can worsen or improve in an unpredictable way over her lifetime and model the retiree’s mortality intensity by a stochastic process. In order to make the decision about the consumption and investment policy more realistic, we assume that the retiree applies a non-exponential discount factor (an exponential discount factor with a small amount of hyperbolic discounting) to value her future income. In other words, we consider an optimization problem by combining four important aspects: asset allocation, sustainable withdrawal, longevity risk and non-exponential discounting. Due to the non-exponential discount factor, we have to solve a time-inconsistent optimization problem. We derive a non-local HJB equation which characterizes the equilibrium optimal investment and consumption strategy. We establish the first-order expansions of the equilibrium value function and the equilibrium strategies by applying expansion techniques. The expansion is performed on the parameter controlling the degree of discounting in the hyperbolic discounting that is added to the exponential discount factors. The first-order equilibrium investment and consumption strategies can be calculated in a feasible way by solving PDEs.     

A two-stage supply chain coordination mechanism considering price sensitive demand and quantity discounts

This paper explores the coordination between a supplier and a buyer within a decentralized supply chain, through the use of quantity discounts in a game theoretic model. Within this model, the players face inventory and pricing decisions. We propose both cooperative and non-cooperative approaches considering that the product traded experiences a price sensitive demand. In the first case, we study the dynamics of the game from the supplier's side as the leader in the negotiation obtaining a Stackelberg equilibrium, and then show how the payoff of this player could still improve from this point. In the second case, a cooperative model is formulated, where decisions are taken simultaneously, emulating a centralized firm, showing the benefits of the cooperation between the players. We further formulate a pricing game, where the buyer is allowed to set different prices to the final customer as a reaction to the supplier's discount decisions. For the latter we investigate the difference between feasibility of implementing a retail discount given a current coordination mechanism and without it. Finally the implications of transportation costs are analyzed in the quantity discount schedule. Our findings are illustrated with a numerical example showing the difference in the players’ payoff in each case and the optimal strategies, comparing in each case our results with existing work.     

Optimal control in nonlinear infinite-dimensional systems with nondifferentiability of two types

We consider the optimal control problem for systems described by nonlinear equations of elliptic type. If the nonlinear term in the equation is smooth and the nonlinearity increases at a comparatively low rate of growth, then necessary conditions for optimality can be obtained by well-known methods. For small values of the nonlinearity exponent in the smooth case, we propose to approximate the state operator by a certain differentiable operator. We show that the solution of the approximate problem obtained by standard methods ensures that the optimality criterion for the initial problem is close to its minimal value. For sufficiently large values of the nonlinearity exponent, the dependence of the state function on the control is nondifferentiable even under smoothness conditions for the operator. But this dependence becomes differentiable in a certain extended sense, which is sufficient for obtaining necessary conditions for optimality. Finally, if there is no smoothness and no restrictions are imposed on the nonlinearity exponent of the equation, then a smooth approximation of the state operator is possible. Next, we obtain necessary conditions for optimality of the approximate problem using the notion of extended differentiability of the solution of the equation approximated with respect to the control, and then we show that the optimal control of the approximated extremum problem minimizes the original functional with arbitrary accuracy.     

Pricing general insurance in a reactive and competitive market

A simple parameterisation is introduced which represents the insurance market’s response to an insurer adopting a pricing strategy determined via optimal control theory. Claims are modelled using a lognormally distributed mean claim size rate, and the market average premium is determined via the expected value principle. If the insurer maximises its expected wealth then the resulting Bellman equation has a moving boundary in state space that determines when it is optimal to stop selling insurance. This stochastic optimisation problem is simplified by the introduction of a stopping time that prevents an insurer leaving and then re-entering the insurance market. Three finite difference schemes are used to verify the existence of a solution to the resulting Bellman equation when there is market reaction. All of the schemes use a front-fixing transformation. If the market reacts, then it is found that the optimal strategy is altered, in that premiums are raised if the strategy is of loss-leading type and lowered if it is optimal for the insurer to set a relatively high premium and sell little insurance.     

Pricing general insurance in a reactive and competitive market

A simple parameterisation is introduced which represents the insurance market’s response to an insurer adopting a pricing strategy determined via optimal control theory. Claims are modelled using a lognormally distributed mean claim size rate, and the market average premium is determined via the expected value principle. If the insurer maximises its expected wealth then the resulting Bellman equation has a moving boundary in state space that determines when it is optimal to stop selling insurance. This stochastic optimisation problem is simplified by the introduction of a stopping time that prevents an insurer leaving and then re-entering the insurance market. Three finite difference schemes are used to verify the existence of a solution to the resulting Bellman equation when there is market reaction. All of the schemes use a front-fixing transformation. If the market reacts, then it is found that the optimal strategy is altered, in that premiums are raised if the strategy is of loss-leading type and lowered if it is optimal for the insurer to set a relatively high premium and sell little insurance.     

基于消费者异质性社交能力的推荐奖励策略

从消费者社交性的视角优化推荐奖励策略。文章基于效用理论,分析消费者的购买和推荐行为。构建了当期利润最大化基本模型和纳入远期利益的扩展模型,优化产品价格和折扣,给出高低折扣策略的适用空间,并探讨消费者社交能力对企业利润和社会绩效的影响。结果显示,消费者的社交能力差异较小时适合采取高折扣策略,否则适宜低折扣策略。当企业考虑远期利益的时候,企业的最优策略是免费策略。随着环境参数变化,最优奖励策略由高折扣调整为低折扣时会带来福利损失,但低折扣策略并不必定导致社会福利的损失。此外,还给出了实践中某些商品价格虚高的经济解释。本研究能够为企业实施推荐计划制定奖励策略提供依据。     

A combination of energy method and spectral analysis for study of equations of gas motion

There have been extensive studies on the large time behavior of solutions to systems on gas motions, such as the Navier-Stokes equations and the Boltzmann equation. Recently, an approach is introduced by combining the energy method and the spectral analysis to the study of the optimal rates of convergence to the asymptotic profiles. In this paper, we will first illustrate this method by using some simple model and then we will present some recent results on the Navier-Stokes equations and the Boltzmann equation. Precisely, we prove the stability of the non-trivial steady state for the Navier-Stokes equations with potential forces and also obtain the optimal rate of convergence of solutions toward the steady state. The same issue was also studied for the Boltzmann equation in the presence of the general time-space dependent forces. It is expected that this approach can also be applied to other dissipative systems in fluid dynamics and kinetic models such as the model system of radiating gas and the Vlasov-Poisson-Boltzmann system.      

Analysis of a two-stage telecommunication supply chain with technology dependent demand

We analyze a two-stage telecommunication supply chain consisting of one operator and one vendor under a multiple period setting. The operator faces a stochastic market demand which depends on technology investment level. The decision variables for the operator are the initial technology investment level and the capacity of the network for each period. The capacity that the operator installs in one period also remains available in subsequent periods. The operator can increase or decrease the available capacity at each period. For this model, an algorithm to find the centralized optimal solution is proposed. A profit sharing contract where firms share both the revenue and operating costs generated throughout the periods along with initial technology investment is suggested. Also a coordinating quantity discount contract where the discount on the price depends on the total installed capacity is designed. The case where the vendor decides on the technology investment level and the operator decides on the capacity of the network is also analyzed and it is shown that this game has a unique Nash equilibrium.     

The optimal order and payment policies for deteriorating items in discount cash flows analysis under the alternatives of conditionally permissible delay in payments and cash discount

This study investigates a deteriorating inventory problem in which the supplier simultaneously offers the retailer either a conditionally permissible delay in payments or a cash discount. In the case of a conditionally permissible delay, if the retailer orders more than a predetermined quantity, then he/she has a grace period to make the full payment. Otherwise, he/she must pay the payment for goods of certain proportion first while receiving the goods and has a grace period to pay off the rest. As to a cash discount, if the retailer pays for the entire amount of the order within a certain short period, then he/she will receive a cash discount from the supplier. In additions, from a financial standpoint, all cash outflows related to the inventory control that occur at different points of time have different values. Hence, it is necessary to take account of the factor of time value of monetary when drafting the replenishment policy. In a word, this paper uses an alternate approach-discount cash flow (DCF) analysis to establish an inventory problem for deteriorating items in which the supplier provides the retailer either a conditionally permissible delay or a cash discount. We then study the necessary and sufficient conditions for finding the optimal solution. Furthermore, we establish several theoretical results to obtain the solution that provides the smallest present value of all future cash flows. Finally, several numerical examples are given to illustrate the results and obtain some managerial insights.     

The golden rule when preferences are time inconsistent

We investigate the classical Ramsey problem of economic growth when the planner uses non-constant discounting. It is well-known that this leads to time inconsistency, so that optimal strategies are no longer implementable. We then define equilibrium strategies to be such that unilateral deviations occurring during a small time interval are penalized. Non-equilibrium strategies are not implementable, so only equilibrium strategies should be considered by a rational planner. We show that there exists such strategies which are (a) smooth, and (b) lead to stationary growth, as in the classical Ramsey model. Finally, we prove an existence and multiplicity result: for logarithmic utility and quasi-exponential discount, there is an interval I such that, for every k in I, there is an equilibrium strategy converging to k. We conclude by giving an example where the planner is led to non-constant discount rates by considerations of intergenerational equity.     

确定性需求下的最优保理融资与生产策略

企业为下游买方提供赊销,由于大量的资金被应收账款占用,企业可能因资金不足而无法生产足够的产品。企业可以通过保理(出售应收账款)进行融资,减小需求损失。在离散时间多周期的确定需求下,使用决策变量描述各周期的系统状态及其状态转移方程,将此问题建模为线性规划。通过分析此问题的结构特点,再提出了一种新颖且等价的建模方法,可以有效减少决策变量和约束条件的数量。在连续时间模型和混合模型中,这种建模方法同样适用,将优化问题写为连续线性规划,极大地降低了优化问题的复杂度。此连续线性规划问题可通过适当的区间划分进行离散化,用分片常量函数代替优化模型中的一般函数(无限维)决策变量,通过求解有限维线性规划得到原问题的可行近似解。最后,通过数值例子分析了贴现率对企业利润的影响。