Optimal risk-sharing under mutually singular beliefs

We focus on the situation in which agents might have mutually singular beliefs in a maxmin expected utility framework. We show the existence of an equilibrium under fairly general conditions. We further demonstrate that the characterization of Pareto optimal allocation is significantly different from the classical situation, where all beliefs are mutually equivalent for each agent. Absent aggregate uncertainty, we prove that with common beliefs among agents, any Pareto optimal allocation is a full insurance under the upper capacities for all agents. But the full insurance feature of all Pareto optimal allocations, if true, does not necessarily ensure common beliefs. Moreover, despite agents have sharing beliefs, a full insurance Pareto optimal allocation could be associated with intricate allocation form.     

Group activity selection problem with approval preferences

We consider a setting where one has to organize one or several group activities for a set of agents. Each agent will participate in at most one activity, and her preferences over activities depend on the number of participants in the activity. The goal is to assign agents to activities based on their preferences in a way that is socially optimal and/or stable. We put forward a general model for this setting, which is a natural generalization of anonymous hedonic games. We then focus on a special case of our model where agents’ preferences are binary, i.e., each agent classifies all pairs of the form ‘(activity, group size)’ into ones that are acceptable and ones that are not. We formulate several solution concepts for this scenario, and study them from the computational point of view, providing hardness results for the general case as well as efficient algorithms for settings where agents’ preferences satisfy certain natural constraints.     

A single-resource revenue management problem with random resource consumptions

We study a single-resource multi-class revenue management problem where the resource consumption for each class is random and only revealed at departure. The model is motivated by cargo revenue management problems in the airline and other shipping industries. We study how random resource consumption distribution affects the optimal expected profit and identify a preference acceptance order on classes. For a special case where the resource consumption for each class follows the same distribution, we fully characterize the optimal control policy. We then propose two easily computable heuristics: (i) a class-independent heuristic through parameter scaling, and (ii) a decomposition heuristic that decomposes the dynamic programming formulation into a collection of one-dimensional problems. We conduct extensive numerical experiments to investigate the performance of the two heuristics and compared them with several widely studied heuristic policies. Our results show that both heuristics work very well, with class-independent heuristic slightly better between the two. In particular, they consistently outperform heuristics that ignore demand and/or resource consumption uncertainty. Our results demonstrate the importance of considering random resource consumption as another problem dimension in revenue management applications.     

常数比例投资组合保险(CPPI)策略 --捐赠型基金投资策略的最优选择

本文采用Merton提出的处理捐赠型基金的连续时间模型的一般框架，分析了在风险资产为几何布朗运动，效用函数为CRRA效用函数，且捐赠型基金有动态最低支出时的最优支出策略和最优投资策略，结果表明存在一条策略基准线，当基金的总资产在策略基准线之上时，基金管理人关于基金支出与投资策略的选择与不存在最低支出的要求时所作出的决策是一样的，但是一旦基金的总资产低于这条策略基准线时，基金管理人便需要考虑到基金将来必要的支出，并实际影响到他对投资策略的选择，此时基金管理人可作的最优选择是：最低的支出和一种为复制幂收益函数期权的CPPI投资策略。     

跳扩散最优控制的随机最大值原理及在金融中的应用

讨论了由金融市场中投资组合和消费选择问题引出的一类最优控制问题,投资者的期望效用是常数相对风险厌恶(CRRA)情形.在跳扩散框架下,利用古典变分法得到了一个局部随机最大值原理.结果应用到最优投资组合和消费选择策略问题,得到了状态反馈形式的显式最优解.     

Single-machine scheduling with learning effect and resource-dependent processing times in the serial-batching production

In this paper, we study a single machine scheduling problem by simultaneously considering the processing method of serial-batching, learning effect, resource-dependent processing times, and setup operations. We consider minimizing the makespan as the objective of the studied problem under the constraint that the total resource consumption does not exceed a given limit. For the special case where the resource allocation is given, we first propose the structural properties for job batching policies and batching sequencing, and an optimal batching policy is derived based on these properties. Then, we develop a novel hybrid GSA–TS algorithm which combines the Gravitational Search Algorithm (GSA) and the Tabu Search (TS) algorithm to solve the general case. Computational experiments with different scales show the effectiveness and efficiency of the proposed algorithm.     

Decentralized and dynamic group formation of reconfigurable agents

Adaptive group formation in dynamic environments performed by heterogeneous swarms of simple agents is an interesting research topic. In this paper we consider an unsupervised scenario where the individuals of the swarm have limited information about their environment as well as limited communication capabilities. The particular case of a multi-agent model with self-organized reconfigurable agents where the agents are confronted with a resource collection task, different movement, and group formation tactics are analyzed experimentally. It is shown that cooperation in groups is profitable for the group members and the optimal group size depends on environmental parameters. Moreover, a simple strategy based on the agents ability to measure their own workload results in an adaptive behavior that influences the size of the groups and increases the performance of the overall system.     

optimal consump0tion of an investment †

We consider a continuous-time stochastic model for the consumption of an invested resource. The optimal consumption policy is derived. A system of prices which stimulate the optimal consumption is identified. The optimal policy and payoff are found explicitly in the case where the interest process has independent increments.     

Optimal Asset Allocation for Retirement Saving: Deterministic Vs. Time Consistent Adaptive Strategies

We consider optimal asset allocation for an investor saving for retirement. The portfolio contains a bond index and a stock index. We use multi-period criteria and explore two types of strategies: deterministic strategies are based only on the time remaining until the anticipated retirement date, while adaptive strategies also consider the investor’s accumulated wealth. The vast majority of financial products designed for retirement saving use deterministic strategies (e.g., target date funds). In the deterministic case, we determine an optimal open loop control using mean-variance criteria. In the adaptive case, we use time consistent mean-variance and quadratic shortfall objectives. Tests based on both a synthetic market where the stock index is modelled by a jump-diffusion process and also on bootstrap resampling of long-term historical data show that the optimal adaptive strategies significantly outperform the optimal deterministic strategy. This suggests that investors are not being well served by the strategies currently dominating the marketplace.     

Mean and variance optimization of non–linear systems and worst–case analysis

In this paper, we consider expected value, variance and worst–case optimization of nonlinear models. We present algorithms for computing optimal expected value, and variance policies, based on iterative Taylor expansions. We establish convergence and consider the relative merits of policies based on expected value optimization and worst–case robustness. The latter is a minimax strategy and ensures optimal cover in view of the worst–case scenario(s) while the former is optimal expected performance in a stochastic setting. Both approaches are used with a small macroeconomic model to illustrate relative performance, robustness and trade-offs between the alternative policies.     

Managing transshipments in a multi-retailer system with approximate policies

Using the right transshipment policy is important when transshipments are exercised under demand uncertainty. Optimal transshipment policy can be quite complex in a multi-firm system as optimal actions depend on all system variables. Moreover, both how to select requested retailer and how to respond to requests are in question. We introduce simple, close-to-optimal heuristic transshipment policies for multiple retailers. We first show that heuristic policies may perform even better than self-optimal policy, which is explained by Braess’s paradox. Then we test the performances of various heuristics with respect to centrally optimal policy. When retailers can observe others’ inventory levels, more effective transshipments can be made. Otherwise, a random selection performs quite well. We also observe that although always-accept respond policy is quite close to centrally optimal in small systems, the performance of pairwise-optimal holdback levels to respond requests is more clear and consistent for larger systems.     

Equilibrium and optimal behavior of customers in Markovian queues with multiple working vacations

This paper studies the customers’ equilibrium and socially optimal joining–balking behavior in single-server Markovian queues with multiple working vacations. Different from the classical vacation policies, the server does not completely stop service but maintains a low service rate in vacation state in case there are customer arrivals. Based on different precision levels of the system information, we discuss the observable queues, the partially observable queues, and the unobservable queues, respectively. For each type of queues, we get both the customers’ equilibrium and socially optimal joining–balking strategies and make numerical comparisons between them. We numerically observe that their equilibrium strategy is unique, and especially, the customers’ equilibrium joining probability in vacation state is not necessarily smaller than that in busy state in the partially observable queues. Moreover, we also find that the customers’ individual behavior always deviates from the social expectation and makes the system more congested.     

Model predictive control,the economy,and the issue of global warming

This study is motivated by the evidence of global warming, which is caused by human activity but affects the efficiency of the economy. We employ the integrated assessment Nordhaus DICE-2007 model (Nordhaus, A question of balance: economic modeling of global warming, Yale University Press, New Haven, 2008). Generally speaking, the framework is that of dynamic optimization of the discounted inter-temporal utility of consumption, taking into account the economic and the environmental dynamics. The main novelty is that several reasonable types of behavior (policy) of the economic agents, which may be non-optimal from the point of view of the global performance but are reasonable form an individual point of view and exist in reality, are strictly defined and analyzed. These include the concepts of “business as usual”, in which an economic agent ignores her impact on the climate change (although adapting to it), and of “free riding with a perfect foresight”, where some economic agents optimize in an adaptive way their individual performance expecting that the others would perform in a collectively optimal way. These policies are defined in a formal and unified way modifying ideas from the so-called “model predictive control”. The introduced concepts are relevant to many other problems of dynamic optimization, especially in the context of resource economics. However, the numerical analysis in this paper is devoted to the evolution of the world economy and the average temperature in the next 150 years, depending on different scenarios for the behavior of the economic agents. In particular, the results show that the “business as usual”, although adaptive to the change of the atmospheric temperature, may lead within 150 years to increase of temperature by 2°C more than the collectively optimal policy.     

Optimal consumption problems in discontinuous markets

We study an extension of Merton’s classical portfolio investment – consumption optimization problem (1969–1970) to a particular case of complete discontinuous market, with a single jump. The market consists of a non-risky asset, a ‘standard risky’ asset and a risky asset with discontinuous price dynamics (e.g. a defaultable bond or a mortality linked security). We consider three different problems of maximization of the expected utility from consumption: in the case when the investment horizon is fixed and finite, when it is finite, but possibly uncertain and when it is infinite. The innovative setting is the second one. In a general stochastic coefficients’ model, we solve the problems and we compare the three optimal consumption rates, finding quite interesting results. In the logarithmic and power utility cases, explicit solutions are provided. Furthermore, the benchmark – constant coefficients’ case is deeply investigated and a partial information setting is also studied in the uncertain time horizon case.     

Optimality of monotonic policies for two-action Markovian decision processes,with applications to control of queues with delayed information

We consider a discrete-time Markov decision process with a partially ordered state space and two feasible control actions in each state. Our goal is to find general conditions, which are satisfied in a broad class of applications to control of queues, under which an optimal control policy is monotonic. An advantage of our approach is that it easily extends to problems with both information and action delays, which are common in applications to high-speed communication networks, among others. The transition probabilities are stochastically monotone and the one-stage reward submodular. We further assume that transitions from different states are coupled, in the sense that the state after a transition is distributed as a deterministic function of the current state and two random variables, one of which is controllable and the other uncontrollable. Finally, we make a monotonicity assumption about the sample-path effect of a pairwise switch of the actions in consecutive stages. Using induction on the horizon length, we demonstrate that optimal policies for the finite- and infinite-horizon discounted problems are monotonic. We apply these results to a single queueing facility with control of arrivals and/or services, under very general conditions. In this case, our results imply that an optimal control policy has threshold form. Finally, we show how monotonicity of an optimal policy extends in a natural way to problems with information and/or action delay, including delays of more than one time unit. Specifically, we show that, if a problem without delay satisfies our sufficient conditions for monotonicity of an optimal policy, then the same problem with information and/or action delay also has monotonic (e.g., threshold) optimal policies.     

Tracking Demands in Optimal Control of Managerial Systems with Continuously-Divisible, Doubly Constrained Resources

The paper addresses problems of allocating continuously divisible resources among multiple production activities. The resources are allowed to be doubly constrained, so that both usage at every point of time and cumulative consumption over a planning horizon are limited as it is often the case in project and production scheduling. The objective is to track changing in time demands for the activities as closely as possible. We propose a general continuous-time model that states the problem in a form of the optimal control problem with non-linear speed-resource usage functions. With the aid of the maximum principle, properties of the solutions are derived to characterize optimal resource usage policies. On the basis of this analytical investigation, numerical scheduling methods are suggested and computationally studied. This revised version was published online in July 2006 with corrections to the Cover Date.     

SOLVING DYNAMIC WILDLIFE RESOURCE OPTIMIZATION PROBLEMS USING REINFORCEMENT LEARNING

ABSTRACT. An important technical component of natural resource management, particularly in an adaptive management context, is optimization. This is used to select the most appropriate management strategy, given a model of the system and all relevant available information. For dynamic resource systems, dynamic programming has been the de facto standard for deriving optimal state‐specific management strategies. Though effective for small‐dimension problems, dynamic programming is incapable of providing solutions to larger problems, even with modern microcomputing technology. Reinforcement learning is an alternative, related procedure for deriving optimal management strategies, based on stochastic approximation. It is an iterative process that improves estimates of the value of state‐specific actions based in interactions with a system, or model thereof. Applications of reinforcement learning in the field of artificial intelligence have illustrated its ability to yield near‐optimal strategies for very complex model systems, highlighting the potential utility of this method for ecological and natural resource management problems, which tend to be of high dimension. I describe the concept of reinforcement learning and its approach of estimating optimal strategies by temporal difference learning. I then illustrate the application of this method using a simple, well‐known case study of Anderson [1975], and compare the reinforcement learning results with those of dynamic programming. Though a globally‐optimal strategy is not discovered, it performs very well relative to the dynamic programming strategy, based on simulated cumulative objective return. I suggest that reinforcement learning be applied to relatively complex problems where an approximate solution to a realistic model is preferable to an exact answer to an oversimplified model.     

Groundwater management: Waiting for a drought*

In this article we present a stylized model for optimal management of an unconfined groundwater resource when the threat of drought exists. The drought is modeled as a stochastic event that hits at an uncertain date and two benchmark management policies are investigated: (a) A policy of optimal dynamic management ignoring the threat of drought; and (b) an economically optimal policy that accounts for the threat of a drought. We show that the optimal predrought steady‐state equilibrium stock size of groundwater under policy b is larger than that under policy (a) Furthermore, we show that an increase in the probability of a drought gives rise to two counteracting effects: One in the direction of a larger predrought steady‐state equilibrium stock size (a recovery effect) and one in the direction of a lower predrought steady‐state equilibrium stock (an extinction effect). We find that the recovery effect dominates the extinction effect. Recommendations for Resource Managers: We analyze two groundwater extraction policies that can be used when a threat of drought exists: (a) Dynamic optimal management ignoring the threat of drought; and (b) dynamic optimal management taking the threat of drought into account. We show that the predrought steady‐state equilibrium stock size of water should be larger under the policy (b) than under policy (a). This conclusion has three implications for resource managers:

• Current groundwater management should take future extraction possibilities into account.

• A resource manager ought to take the threat of drought into account in groundwater management.

• A buffer stock of water should be built‐up before the drought to be drawn upon during the event.

     

Practical Inventory Replenishment Policies for a Linear Trend in Demand Followed by a Period of Steady Demand

Inventory policies are considered for the situation of a deterministic linear trend in demand followed by a period of constant demand: an idealised form of a common situation in spares provisioning. An analytic optimal solution is derived and its practical shortcomings discussed. The performance of practical inventory policies for this situation are compared and a policy based on the well known economic batch quantity, suitably modified for linear trend, is shown to give small cost penalties when compared to the analytic solution. The good performance of the policy based on the economic batch quantity is maintained, even when used on examples which simulate situations where the parameters of demand are not known exactly. Indeed the method is then superior to all others tried, including the "optimal" analytic method.     

Optimal balanced control for call centers

In this paper we study the optimal assignment of tasks to agents in a call center. For this type of problem, typically, no single deterministic and stationary (i.e., state independent and easily implementable) policy yields the optimal control, and mixed strategies are used. Other than finding the optimal mixed strategy, we propose to optimize the performance over the set of ??balanced?? deterministic periodic non-stationary policies. We provide a stochastic approximation algorithm that allows to find the optimal balanced policy by means of Monte Carlo simulation. As illustrated by numerical examples, the optimal balanced policy outperforms the optimal mixed strategy.