ADAPTIVE MANAGEMENT OF HARVEST RATES IN THE PRESENCE OF A RISK AVERSE UTILITY FUNCTION

Previous research on adaptive harvest strategies has suggested that it should rarely be optimal to introduce deliberate, probing variations in harvest rates in order to reduce uncertainty about production parameters, when there is strong aversion to the risk of low catches. Here we show that this suggestion is incorrect for cases where the stock dynamics can be approximated by a simple power model and the objective is to maximize the sum over time of discounted logarithms of catches. In such cases, the objective function has a simple form that permits easy comparison between passively adaptive (certainty equivalent or cautious) versus actively adaptive policies involving probing variation in harvest rates. Probing is favored when there is high uncertainty about the power parameter that represents density dependence in the stock dynamics.     

OPTIMAL FISH HARVESTING FOR A POPULATION MODELED BY A NONLINEAR PARABOLIC PARTIAL DIFFERENTIAL EQUATION

As the human population continues to grow, there is a need for better management of our natural resources in order for our planet to be able to produce enough to sustain us. One important resource we must consider is marine fish populations. We use the tool of optimal control to investigate harvesting strategies for maximizing yield of a fish population in a heterogeneous, finite domain. We determine whether these solutions include no‐take marine reserves as part of the optimal solution. The fishery stock is modeled using a nonlinear, parabolic partial differential equation with logistic growth, movement by diffusion and advection, and with Robin boundary conditions. The objective for the problem is to find the harvest rate that maximizes the discounted yield. Optimal harvesting strategies are found numerically.     

ENVIRONMENTAL CHANGE AND THE CONTRIBUTION OF BIODIVERSITY TO ECOSYSTEM ADAPTATION

Abstract This paper develops a measure of the contribution of biodiversity in enhancing ecosystem performance that is subject to environmental fluctuation. The analysis draws from an ecological model that relates high phenotypic variance with lower short‐term productivity (due to the presence of suboptimal species) and higher long‐term productivity (due to better ability to respond to environmental fluctuations). This feature, which is a notable extension to existing economic‐ecological models of biodiversity, enables assessment of the interactions between diversity and a range of environmental fluctuations to highlight that biodiversity could be rendered economically disadvantageous when environmental fluctuation is insufficient. The resulting economic‐ecological model generates discounted present value of harvests for an ecosystem with diverse set of species. This value is compared with the harvest value of a similar economic‐ecological model with no diversity and that of an ecosystem where the dynamics of phenotypes in response to environmental fluctuations is disregarded. The results show that diversity positively contributes to the performance of ecosystems subject to sufficiently large environmental fluctuation. In addition, neglecting an ecosystem's increasing ability to adapt to match environmental conditions is also shown to be more costly than having no diversity in an otherwise identical ecosystem.     

Continuous-Time Markov Decision Processes with State-Dependent Discount Factors

We consider continuous-time Markov decision processes in Polish spaces. The performance of a control policy is measured by the expected discounted reward criterion associated with state-dependent discount factors. All underlying Markov processes are determined by the given transition rates which are allowed to be unbounded, and the reward rates may have neither upper nor lower bounds. By using the dynamic programming approach, we establish the discounted reward optimality equation (DROE) and the existence and uniqueness of its solutions. Under suitable conditions, we also obtain a discounted optimal stationary policy which is optimal in the class of all randomized stationary policies. Moreover, when the transition rates are uniformly bounded, we provide an algorithm to compute (or?at least to approximate) the discounted reward optimal value function as well as a discounted optimal stationary policy. Finally, we use an example to illustrate our results. Specially, we first derive an explicit and exact solution to the DROE and an explicit expression of a discounted optimal stationary policy for such an example.     

Analysis for some properties of discrete time Markov decision processes

This paper investigates properties of the optimality equation and optimal policies in discrete time Markov decision processes with expected discounted total rewards under weak conditions that the model is well defined and the optimality equation is true. The optimal value function is characterized as a solution of the optimality equation and the structure of optimal policies is also given.     

Sensitivity of constrained Markov decision processes

We consider the optimization of finite-state, finite-action Markov decision processes under constraints. Costs and constraints are of the discounted or average type, and possibly finite-horizon. We investigate the sensitivity of the optimal cost and optimal policy to changes in various parameters. We relate several optimization problems to a generic linear program, through which we investigate sensitivity issues. We establish conditions for the continuity of the optimal value in the discount factor. In particular, the optimal value and optimal policy for the expected average cost are obtained as limits of the dicounted case, as the discount factor goes to one. This generalizes a well-known result for the unconstrained case. We also establish the continuity in the discount factor for certain non-stationary policies. We then discuss the sensitivity of optimal policies and optimal values to small changes in the transition matrix and in the instantaneous cost functions. The importance of the last two results is related to the performance of adaptive policies for constrained MDP under various cost criteria [3,5]. Finally, we establish the convergence of the optimal value for the discounted constrained finite horizon problem to the optimal value of the corresponding infinite horizon problem.     

Compactness of the space of non-randomized policies in countable-state sequential decision processes

For sequential decision processes with countable state spaces, we prove compactness of the set of strategic measures corresponding to nonrandomized policies. For the Borel state case, this set may not be compact (Piunovskiy, Optimal control of random sequences in problems with constraints. Kluwer, Boston, p. 170, 1997) in spite of compactness of the set of strategic measures corresponding to all policies (Schäl, On dynamic programming: compactness of the space of policies. Stoch Processes Appl 3(4):345–364, 1975b; Balder, On compactness of the space of policies in stochastic dynamic programming. Stoch Processes Appl 32(1):141–150, 1989). We use the compactness result from this paper to show the existence of optimal policies for countable-state constrained optimization of expected discounted and nonpositive rewards, when the optimality is considered within the class of nonrandomized policies. This paper also studies the convergence of a value-iteration algorithm for such constrained problems.     

Managing transshipments in a multi-retailer system with approximate policies

Using the right transshipment policy is important when transshipments are exercised under demand uncertainty. Optimal transshipment policy can be quite complex in a multi-firm system as optimal actions depend on all system variables. Moreover, both how to select requested retailer and how to respond to requests are in question. We introduce simple, close-to-optimal heuristic transshipment policies for multiple retailers. We first show that heuristic policies may perform even better than self-optimal policy, which is explained by Braess’s paradox. Then we test the performances of various heuristics with respect to centrally optimal policy. When retailers can observe others’ inventory levels, more effective transshipments can be made. Otherwise, a random selection performs quite well. We also observe that although always-accept respond policy is quite close to centrally optimal in small systems, the performance of pairwise-optimal holdback levels to respond requests is more clear and consistent for larger systems.     

Continuous time Markovian decision processes average return criterion

Continuous time Markovian decision models with countable state space are investigated. The existence of an optimal stationary policy is established for the expected average return criterion function. It is shown that the expected average return can be expressed as an expected discounted return of a related Markovian decision process. A policy iteration method is given which converges to an optimal deterministic policy, the policy so obtained is shown optimal over all Markov policies.     

How to correctly calculate discounted healthcare costs and benefits

A feature of a healthcare policy (such as screening) with interventions at specific ages is that when it is introduced, part of the population is too old to participate in the full programme. This fact changes the formulae to be used for cost and benefit discounting in a non-intuitive way. General formulae are derived for the expected discounted costs and benefits of such health promotion policies, for a stationary population. Correct ways to calculate discounted costs and benefits via simulation are also described. The formulae have some surprising properties, for example the relative cost of two health policies does not depend on the discounting rate. They are also relevant to the ongoing debate over the correct discounting rate for benefits. It is shown that when health benefits follow quickly on treatments, varying the discounting rate for health benefits is merely equivalent to rescaling the cash value of a benefit. It is only when benefit follows long after treatment that the problem of choosing an appropriate discount rate for benefits cannot be simplified.     

A stochastic control model of investment,production, and consumption on a finite horizon

In this paper, we consider a stochastic control problem on a finite time horizon. The unit price of capital obeys a logarithmic Brownian motion, and the income from production is also subject to the random Brownian fluctuations. The goal is to choose optimal investment and consumption policies to maximize the finite horizon expected discounted hyperbolic absolute risk aversion utility of consumption. A dynamic programming principle is used to derive a time‐dependent Hamilton–Jacobi–Bellman equation. The Leray–Schauder fixed point theorem is used to obtain existence of solution of the HJB equation. At last, we derive the optimal investment and consumption policies by the verification theorem. The main contribution in this paper is the use of PDE technique to the finite time problem for obtaining optimal polices. Copyright © 2014 John Wiley & Sons, Ltd.     

TIMBER SUPPLY UNDER DEMAND UNCERTAINTY: WELFARE GAINS FROM PERFECT COMPETITION WITH RATIONAL EXPECTATIONS

ABSTRACT. This paper presents a noneconometric approach to estimating the short‐run timber supply function based on optimal harvest decisions. Determination of optimal harvest levels and estimation of supply function coefficients are integrated into one step by incorporating a parametric short‐run timber supply function into the harvest decision model. In this manner we convert the original harvest decision model into a new optimization problem with the supply function coefficients functioning as “decision variables.” Optimal solution to the new decision model gives the coefficients of the short‐run supply function and, indirectly, the optimal harvest levels. This approach enables us to develop stochastic models of the timber market that are particularly useful for forest sector analysis involving comparison of alternative institutional regimes or policy proposals and when the timber market is affected by stochastic variables. For demonstration purposes, we apply this method to compare the performances of two timber market regimes (perfect competition and monopoly) under demand uncertainty, using the Swedish data. The results show that the expected timber price is 22 percent lower and the expected annual timber supply is 43 percent higher in the competitive market than in the monopoly market. This confirms the theoretical result that monopoly reduces supply and increases price. The expected social welfare gain from perfect competition over monopoly is about 24 percent.     

On the Gerber-Shiu Function and Optimal Dividend Strategy for a Thinning Risk Model

In this paper, the risk model under constant dividend barrier strategy is studied, in which the premium income follows a compound Poisson process and the arrival of the claims is a p-thinning process of the premium arrival process. The integral equations with boundary conditions for the expected discounted aggregate dividend payments and the expected discounted penalty function until ruin are derived. In addition, the explicit expressions for the Laplace transform of the ruin time and the expected aggregate discounted dividend payments until ruin are given when the individual stochastic premium amount and claim amount are exponentially distributed. Finally, the optimal barrier is presented under the condition of maximizing the expectation of the difference between discounted aggregate dividends until ruin and the deficit at ruin.     

Constrained Markov decision processes with total cost criteria: Occupation measures and primal LP

This paper is the third in a series on constrained Markov decision processes (CMDPs) with a countable state space and unbounded cost. In the previous papers we studied the expected average and the discounted cost. We analyze in this paper the total cost criterion. We study the properties of the set of occupation measures achieved by different classes of policies; we then focus on stationary policies and on mixed deterministic policies and present conditions under which optimal policies exist within these classes. We conclude by introducing an equivalent infinite Linear Program.     

Optimization problems for general simple population with n-impulsive harvest

In this paper, n-impulsive harvest problems of general simple population are discussed by models with Dirac function. The optimal impulsive harvest policies to protect the renewable resource better are obtained under conditions of fixed quantity per impulsive harvest. Then, a concept of the sequence for ?-optimal harvest moments for general simple population is presented which is beneficial to protect resource better and sustainable development. Finally, we apply the conclusions to some special models.     

Controlled jump processes

Finite and infinite planning horizon Markov decision problems are formulated for a class of jump processes with general state and action spaces and controls which are measurable functions on the time axis taking values in an appropriate metrizable vector space. For the finite horizon problem, the maximum expected reward is the unique solution, which exists, of a certain differential equation and is a strongly continuous function in the space of upper semi-continuous functions. A necessary and sufficient condition is provided for an admissible control to be optimal, and a sufficient condition is provided for the existence of a measurable optimal policy. For the infinite horizon problem, the maximum expected total reward is the fixed point of a certain operator on the space of upper semi-continuous functions. A stationary policy is optimal over all measurable policies in the transient and discounted cases as well as, with certain added conditions, in the positive and negative cases.     

Policy structure for discrete time Markov chain disorder problems

Markov chain disorder problems are partially observed Markov decision problems where the decision maker must formulate a policy of response to an unobservable transition to an ‘undesirable state’. This paper introduces a structural property for policies—the likelihood consistency property. For certain varieties of Markov chain disorder problems, policies which minimize the expected discounted present cost possess the likelihood consistency property. This extends the list of known policy structure results for partially observed Markov decision processes, and introduces a class of policies which can be used when the ‘optimal’ policy is unattainable or undesirable.     

Markov: A methodology for the solution of infinite time horizon markov decision processes

Algorithms are described for determining optimal policies for finite state, finite action, infinite discrete time horizon Markov decision processes. Both value-improvement and policy-improvement techniques are used in the algorithms. Computing procedures are also described. The algorithms are appropriate for processes that are either finite or infinite, deterministic or stochastic, discounted or undiscounted, in any meaningful combination of these features. Computing procedures are described in terms of initial data processing, bound improvements, process reduction, and testing and solution. Application of the methodology is illustrated with an example involving natural resource management. Management implications of certain hypothesized relationships between mallard survival and harvest rates are addressed by applying the optimality procedures to mallard population models.     

An optimal inspection and replacement policy under incomplete state information

This paper investigates an optimal inspection and replacement problem for a discrete-time Markovian deterioration system. It is assumed that the system is monitored incompletely by a certain mechanism which gives the decision maker some information about the exact state of the system. The problem is to obtain an optimal inspection and replacement policy minimizing the expected total discounted cost over an infinite horizon and formulated as a partially observable Markov decision process. Furthermore, under some reasonable conditions reflecting the practical meaning of the deterioration, it is shown that there exists an optimal inspection and replacement policy in the class of monotonic four-region policies.     

Variance minimization for constrained discounted continuous-time MDPs with exponentially distributed stopping times

This paper deals with minimization of the variances of the total discounted costs for constrained Continuous-Time Markov Decision Processes (CTMDPs). The costs consist of cumulative costs incurred between jumps and instant costs incurred at jump epochs. We interpret discounting as an exponentially distributed stopping time. According to existing theory, for the expected total discounted costs optimal policies exist in the forms of randomized stationary and switching stationary policies. While the former is typically unique, the latter forms a finite set whose number of elements grows exponentially with the number of constraints. This paper investigates the problem when the process stops immediately after the first jump. For costs up to the first jump we provide an index for selection of actions by switching stationary policies and show that the indexed switching policy achieves a smaller variance than the randomized stationary policy. For problems without instant costs, the indexed switching policy achieves the minimum variance of costs up to the first jump among all the equivalent switching policies.