Stochastic control processes and management science

In the stochastic control processes, the criterion which minimizes the probability that the state variables of the overall stage exceed the fixed level is probably of greater importance. Although it does not resolve the arbitrariness inherent in the choice of a criterion of performance, this is a conforming result. The paper is divided into three parts. First, the mathematical formulation, the theorem of existence and uniqueness of solution, and the optimal policy in the equations created from our stochastic control processes are considered. Second, problems involving some stochastic control processes in a fuzzy environment are considered. Third, applications of the inventory control problem, the statistical quality control processes, and a control process connected with the Van der Pol equation are presented. Finally, the future problems are discussed.     

On the Hungarian inventory control model

In this paper we recall and further develop an inventory model formulated by the author [Prékopa, A., 1965. Reliability equation for an inventory problem and its asymptotic solutions. In: Prékopa, A. (Ed.), Colloquia Applied Mathematics in Economics. Publ. House of the Hung. Acad. Sci., Budapest, pp. 317–327; Prékopa, A., 1973. Generalizations of the theorems of Smirnov with application to a reliability type inventory problem. Math. Operationsforschung und Stat. 4, 283–297] and Ziermann [Ziermann, M., 1964. Application of Smirnov’s theorems for an inventory control problem. Publications of the Mathematical Institute of the Hungarian Academy of Sciences Ser. B 8, 509–518] that has had wide application in Hungary and elsewhere. The basic assumption made in connection with this model is that the delivery of the ordered amount takes place in an interval, according to some random process, rather than at one time epoch. The problem is to determine that minimum level of safety stock, that ensures continuous production, without disruption, by a prescribed high probability. The model is further developed first by its combination with another inventory control model, the order up to S model and then, by the formulations of a static and a dynamic type stochastic programming models.     

Uniqueness of solution of production control problem in a manufacturing system with degenerate demand

This paper studies the production inventory problem of minimizing the expected discounted present value of production cost control in a manufacturing system with degenerate stochastic demand. We establish the existence of a unique solution of the Hamilton-Jacobi-Bellman (HJB) equation associated with this problem. The optimal control is given by a solution to the corresponding HJB equation.     

Optimal pricing,production, and inventory for deteriorating items under demand uncertainty: The finite horizon case

This paper aims to investigate the joint dynamic pricing and production decisions of deteriorating items with uncertain demand over a finite selling season, where the demand is price sensitive and the potential demand is characterized by a stochastic process. The stocks deteriorate physically at a constant fraction of the on-hand inventory. A joint dynamic pricing and production problem to maximize the total expected profit is modeled as a stochastic optimal control problem. We derive the closed-form solutions, which are in time-dependent linear feedback form of the inventory level when it is either positive or negative. It is shown that the manufacturer always benefits from a reduction in the volatility of potential market demand. In addition, to highlight the effectiveness of the joint dynamic strategy, we also consider the case of optimal production with a static price. A numerical example is presented to illustrate the validity of the optimal control policy, and sensitivity analysis on major parameters is performed to provide more managerial insights into deteriorating items.     

不完备质量下的报童问题及其评估机制

在产品质量不完备的环境下,考虑了需求依赖于质量水平的报童问题。本文主要利用马氏理论刻画质量水平与需求之间关联性的动态演变过程,并将"不完备质量"的决策理念纳入报童问题的理论框架,进而提出了新的随机库存系统的优化模型及其决策机制。同时,利用随机质量过程中的首达性、遍历性、不可约性等基本属性,构建了随机库存系统在运作和管理过程中的可靠性及其收益评估机制。模型的相关结论表明:在不完备质量的环境下,零售商的最优订购决策是由各个质量状态的转移概率所确定,若由质量水平的波动性所导出的随机过程为不可约遍历马氏链时,则库存系统的决策机制具有良好的稳定性。     

Performance analysis and design of supply chains: a Petri net approach

In this paper, we investigated a dynamic modelling technique for analysing supply chain networks using generalised stochastic Petri nets (GSPNs). The customer order arrival process is assumed to be Poisson and the service processes at the various facilities of the supply chain are assumed to be exponential. Our model takes into account both the procurement process and delivery logistics that exist between any two members of the supply chain. We compare the performance of two production planning and control policies, the make-to-stock and the assemble-to-order systems in terms of total cost which is the sum of inventory carrying cost and cost incurred due to delayed deliveries. We formulate and solve the decoupling point location problem in supply chains as a total relevant cost (sum of inventory carrying cost and the delay costs) minimisation problem. We use the framework of integrated GSPN-queuing network modelling—with the GSPN at the higher level and a generalised queuing network at the lower level—to solve the decoupling point location problem.     

A stochastic multi-period industrial hazardous waste location-routing problem: Integrating NSGA-II and Monte Carlo simulation

The present study extends a multi-objective mathematical model in the context of industrial hazardous waste management, which covers the integrated decisions of three levels with locating, vehicle routing, and inventory control. Analyzing these decisions simultaneously not only may lead to the most effective structure in the waste management network, but also may reduce the potential risk of managing the hazardous waste. Furthermore, because of the inherent complexity of the waste management system, uncertainty is inevitable and should be acknowledged to guarantee reliability in the decision-making process. From this perspective, the proposed model is novel in the following three aspects: (1) shifting from a deterministic to stochastic environment; (2) considering a multi-period planning horizon; and (3) incorporating the inventory decisions into the problem. The problem is formulated as a multi-objective stochastic Mixed-Integer Nonlinear Programming (MINLP) model, which can be easily converted into a MILP one. In terms of methodological contribution, a new simheuristic approach that is an integration of Non-Dominated Sorting Genetic Algorithm-II (NSGA-II) and Monte Carlo simulation is developed to overcome the stochastic combinatorial optimization problem of this study. Our findings verify the efficiency of the proposed approach as it is able to find a high-quality solution within a relatively reasonable computational time.     

A multi-product continuous review inventory system with stochastic demand,backorders, and a budget constraint

Common characteristics of inventory systems include uncertain demand and restrictions such as budgetary or storage space constraints. Several authors have examined budget constrained multi-item stochastic inventory systems controlled by continuous review policies without considering marginal shortage costs. Existing models assume that purchasing costs are paid at the time an order is placed, which is not always the case since in some systems purchasing costs are paid when orders arrive. In the latter case the maximum investment in inventory is random since the inventory level when an order arrives is a random variable. Hence payment of purchasing costs on delivery yields a stochastic budget constraint for inventory. This paper models a multi-item stochastic inventory system with backordered shortages when estimation of marginal backorder cost is available, and payment is due upon order arrival. The budget constraint can easily be converted into a storage constraint.     

A stochastic inventory model

Summary In some cases arising in certain industries or military installations not only the demand for a particular commodity is a stochastic variable but its supply as well. In these cases it is convenient to consider the inventory level resulting from the interaction of supply and demand as a third stochastic variable. The variation of the inventory level in time can then be considered as a stochastic process. If this process is ergodic, the total inventory cost over a certain timeT may be represented as a function of the mean inventory level. This mean level can then be manipulated in such a way as to minimize the total inventory cost.

---

Zusammenfassung Es kommt vor, daß in gewissen Industriezweigen sowohl der Verbrauch, als auch die Anlieferung eines bestimmten Gutes stochastische Variable sind. In solchen Fällen ist es zweckmäßig, wenn man die aus der Zusammenwirkung von Verbrauch und Anlieferung resultierende Vorratsmenge als eine dritte stochastische Variable einführt. Man kann dann die Oszillationen der Vorratsmenge in der Zeit als einen stochastischen Prozeß auffassen. Falls dieser Prozeß ergodisch ist, können die gesamten Vorratshaltungskosten für eine bestimmte ZeitT dargestellt werden als eine Funktion der mittleren Vorratsmenge. Diese mittlere Vorratsmenge kann dann so bestimmt werden, daß sie die gesamten Vorratshaltungskosten minimalisiert.

---

---

SHAPE Air Defence Technical Centre. Formerly with Tidewater Oil Company, Los Angeles, California, where the present problem was originally investigated.

---

Vorgel. v.:J. Nitsche.     

A Deterministic Approach To Stochastic Optimal Control With Application To Anticipative Control

Using the decomposition of solution of SDE, we consider the stochastic optimal control problem with anticipative controls as a family of deterministic control problems parametrized by the paths of the driving Wiener process and of a newly introduced Lagrange multiplier stochastic process (nonanticipativity equality constraint). It is shown that the value function of these problems is the unique global solution of a robust equation (random partial differential equation) associated to a linear backward Hamilton-Jacobi-Bellman stochastic partial differential equation (HJB SPDE). This appears as limiting SPDE for a sequence of random HJB PDE's when linear interpolation approximation of the Wiener process is used. Our approach extends the Wong-Zakai type results [20] from SDE to the stochastic dynamic programming equation by showing how this arises as average of the limit of a sequence of deterministic dynamic programming equations. The stochastic characteristics method of Kunita [13] is used to represent the value function. By choosing the Lagrange multiplier equal to its nonanticipative constraint value the usual stochastic (nonanticipative) optimal control and optimal cost are recovered. This suggests a method for solving the anticipative control problems by almost sure deterministic optimal control. We obtain a PDE for the “cost of perfect information” the difference between the cost function of the nonanticipative control problem and the cost of the anticipative problem which satisfies a nonlinear backward HJB SPDE. Poisson bracket conditions are found ensuring this has a global solution. The cost of perfect information is shown to be zero when a Lagrangian submanifold is invariant for the stochastic characteristics. The LQG problem and a nonlinear anticipative control problem are considered as examples in this framework     

Optimal investment–consumption problem: Post-retirement with minimum guarantee

We study the optimal investment–consumption problem for a member of defined contribution plan during the decumulation phase. For a fixed annuitization time, to achieve higher final annuity, we consider a variable consumption rate. Moreover, to have a minimum guarantee for the final annuity, a safety level for the wealth process is considered. To solve the stochastic optimal control problem via dynamic programming, we obtain a Hamilton–Jacobi–Bellman (HJB) equation on a bounded domain. The existence and uniqueness of classical solutions are proved through the dual transformation. We apply the finite difference method to find numerical approximations of the solution of the HJB equation. Finally, the simulation results for the optimal investment–consumption strategies, optimal wealth process and the final annuity for different admissible ranges of consumption are given. Furthermore, by taking into account the market present value of the cash flows before and after the annuitization, we compare the outcomes of different scenarios.     

伊藤-泊松型随机微分方程的线性二次控制

本文研究伊藤-泊松型随机微分方程的线性二次控制问题,利用动态规划方法、伊藤公式等技巧,通过解HJB方程,我们得到了随机Riccati方程及另外两个微分方程,求出控制变量,解决了线性二次最优控制最优问题.     

具有两类需求服务的随机库存系统控制策略研究

研究具有两类顾客排队需求服务的随机库存系统.系统采取(s,Q)补货策略且当库存水平下降到安全库存s时,到达的第二类顾客以概率P得到服务.首先,建立库存水平状态转移方程并通过递推算法求解获得库存水平稳态概率分布和系统稳态指标;接下来,构建库存成本函数;最后,采用数值试验的方法研究该库存系统的最优控制策略并考察系统参数的敏感性.     

Finding and identifying optimal inventory levels for systems with common components

In this article, we consider the problem of finding the optimal inventory level for components in an assembly system where multiple products share common components in the presence of random demand. Previously, solution procedures that identify the optimal inventory levels for components in a component commonality problem have been considered for two product or one common component systems. We will here extend this to a three products system considering any number of common components. The inventory problem considered is modeled as a two stage stochastic recourse problem where the first stage is to set the inventory levels to maximize expected profit while the second stage is to allocate components to products after observing demand. Our main contribution, and the main focus of this paper, is the outline of a procedure that finds the gradient for the stochastic problem, such that an optimal solution can be identified and a gradient based search method can be used to find the optimal solution.     

Theoretical solution versus industry standard: Optimal leverage function for CPDOs

In this paper, we derive an optimal leverage function for Constant Proportion Debt Obligations (CPDOs) by using stochastic control techniques. The investor’s goal is to maximise redemption of capital at maturity. The control variable of the problem is the leverage process, i.e. the time dependent notional exposure to the underlying risky index/portfolio. The control problem is solved explicitly with the help of the Legendre transform applied to the HJB equation of stochastic control. A closed form solution is given for the optimal leverage. Contrary to the industry practise, the optimal leverage derived in this paper is a non-linear, bell-shaped function of the CPDO assets value.     

Stochastic Linear Quadratic Optimal Control Problems

This paper is concerned with the stochastic linear quadratic optimal control problem (LQ problem, for short) for which the coefficients are allowed to be random and the cost functional is allowed to have a negative weight on the square of the control variable. Some intrinsic relations among the LQ problem, the stochastic maximum principle, and the (linear) forward—backward stochastic differential equations are established. Some results involving Riccati equation are discussed as well. Accepted 15 May 2000. Online publication 1 December 2000     

A maximum principle for optimal control problem of fully coupled forward-backward stochastic systems with partial information

The paper is concerned with a stochastic optimal control problem in which the controlled system is described by a fully coupled nonlinear forward-backward stochastic differential equation driven by a Brownian motion. It is required that all admissible control processes are adapted to a given subfiltration of the filtration generated by the underlying Brownian motion. For this type of partial information control, one sufficient (a verification theorem) and one necessary conditions of optimality are proved. The control domain need to be convex and the forward diffusion coefficient of the system can contain the control variable. This work was partially supported by Basic Research Program of China (Grant No. 2007CB814904), National Natural Science Foundation of China (Grant No. 10325101) and Natural Science Foundation of Zhejiang Province (Grant No. Y605478, Y606667)     

Continuous-review inventory problem with random supply interruptions

This paper considers a continuous-review stochastic inventory problem with random demand and random lead-time where supply may be disrupted due to machine breakdowns, strikes or other randomly occurring events. The supplier availability is modelled as a semi-Markov process (more specifically, as an alternating renewal process). The standard (q, r) policy is used when the supplier is available (ON), i.e., when the inventory position reaches the reorder point r, q units are ordered to raise the inventory position to the target level of R = q + r. The form of the policy changes when the supplier becomes unavailable (OFF) in which case orders cannot be placed when the reorder point r is reached. However, as soon as the supplier becomes available again one orders enough to bring the inventory position up to the target level of R. The regenerative cycles are identified by observing the inventory position process. We construct the average cost per time objective function using the renewal reward theorem. It is assumed that the duration of the ON period is E_k (i.e., k-stage Erlangian) and the OFF period is general. In analogy with queuing notation we call this an E_k/G system. By employing the ‘method of stages’, we obtain a problem with a larger state space for the ON/OFF stochastic process; but the resulting ON process can now be analyzed using Markovian techniques. For asymptotic values of q, the objective function assumes a particularly simple form which is shown to be convex under mild restrictions on the density functions of demand. Numerical examples illustrate the results.     

Inventory levels to stop and restart a single machine producing one product

A central problem in production planning is the coordination of the production rate with the inventory level in order to find a suitable compromise between the inventory on hand, the frequency of changes in the production rate and customer service. This paper deals with an one product production/inventory problem with an intermittently operating production facility controlled by inventory levels to shut down and restart production. The demand process is a compound Poisson process and a service level constraint is imposed on the fraction of demand to be met directly from stock on hand. The paper presents a tractable two-moments approximation for the control rule for starting up and shutting down the production.     

Analysis of a single-stage,single-product,stochastic, MRP-controlled assembly system

This paper models a single-stage, single-product, stochastic assembly system, operating according to an Materials Requirements Planning controlled (MRP) ordering philosophy. It deals explicitly with the underlying stochastic process that describes the end-product inventory position, enabling production lead times to be treated as independent and generally distributed random variables. The inventory position process is identified as a Markov renewal process, and this structure is exploited to determine system performance measures such as average inventory level, average backorder level, and the probability distribution of the end-product inventory position. An example, which demonstrates the type of analysis possible, focuses on quantifying the effect of kitting on the availability of end-products.