Reinforcement learning for joint pricing,lead-time and scheduling decisions in make-to-order systems

The paper investigates a problem faced by a make-to-order (MTO) firm that has the ability to reject or accept orders, and set prices and lead-times to influence demands. Inventory holding costs for early completed orders, tardiness costs for late delivery orders, order rejection costs, manufacturing variable costs, and fixed costs are considered. In order to maximize the expected profits in an infinite planning horizon with stochastic demands, the firm needs to make decisions from the following aspects: which orders to accept or reject, the trade-off between price and lead-time, and the potential for increased demand against capacity constraints. We model the problem as a Semi-Markov Decision Problem (SMDP) and develop a reinforcement learning (RL) based Q-learning algorithm (QLA) for the problem. In addition, we build a discrete-event simulation model to validate the performance of the QLA, and compare the experimental results with two benchmark policies, the First-Come-First-Serve (FCFS) policy and a threshold heuristic policy. It is shown that the QLA outperforms the existing policies.     

Insurance valuation: A computable multi-period cost-of-capital approach

We present an approach to market-consistent multi-period valuation of insurance liability cash flows based on a two-stage valuation procedure. First, a portfolio of traded financial instrument aimed at replicating the liability cash flow is fixed. Then the residual cash flow is managed by repeated one-period replication using only cash funds. The latter part takes capital requirements and costs into account, as well as limited liability and risk averseness of capital providers. The cost-of-capital margin is the value of the residual cash flow. We set up a general framework for the cost-of-capital margin and relate it to dynamic risk measurement. Moreover, we present explicit formulas and properties of the cost-of-capital margin under further assumptions on the model for the liability cash flow and on the conditional risk measures and utility functions. Finally, we highlight computational aspects of the cost-of-capital margin, and related quantities, in terms of an example from life insurance.     

A detailed workforce planning model including non-linear dependence of capacity on the size of the staff and cash management

This paper introduces an original planning model which integrates production, human resources and cash management decisions, taking into account the consequences that decisions in one area may have on other areas and allowing all these areas to be coordinated. The most relevant characteristics of the planning problem are: (1) production capacity is a non-linear function of the size of the staff; (2) firing costs may depend on the worker who is fired; (3) working time is managed under a working time account (WTA) scheme, so positive balances must be paid to workers who leave the company; (4) there is a learning period for hired workers; and (5) cash management is included. A mixed integer linear program is designed to solve the problem. Despite the size and complexity of the model, it can be solved in a reasonable time. A numerical example, the main results of a computational experiment and a sensibility analysis illustrate the performance and benefits of the model.     

Optimal policy for a dynamic,non-stationary,stochastic inventory problem with capacity commitment

This paper studies a single-product, dynamic, non-stationary, stochastic inventory problem with capacity commitment, in which a buyer purchases a fixed capacity from a supplier at the beginning of a planning horizon and the buyer’s total cumulative order quantity over the planning horizon is constrained with the capacity. The objective of the buyer is to choose the capacity at the beginning of the planning horizon and the order quantity in each period to minimize the expected total cost over the planning horizon. We characterize the structure of the minimum sum of the expected ordering, storage and shortage costs in a period and thereafter and the optimal ordering policy for a given capacity. Based on the structure, we identify conditions under which a myopic ordering policy is optimal and derive an equation for the optimal capacity commitment. We then use the optimal capacity and the myopic ordering policy to evaluate the effect of the various parameters on the minimum expected total cost over the planning horizon.     

Capacity planning in networks of queues with manufacturing applications

This paper addresses capacity planning in systems that can be modeled as a network of queues. More specifically, we present an optimization model and solution methods for the minimum cost selection of capacity at each node in the network such that a set of system performance constraints is satisfied. Capacity is controlled through the mean service rate at each node. To illustrate the approach and how queueing theory can be used to measure system performance, we discuss a manufacturing model that includes upper limits on product throughput times and work-in-process in the system. Methods for solving capacity planning problems with continuous and discrete capacity options are discussed. We focus primarily on the discrete case with a concave cost function, allowing fixed charges and costs exhibiting economies of scale with respect to capacity to be handled.     

3.1. OR in banking

In this paper, we describe a deterministic multiperiod capacity expansion model in which a single facility serves the demand for many products. Potential applications for the model can be found in the capacity expansion planning of communication systems as well as in the production planning of heavy process industries. The model assumes that each capacity unit simultaneously serves a prespecified (though not necessarily integer) number of demand units of each product. Costs considered include capacity expansion costs, idle capacity holding costs, and capacity shortage costs. All cost functions are assumed to be nondecreasing and concave. Given the demand for each product over the planning horizon, the objective is to find the capacity expansion policy that minimizes the total cost incurred. We develop a dynamic programming algorithm that finds optimal policies. The required computational effort is a polynomial function of the number of products and the number of time periods. When the number of products equals one, the algorithm reduces to the well-known algorithm for the classical dynamic lot size problem.     

Production lot-sizing with dynamic capacity adjustment

In this paper we study a single-item lot-sizing model in which production capacity can be adjusted from time to time. There are a number of different production capacity levels available to be acquired in each period, where each capacity level is assumed to be a multiple of a base capacity unit. To reduce the waste of excess of capacity but guarantee meeting the demand, it is important to decide which level of capacity should be acquired and how many units of the item should be produced for every period in the planning horizon. Capacity adjustment cost incurs when capacity acquired in the current period differs from the one acquired in the previous period. Capacity acquisition costs, capacity adjustment costs, and production costs in each period are all time-varying and depend on the capacity level acquired in that period. Backlogging is allowed. Both production costs and inventory costs are assumed to be general concave. We provide optimal properties and develop an efficient exact algorithm for the general model. For the special cases with zero capacity adjustment costs or fixed-plus-linear production costs, we present a faster exact algorithm. Computational experiments show that our algorithm is able to solve medium-size instances for the general model in a few seconds, and that cost can be reduced significantly through flexible capacity adjustment.     

Tactical level planning in float glass manufacturing with co-production,random yields and substitutable products

We investigate tactical level planning problems in float glass manufacturing. Float glass manufacturing is a process that has some unique properties such as uninterruptible production, random yields, partially controllable co-production compositions, complex relationships in sequencing of products, and substitutable products. Furthermore, changeover times and costs are very high, and production speed depends significantly on the product mix. These characteristics render measurement and management of the production capacity difficult. The motivation for this study is a real life problem faced at Trakya Cam in Turkey. Trakya Cam has multiple geographically separated production facilities. Since transportation of glass is expensive, logistics costs are high. In this paper, we consider multi-site aggregate planning, and color campaign duration and product mix planning. We develop a decision support system based on several mixed integer linear programming models in which production and transportation decisions are made simultaneously. The system has been fully implemented, and has been in use at Trakya Cam since 2005.     

A fuzzy stochastic single-period model for cash management

The major purpose of this paper is to apply a stochastic single-period inventory management approach to analyze optimal cash management policies with fuzzy cash demand based on fuzzy integral method so that total cost is minimized. We will find that, after defuzzification, the cash-raising amounts and the total costs between the fuzzy case and the crisp case are slightly different when the variation of cash demand is small. As a result, we point out that the fuzzy stochastic single-period model is one extension of the crisp models. In any case, one may conclude that a conscientious analysis in fuzzy mathematics like that presented in this paper provides a financial decision maker with a deeper insight into the more real cash management problem.     

Heuristics for Distribution Network Design in Telecommunication

A distribution network problem arises in a lower level of an hierarchical modeling approach for telecommunication network planning. This paper describes a model and proposes a lagrangian heuristic for designing a distribution network. Our model is a complex extension of a capacitated single commodity network design problem. We are given a network containing a set of sources with maximum available supply, a set of sinks with required demands, and a set of transshipment points. We need to install adequate capacities on the arcs to route the required flow to each sink, that may be an intermediate or a terminal node of an arborescence. Capacity can only be installed in discrete levels, i.e., cables are available only in certain standard capacities. Economies of scale induce the use of a unique higher capacity cable instead of an equivalent set of lower capacity cables to cover the flow requirements of any link. A path from a source to a terminal node requires a lower flow in the measure that we are closer to the terminal node, since many nodes in the path may be intermediate sinks. On the other hand, the reduction of cable capacity levels across any path is inhibited by splicing costs. The objective is to minimize the total cost of the network, given by the sum of the arc capacity (cables) costs plus the splicing costs along the nodes. In addition to the limited supply and the node demand requirements, the model incorporates constraints on the number of cables installed on each edge and the maximum number of splices at each node. The model is a NP-hard combinatorial optimization problem because it is an extension of the Steiner problem in graphs. Moreover, the discrete levels of cable capacity and the need to consider splicing costs increase the complexity of the problem. We include some computational results of the lagrangian heuristics that works well in the practice of computer aided distribution network design.     

A stochastic programming approach to cash management in banking

The treasurer of a bank is responsible for the cash management of several banking activities. In this work, we focus on two of them: cash management in automatic teller machines (ATMs), and in the compensation of credit card transactions. In both cases a decision must be taken according to a future customers demand, which is uncertain. From historical data we can obtain a discrete probability distribution of this demand, which allows the application of stochastic programming techniques. We present stochastic programming models for each problem. Two short-term and one mid-term models are presented for ATMs. The short-term model with fixed costs results in an integer problem which is solved by a fast (i.e. linear running time) algorithm. The short-term model with fixed and staircase costs is solved through its MILP equivalent deterministic formulation. The mid-term model with fixed and staircase costs gives rise to a multi-stage stochastic problem, which is also solved by its MILP deterministic equivalent. The model for compensation of credit card transactions results in a closed form solution. The optimal solutions of those models are the best decisions to be taken by the bank, and provide the basis for a decision support system.     

Inventory control with an order-time constraint: optimality,uniqueness and significance

This paper analyzes a stochastic inventory problem with an order-time constraint that restricts the times at which a manufacturer places new orders to a supplier. This constraint stems from the limited upstream capacity in a supply chain, such as production capacity at a supplier or transportation capacity between a supplier and a manufacturer. Consideration of limited upstream capacity extends the classical inventory literature that unrealistically assumes infinite supplier/transporter capacity. But this consideration increases the complexity of the problem. We study the constraint under a Poisson demand process and allow for a fixed ordering cost. In presence of the constraint, we establish the optimality of an (s,S) policy under both the discounted and average cost objectives. Under the average cost objective, we show the uniqueness of the order-up-to level S. We numerically compare our model with the classical unconstrained model. We report significant savings in costs that can be achieved by using our model when the order time is constrained.     

Notes on the single route lateral transhipment problem

Previous research has analyzed deterministic and stochastic models of lateral transhipments between different retailers in a supply chain. In these models the analysis assumes given fixed transhipment costs and determines under which situations (magnitudes of excess supply and demand at various retailers) the transhipment is profitable. However, in reality, these depend on aspects like the distance between retailers or the transportation mode chosen. In many situations, combining the transhipments may save transportation costs. For instance, one or more vehicle routes may be used to redistribute the inventory of the potential pickup and delivery stations. This can be done in any sequence as long as the vehicle capacity is not violated and there is enough load on the vehicle to satisfy demand. The corresponding problem is an extension of the one-commodity pickup and delivery traveling salesman and the pickup and delivery vehicle routing problem. When ignoring the routing aspect and assuming given fixed costs, transhipment is only profitable if the quantities are higher than a certain threshold. In contrast to that, the selection of visited retailers is dependent on the transportation costs of the tour and therefore the selected retailers are interrelated. Hence the problem also has aspects of a (team) orienteering problem. The main contribution is the discussion of the tour planning aspects for lateral transhipments which may be valuable for in-house planning but also for price negotiations with external contractors. A mixed integer linear program for the single route and single commodity version is presented and an improved LNS framework to heuristically solve the problem is introduced. Furthermore, the effect of very small load capacity on the structure of optimal solutions is discussed.     

Order Quantities with Temporary Price Reductions

The temporary price-change problem is studied, in which the objective is to minimize discounted cash flows. As pointed out by Goyal in an earlier paper, only the cash transactions at purchase times (i.e. the payments for the goods and the ordering costs) were considered. The cash flows associated with `inventory maintenance' costs which occur more or less continuously over time were neglected, which changes the structure of the model. Examples of these costs include storage, insurance, record-keeping, deterioration and obsolescence costs. In this paper, these continuously generated cash flows are included in the analysis, thereby making the new model more applicable to practical situations. This model is of interest because order-quantity decisions often must be made under conditions of both temporary price reductions and/or imminent price increases. These changes occur frequently in practice.     

Linear programming models with planned lead times for supply chain operations planning

This paper contributes to the development of models for capacity constrained Supply Chain Operations Planning (SCOP). We focus on production environments with arbitrary supply chain structures. The demand for the end items is assumed to be exogenously determined. We solve the SCOP problem with Linear Programming models using planned lead times with multi-period capacity consumption. Using planned lead times increases the reliability of the communication between SCOP and Scheduling with regard to the feasibility of the planning. Planned lead times also reduce the nervousness in the system. We model capacity constraints on the quantity of items that can be assembled within a time interval. In particular, items can be assigned to multiple resources. We discuss two LP approaches which plan the production of items so that a sum of inventory costs and costs due to backordering is minimized.     

Multi-Item, Multi-Facility Supply Chain Planning: Models, Complexities, and Algorithms

We propose a planning model for products manufactured across multiple manufacturing facilities sharing similar production capabilities. The need for cross-facility capacity management is most evident in high-tech industries that have capital-intensive equipment and a short technology life cycle. We propose a multicommodity flow network model where each commodity represents a product and the network structure represents manufacturing facilities in the supply chain capable of producing the products. We analyze in depth the product-level (single-commodity, multi-facility) subproblem when the capacity constraints are relaxed. We prove that even the general-cost version of this uncapacitated subproblem is NP-complete. We show that there exists an optimization algorithm that is polynomial in the number of facilities, but exponential in the number of periods. We further show that under special cost structures the shortest-path algorithm could achieve optimality. We analyze cases when the optimal solution does not correspond to a source-to-sink path, thus the shortest path algorithm would fail. To solve the overall (multicommodity) planning problem we develop a Lagrangean decomposition scheme, which separates the planning decisions into a resource subproblem, and a number of product-level subproblems. The Lagrangean multipliers are updated iteratively using a subgradient search algorithm. Through extensive computational testing, we show that the shortest path algorithm serves as an effective heuristic for the product-level subproblem (a mixed integer program), yielding high quality solutions with only a fraction (roughly 2%) of the computer time.     

Analysis of the stochastic cash balance problem using a level crossing technique

The simple cash management problem includes the following considerations: the opportunity cost of holding too much cash versus the penalty cost of not having enough cash to meet current needs; the cost incurred (or profit generated) when making changes to cash levels by increasing or decreasing them when necessary; the uncertainty in timing and magnitude of cash receipts and cash disbursements; and the type of control policy that should be used to minimize the required level of cash balances and related costs. In this paper, we study a version of this problem in which cash receipts and cash disbursements occur according to two independent compound Poisson processes. The cash balance is monitored continuously and an order-point, order-up-to-level, and keep-level \( \left( {s, S, M} \right) \) policy is used to monitor the content, where \( s \le S \le M \). That is, (a) if, at any time, the cash level is below s, an order is immediately placed to raise the level to S; (b) if the cash level is between s and M, no action is taken; (c) if the cash level is greater than M, the amount in excess of M is placed into an earning asset. We seek to minimize the expected total costs per unit time of running the cash balance. We use a level-crossing approach to develop a solution procedure for finding the optimal policy parameters and costs. Several numerical examples are given to illustrate the tradeoffs.     

现金持有动态调整机制——基于动态面板模型的实证分析

本文通过建立动态面板数据模型,系统地研究了中国企业现金持有的动态调整机制。发现:第一,企业存在最优目标现金持有水平,现金持有策略倾向于围绕这一目标进行动态调整,但是我国资本市场动态调整的成本较高;第二,现金持有不足企业的调整速度快于超额现金持有企业的调整速度,这说明,相对于超额现金持有带来的代理成本负面影响,企业更加倾向于关注现金持有不足所产生的流动性短缺风险。并且统计过程表明系统广义矩方法在动态面板模型估计中具有合理性。     

On upper bounds of sequential stochastic production planning problems

In this paper we consider a class of problems that determine production, inventory and work force levels for a firm in order to meet fluctuating demand requirements. A production planning problem arises because of the need to match, at the firm level, supply and demand efficiently. In practice, the two common approaches to counter demand uncertainties are (i) carrying a constant safety stock from period to period, and (ii) planning with a rolling horizon. Under the rolling horizon (or sequential) strategy the planning model is repeatedly solved, usually at the end of every time period, as new information becomes available and is used to update the model parameters. The costs associated with a rolling horizon strategy are hard to compute a priori because the solution of the model in any intermediate time period depends on the actual demands of the previous periods.In this paper we derive two a priori upper bounds on the costs for a class of production planning problems under the rolling horizon strategy. These upper bounds are derived by establishing correspondences between the rolling horizon problems and related deterministic programs. One of the upper bounds is obtained through Lagrangian relaxation of the service level constraint. We propose refinements to the non-Lagrangian bounds and present limited computational results. Extensions of the main results to the multiple item problems are also discussed. The results of this paper are intended to support production managers in estimating the production costs and value of demand information under a rolling horizon strategy.     

Cash management cost reduction using data mining to forecast cash demand and LP to optimize resources

This paper presents a two step model aimed at reducing cash management costs in a bank’s branch. First, data mining was used to forecast daily cash demand, comparing an ARMA-ARCH model with a neural network. Secondly, using the prior result, a linear programming model was solved. The optimal allocation of resources, i.e., cash collections and supplies was estimated showing that the model can be a helpful tool to support the determination of collections and supplies at the bank branch.