Optimal policy for minimizing risk models in Markov decision processes

We consider the minimizing risk problems in discounted Markov decisions processes with countable state space and bounded general rewards. We characterize optimal values for finite and infinite horizon cases and give two sufficient conditions for the existence of an optimal policy in an infinite horizon case. These conditions are closely connected with Lemma 3 in White (1993), which is not correct as Wu and Lin (1999) point out. We obtain a condition for the lemma to be true, under which we show that there is an optimal policy. Under another condition we show that an optimal value is a unique solution to some optimality equation and there is an optimal policy on a transient set.     

Time consistency and risk averse dynamic decision models: Definition,interpretation and practical consequences

This paper aims at resolving a major obstacle to practical usage of time-consistent risk-averse decision models. The recursive objective function, generally used to ensure time consistency, is complex and has no clear/direct interpretation. Practitioners rather choose a simpler and more intuitive formulation, even though it may lead to a time inconsistent policy. Based on rigorous mathematical foundations, we impel practical usage of time consistent models as we provide practitioners with an intuitive economic interpretation for the referred recursive objective function. We also discourage time-inconsistent models by arguing that the associated policies are sub-optimal. We developed a new methodology to compute the sub-optimality gap associated with a time-inconsistent policy, providing practitioners with an objective method to quantify practical consequences of time inconsistency. Our results hold for a quite general class of problems and we choose, without loss of generality, a CVaR-based portfolio selection application to illustrate the developed concepts.     

Confidence structures in decision making

Decision making is defined in terms of four elements: the set of decisions, the set of outcomes for each decision, a set-valued criterion function, and the decision maker's value judgment for each outcome. Various confidence structures are defined, which give the decision maker's confidence of a given decision leading to a particular outcome. The relation of certain confidence structures to Bayesian decision making and to membership functions in fuzzy set theory is established. A number of schemes are discussed for arriving atbest decisions, and some new types of domination structures are introduced.This research was partly supported by Project No. NR-047-021, ONR Contract No. N-00014-75-C-0569 with the Center for Cybernetic Studies, The University of Texas, Austin, Texas, and by ONR Contract No. N-00014-69-A-0200-1012 with the University of California, Berkeley, California.     

Correlation order, merging and diversification

We investigate the influence of the dependence between random losses on the shortfall and on the diversification benefit that arises from merging these losses.We prove that increasing the dependence between losses, expressed in terms of correlation order, has an increasing effect on the shortfall, expressed in terms of an appropriate integral stochastic order. Furthermore, increasing the dependence between losses decreases the diversification benefit.We also consider merging comonotonic losses and show that even in this extreme case a strictly positive diversification benefit will often arise.     

Effects of risk aversion and decision preference on equilibriums in supply chain finance incorporating bank credit with credit guarantee

We constructed a Stackelberg game in a supply chain finance (SCF) system including a manufacturer, a capital‐constrained retailer, and a bank that provides loans on the basis of the manufacturer's credit guarantee. To emphasize the financial service providers' risks, we assumed that both the bank and the manufacturer are risk‐averse and formulated trade‐off objective functions for both of them as the convex combination of the expected profit and conditional value‐at‐risk. To explore the effects of the risk preferences and decision preferences on SCF equilibriums, we mathematically analyzed the optimal order quantities, wholesale prices, and interest rates under different risk preference scenarios and performed numerical analyses to quantify the effects. We found that incorporating bank credit with a credit guarantee can effectively balance the retailer's financing risk between the bank and the manufacturer through interest rate charging and wholesale pricing. Moreover, SCF equilibriums with risk aversion are highly affected by the degree of both the lender's and guarantor's risk tolerance in regard to the borrower's default probability and will be more conservative than those in the risk‐neutral cases that only maximize expected profit.     

Risk models with dependence between claim occurrences and severities for Atlantic hurricanes

In the line of Cossette et al. (2003), we adapt and refine known Markovian-type risk models of Asmussen (1989) and Lu and Li (2005) to a hurricane risk context. These models are supported by the findings that El Niño/Southern Oscillation (as well as other natural phenomena) influence both the number of hurricanes and their strength. Hurricane risk is thus broken into three components: frequency, intensity and damage where the first two depend on the state of the Markov chain and intensity influences the amount of damage to an individual building. The proposed models are estimated with Florida hurricane data and several risk measures are computed over a fictitious portfolio.     

Multivariate time changes for Lévy asset models: Characterization and calibration

We build a theoretical framework for multivariate subordination of Brownian motions, with a common and an idiosyncratic component. This follows economic intuition and introduces generalizations of some well known multivariate Lévy processes for financial applications: the compound Poisson, NIG, Variance Gamma and CGMY. In most cases we obtain the characteristic function in closed form. The extension is first kept parsimonious, by adding one parameter only. The empirical fit of (linear) dependence is then increased, by allowing for dependent Brownian motions.     

Optimal threshold probability and expectation in semi-Markov decision processes

We consider undiscounted semi-Markov decision process with a target set and our main concern is a problem minimizing threshold probability. We formulate the problem as an infinite horizon case with a recurrent class. We show that an optimal value function is a unique solution to an optimality equation and there exists a stationary optimal policy. Also several value iteration methods and a policy improvement method are given in our model. Furthermore, we investigate a relationship between threshold probabilities and expectations for total rewards.     

The order of a zero of a Wronskian and the theory of linear dependence

Every root of the top Wronskian of a Wronskian matrix whose rank at the root is equal to the number of columns, is of integer order even if the highest derivatives exist only at the root. If the rank of a Wronskian matrix is constant and smaller than the number of rows, then the number of independent linear relations between the functions in the first row is equal to the number of functions minus the rank. These results were proved under additional assumptions by Bôcher, Curtiss, and Moszner. Their proofs are simplified.     

Possibilities of solution in stochastic decision models with recursive reward functions

A stochastic decision model with general, non-necessarily additive reward function is considered and essential properties of such reward functions are formulated which only allow a successive proceeding in the sense of dynamic programming. Conditions for recursive—additive reward functions are given which ensure the existence of optimal strategies and the usability of value iteration to find an optimal policy and the optimal total reward.     

Entropic risk measures and their comparative statics in portfolio selection: Coherence vs. convexity

We conduct a decision-theoretic analysis of optimal portfolio choices and, in particular, their comparative statics under two types of entropic risk measures, the coherent entropic risk measure (CERM) and the convex entropic risk measure (ERM). Starting with the portfolio selection between a risky and a risk free asset (framework of Arrow (1965) and Pratt (1964)), we find a restrictive all-or-nothing investment decision under the CERM, while the ERM yields diversification. We then address a portfolio problem with two risky assets, and provide comparative statics with respect to the investor’s risk aversion (framework of Ross (1981)). Here, both the CERM and the ERM exhibit closely interrelated inconsistencies with respect to the interpretation of their risk parameters as a measure of risk aversion: for any two investors with different risk parameters, it may happen that the investor with the higher risk parameter invests more in the riskier one of the two assets. Finally, we analyze the portfolio problem “risky vs. risk free” in the presence of an independent background risk, and analyze the effect of changes in this background risk (framework of Gollier and Pratt (1996)). Again, we find questionable predictions: under the CERM, the optimal risky investment is always increasing instead of decreasing when a background risk is introduced, while under the ERM it remains unaffected.     

Aggregation and disaggregation in Markov decision models for inventory control

In this paper the possibility is investigated of using aggregation in the action space for some Markov decision processes of inventory control type. For the standard (s, S) inventory control model the policy improvement procedure can be executed in a very efficient way, therefore, aggregation in the action space is not of much use. However, in situations where the decisions have some aftereffect and, hence, the old decision has to be incorporated in the state, it might be rewarding to aggregate actions. Some variants for aggregation and disaggregation are formulated and analyzed. Numerical evidence is presented.     

Generalization of domination structures and nondominated solutions in multicriteria decision making

The concepts of domination structures and nondominated solutions are important in tackling multicriteria decision problems. We relax Yu's requirement that the domination structure at each point of the criteria space be a convex cone (Ref. 1) and give results concerning the set of nondominated solutions for the case where the domination structure at each point is a convex set. A practical necessity for such a generalization is discussed. We also present conditions under which a locally nondominated solution is also a globally nondominated solution.     

Complex structures and composite models — An essay on methodology

A principal desideratum of a methodology adequate for the task of constructing models of complex military, socio-political phenomena, typified in C3 + I systems as found in theories of command and control, is that such a methodology be capable of capturing both the quantitative and qualitative features that such phenomena exhibit in real-world occurrences. It is our contention that current state-of-the-art tools are deficient in this regard. Which is to say, that while real-world occurrences of complex phenomena clearly exhibit both quantitative and qualitative features, most models concentrate on one aspect, to the exclusion of the other.An approach to the above-mentioned deficiency is outlined in this paper. A program is set forth that treats models of complex phenomena as formal symbolic entities in a first-order language, which in turn is comprised of both logical and proper sets of axioms. Such entities, termed composite structures, permit the distinct, but interrelated, portrayal of the quantitative and qualitative features of a given model within a unified framework of discourse.     

On a decision rule using dichotomies for identifying the nonnegligible parameter in certain linear models

Consider the class of linear models (with uncorrelated observation, each having variance σ²), in which it is known that at most k (location) parameters are negligible, but it is not known which are negligible. The problem is to identify the nonnegligible parameters. In this paper, for k = 1, and under certain restrictions on the model, a technique is developed for solving this problem, which has the feature of requiring (in an information theoretic sense) the minimum amount of computation. (It can “search through” 2^m objects, using m “steps.”) The technique consists of dichotomizing the set of parameters (one known subset possibly containing the nonnegligible element, and the other not), using chi-square variables. A method for computing the probability that the correct parameter is identified, is presented, and an important application to factorial search designs is established.     

Rainflow analysis in Coastal Engineering using switching second order Markov models

The paper deals with the use of Markov and switching Markov chain models of turning points to reproduce random sets of sea states. The advantages of these models are emphasized and compared with existing models based on wave height records, indicating that long and short range and period cycles are included, while the wave height records ignore this important information from the point of view of damage accumulation. Existing models for first order Markov processes are extended to the case of second order processes and closed formulas are given to derive the rainflow matrices of these processes. Finally, one illustrative example of application is given.     

Local dependence functions for some families of bivariate distributions and total positivity

The purpose of this paper is to investigate a very useful application of a certain local dependence function γ_f(x,y), which was considered recently by Holland and Wang [20]. An interesting property of γ_f(x,y) is that the underlying joint density f(x,y) is TP₂ (that is, totally positive of order 2) if and only if . This gives an elegant way to investigate the TP₂ property of any bivariate distribution. For the Saramanov family, the Ali-Mikhail-Haq family of bivariate distributions and the family of bivariate elliptical distributions, we derive the local dependence function and obtain conditions for f(x,y) to be TP₂. These families are quite rich and include many other large classes of bivariate distributions as their special cases. Similar conditions are obtained for bivariate distributions with exponential conditionals and bivariate distributions with Pareto conditionals.