Single Sample Path-Based Optimization of Markov Chains

Motivated by the needs of on-line optimization of real-world engineering systems, we studied single sample path-based algorithms for Markov decision problems (MDP). The sample path used in the algorithms can be obtained by observing the operation of a real system. We give a simple example to explain the advantages of the sample path-based approach over the traditional computation-based approach: matrix inversion is not required; some transition probabilities do not have to be known; it may save storage space; and it gives the flexibility of iterating the actions for a subset of the state space in each iteration. The effect of the estimation errors and the convergence property of the sample path-based approach are studied. Finally, we propose a fast algorithm, which updates the policy whenever the system reaches a particular set of states and prove that the algorithm converges to the true optimal policy with probability one under some conditions. The sample path-based approach may have important applications to the design and management of engineering systems, such as high speed communication networks.This work was supported in part by     

A multi-parametric programming approach for multilevel hierarchical and decentralised optimisation problems

In this paper, we outline the foundations of a general global optimisation strategy for the solution of multilevel hierarchical and general decentralised multilevel problems, based on our recent developments on multi-parametric programming and control theory. The core idea is to recast each optimisation subproblem, present in the hierarchy, as a multi-parametric programming problem, with parameters being the optimisation variables belonging to the remaining subproblems. This then transforms the multilevel problem into single-level linear/convex optimisation problems. For decentralised systems, where more than one optimisation problem is present at each level of the hierarchy, Nash equilibrium is considered. A three person dynamic optimisation problem is presented to illustrate the mathematical developments.     

A Bayesian approach to seafloor classification using multi-beam echo-sounder backscatter data

Seafloor classification using acoustic remote sensing techniques is an attractive approach due to its high-coverage capabilities and limited costs. The multi-beam echo-sounder (MBES) system provides high-resolution bathymetry and backscatter information with 100% coverage. In this paper, we present a seafloor classification method that employs the MBES backscatter data. The method uses the averaged backscatter data per beam. It, therefore, is independent on the quality of the MBES calibration. Also, its performance is insensitive to seafloor type variation along the MBES swathe and corrections for the angular dependence of the backscatter are not needed. The method accounts for the ping-to-ping variability of the backscatter intensity. It estimates both the number of seafloor types present in the survey area and the probability density function for the backscatter strength at a certain angle for each of the seafloor types. Application of the method to MBES backscatter data acquired in a well-known test area in the North Sea shows very good agreement with available ground truth. The method’s discriminatory performance for this area is demonstrated to be comparable to that of taking samples of the sediment. All seafloor types known to be present in the area are resolved for. Application of the method to the Stanton bank data set shows clearly separable areas that differ in seafloor composition.     

A practical weight sensitivity algorithm for goal and multiple objective programming

This paper presents a weight sensitivity algorithm that can be used to investigate a portion of weight space of interest to the decision maker in a goal or multiple objective programme. The preferential information required from the decision maker is an initial estimate of their starting solution, with an equal weights solution being used as a default if this is not available, and preference information that will define the portion of weight space on which the sensitivity analysis is to be conducted. The different types of preferential information and how they are incorporated by the algorithm are discussed. The output of the algorithm is a set of distinct solutions that characterise the portion of weight space searched. The possible different output requirements of decision makers are detailed in the context of the algorithm.The methodology is demonstrated on two examples, one hypothetical and the other relating to predicting cinema-going behaviour. Conclusions and avenues for future research are given.     

决策树分类模型在工程项目评标风险预警中的应用

本文将数据挖掘中的决策树分类方法运用到工程项目评标数据分析,从200多个天津市工程项目招投标打分数据中,随机抽取15个招投标项目中的67个承包商的评标专家打分数据进行分析,得到中标承包商技术和商务评分分界点,进而得到工程项目潜在风险的预警阈值,然后借助因子分析辨识出风险来源并进行预警。     

连续时间马尔可夫决策过程的折扣模型

本文考虑的是转移速率族任意且费用率函数可能无界的连续时间马尔可夫决策过程的折扣模型．放弃了传统的要求相应于每个策略的　Ｑ　－过程唯一等条件,而首次考虑相应每个策略的　Ｑ　－过程不一定唯一,　转移速率族也不一定保守,　费用率函数可能无界,　且允许行动空间非空任意的情形．　本文首次用"α－折扣费用最优不等式"更新了传统的α－折扣费用最优方程,并用"最优不等式"和新的方法,不仅证明了传统的主要结果即最优平稳策略的存在性,　而且还进一步探讨了（　∈＞０　　）－最优平稳策略,具有单调性质的最优平稳策略,　以及（∈≥０）　－最优决策过程的存在性,　得到了一些有意义的新结果．　最后,　提供了一个迁移率受控的生灭系统例子,　它满足本文的所有条件,　而传统的假设（见文献［１－１４］）均不成立．     

Fuzzy Markov Chains and Decision-Making

Decision-making in an environment of uncertainty and imprecision for real-world problems is a complex task. In this paper it is introduced general finite state fuzzy Markov chains that have a finite convergence to a stationary (may be periodic) solution. The Cesaro average and the -potential for fuzzy Markov chains are defined, then it is shown that the relationship between them corresponds to the Blackwell formula in the classical theory of Markov decision processes. Furthermore, it is pointed out that recurrency does not necessarily imply ergodicity. However, if a fuzzy Markov chain is ergodic, then the rows of its ergodic projection equal the greatest eigen fuzzy set of the transition matrix. Then, the fuzzy Markov chain is shown to be a robust system with respect to small perturbations of the transition matrix, which is not the case for the classical probabilistic Markov chains. Fuzzy Markov decision processes are finally introduced and discussed.     

A Graded Quadrivalent Logic for Ordinal Preference Modelling: Loyola–Like Approach

We extend a quadrivalent logic of Belnap to graded truth values in order to handle graded relevance of positive and negative arguments provided in preferential information concerning ranking of a finite set of alternatives. This logic is used to design the preference modelling and exploitation phases of decision aiding with respect to the ranking problem. The graded arguments are presented on an ordinal scale and their aggregation leads to preference model in form of four graded outranking relations (true, false, unknown and contradictory). The exploitation procedure combines the min-scoring procedure with the leximin rule. Aggregation of positive and negative arguments as well as exploitation of the resulting outranking relations is concordant with an advice given by St. Ignatius of Loyola (1548) how to make a good choice.     

平面多项式系统及其相伴系统

引进平面多项式系统的相伴系统的概念 ,分析了多项式系统与其相伴系统之间在极限环不存在性 ,唯一性 ,中心焦点的判定等问题上的相似之处和不同之处 ,提出了几个公开问题 .     

Analysis of multiobjective decision problems by the indifference band approach

A new approach, based on indifference band, for analyzing problems with multiple objectives is described. The relations of this approach to some previous results are given. Methods for generating nondominated solutions are supplied.