信息熵公理及其在决策分析中的应用

针对信息量是消息发生前的不确定性给出一个直观测量信息量公式.为了克服Shannon熵的局限性和分析信息度量本质,借鉴距离空间理论中度量公理定义的思路,通过非负性、对称性、次可加和极大性给出信息熵的公理化新定义.将Shannon熵、直观信息熵和β-熵等不同形式的信息度量统一在同一公理化结构下.应用直观信息熵公式仅采用四则运算进行决策树分析,避免了利用Shannon熵公式的对数运算.     

二维曲线型随机变量的信息度量

主要在二维曲线型随机变量的基础上,引入了信息论中常讨论的信息量和信息熵,尤其是条件熵的建立.     

物理熵、信息熵及其演化方程

在非平衡统计物理熵演化方程和熵产生率简明公式的启示下,发展了动态信息熵理论,建立了信息熵密度在时间和态变量空间变化的非线性演化方程.它的数学形式和物理意义与物理熵演化方程类似：信息熵密度的变化同样是由漂移、扩散和产生三者共同引起的.信息熵产生率简明公式与物理熵的亦类似.进而研究了物理熵和信息熵的同异及这两种统计熵的可能统一,讨论了熵增加原理、平衡态最大熵原理和最大信息熵原理三者相互间及其与熵演化方程之间的关联.     

基于信息熵的空战目标威胁评估

空战目标威胁评估是协同多目标攻击空战决策的关键技术之一.针对传统空战目标威胁评估方法在确定权重系数方面的缺陷和不足,提出了运用信息熵方法来解决空战目标威胁评估问题.给出了信息熵方法确定威胁指标权重的具体步骤,建立了基于信息熵的空战目标威胁评估模型,并进行了仿真研究,仿真结果表明该方法是合理和有效的.     

基于信息熵的属性约简在多属性决策中的应用

属性约简是在信息系统中的一个重要操作.分类是属性约简的基础,且直接在大数据集上进行属性约简往往存在效率低下的问题.以分类为基础提出了一种基于信息熵的信息系统属性约简算法.算法通过信息熵的计算,在属性约简的同时对原信息系统逐层分解,从而实现了属性的约简并缩小了搜索空间.提出了依据信息熵来确定属性的不必要性及简约属性集,应用在多属性决策中所带来的优势.     

离散信道容量的迭代算法

引入信息量偏差概念，给出平均交互信息量关于输入概率的增量公式，设计出离散信道容量线性乘法迭代和线性常系数迭代算法，它们都优于现有的指数迭代算法．并证明在所有单步迭代算法中它们几乎是最好的算法.     

信息熵方程求解算法及其应用

给定一个离散且有限随机变量的信息熵,求其对应的概率分布需要解多元非线性方程,文中提出了一个将n元信息熵方程化为至多(n-1)个一元非线性方程求解的算法,证明了算法的正确性,给出了算法误差估计;运用熵方程求解算法设计了一种基于信息熵的文本数字水印方案.     

信息熵在粗糙集信息检索模型中的应用

在信息检索过程中,由于文档中存在大量的多义和近义现象,导致不确定性出现,这将影响检索的性能.为此我们采用信息熵和粗糙集理论来处理这类不确定性问题.首先计算训练文档集中的词之间的信息熵,对信息熵做模糊聚类来构造词之间的等价关系,然后借助于该等价关系提出并实现了一个以粗糙集上下近似为基础的信息检索模型,通过实验的测试,该模型能够提高信息检索的效率.     

基于小波分形理论的股价指数信息量测度研究

本把小波分析和分形理论引入到股价指数时间序列的分析中，给出了股价指数波动复杂性的信息量测度方法——信息熵和分形维方法。通过对上证综指和深证成指1994年1月3日至2002年3月4日期间的数据进行的实证分析显示，两种方法均能刻画股价指数波动的复杂程度，这对初步了解我国股市场的波动规律有一定的实际意义。     

同一水平下模糊规则信息熵的描述

给出了模糊知识系统及模糊决策逻辑公式的定义,在此基础上描述了模糊决策逻辑公式及模糊知识系统下模糊规则的信息熵,讨论了模糊规则信息熵的相关性质;其次,利用模糊规则信息熵对模糊规则进行了分类、评价,从而为建立合理的模糊系统提供了一种有效的判定方法;最后,通过实例验证了所提出理论的正确性.     

Multiscale measures of equilibrium on finite dynamic systems

This article presents a new method for the study of the evolution of dynamic systems based on the notion of quantity of information. The system is divided into elementary cells and the quantity of information is studied with respect to the cell size. We have introduced an analogy between quantity of information and entropy, and defined the intrinsic entropy as the entropy of the whole system independent of the size of the cells. It is shown that the intrinsic entropy follows a Gaussian probability density function (PDF) and thereafter, the time needed by the system to reach equilibrium is a random variable. For a finite system, statistical analyses show that this entropy converges to a state of equilibrium and an algorithmic method is proposed to quantify the time needed to reach equilibrium for a given confidence interval level. A Monte-Carlo simulation of diffusion of A* atoms in A is then provided to illustrate the proposed simulation. It follows that the time to reach equilibrium for a constant error probability, t_e, depends on the number, n, of elementary cells as: t_e∝n^2.22_±0.06. For an infinite system size (n infinite), the intrinsic entropy obtained by statistical modelling is a pertinent characteristic number of the system at the equilibrium.     

Fuzzy Entropy： Axiomatic Definition and Neural Networks Model

The measure of uncertainty is adopted as a measure of information. The measures of fuzziness are known as fuzzy information measures. The measure of a quantity of fuzzy information gained from a fuzzy set or fuzzy system is known as fuzzy entropy. Fuzzy entropy has been focused and studied by many researchers in various fields. In this paper, firstly, the axiomatic definition of fuzzy entropy is discussed. Then, neural networks model of fuzzy entropy is proposed, based on the computing capability of neural networks. In the end, two examples are discussed to show the efficiency of the model.     

Information and dynamical systems: a concrete measurement on sporadic dynamics

We present a method for the study of dynamical systems based on the notion of quantity of information. Measuring the quantity of information of a string by using data compression algorithms, it is possible to give a notion of orbit complexity of dynamical systems. In compact ergodic dynamical systems, entropy is almost everywhere equal to orbit complexity. We have introduced a new compression algorithm called CASToRe which allows a direct estimation of the information content of the orbits in the 0-entropy case. The method is applied to a sporadic dynamical system (Manneville map).     

不完全信息下损失风险的研究

在获得损失分布不完全信息情况下,提出用方差和熵共同度量损失风险的方法.在不完全信息条件下,通过最大熵原理在最不确定的情况下得到最大熵损失分布,并获得了损失分布的熵函数值.用熵值度量损失分布对于均匀分布的离散程度,从而度量概率波动带来的风险;用方差度量损失对于均值的离散程度,从而度量状态波动带来的风险.由于熵是与损失变量更高阶矩信息相联系的,所以新方法是从更全面的角度对损失风险的预测.通过算例,进一步看出在获得高阶矩信息下,熵参与风险度量的必要性.     

Relative entropy between quantum ensembles

Relative entropy between two quantum states, which quantifies to what extent the quantum states can be distinguished via whatever methods allowed by quantum mechanics, is a central and fundamental quantity in quantum information theory. However, in both theoretical analysis (such as selective measurements) and practical situations (such as random experiments), one is often encountered with quantum ensembles, which are families of quantum states with certain prior probability distributions. How can we quantify the quantumness and distinguishability of quantum ensembles? In this paper, by use of a probabilistic coupling technique, we propose a notion of relative entropy between quantum ensembles, which is a natural generalization of the relative entropy between quantum states. This generalization enjoys most of the basic and important properties of the original relative entropy. As an application, we use the notion of relative entropy between quantum ensembles to define a measure for quantumness of quantum ensembles. This quantity may be useful in quantum cryptography since in certain circumstances it is desirable to encode messages in quantum ensembles which are the most quantum, thus the most sensitive to eavesdropping. By use of this measure of quantumness, we demonstrate that a set consisting of two pure states is the most quantum when the states are 45° apart.     

水资源配置方案综合评价中的信息损失测度研究

水资源配置的多属性特征使得在方案综合评价中依据不同的聚合方法,能够获得不完全一致的方案排序结果。本文从水资源配置方案评价的决策矩阵及方案排序结果中的信息量大小入手,利用信息熵理论和斯皮尔曼等级相关系数提出衡量水资源配置方案综合评价中信息损失的香农斯皮尔曼测度。并构造由2种权重(熵权法权重、证据理论权重)与3种评价模型(简单加权法、模糊优选法、TOPSIS)组合而成的6种综合评价模型。最后,将香农斯皮尔曼测度运用到天津市水资源配置方案综合评价过程中的信息损失的衡量,并与文献中的综合评价模型所得结果进行比较。结果表明,采用证据理论权重-模糊优选评价模型所得评价结果的绝对信息损失和相对信息损失最小。通过水资源配置方案综合评价中的信息损失测度研究,能够使得水资源配置方案综合评价过程更加透明,并为决策部门选取理想的水资源配置方案综合评价方法提供决策支持。     

A maximum entropy approach to the newsvendor problem with partial information

In this paper, we consider the newsvendor model under partial information, i.e., where the demand distribution D is partly unknown. We focus on the classical case where the retailer only knows the expectation and variance of D. The standard approach is then to determine the order quantity using conservative rules such as minimax regret or Scarf’s rule. We compute instead the most likely demand distribution in the sense of maximum entropy. We then compare the performance of the maximum entropy approach with minimax regret and Scarf’s rule on large samples of randomly drawn demand distributions. We show that the average performance of the maximum entropy approach is considerably better than either alternative, and more surprisingly, that it is in most cases a better hedge against bad results.     

信息熵增量最小化准则在供应链中的应用

本文研究了在考虑利润最大化准则和信息熵增量最小准则下，既能满足了利润最大化，又要增加利润可得性，销售商如何确定订购量的问题。数字实验表明：考虑双重准则得到的策略比仅考虑利润最大化准则得到的策略更好，可以使供应链、销售商及制造商都受益。     

Invariance Entropy

In this paper, the concept of invariance entropy for continuous-time control systems is introduced. This quantity measures the minimal data rate necessary to render a compact subset of the state space invariant by a coder-controller device. Invariance entropy is an intrinsic quantity of the open-loop system and shares several properties with the classical entropy quantities in the field of dynamical systems. Some of these properties, in particular relations to Lyapunov exponents, are pointed out. (© 2011 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim)