Information-Theoretic Method for Assessing the Quality of Translations

In recent years, the task of translating from one language to another has attracted wide attention from researchers due to numerous practical uses, ranging from the translation of various texts and speeches, including the so-called “machine” translation, to the dubbing of films and numerous other video materials. To study this problem, we propose to use the information-theoretic method for assessing the quality of translations. We based our approach on the classification of sources of text variability proposed by A.N. Kolmogorov: information content, form, and unconscious author’s style. It is clear that the unconscious “author’s” style is influenced by the translator. So researchers need special methods to determine how accurately the author’s style is conveyed, because it, in a sense, determines the quality of the translation. In this paper, we propose a method that allows us to estimate the quality of translation from different translators. The method is used to study translations of classical English-language works into Russian and, conversely, Russian classics into English. We successfully used this method to determine the attribution of literary texts.     

Hierarchical cluster-tendency analysis of the group structure in the foreign exchange market

A hierarchical cluster-tendency (HCT) method in analyzing the group structure of networks of the global foreign exchange (FX) market is proposed by combining the advantages of both the minimal spanning tree (MST) and the hierarchical tree (HT). Fifty currencies of the top 50 World GDP in 2010 according to World Bank’s database are chosen as the underlying system. By using the HCT method, all nodes in the FX market network can be “colored” and distinguished. We reveal that the FX networks can be divided into two groups, i.e., the Asia-Pacific group and the Pan-European group. The results given by the hierarchical cluster-tendency method agree well with the formerly observed geographical aggregation behavior in the FX market. Moreover, an oil-resource aggregation phenomenon is discovered by using our method. We find that gold could be a better numeraire for the weekly-frequency FX data.     

GDP vs. LDP: A Survey from the Perspective of Information-Theoretic Channel

The existing work has conducted in-depth research and analysis on global differential privacy (GDP) and local differential privacy (LDP) based on information theory. However, the data privacy preserving community does not systematically review and analyze GDP and LDP based on the information-theoretic channel model. To this end, we systematically reviewed GDP and LDP from the perspective of the information-theoretic channel in this survey. First, we presented the privacy threat model under information-theoretic channel. Second, we described and compared the information-theoretic channel models of GDP and LDP. Third, we summarized and analyzed definitions, privacy-utility metrics, properties, and mechanisms of GDP and LDP under their channel models. Finally, we discussed the open problems of GDP and LDP based on different types of information-theoretic channel models according to the above systematic review. Our main contribution provides a systematic survey of channel models, definitions, privacy-utility metrics, properties, and mechanisms for GDP and LDP from the perspective of information-theoretic channel and surveys the differential privacy synthetic data generation application using generative adversarial network and federated learning, respectively. Our work is helpful for systematically understanding the privacy threat model, definitions, privacy-utility metrics, properties, and mechanisms of GDP and LDP from the perspective of information-theoretic channel and promotes in-depth research and analysis of GDP and LDP based on different types of information-theoretic channel models.     

An Information-Theoretic Analysis of the Cost of Decentralization for Learning and Inference under Privacy Constraints

In vertical federated learning (FL), the features of a data sample are distributed across multiple agents. As such, inter-agent collaboration can be beneficial not only during the learning phase, as is the case for standard horizontal FL, but also during the inference phase. A fundamental theoretical question in this setting is how to quantify the cost, or performance loss, of decentralization for learning and/or inference. In this paper, we study general supervised learning problems with any number of agents, and provide a novel information-theoretic quantification of the cost of decentralization in the presence of privacy constraints on inter-agent communication within a Bayesian framework. The cost of decentralization for learning and/or inference is shown to be quantified in terms of conditional mutual information terms involving features and label variables.     

BHGAttN: A Feature-Enhanced Hierarchical Graph Attention Network for Sentiment Analysis

Recently, with the rise of deep learning, text classification techniques have developed rapidly. However, the existing work usually takes the entire text as the modeling object and pays less attention to the hierarchical structure within the text, ignoring the internal connection between the upper and lower sentences. To address these issues, this paper proposes a Bert-based hierarchical graph attention network model (BHGAttN) based on a large-scale pretrained model and graph attention network to model the hierarchical relationship of texts. During modeling, the semantic features are enhanced by the output of the intermediate layer of BERT, and the multilevel hierarchical graph network corresponding to each layer of BERT is constructed by using the dependencies between the whole sentence and the subsentence. This model pays attention to the layer-by-layer semantic information and the hierarchical relationship within the text. The experimental results show that the BHGAttN model exhibits significant competitive advantages compared with the current state-of-the-art baseline models.     

Asymptotic Information-Theoretic Detection of Dynamical Organization in Complex Systems

The identification of emergent structures in complex dynamical systems is a formidable challenge. We propose a computationally efficient methodology to address such a challenge, based on modeling the state of the system as a set of random variables. Specifically, we present a sieving algorithm to navigate the huge space of all subsets of variables and compare them in terms of a simple index that can be computed without resorting to simulations. We obtain such a simple index by studying the asymptotic distribution of an information-theoretic measure of coordination among variables, when there is no coordination at all, which allows us to fairly compare subsets of variables having different cardinalities. We show that increasing the number of observations allows the identification of larger and larger subsets. As an example of relevant application, we make use of a paradigmatic case regarding the identification of groups in autocatalytic sets of reactions, a chemical situation related to the origin of life problem.     

A Generalized Information-Theoretic Approach for Bounding the Number of Independent Sets in Bipartite Graphs

This paper studies the problem of upper bounding the number of independent sets in a graph, expressed in terms of its degree distribution. For bipartite regular graphs, Kahn (2001) established a tight upper bound using an information-theoretic approach, and he also conjectured an upper bound for general graphs. His conjectured bound was recently proved by Sah et al. (2019), using different techniques not involving information theory. The main contribution of this work is the extension of Kahn’s information-theoretic proof technique to handle irregular bipartite graphs. In particular, when the bipartite graph is regular on one side, but may be irregular on the other, the extended entropy-based proof technique yields the same bound as was conjectured by Kahn (2001) and proved by Sah et al. (2019).     

On the Existence of XOR-Based Codes for Private Information Retrieval with Private Side Information

We consider the problem of Private Information Retrieval with Private Side Information (PIR-PSI), wherein the privacy of the demand and the side information are jointly preserved. Although the capacity of the PIR-PSI setting is known, we observe that the underlying capacity-achieving code construction uses Maximum Distance Separable (MDS) codes therefore contributing to high computational complexity when retrieving the demand. Pointing at this drawback of MDS-based PIR-PSI codes, we propose XOR-based PIR-PSI codes for a simple yet non-trivial setting of two non-colluding databases and two side information files at the user. Although our codes offer substantial reduction in complexity when compared to MDS-based codes, the code-rate marginally falls short of the capacity of the PIR-PSI setting. Nevertheless, we show that our code-rate is strictly higher than that of XOR-based codes for PIR with no side information. As a result, our codes can be useful when privately downloading a file especially after having downloaded a few other messages privately from the same database at an earlier time-instant.     

A New Statistical Aspect of the Cluster Variation Method for Lattice Systems

This paper presents an alternative statistical way to derive the cluster variation method (CVM) for lattice systems. The formulation is developed for a series of different clusters, each of which is the largest overlap cluster between two clusters of the next larger type. We arrive at the CVM expression of the lattice configuration factor by deriving the number of different ways of distributing clusters of a selected type in the lattice so that they overlap each other at the largest overlap clusters in a physically correct manner. The essential assumption employed is that individual overlapping events are statistically independent of each other. This reveals a new statistical aspect of the CVM: The CVM is based on a Bethe tree of clusters of the selected type.     

基于高阶叠层矢量基函数的E-H时域有限元方法分析谐振腔和波导结构

将高阶叠层矢量基函数用于E-H时域有限元方法,电场和磁场用相同的基函数展开并同时求解,时间离散采用Crank-Nicolson差分格式使得时间步长的选取摆脱稳定性条件的限制,同时采用完美匹配层来截断计算区域.对三维谐振腔及波导结构进行数值模拟与分析,结果表明,相较于低阶基函数,高阶叠层矢量基函数可以有效提高E-H时域有限元方法的计算精度.     

Unsupervised Hierarchical Classification Approach for Imprecise Data in the Breast Cancer Detection

(1) Background: in recent years, a lot of the research of statistical methods focused on the classification problem in presence of imprecise data. A particular case of imprecise data is the interval-valued data. Following this research line, in this work a new hierarchical classification technique for multivariate interval-valued data is suggested for diagnosis of the breast cancer; (2) Methods: an unsupervised hierarchical classification method for imprecise multivariate data (called HC-ID) is performed for diagnosis of breast cancer (i.e., to discriminate between benign or malignant masses) and the results have been compared with the conventional (unsupervised) hierarchical classification approach (HC); (3) Results: the application on real data shows that the HC-ID procedure performs better HC procedure in terms of accuracy (HC-ID = 0.80, HC = 0.66) and sensitivity (HC-ID = 0.61, HC = 0.08). In the results obtained by the usual procedure, there is a high degree of false-negative (i.e., benign cancer diagnosis in malignant status) affected by the high degree of variability (i.e., uncertainty) characterizing the worst data.     

Boltzmann Configurational Entropy Revisited in the Framework of Generalized Statistical Mechanics

As known, a method to introduce non-conventional statistics may be realized by modifying the number of possible combinations to put particles in a collection of single-particle states. In this paper, we assume that the weight factor of the possible configurations of a system of interacting particles can be obtained by generalizing opportunely the combinatorics, according to a certain analytical function f{π}(n) of the actual number of particles present in every energy level. Following this approach, the configurational Boltzmann entropy is revisited in a very general manner starting from a continuous deformation of the multinomial coefficients depending on a set of deformation parameters {π}. It is shown that, when f{π}(n) is related to the solutions of a simple linear difference–differential equation, the emerging entropy is a scaled version, in the occupational number representation, of the entropy of degree (κ,r) known, in the framework of the information theory, as Sharma–Taneja–Mittal entropic form.     

A Study on the New Agegraphic Dark Energy Model in the Framework of Generalized Uncertainty Principle for Different Scale Factors

In this paper, we consider the New Agegraphic Dark Energy (NADE) model interacting with pressureless Dark Matter (DM) in the framework of generalized uncertainty principle. We consider different expressions of the scale factor a(t) pertaining to the emergent, the intermediate and the logamediate scenarios of the universe. We have derived the expressions for various cosmological parameters in all the three cases and plotted the equation of state (EoS) parameter ω _D and squared speed of the sound $v_{s}^{2}$ to check the stability of the model in each case. We have observed that for emergent and intermediate cases, the EoS parameter has a quintom-like behavior and in the logamediate case it has quintessence-like behavior. The negative squared speed of sound in all of the three cases has indicated that the model is classically unstable for each choice of scale factor.     

Enhancement of the Quantum Coherence in a Hierarchical Environment System by Weak Measurement and Weak Measurement Reversal

The quantum coherence of a two-qubit system in which each qubit is coupled to its own hierarchical environment is studied. The effect of the hierarchical environment is explored in order to improve the coherence effectively. It is discovered that the dynamics of coherence can be manipulated by the number N of the second-layer cavities ($m_n$), the coupling strengths Ω₀, Ω, κ, respectively for the qubit-cavity (m₀), nearest-neighbor, and m₀-$m_n$ cavities. It depends also on the decay rate Γ of second-layer cavities. Furthermore, an effective scheme for enhancing coherence is proposed by using weak measurement and weak measurement reversal. The explicit conditions of the optimal measurement strengths to improve the coherence comparatively are derived. It is seen that the final stable value of the coherence is independent of the number N of second-layer cavities but is related to the weak measurement strength m when the coupling strength between two nearest-neighbor cavities is taken into account.     

Estimation of the Covariance Matrix in Hierarchical Bayesian Spatio-Temporal Modeling via Dimension Expansion

Ozone concentrations are key indicators of air quality. Modeling ozone concentrations is challenging because they change both spatially and temporally with complicated structures. Missing data bring even more difficulties. One of our interests in this paper is to model ozone concentrations in a region in the presence of missing data. We propose a method without any assumptions on the correlation structure to estimate the covariance matrix through a dimension expansion method for modeling the semivariograms in nonstationary fields based on the estimations from the hierarchical Bayesian spatio-temporal modeling technique (Le and Zidek). Further, we apply an entropy criterion (Jin et al.) based on a predictive model to decide if new stations need to be added. This entropy criterion helps to solve the environmental network design problem. For demonstration, we apply the method to the ozone concentrations at 25 stations in the Pittsburgh region studied. The comparison of the proposed method and the one is provided through leave-one-out cross-validation, which shows that the proposed method is more general and applicable.     

A Testable Theory for the Emergence of the Classical World

The transition from the quantum to the classical world is not yet understood. Here, we take a new approach. Central to this is the understanding that measurement and actualization cannot occur except on some specific basis. However, we have no established theory for the emergence of a specific basis. Our framework entails the following: (i) Sets of N entangled quantum variables can mutually actualize one another. (ii) Such actualization must occur in only one of the 2^N possible bases. (iii) Mutual actualization progressively breaks symmetry among the 2^N bases. (iv) An emerging “amplitude” for any basis can be amplified by further measurements in that basis, and it can decay between measurements. (v) The emergence of any basis is driven by mutual measurements among the N variables and decoherence with the environment. Quantum Zeno interactions among the N variables mediates the mutual measurements. (vi) As the number of variables, N, increases, the number of Quantum Zeno mediated measurements among the N variables increases. We note that decoherence alone does not yield a specific basis. (vii) Quantum ordered, quantum critical, and quantum chaotic peptides that decohere at nanosecond versus femtosecond time scales can be used as test objects. (viii) By varying the number of amino acids, N, and the use of quantum ordered, critical, or chaotic peptides, the ratio of decoherence to Quantum Zeno effects can be tuned. This enables new means to probe the emergence of one among a set of initially entangled bases via weak measurements after preparing the system in a mixed basis condition. (ix) Use of the three stable isotopes of carbon, oxygen, and nitrogen and the five stable isotopes of sulfur allows any ten atoms in the test protein to be discriminably labeled and the basis of emergence for those labeled atoms can be detected by weak measurements. We present an initial mathematical framework for this theory, and we propose experiments.     

A Reduced System of Linear Algebraic Equations for Giving Soliton Solutions of the Generalized ZS/AKNS Systems

A simple method for finding soliton solutions of the generaked ZS/AKNS systems whose Lax pairs are matrices with high orders is considered. An explicit expreesion of transformation between the Jost solution relating to the (n-1)-soliton solution and that relating to the n-soliton solution is found. A reduced system of N algebraic equations for giving N soliton solutions is deduced, it has an identical form no matter how high the order of matrices of the Lax pain is.     

On the Value of Information in Status Update Systems

The age of information (AoI) is now well established as a metric that measures the freshness of information delivered to a receiver from a source that generates status updates. This paper is motivated by the inherent value of packets arising in many cyber-physical applications (e.g., due to precision of the information content or an alarm message). In contrast to AoI, which considers all packets are of equal importance or value, we consider status update systems with update packets carrying values as well as their generated time stamps. A status update packet has a random initial value at the source and a deterministic deadline after which its value vanishes (called ultimate staleness). In our model, the value of a packet either remains constant until the deadline or decreases in time (even after reception) starting from its generation to the deadline when it vanishes. We consider two metrics for the value of information (VoI) at the receiver: sum VoI is the sum of the current values of all packets held by the receiver, whereas packet VoI is the value of a packet at the instant it is delivered to the receiver. We investigate various queuing disciplines under potential dependence between value and service time and provide closed form expressions for both average sum VoI and packet VoI at the receiver. Numerical results illustrate the average VoI for different scenarios and relations between average sum VoI and average packet VoI.     

Improved Treatment of the Independent Variables for the Deployment of Model Selection Criteria in the Analysis of Complex Systems

Model selection criteria are widely used to identify the model that best represents the data among a set of potential candidates. Amidst the different model selection criteria, the Bayesian information criterion (BIC) and the Akaike information criterion (AIC) are the most popular and better understood. In the derivation of these indicators, it was assumed that the model’s dependent variables have already been properly identified and that the entries are not affected by significant uncertainties. These are issues that can become quite serious when investigating complex systems, especially when variables are highly correlated and the measurement uncertainties associated with them are not negligible. More sophisticated versions of this criteria, capable of better detecting spurious relations between variables when non-negligible noise is present, are proposed in this paper. Their derivation is obtained starting from a Bayesian statistics framework and adding an a priori Chi-squared probability distribution function of the model, dependent on a specifically defined information theoretic quantity that takes into account the redundancy between the dependent variables. The performances of the proposed versions of these criteria are assessed through a series of systematic simulations, using synthetic data for various classes of functions and noise levels. The results show that the upgraded formulation of the criteria clearly outperforms the traditional ones in most of the cases reported.     

Application of the Generalized Differential Quadrature Method in Solving Burgers'Equations

The aim of this paper is to obtain numerical solutions of the one-dimensional, two-dimensional and coupled Burgers' equations through the generalized differential quadrature method (GDQM). The polynomial-based differential quadrature (PDQ) method is employed and the obtained system of ordinary differential equations is solved via the total variation diminishing Runge-Kutta (TVD-RK) method. The numerical solutions are satisfactorily coincident with
the exact solutions. The method can compete against the methods applied in the literature.