Practical considerations of permutation entropy

More than ten years ago Bandt and Pompe introduced a new measure to quantify complexity in measured time series. During these ten years, this measure has been modified and extended. In this review we will give a brief introduction to permutation entropy, explore the different fields of utilization where permutation entropy has been applied and provide a guide on how to choose appropriate parameters for different applications of permutation entropy.     

Entropy Sources Based on Silicon Chips: True Random Number Generator and Physical Unclonable Function

Entropy is a measure of uncertainty or randomness. It is the foundation for almost all cryptographic systems. True random number generators (TRNGs) and physical unclonable functions (PUFs) are the silicon primitives to respectively harvest dynamic and static entropy to generate random bit streams. In this survey paper, we present a systematic and comprehensive review of different state-of-the-art methods to harvest entropy from silicon-based devices, including the implementations, applications, and the security of the designs. Furthermore, we conclude the trends of the entropy source design to point out the current spots of entropy harvesting.     

Some Parameterized Quantum Midpoint and Quantum Trapezoid Type Inequalities for Convex Functions with Applications

Quantum information theory, an interdisciplinary field that includes computer science, information theory, philosophy, cryptography, and entropy, has various applications for quantum calculus. Inequalities and entropy functions have a strong association with convex functions. In this study, we prove quantum midpoint type inequalities, quantum trapezoidal type inequalities, and the quantum Simpson’s type inequality for differentiable convex functions using a new parameterized q-integral equality. The newly formed inequalities are also proven to be generalizations of previously existing inequities. Finally, using the newly established inequalities, we present some applications for quadrature formulas.     

Entropy Production in Exactly Solvable Systems

The rate of entropy production by a stochastic process quantifies how far it is from thermodynamic equilibrium. Equivalently, entropy production captures the degree to which global detailed balance and time-reversal symmetry are broken. Despite abundant references to entropy production in the literature and its many applications in the study of non-equilibrium stochastic particle systems, a comprehensive list of typical examples illustrating the fundamentals of entropy production is lacking. Here, we present a brief, self-contained review of entropy production and calculate it from first principles in a catalogue of exactly solvable setups, encompassing both discrete- and continuous-state Markov processes, as well as single- and multiple-particle systems. The examples covered in this work provide a stepping stone for further studies on entropy production of more complex systems, such as many-particle active matter, as well as a benchmark for the development of alternative mathematical formalisms.     

Numerical and experimental investigation of the effect of filtering on chaotic symbolic dynamics

Motivated by the practical consideration of the measurement of chaotic signals in experiments or the transmission of these signals through a physical medium, we investigate the effect of filtering on chaotic symbolic dynamics. We focus on the linear, time-invariant filters that are used frequently in many applications, and on the two quantities characterizing chaotic symbolic dynamics: topological entropy and bit-error rate. Theoretical consideration suggests that the topological entropy is invariant under filtering. Since computation of this entropy requires that the generating partition for defining the symbolic dynamics be known, in practical situations the computed entropy may change as a filtering parameter is changed. We find, through numerical computations and experiments with a chaotic electronic circuit, that with reasonable care the computed or measured entropy values can be preserved for a wide range of the filtering parameter.     

A new complexity measure for time series analysis and classification

Complexity measures are used in a number of applications including extraction of information from data such as ecological time series, detection of non-random structure in biomedical signals, testing of random number generators, language recognition and authorship attribution etc. Different complexity measures proposed in the literature like Shannon entropy, Relative entropy, Lempel-Ziv, Kolmogrov and Algorithmic complexity are mostly ineffective in analyzing short sequences that are further corrupted with noise. To address this problem, we propose a new complexity measure ETC and define it as the “Effort To Compress” the input sequence by a lossless compression algorithm. Here, we employ the lossless compression algorithm known as Non-Sequential Recursive Pair Substitution (NSRPS) and define ETC as the number of iterations needed for NSRPS to transform the input sequence to a constant sequence. We demonstrate the utility of ETC in two applications. ETC is shown to have better correlation with Lyapunov exponent than Shannon entropy even with relatively short and noisy time series. The measure also has a greater rate of success in automatic identification and classification of short noisy sequences, compared to entropy and a popular measure based on Lempel-Ziv compression (implemented by Gzip).     

Trace Functions with Applications in Quantum Physics

We consider both known and not previously studied trace functions with applications in quantum physics. By using perspectives we obtain convexity statements for different notions of residual entropy, including the entropy gain of a quantum channel studied by Holevo and others. We give new proofs of Carlen-Lieb’s concavity/convexity theorems for certain trace functions, by making use of the theory of operator monotone functions. We then apply these methods in a study of new classes of trace functions.     

How hidden are hidden processes? A primer on crypticity and entropy convergence

We investigate a stationary process's crypticity--a measure of the difference between its hidden state information and its observed information--using the causal states of computational mechanics. Here, we motivate crypticity and cryptic order as physically meaningful quantities that monitor how hidden a hidden process is. This is done by recasting previous results on the convergence of block entropy and block-state entropy in a geometric setting, one that is more intuitive and that leads to a number of new results. For example, we connect crypticity to how an observer synchronizes to a process. We show that the block-causal-state entropy is a convex function of block length. We give a complete analysis of spin chains. We present a classification scheme that surveys stationary processes in terms of their possible cryptic and Markov orders. We illustrate related entropy convergence behaviors using a new form of foliated information diagram. Finally, along the way, we provide a variety of interpretations of crypticity and cryptic order to establish their naturalness and pervasiveness. This is also a first step in developing applications in spatially extended and network dynamical systems.     

Von Neumann entropy in a Rashba-Dresselhaus nanodot; dynamical electronic spin-orbit entanglement

The main purpose of the present article is to report the characteristics of von Neumann entropy, thereby, the electronic hybrid entanglement, in the heterojunction of two semiconductors, with due attention to the Rashba and Dresselhaus spin-orbit interactions. To this end, we cast the von Neumann entropy in terms of spin polarization and compute its time evolution; with a vast span of applications. It is assumed that gate potentials are applied to the heterojunction, providing a two dimensional parabolic confining potential (forming an isotropic nanodot at the junction), as well as means of controlling the spin-orbit couplings. The spin degeneracy is also removed, even at electronic zero momentum, by the presence of an external magnetic field which, in turn, leads to the appearance of Landau states. We then proceed by computing the time evolution of the corresponding von Neumann entropy from a separable (spin-polarized) initial state. The von Neumann entropy, as we show, indicates that electronic hybrid entanglement does occur between spin and two-dimensional Landau levels. Our results also show that von Neumann entropy, as well as the degree of spin-orbit entanglement, periodically collapses and revives. The characteristics of such behavior; period, amplitude, etc., are shown to be determined from the controllable external agents. Moreover, it is demonstrated that the phenomenon of collapse-revivals’ in the behavior of von Neumann entropy, equivalently, electronic hybrid entanglement, is accompanied by plateaus (of great importance in quantum computation schemes) whose durations are, again, controlled by the external elements. Along these lines, we also make a comparison between effects of the two spin-orbit couplings on the entanglement (von Neumann entropy) characteristics. The finer details of the electronic hybrid entanglement, which may be easily verified through spin polarization measurements, are also accreted and discussed. The novel results of the present article, with potent applications in the field of quantum information processing, provide a deeper understanding of the electronic von Neumann entropy and hybrid entanglement that occurs in two-dimensional nanodots.     

Towards information theory for q-nonextensive statistics without q-deformed distributions

In this paper we extend our recent results [P. Jizba, T. Arimitsu Physica A 340 (2004) 110] on q-nonextensive statistics with non-Tsallis entropies. In particular, we combine an axiomatics of Rényi with the q-deformed version of Khinchin axioms to obtain the entropy which accounts both for systems with embedded self-similarity and q-nonextensivity. We find that this entropy can be uniquely solved in terms of a one-parameter family of information measures. The corresponding entropy maximizer is expressible via a special function known under the name of the Lambert W-function. We analyze the corresponding “high” and “low-temperature” asymptotics and make some remarks on the possible applications.     

Geometric Partition Entropy: Coarse-Graining a Continuous State Space

Entropy is re-examined as a quantification of ignorance in the predictability of a one dimensional continuous phenomenon. Although traditional estimators for entropy have been widely utilized in this context, we show that both the thermodynamic and Shannon’s theory of entropy are fundamentally discrete, and that the limiting process used to define differential entropy suffers from similar problems to those encountered in thermodynamics. In contrast, we consider a sampled data set to be observations of microstates (unmeasurable in thermodynamics and nonexistent in Shannon’s discrete theory), meaning, in this context, it is the macrostates of the underlying phenomenon that are unknown. To obtain a particular coarse-grained model we define macrostates using quantiles of the sample and define an ignorance density distribution based on the distances between quantiles. The geometric partition entropy is then just the Shannon entropy of this finite distribution. Our measure is more consistent and informative than histogram-binning, especially when applied to complex distributions and those with extreme outliers or under limited sampling. Its computational efficiency and avoidance of negative values can also make it preferable to geometric estimators such as k-nearest neighbors. We suggest applications that are unique to this estimator and illustrate its general utility through an application to time series in the approximation of an ergodic symbolic dynamics from limited observations.     

The Fisher-Hartwig Formula and Entanglement Entropy

Toeplitz matrices have applications to different problems of statistical mechanics. Recently it was used for calculation of entanglement entropy in spin chains. In the paper we review these recent developments. We use the Fisher-Hartwig formula, as well as the recent results concerning the asymptotics of the block Toeplitz determinants, to calculate entanglement entropy of large block of spins in the ground state of XY spin chain.     

Design and Test of an Integrated Random Number Generator with All-Digital Entropy Source

In the cybersecurity field, the generation of random numbers is extremely important because they are employed in different applications such as the generation/derivation of cryptographic keys, nonces, and initialization vectors. The more unpredictable the random sequence, the higher its quality and the lower the probability of recovering the value of those random numbers for an adversary. Cryptographically Secure Pseudo-Random Number Generators (CSPRNGs) are random number generators (RNGs) with specific properties and whose output sequence has such a degree of randomness that it cannot be distinguished from an ideal random sequence. In this work, we designed an all-digital RNG, which includes a Deterministic Random Bit Generator (DRBG) that meets the security requirements for cryptographic applications as CSPRNG, plus an entropy source that showed high portability and a high level of entropy. The proposed design has been intensively tested against both NIST and BSI suites to assess its entropy and randomness, and it is ready to be integrated into the European Processor Initiative (EPI) chip.     

Distributions and channel capacities in generalized statistical mechanics

The purpose of this note is twofold. Firstly, we consider generalizations of Shannon's entropy and its applications to thermodynamics based on extensivity considerations. Secondly, we apply the generalized entropy formalism to deriving various generalised channel capacities. We arrive at some surprising conclusions of systems achieving “super-capacitance” or “sub-capacitance” depending on the circumstances. These results suggest the possibility of improving the conventional Shannon capacity by using physical systems obeying more generalized statistics, but also predict behaviour which is hard to reconcile with experience.     

Estimation for Entropy and Parameters of Generalized Bilal Distribution under Adaptive Type II Progressive Hybrid Censoring Scheme

Entropy measures the uncertainty associated with a random variable. It has important applications in cybernetics, probability theory, astrophysics, life sciences and other fields. Recently, many authors focused on the estimation of entropy with different life distributions. However, the estimation of entropy for the generalized Bilal (GB) distribution has not yet been involved. In this paper, we consider the estimation of the entropy and the parameters with GB distribution based on adaptive Type-II progressive hybrid censored data. Maximum likelihood estimation of the entropy and the parameters are obtained using the Newton–Raphson iteration method. Bayesian estimations under different loss functions are provided with the help of Lindley’s approximation. The approximate confidence interval and the Bayesian credible interval of the parameters and entropy are obtained by using the delta and Markov chain Monte Carlo (MCMC) methods, respectively. Monte Carlo simulation studies are carried out to observe the performances of the different point and interval estimations. Finally, a real data set has been analyzed for illustrative purposes.     

Modified Hilbert Curve for Rectangles and Cuboids and Its Application in Entropy Coding for Image and Video Compression

In our previous work, by combining the Hilbert scan with the symbol grouping method, efficient run-length-based entropy coding was developed, and high-efficiency image compression algorithms based on the entropy coding were obtained. However, the 2-D Hilbert curves, which are a critical part of the above-mentioned entropy coding, are defined on squares with the side length being the powers of 2, i.e., 2ⁿ, while a subband is normally a rectangle of arbitrary sizes. It is not straightforward to modify the Hilbert curve from squares of side lengths of 2ⁿ to an arbitrary rectangle. In this short article, we provide the details of constructing the modified 2-D Hilbert curve of arbitrary rectangle sizes. Furthermore, we extend the method from a 2-D rectangle to a 3-D cuboid. The 3-D modified Hilbert curves are used in a novel 3-D transform video compression algorithm that employs the run-length-based entropy coding. Additionally, the modified 2-D and 3-D Hilbert curves introduced in this short article could be useful for some unknown applications in the future.     

Information Entropy in Chemistry: An Overview

Basic applications of the information entropy concept to chemical objects are reviewed. These applications deal with quantifying chemical and electronic structures of molecules, signal processing, structural studies on crystals, and molecular ensembles. Recent advances in the mentioned areas make information entropy a central concept in interdisciplinary studies on digitalizing chemical reactions, chemico-information synthesis, crystal engineering, as well as digitally rethinking basic notions of structural chemistry in terms of informatics.     

Stationary open systems: A brief review on contemporary theories on irreversibility

Open systems are very important in science and engineering for their applications and the analysis of the real word. At their steady state, two apparently opposed principles for their rate of entropy production have been proposed: the minimum entropy production rate and the maximum entropy production, useful in the analysis of dissipation and irreversibility of different processes in physics, chemistry, biology and engineering. Both principles involve an extremum of the rate of the entropy production at the steady state under non-equilibrium conditions. On the other hand, in engineering thermodynamics, dissipation and irreversibility are analyzed using the entropy generation, for which there exist two principle of extrema too, the minimum and the maximum principle. Finally, oppositions to the extrema principle have been developed too. In this paper, all these extrema principles will be analyzed in order to point out the relations among them and a synthesis useful in engineering applications, in physical and chemical process analysis and in biology and biotechnology will be proposed.     

New features of entanglement and other applications of a two-qubit system

In this paper, we examine a class of special cases of a two-qubit system under time evolution, in the presence of local squeezed reservoirs, we report some results related to the occurrence of so-called entanglement sudden death (ESD) of two cases of the class, which are initially entangled. In addition, we calculate the information entropies, the purity and the single-atom entropy squeezing of the bipartite system. Finally, we discuss the effects of the local squeezed vacuum fields on the entanglement and other applications of the bipartite system.