Optimal Randomness for Stochastic Configuration Network (SCN) with Heavy-Tailed Distributions

Stochastic Configuration Network (SCN) has a powerful capability for regression and classification analysis. Traditionally, it is quite challenging to correctly determine an appropriate architecture for a neural network so that the trained model can achieve excellent performance for both learning and generalization. Compared with the known randomized learning algorithms for single hidden layer feed-forward neural networks, such as Randomized Radial Basis Function (RBF) Networks and Random Vector Functional-link (RVFL), the SCN randomly assigns the input weights and biases of the hidden nodes in a supervisory mechanism. Since the parameters in the hidden layers are randomly generated in uniform distribution, hypothetically, there is optimal randomness. Heavy-tailed distribution has shown optimal randomness in an unknown environment for finding some targets. Therefore, in this research, the authors used heavy-tailed distributions to randomly initialize weights and biases to see if the new SCN models can achieve better performance than the original SCN. Heavy-tailed distributions, such as Lévy distribution, Cauchy distribution, and Weibull distribution, have been used. Since some mixed distributions show heavy-tailed properties, the mixed Gaussian and Laplace distributions were also studied in this research work. Experimental results showed improved performance for SCN with heavy-tailed distributions. For the regression model, SCN-Lévy, SCN-Mixture, SCN-Cauchy, and SCN-Weibull used less hidden nodes to achieve similar performance with SCN. For the classification model, SCN-Mixture, SCN-Lévy, and SCN-Cauchy have higher test accuracy of 91.5%, 91.7% and 92.4%, respectively. Both are higher than the test accuracy of the original SCN.     

Skellam Type Processes of Order k and Beyond

In this article, we introduce the Skellam process of order k and its running average. We also discuss the time-changed Skellam process of order k. In particular, we discuss the space-fractional Skellam process and tempered space-fractional Skellam process via time changes in Skellam process by independent stable subordinator and tempered stable subordinator, respectively. We derive the marginal probabilities, Lévy measures, governing difference-differential equations of the introduced processes. Our results generalize the Skellam process and running average of Poisson process in several directions.     

Accumulation of Particles and Formation of a Dissipative Structure in a Nonequilibrium Bath

The standard textbooks contain good explanations of how and why equilibrium thermodynamics emerges in a reservoir with particles that are subjected to Gaussian noise. However, in systems that convert or transport energy, the noise is often not Gaussian. Instead, displacements exhibit an α-stable distribution. Such noise is commonly called Lévy noise. With such noise, we see a thermodynamics that deviates from what traditional equilibrium theory stipulates. In addition, with particles that can propel themselves, so-called active particles, we find that the rules of equilibrium thermodynamics no longer apply. No general nonequilibrium thermodynamic theory is available and understanding is often ad hoc. We study a system with overdamped particles that are subjected to Lévy noise. We pick a system with a geometry that leads to concise formulae to describe the accumulation of particles in a cavity. The nonhomogeneous distribution of particles can be seen as a dissipative structure, i.e., a lower-entropy steady state that allows for throughput of energy and concurrent production of entropy. After the mechanism that maintains nonequilibrium is switched off, the relaxation back to homogeneity represents an increase in entropy and a decrease of free energy. For our setup we can analytically connect the nonequilibrium noise and active particle behavior to entropy decrease and energy buildup with simple and intuitive formulae.     

A High-Efficiency Spectral Method for Two-Dimensional Ocean Acoustic Propagation Calculations

The accuracy and efficiency of sound field calculations highly concern issues of hydroacoustics. Recently, one-dimensional spectral methods have shown high-precision characteristics when solving the sound field but can solve only simplified models of underwater acoustic propagation, thus their application range is small. Therefore, it is necessary to directly calculate the two-dimensional Helmholtz equation of ocean acoustic propagation. Here, we use the Chebyshev–Galerkin and Chebyshev collocation methods to solve the two-dimensional Helmholtz model equation. Then, the Chebyshev collocation method is used to model ocean acoustic propagation because, unlike the Galerkin method, the collocation method does not need stringent boundary conditions. Compared with the mature Kraken program, the Chebyshev collocation method exhibits a higher numerical accuracy. However, the shortcoming of the collocation method is that the computational efficiency cannot satisfy the requirements of real-time applications due to the large number of calculations. Then, we implemented the parallel code of the collocation method, which could effectively improve calculation effectiveness.     

Rényi Cross-Entropy Measures for Common Distributions and Processes with Memory

Two Rényi-type generalizations of the Shannon cross-entropy, the Rényi cross-entropy and the Natural Rényi cross-entropy, were recently used as loss functions for the improved design of deep learning generative adversarial networks. In this work, we derive the Rényi and Natural Rényi differential cross-entropy measures in closed form for a wide class of common continuous distributions belonging to the exponential family, and we tabulate the results for ease of reference. We also summarise the Rényi-type cross-entropy rates between stationary Gaussian processes and between finite-alphabet time-invariant Markov sources.     

Stochastic Order and Generalized Weighted Mean Invariance

In this paper, we present order invariance theoretical results for weighted quasi-arithmetic means of a monotonic series of numbers. The quasi-arithmetic mean, or Kolmogorov–Nagumo mean, generalizes the classical mean and appears in many disciplines, from information theory to physics, from economics to traffic flow. Stochastic orders are defined on weights (or equivalently, discrete probability distributions). They were introduced to study risk in economics and decision theory, and recently have found utility in Monte Carlo techniques and in image processing. We show in this paper that, if two distributions of weights are ordered under first stochastic order, then for any monotonic series of numbers their weighted quasi-arithmetic means share the same order. This means for instance that arithmetic and harmonic mean for two different distributions of weights always have to be aligned if the weights are stochastically ordered, this is, either both means increase or both decrease. We explore the invariance properties when convex (concave) functions define both the quasi-arithmetic mean and the series of numbers, we show its relationship with increasing concave order and increasing convex order, and we observe the important role played by a new defined mirror property of stochastic orders. We also give some applications to entropy and cross-entropy and present an example of multiple importance sampling Monte Carlo technique that illustrates the usefulness and transversality of our approach. Invariance theorems are useful when a system is represented by a set of quasi-arithmetic means and we want to change the distribution of weights so that all means evolve in the same direction.     

Spectral Properties of Effective Dynamics from Conditional Expectations

The reduction of high-dimensional systems to effective models on a smaller set of variables is an essential task in many areas of science. For stochastic dynamics governed by diffusion processes, a general procedure to find effective equations is the conditioning approach. In this paper, we are interested in the spectrum of the generator of the resulting effective dynamics, and how it compares to the spectrum of the full generator. We prove a new relative error bound in terms of the eigenfunction approximation error for reversible systems. We also present numerical examples indicating that, if Kramers–Moyal (KM) type approximations are used to compute the spectrum of the reduced generator, it seems largely insensitive to the time window used for the KM estimators. We analyze the implications of these observations for systems driven by underdamped Langevin dynamics, and show how meaningful effective dynamics can be defined in this setting.     

Causal Inference in Time Series in Terms of Rényi Transfer Entropy

Uncovering causal interdependencies from observational data is one of the great challenges of a nonlinear time series analysis. In this paper, we discuss this topic with the help of an information-theoretic concept known as Rényi’s information measure. In particular, we tackle the directional information flow between bivariate time series in terms of Rényi’s transfer entropy. We show that by choosing Rényi’s parameter α, we can appropriately control information that is transferred only between selected parts of the underlying distributions. This, in turn, is a particularly potent tool for quantifying causal interdependencies in time series, where the knowledge of “black swan” events, such as spikes or sudden jumps, are of key importance. In this connection, we first prove that for Gaussian variables, Granger causality and Rényi transfer entropy are entirely equivalent. Moreover, we also partially extend these results to heavy-tailed α-Gaussian variables. These results allow establishing a connection between autoregressive and Rényi entropy-based information-theoretic approaches to data-driven causal inference. To aid our intuition, we employed the Leonenko et al. entropy estimator and analyzed Rényi’s information flow between bivariate time series generated from two unidirectionally coupled Rössler systems. Notably, we find that Rényi’s transfer entropy not only allows us to detect a threshold of synchronization but it also provides non-trivial insight into the structure of a transient regime that exists between the region of chaotic correlations and synchronization threshold. In addition, from Rényi’s transfer entropy, we could reliably infer the direction of coupling and, hence, causality, only for coupling strengths smaller than the onset value of the transient regime, i.e., when two Rössler systems are coupled but have not yet entered synchronization.     

Potential Well in Poincaré Recurrence

From a physical/dynamical system perspective, the potential well represents the proportional mass of points that escape the neighbourhood of a given point. In the last 20 years, several works have shown the importance of this quantity to obtain precise approximations for several recurrence time distributions in mixing stochastic processes and dynamical systems. Besides providing a review of the different scaling factors used in the literature in recurrence times, the present work contributes two new results: (1) For ϕ-mixing and ψ-mixing processes, we give a new exponential approximation for hitting and return times using the potential well as the scaling parameter. The error terms are explicit and sharp. (2) We analyse the uniform positivity of the potential well. Our results apply to processes on countable alphabets and do not assume a complete grammar.     

Rényi Entropies of Multidimensional Oscillator and Hydrogenic Systems with Applications to Highly Excited Rydberg States

The various facets of the internal disorder of quantum systems can be described by means of the Rényi entropies of their single-particle probability density according to modern density functional theory and quantum information techniques. In this work, we first show the lower and upper bounds for the Rényi entropies of general and central-potential quantum systems, as well as the associated entropic uncertainty relations. Then, the Rényi entropies of multidimensional oscillator and hydrogenic-like systems are reviewed and explicitly determined for all bound stationary position and momentum states from first principles (i.e., in terms of the potential strength, the space dimensionality and the states’s hyperquantum numbers). This is possible because the associated wavefunctions can be expressed by means of hypergeometric orthogonal polynomials. Emphasis is placed on the most extreme, non-trivial cases corresponding to the highly excited Rydberg states, where the Rényi entropies can be amazingly obtained in a simple, compact, and transparent form. Powerful asymptotic approaches of approximation theory have been used when the polynomial’s degree or the weight-function parameter(s) of the Hermite, Laguerre, and Gegenbauer polynomials have large values. At present, these special states are being shown of increasing potential interest in quantum information and the associated quantum technologies, such as e.g., quantum key distribution, quantum computation, and quantum metrology.     

熔锥型熊猫光纤耦合器的传输特性分析

熔锥型保偏耦合器的传输特性决定了其光学原理与功能的实现。为了从理论上分析各参量对保偏耦合器传输特性的影响,基于热-结构-电磁多物理场耦合理论,建立了熔锥型熊猫光纤耦合器双锥模型,应用有限元法,分析了各参量对熊猫光纤耦合器传输特性的影响。结果表明应力区与包层折射率差影响耦合区纵向电磁场的分布,折射率差越大,纵向电磁场分布的变形也越大;HE1x1模和HEy11模的传播常量对偏振主轴角度差不敏感,偏振主轴角度差是通过耦合系数进而影响保偏光纤耦合器的消光比的;HE1x1模的传播常量对熔锥的熔锥区横截面椭圆率比较敏感,横截面椭圆率变化6.67%时Δβx11变化0.14%,计算结果表明当熔锥区横截面椭圆率为0.56时可获得较高性能的耦合器。     

First-Principle Derivation of Single-Photon Entropy and Maxwell–Jüttner Velocity Distribution

This work is devoted to deriving the entropy of a single photon in a beam of light from first principles. Based on the quantum processes of light–matter interaction, we find that, if the light is not in equilibrium, there are two different ways, depending on whether the photon is being added or being removed from the light, of defining the single-photon entropy of this light. However, when the light is in equilibrium at temperature T, the two definitions are equivalent and the photon entropy of this light is hν/T. From first principles, we also re-derive the Jüttner velocity distribution showing that, even without interatomic collisions, two-level atoms will relax to the state satisfying the Maxwell–Jüttner velocity distribution when they are moving in blackbody radiation fields.     

Some Analytical Properties of the Model for Stochastic Evolutionary Games in Finite Populations with Non-uniform Interaction Rate

Traditional evolutionary games assume uniform interaction rate, which means that the rate at which individuals meet and interact is independent of their strategies. But in some systems, especially biological systems, the players interact with each other discriminately. Taylor and Nowak (2006) were the first to establish the corresponding non-uniform interaction rate model by allowing the interaction rates to depend on strategies. Their model is based on replicator dynamics which assumes an infinite size population. But in reality, the number of individuals in the population is always finite, and there will be some random interference in the individuals' strategy selection process. Therefore, it is more practical to establish the corresponding stochastic evolutionary model in finite populations. In fact, the analysis of evolutionary games in a finite size population is more difficult. Just as Taylor and Nowak said in the outlook section of their paper, "The analysis of non-uniform interaction rates should be extended to stochastic game dynamics of finite populations." In this paper, we are exactly doing this work. We extend Taylor and Nowak's model from infinite to finite case, especially focusing on the influence of non-uniform connection characteristics on the evolutionary stable state of the system. We model the strategy evolutionary process of the population by a continuous ergodic Markov process. Based on the limit distribution of the process, we can give the evolutionary stable state of the system. We make a complete classification of the symmetric 2×2 games. For each case game, the corresponding limit distribution of the Markov-based process is given when noise intensity is small enough. In contrast with most literatures in evolutionary games using the simulation method, all our results obtained are analytical. Especially, in the dominant-case game, coexistence of the two strategies may become evolutionary stable states in our model. This result can be used to explain the emergence of cooperation in the Prisoner is Dilemma Games to some extent. Some specific examples are given to illustrate our results.     

A Generalization of the Concavity of Rényi Entropy Power

Recently, Savaré-Toscani proved that the Rényi entropy power of general probability densities solving the p-nonlinear heat equation in Rn is a concave function of time under certain conditions of three parameters n,p,μ , which extends Costa’s concavity inequality for Shannon’s entropy power to the Rényi entropy power. In this paper, we give a condition Φ(n,p,μ) of n,p,μ under which the concavity of the Rényi entropy power is valid. The condition Φ(n,p,μ) contains Savaré-Toscani’s condition as a special case and much more cases. Precisely, the points (n,p,μ) satisfying Savaré-Toscani’s condition consist of a two-dimensional subset of R3 , and the points satisfying the condition Φ(n,p,μ) consist a three-dimensional subset of R3 . Furthermore, Φ(n,p,μ) gives the necessary and sufficient condition in a certain sense. Finally, the conditions are obtained with a systematic approach.     

Some Analytical Properties of the Model for Stochastic Evolutionary Games in Finite Populations with Non-uniform Interaction Rate

Traditional evolutionary games assume uniform interaction rate, which means that the rate at which individuals meet and interact is independent of their strategies. But in some systems, especially biological systems, the players interact with each other discriminately. Taylor and Nowak (2006) were the first to establish the corresponding non-uniform interaction rate model by allowing the interaction rates to depend on strategies. Their model is based on replicator dynamics which assumes an infinite size population. But in reality, the number of individuals in the population is always finite, and there will be some random interference in the individuals' strategy selection process. Therefore, it is more practical to establish the corresponding stochastic evolutionary model in finite populations. In fact, the analysis of evolutionary games in a finite size population is more difficult. Just as Taylor and Nowak said in the outlook section of their paper, "The analysis of non-uniform interaction rates should be extended to stochastic game dynamics of finite populations." In this paper, we are exactly doing this work. We extend Taylor and Nowak's model from infinite to finite case, especially focusing on the influence of non-uniform connection characteristics on the evolutionary stable state of the system. We model the strategy evolutionary process of the population by a continuous ergodic Markov process. Based on the limit distribution of the process, we can give the evolutionary stable state of the system. We make a complete classification of the symmetric 2×2 games. For each case game, the corresponding limit distribution of the Markov-based process is given when noise intensity is small enough. In contrast with most literatures in evolutionary games using the simulation method, all our results obtained are analytical. Especially, in the dominant-case game, coexistence of the two strategies may become evolutionary stable states in our model. This result can be used to explain the emergence of cooperation in the Prisoner is Dilemma Games to some extent. Some specific examples are given to illustrate our results.     

Non-Hermitian Generalization of Rényi Entropy

From their conception to present times, different concepts and definitions of entropy take key roles in a variety of areas from thermodynamics to information science, and they can be applied to both classical and quantum systems. Among them is the Rényi entropy. It is able to characterize various properties of classical information with a unified concise form. We focus on the quantum counterpart, which unifies the von Neumann entropy, max- and min-entropy, collision entropy, etc. It can only be directly applied to Hermitian systems because it usually requires that the density matrices is normalized. For a non-Hermitian system, the evolved density matrix may not be normalized; i.e., the trace can be larger or less than one as the time evolution. However, it is not well-defined for the Rényi entropy with a non-normalized probability distribution relevant to the density matrix of a non-Hermitian system, especially when the trace of the non-normalized density matrix is larger than one. In this work, we investigate how to describe the Rényi entropy for non-Hermitian systems more appropriately. We obtain a concisely and generalized form of α-Rényi entropy, which we extend the unified order-α from finite positive real numbers to zero and infinity. Our generalized α-Rényi entropy can be directly calculated using both of the normalized and non-normalized density matrices so that it is able to describe non-Hermitian entropy dynamics. We illustrate the necessity of our generalization by showing the differences between ours and the conventional Rényi entropy for non-Hermitian detuning two-level systems.     

From Rényi Entropy Power to Information Scan of Quantum States

In this paper, we generalize the notion of Shannon’s entropy power to the Rényi-entropy setting. With this, we propose generalizations of the de Bruijn identity, isoperimetric inequality, or Stam inequality. This framework not only allows for finding new estimation inequalities, but it also provides a convenient technical framework for the derivation of a one-parameter family of Rényi-entropy-power-based quantum-mechanical uncertainty relations. To illustrate the usefulness of the Rényi entropy power obtained, we show how the information probability distribution associated with a quantum state can be reconstructed in a process that is akin to quantum-state tomography. We illustrate the inner workings of this with the so-called “cat states”, which are of fundamental interest and practical use in schemes such as quantum metrology. Salient issues, including the extension of the notion of entropy power to Tsallis entropy and ensuing implications in estimation theory, are also briefly discussed.     

Understanding the Nature of the Long-Range Memory Phenomenon in Socioeconomic Systems

In the face of the upcoming 30th anniversary of econophysics, we review our contributions and other related works on the modeling of the long-range memory phenomenon in physical, economic, and other social complex systems. Our group has shown that the long-range memory phenomenon can be reproduced using various Markov processes, such as point processes, stochastic differential equations, and agent-based models—reproduced well enough to match other statistical properties of the financial markets, such as return and trading activity distributions and first-passage time distributions. Research has lead us to question whether the observed long-range memory is a result of the actual long-range memory process or just a consequence of the non-linearity of Markov processes. As our most recent result, we discuss the long-range memory of the order flow data in the financial markets and other social systems from the perspective of the fractional Lèvy stable motion. We test widely used long-range memory estimators on discrete fractional Lèvy stable motion represented by the auto-regressive fractionally integrated moving average (ARFIMA) sample series. Our newly obtained results seem to indicate that new estimators of self-similarity and long-range memory for analyzing systems with non-Gaussian distributions have to be developed.     

Physical Properties of Schwarzschild–deSitter Event Horizon Induced by Stochastic Quantum Gravity

A new type of quantum correction to the structure of classical black holes is investigated. This concerns the physics of event horizons induced by the occurrence of stochastic quantum gravitational fields. The theoretical framework is provided by the theory of manifestly covariant quantum gravity and the related prediction of an exclusively quantum-produced stochastic cosmological constant. The specific example case of the Schwarzschild–deSitter geometry is looked at, analyzing the consequent stochastic modifications of the Einstein field equations. It is proved that, in such a setting, the black hole event horizon no longer identifies a classical (i.e., deterministic) two-dimensional surface. On the contrary, it acquires a quantum stochastic character, giving rise to a frame-dependent transition region of radial width δr between internal and external subdomains. It is found that: (a) the radial size of the stochastic region depends parametrically on the central mass M of the black hole, scaling as δr∼M3; (b) for supermassive black holes δr is typically orders of magnitude larger than the Planck length lP. Instead, for typical stellar-mass black holes, δr may drop well below lP. The outcome provides new insight into the quantum properties of black holes, with implications for the physics of quantum tunneling phenomena expected to arise across stochastic event horizons.