A scalable framework for the solution of stochastic inverse problems using a sparse grid collocation approach

Experimental evidence suggests that the dynamics of many physical phenomena are significantly affected by the underlying uncertainties associated with variations in properties and fluctuations in operating conditions. Recent developments in stochastic analysis have opened the possibility of realistic modeling of such systems in the presence of multiple sources of uncertainties. These advances raise the possibility of solving the corresponding stochastic inverse problem: the problem of designing/estimating the evolution of a system in the presence of multiple sources of uncertainty given limited information.A scalable, parallel methodology for stochastic inverse/design problems is developed in this article. The representation of the underlying uncertainties and the resultant stochastic dependant variables is performed using a sparse grid collocation methodology. A novel stochastic sensitivity method is introduced based on multiple solutions to deterministic sensitivity problems. The stochastic inverse/design problem is transformed to a deterministic optimization problem in a larger-dimensional space that is subsequently solved using deterministic optimization algorithms. The design framework relies entirely on deterministic direct and sensitivity analysis of the continuum systems, thereby significantly enhancing the range of applicability of the framework for the design in the presence of uncertainty of many other systems usually analyzed with legacy codes. Various illustrative examples with multiple sources of uncertainty including inverse heat conduction problems in random heterogeneous media are provided to showcase the developed framework.     

Tiling solutions for optimal biological sensing

Biological systems, from cells to organisms, must respond to the ever-changing environment in order to survive and function. This is not a simple task given the often random nature of the signals they receive, as well as the intrinsically stochastic, many-body and often self-organized nature of the processes that control their sensing and response and limited resources. Despite a wide range of scales and functions that can be observed in the living world, some common principles that govern the behavior of biological systems emerge. Here I review two examples of very different biological problems: information transmission in gene regulatory networks and diversity of adaptive immune receptor repertoires that protect us from pathogens. I discuss the trade-offs that physical laws impose on these systems and show that the optimal designs of both immune repertoires and gene regulatory networks display similar discrete tiling structures. These solutions rely on locally non-overlapping placements of the responding elements (genes and receptors) that, overall, cover space nearly uniformly.     

Intrusive Galerkin methods with upwinding for uncertain nonlinear hyperbolic systems

This paper deals with stochastic spectral methods for uncertainty propagation and quantification in nonlinear hyperbolic systems of conservation laws. We consider problems with parametric uncertainty in initial conditions and model coefficients, whose solutions exhibit discontinuities in the spatial as well as in the stochastic variables. The stochastic spectral method relies on multi-resolution schemes where the stochastic domain is discretized using tensor-product stochastic elements supporting local polynomial bases. A Galerkin projection is used to derive a system of deterministic equations for the stochastic modes of the solution. Hyperbolicity of the resulting Galerkin system is analyzed. A finite volume scheme with a Roe-type solver is used for discretization of the spatial and time variables. An original technique is introduced for the fast evaluation of approximate upwind matrices, which is particularly well adapted to local polynomial bases. Efficiency and robustness of the overall method are assessed on the Burgers and Euler equations with shocks.     

Power-law distribution of gene expression fluctuations

Large-scale genomic technologies has opened new possibilities to infer gene regulatory networks from time series data. Here, we investigate the relationship between the dynamic information of gene expression in time series and the underlying network structure. First, our results show that the distribution of gene expression fluctuations (i.e., standard deviation) follows a power-law. This finding indicates that while most genes exhibit a relatively low variation in expression level, a few genes are revealed as highly variable genes. Second, we propose a stochastic model that explains the emergence of this power-law behavior. The model derives a relationship that connects the standard deviation (variance) of each node to its degree. In particular, it allows us to identify a global property of the underlying genetic regulatory network, such as the degree exponent, by only computing dynamic information. This result not only offers an interesting link to explore the topology of real systems without knowing the real structure but also supports earlier findings showing that gene networks may follow a scale-free distribution.     

Theory of slightly fluctuating ratchets

We consider a Brownian particle moving in a slightly fluctuating potential. Using the perturbation theory on small potential fluctuations, we derive a general analytical expression for the average particle velocity valid for both flashing and rocking ratchets with arbitrary, stochastic or deterministic, time dependence of potential energy fluctuations. The result is determined by the Green’s function for diffusion in the time-independent part of the potential and by the features of correlations in the fluctuating part of the potential. The generality of the result allows describing complex ratchet systems with competing characteristic times; these systems are exemplified by the model of a Brownian photomotor with relaxation processes of finite duration.     

Stochastic dynamics of nanoscale mechanical oscillators immersed in a viscous fluid

The stochastic response of nanoscale oscillators of arbitrary geometry immersed in a viscous fluid is studied. Using the fluctuation-dissipation theorem, it is shown that deterministic calculations of the governing fluid and solid equations can be used in a straightforward manner to directly calculate the stochastic response that would be measured in experiment. We use this approach to investigate the fluid coupled motion of single and multiple cantilevers with experimentally motivated geometries.     

Interval-stochastic thermal processes in electronic systems: Analysis and modeling

Mathematical and computermodeling of thermal processes, applied presently in thermal design of electronic systems, is based on the assumption that the factors determining the thermal processes are completely known and uniquely determined, that is, they are deterministic.Meanwhile, practice shows that the determining factors are of indeterminate interval-stochastic character. Moreover, thermal processes in electronic systems are stochastic and nonlinearly depend on both the stochastic determining factors and on the temperatures of electronics elements and environment. At present, the literature does not present methods of mathematical modeling of nonstationary, stochastic, nonlinear, interval-stochastic thermal processes in electronic systems to model thermal processes, which satisfy all the above-listed requirements to modeling adequacy. The present paper develops a method of mathematical and computer modeling of the nonstationary intervalstochastic nonlinear thermal processes in electronic systems. The method is based on obtaining equations describing the dynamics of time variation of statistical measures (expectations, variances, covariances) of temperature of electronic system elements with given statistical measures of the initial interval-stochastic determining factors.     

Intrinsic noise and division cycle effects on an abstract biological oscillator

Oscillatory dynamics are common in biological pathways, emerging from the coupling of positive and negative feedback loops. Due to the small numbers of molecules typically contained in cellular volumes, stochastic effects may play an important role in system behavior. Thus, for moderate noise strengths, stochasticity has been shown to enhance signal-to-noise ratios or even induce oscillations in a class of phenomena referred to as "stochastic resonance" and "coherence resonance," respectively. Furthermore, the biological oscillators are subject to influences from the division cycle of the cell. In this paper we consider a biologically relevant oscillator and investigate the effect of intrinsic noise as well as division cycle which encompasses the processes of growth, DNA duplication, and cell division. We first construct a minimal reaction network which can oscillate in the presence of large or negligible timescale separation. We then derive corresponding deterministic and stochastic models and compare their dynamical behaviors with respect to (i) the extent of the parameter space where each model can exhibit oscillatory behavior and (ii) the oscillation characteristics, namely, the amplitude and the period. We further incorporate division cycle effects on both models and investigate the effect of growth rate on system behavior. Our results show that in the presence but not in the absence of large timescale separation, coherence resonance effects result in extending the oscillatory region and lowering the period for the stochastic model. When the division cycle is taken into account, the oscillatory region of the deterministic model is shown to extend or shrink for moderate or high growth rates, respectively. Further, under the influence of the division cycle, the stochastic model can oscillate for parameter sets for which the deterministic model does not. The division cycle is also found to be able to resonate with the oscillator, thereby enhancing oscillation robustness. The results of this study can give valuable insight into the complex interplay between oscillatory intracellular dynamics and various noise sources, stemming from gene expression, cell growth, and division.     

Dynamical criteria for the evolution of the stochastic dimensionality in flows with uncertainty

We estimate and study the evolution of the dominant dimensionality of dynamical systems with uncertainty governed by stochastic partial differential equations, within the context of dynamically orthogonal (DO) field equations. Transient nonlinear dynamics, irregular data and non-stationary statistics are typical in a large range of applications such as oceanic and atmospheric flow estimation. To efficiently quantify uncertainties in such systems, it is essential to vary the dimensionality of the stochastic subspace with time. An objective here is to provide criteria to do so, working directly with the original equations of the dynamical system under study and its DO representation. We first analyze the scaling of the computational cost of these DO equations with the stochastic dimensionality and show that unlike many other stochastic methods the DO equations do not suffer from the curse of dimensionality. Subsequently, we present the new adaptive criteria for the variation of the stochastic dimensionality based on instantaneous (i) stability arguments and (ii) Bayesian data updates. We then illustrate the capabilities of the derived criteria to resolve the transient dynamics of two 2D stochastic fluid flows, specifically a double-gyre wind-driven circulation and a lid-driven cavity flow in a basin. In these two applications, we focus on the growth of uncertainty due to internal instabilities in deterministic flows. We consider a range of flow conditions described by varied Reynolds numbers and we study and compare the evolution of the uncertainty estimates under these varied conditions.     

Mutual information of population codes and distance measures in probability space

We studied the mutual information between a stimulus and a system consisting of stochastic, statistically independent elements that respond to a stimulus. Using statistical mechanical methods the properties of the mutual information (MI) in the limit of a large system size N are calculated. For continuous valued stimuli, the MI increases logarithmically with N and is related to the log of the Fisher information of the system. For discrete stimuli the MI saturates exponentially with N. We find that the exponent of saturation of the MI is the Chernoff distance between response probabilities that are induced by different stimuli.     

Quantum dynamical entropies for classical stochastic systems

We compare two proposals for the dynamical entropy of quantum deterministic systems (CNT and AFL) by studying their extensions to classical stochastic systems. We show that the natural measurement procedure leads to a simple explicit expression for the stochastic dynamical entropy with a clear information-theoretical interpretation. Finally, we compare our construction with other recent proposals.     

Enhanced convergence and robust performance of randomized dynamical decoupling

We demonstrate the advantages of randomization in coherent quantum dynamical control. For systems which are either time-varying or require decoupling cycles involving a large number of operations, we find that simple randomized protocols offer superior convergence and stability as compared to deterministic counterparts. In addition, we show how randomization may allow us to outperform purely deterministic schemes at long times, including combinatorial and concatenated methods. General criteria for optimally interpolating between deterministic and stochastic design are proposed and illustrated in explicit decoupling scenarios relevant to quantum information storage.     

Regularities unseen,randomness observed: levels of entropy convergence

We study how the Shannon entropy of sequences produced by an information source converges to the source's entropy rate. We synthesize several phenomenological approaches to applying information theoretic measures of randomness and memory to stochastic and deterministic processes by using successive derivatives of the Shannon entropy growth curve. This leads, in turn, to natural measures of apparent memory stored in a source and the amounts of information that must be extracted from observations of a source in order for it to be optimally predicted and for an observer to synchronize to it. To measure the difficulty of synchronization, we define the transient information and prove that, for Markov processes, it is related to the total uncertainty experienced while synchronizing to a process. One consequence of ignoring a process's structural properties is that the missed regularities are converted to apparent randomness. We demonstrate that this problem arises particularly for settings where one has access only to short measurement sequences. Numerically and analytically, we determine the Shannon entropy growth curve, and related quantities, for a range of stochastic and deterministic processes. We conclude by looking at the relationships between a process's entropy convergence behavior and its underlying computational structure.     

Fluctuation Resonance of Feed Forward Loops in Gene Regulatory Networks

应用化学主方程和线性涨落近似方法,重点研究了前馈环路(FFL)对外界输入弱信号的响应,特别考察了它的涨落共振现象．研究发现Z基因的FR行为很大程度上依赖于FFL的协同性：协同FFL中Z的FR曲线呈明显的单峰,而非协同FFL中该曲线出现明显双峰．由于振荡信号常常在实际应用中用来探测网络的调控结构,因此可以利用涨落共振曲线的定性差别来区分FFL网络的性能．     

Estimating optimal partitions for stochastic complex systems

Partitions provide simple symbolic representations for complex systems. For a deterministic system, a generating partition establishes one-to-one correspondence between an orbit and the infinite symbolic sequence generated by the partition. For a stochastic system, however, a generating partition does not exist. In this paper, we propose a method to obtain a partition that best specifies the locations of points for a time series generated from a stochastic system by using the corresponding symbolic sequence under a constraint of an information rate. When the length of the substrings is limited with a finite length, the method coincides with that for estimating a generating partition from a time series generated from a deterministic system. The two real datasets analyzed in Kennel and Buhl, Phys. Rev. Lett. 91, 084102 (2003), are reanalyzed with the proposed method to understand their underlying dynamics intuitively.