Near-Optimal Controls of Discrete-Time Dynamic Systems Driven by Singularly-Perturbed Markov Chains

This work is devoted to near-optimal controls of large-scale discrete-time nonlinear dynamic systems driven by Markov chains; the underlying problem is to minimize an expected cost function. Our main goal is to reduce the complexity of the underlying systems. To achieve this goal, discrete-time control models under singularly-perturbed Markov chains are introduced. Using a relaxed control representation, our effort is devoted to finding near-optimal controls. Lumping the states in each irreducible class into a single state gives rise to a limit system. Applying near-optimal controls of the limit system to the original system, near-optimal controls of the original system are derived.     

Nonparametric estimation and adaptive control in a class of finite Markov decision chains

We consider a class of Markov decision processes withfinite state and action spaces which, essentially, is determined by the following condition: The state space isirreducible under the action of any stationary policy. However, except by this restriction, the transition law iscompletely unknown to the controller. In this context, we find a set of policies under which thefrequency estimators of the transition law are strongly consistent and then, this result is applied to constructadaptive asymptotically discount-optimal policies.Dedicated to Professor Truman O. Lewis, on the occasion of his sixtieth birthdayThis research was supported in part by the Third World Academy of Sciences (TWAS) under Grant TWAS RG MP 898-152, and in part by the Consejo Nacional de Ciencia y Tecnología (CONACYT) under Grant A128CCOEO550 (MT-2).     

A numerically stable algorithm for two server queue models

In this paper, we consider a queueing system in which there are two exponential servers, each having his own queue, and arriving customers will join the shorter queue. Based on the results given in Flatto and McKean, we rewrite the formula for the probability that there are exactlyk customers in each queue, wherek = 0, 1,…. This enables us to present an algorithm for computing these probabilities and then to find the joint distribution of the queue lengths in the system. A program and numerical examples are given.     

The effect of multiple optima on the simple GA run-time complexity

Genetic algorithms are stochastic search algorithms that have been applied to optimization problems. In this paper we analyze the run-time complexity of a genetic algorithm when we are interested in one of a set of distinguished solutions. One such case occurs when multiple optima exist. We define the worst case scenario and derive a probabilistic worst case bound on the number of iterations required to find one of these multiple solutions of interest.     

Colliding stacks: A large deviations analysis

We analyze the performance of a prototypical scheme for shared storage allocation: two initially empty stacks evolving in a contiguous block of memory of size m. We treat the case in which the stacks are more likely to shrink than grow, but with the probabilities of insertion and deletion allowed to depend arbitrarily on stack height as a fraction of m. New results are obtained on the m → ∞ asymptotics of the stack collision time, and of the final stack heights. The results of Wentzell and Freidlin on the large deviations of Markov chains are used, and the relation of their formalism to the Hamiltonian formulation of classical mechanics is emphasized. Certain results on higher-order asymptotics follow from WKB expansions.     

利用微悬臂梁表面应力研究聚N-异丙基丙烯酰胺分子的构象转变

提出了一种基于微悬臂梁传感技术研究大分子折叠/构象转变的新方法.通过分子自组装的方法将热敏性的聚N-异丙基丙烯酰胺(PNIPAM)分子链修饰到微悬臂梁的单侧表面，用光杠杆技术检测温度在20—40℃之间变化时由于微悬臂梁上的PNIPAM分子在水中的构象转变所引起的微悬臂梁变形.实验结果显示：在升温过程中，微悬臂梁的表面应力发生了变化并且导致微悬臂梁产生了弯曲变形，这个过程对应着微悬臂梁上的PNIPAM分子从无规线团构象到塌缩小球构象的构象转变.在降温过程中，微悬臂梁发生了反方向的弯曲变形，这对应着PNIPA 关键词： 构象转变 聚N-异丙基丙烯酰胺分子链 表面应力 微悬臂梁     

火工品可靠性试验数据的综合分析方法

升降法试验数据和固定刺激量下的成败型试验数据,是两种最常见的火工品可靠性试验数据．我们应用Markov链,研究了升降法试验数据下,感度分布参数的极大似然估计的特性．在此基础上,应用Bootstrap方法和Bayes方法,给出了综合分析两种试验数据的方法．最后,将该方法应用于520底火的可靠性鉴定,得出了有益的结论．     

Existence results for second order boundary value problem of impulsive dynamic equations on time scales

In this paper, Schaefer's fixed point theorem and nonlinear alternative of Leray-Schauder type are used to investigate the existence of solutions for second order boundary value problem for impulsive dynamic equations on time scales.     

A simple Markov chain structure for the evolution of credit ratings

We focus on continuous Markov chains as a model to describe the evolution of credit ratings. In this work it is checked whether a simple, tridiagonal type of generator provides a good approximation to a general one. Three different tridiagonal approximations are proposed and their performance is checked against two generators, corresponding to a volatile and a stable period, respectively. Copyright © 2007 John Wiley & Sons, Ltd.