Application of a Near-Optimal Reinforcement Learning Controller to a Robotics Problem in Manufacturing: A Hybrid Approach

Optimization theory provides a framework for determining the best decisions or actions with respect to some mathematical model of a process. This paper focuses on learning to act in a near-optimal manner through reinforcement learning for problems that either have no model or the model is too complex. One approach to solving this class of problems is via approximate dynamic programming. The application of these methods are established primarily for the case of discrete state and action spaces. In this paper we develop efficient methods of learning which act in complex systems with continuous state and action spaces. Monte-Carlo approaches are employed to estimate function values in an iterative, incremental procedure. Derivative-free line search methods are used to obtain a near-optimal action in the continuous action space for a discrete subset of the state space. This near-optimal control policy is then extended to the entire continuous state space via a fuzzy additive model. To compensate for approximation errors, a modified procedure for perturbing the generated control policy is developed. Convergence results under moderate assumptions and stopping criteria are established.     

Dielectronic Recombination of Very Heavy Lithiumlike Ions

Hyperfine Interactions - An overview of measurements of dielectronic recombination (DR) with the heaviest lithiumlike ions is presented. The experiments have been carried out at the heavy ion...     

Interface analysis of the system Si/YBa2Cu3O7?x

Summary The influence of different preparation conditions and substrate surface orientations on the superconducting properties of thin YBa₂Cu₃O_7–x (YBCO) films on silicon was studied. Comparative electrical and surface spectroscopic measurements were performed. SAM and SIMS depth profile analysis show an enrichment of barium at the interface between the superconductor and silicon for samples with T_c<76 K. Comparison with XPS data obtained for thin silicon films on YBCO indicates the formation of barium and yttrium silicates at the interface under these conditions.     

Existence of a solution for two phase flow in porous media: The case that the porosity depends on the pressure

In this paper we prove the existence of a solution of a coupled system involving a two phase incompressible flow in the ground and the mechanical deformation of the porous medium where the porosity is a function of the global pressure. The model is strongly coupled and involves a nonlinear degenerate parabolic equation. In order to show the existence of a weak solution, we consider a sequence of related uniformly parabolic problems and apply the Schauder fixed point theorem to show that they possess a classical solution. We then prove the relative compactness of sequences of solutions by means of the Fréchet-Kolmogorov theorem; this yields the convergence of a subsequence to a weak solution of the parabolic system.     

Self-similar solutions and large time asymptotics for the dissipative quasi-geostrophic equation

We analyze the well-posedness of the initial value problem for the dissipative quasi-geostrophic equations in the subcritical case. Mild solutions are obtained in several spaces with the right homogeneity to allow the existence of self-similar solutions. While the only small self-similar solution in the strong L^p{\cal L}^{p} space is the null solution, infinitely many self-similar solutions do exist in weak- L^p{\cal L}^{p} spaces and in a recently introduced [7] space of tempered distributions. The asymptotic stability of solutions is obtained in both spaces, and as a consequence, a criterion of self-similarity persistence at large times is obtained.     

How to work with one-dimensional quadratic maps

We analyse some identifiers which can univocally identify hyperbolic components and Misiurewicz points of one-dimensional quadratic maps. After seeing the equivalence among the different identifiers and how to go from one to another, we show which are the best for some specific tasks. Likewise, we present the analytic expressions, some of them shown for the first time in this paper, to calculate these identifiers. Some experimental considerations are taken into account.     

NMR study of products of thermal transformation of substituted N-aryl-o-quinoneimines

Products of thermal transformation of substituted N-aryl-o-quinoneimines were studied using NMR spectroscopy. The formation of 4aH-phenoxazine, which was further dimerized by the Diels—Alder reaction, was established.