Application of a Near-Optimal Reinforcement Learning Controller to a Robotics Problem in Manufacturing: A Hybrid Approach

Optimization theory provides a framework for determining the best decisions or actions with respect to some mathematical model of a process. This paper focuses on learning to act in a near-optimal manner through reinforcement learning for problems that either have no model or the model is too complex. One approach to solving this class of problems is via approximate dynamic programming. The application of these methods are established primarily for the case of discrete state and action spaces. In this paper we develop efficient methods of learning which act in complex systems with continuous state and action spaces. Monte-Carlo approaches are employed to estimate function values in an iterative, incremental procedure. Derivative-free line search methods are used to obtain a near-optimal action in the continuous action space for a discrete subset of the state space. This near-optimal control policy is then extended to the entire continuous state space via a fuzzy additive model. To compensate for approximation errors, a modified procedure for perturbing the generated control policy is developed. Convergence results under moderate assumptions and stopping criteria are established.     

Synthesis,characterization, and micellization of an epoxy‐based amphiphilic diblock copolymer of ϵ‐caprolactone and glycidyl methacrylate by enzymatic ring‐opening polymerization and atom transfer radical polymerization

Amphiphilic diblock copolymer polycaprolactone‐block‐poly(glycidyl methacrylate) (PCL‐b‐PGMA) was synthesized via enzymatic ring‐opening polymerization (eROP) and atom transfer radical polymerization (ATRP). Methanol first initiated eROP of ?‐caprolactone (?‐CL) in the presence of biocatalyst Novozyme‐435 under anhydrous conditions. The resulting monohydroxyl‐terminated polycaprolactone (PCL–OH) was subsequently converted to a bromine‐ended macroinitiator (PCL–Br) for ATRP by esterification with α‐bromopropionyl bromide. PCL‐b‐PGMA diblock copolymers were synthesized in a subsequent ATRP of glycidyl methacrylate (GMA). A kinetic analysis of ATRP indicated a living/controlled radical process. The macromolecular structures were characterized for PCL–OH, PCL–Br, and the block copolymers by means of nuclear magnetic resonance, gel permeation chromatography, and infrared spectroscopy. Differential scanning calorimetry and wide‐angle X‐ray diffraction analyses indicated that the copolymer composition (?‐CL/GMA) had a great influence on the thermal properties. The well‐defined, amphiphilic diblock copolymer PCL‐b‐PGMA self‐assembled into nanoscale micelles in aqueous solutions, as investigated by dynamic light scattering and transmission electron microscopy. © 2007 Wiley Periodicals, Inc. J Polym Sci Part A: Polym Chem 45: 5037–5049, 2007     

2‐Cyanoprop‐2‐yl dithiobenzoate mediated reversible addition–fragmentation chain transfer polymerization of acrylonitrile targeting a polymer with a higher molecular weight

The reversible addition–fragmentation chain transfer (RAFT) polymerization of acrylonitrile (AN) mediated by 2‐cyanoprop‐2‐yl dithiobenzoate was first applied to synthesize polyacrylonitrile (PAN) with a high molecular weight up to 32,800 and a polydispersity index as low as 1.29. The key to success was ascribed to the optimization of the experimental conditions to increase the fragmentation reaction efficiency of the intermediate radical. In accordance with the atom transfer radical polymerization of AN, ethylene carbonate was also a better solvent candidate for providing higher controlled/living RAFT polymerization behaviors than dimethylformamide and dimethyl sulfoxide. The various experimental parameters, including the temperature, the molar ratio of dithiobenzoate to the initiator, the molar ratio of the monomer to dithiobenzoate, the monomer concentration, and the addition of the comonomer, were varied to improve the control of the molecular weight and polydispersity index. The molecular weights of PANs were validated by gel permeation chromatography along with a universal calibration procedure and intrinsic viscosity measurements. ¹H NMR analysis confirmed the high chain‐end functionality of the resultant polymers. © 2007 Wiley Periodicals, Inc. J Polym Sci Part A: Polym Chem 45: 1272–1281, 2007     

Studies on O-,N-Acylation of 3-Hydroxyisoxazole and Their Regioselective Synthesis(Ⅰ)——O-,N-acylation of 5-Methyl-3-hydroxyisoxazole with 2,2-Dimethyl 3-Substituted Cyclopropanecarboxylic Chlorides

The reactions of 5-methyl-3-hydroxyisoxazole with 2, 2-dimethyl, 3-substituted cyclopropanecarboxylic chlorides give O-acyl-5-methyl-3-hydroxyisoxazole and N-acyl-5-methylisoxazolin-3-one derivatives. The ratio of O-acyl to N-acyl product depends upon the acylation reagents. O-acyl derivatives can be converted to the N-acyl compounds by isomerization under acidic conditions or heating.     

An approximate solution to the integral radiative transfer equation in an optically thick slab

We consider the problem of solving the integral form of the radiative transfer equation in an atmosphere with optical thickness τ₀?1. We propose two methods transforming this problem to a finite set of the independent problems of the same type set in an atmosphere with optical thickness much less then τ₀. The error estimates are derived. Copyright © 2007 John Wiley & Sons, Ltd.     

Homogeneous immunoassays

Applied Biochemistry and Biotechnology -     

二维非正交坐标斜方格金属光子带隙结构

金属光子带隙结构在高能加速器、微波真空电子器件和太赫兹波源等方面具有重要的应用前景.基于实空间传输矩阵理论，详细研究了非正交坐标系下二维斜方格金属光子带隙结构，给出了计算横电模、横磁模完全带隙结构的一般公式，并分析了填充系数、任意斜角及金属柱横截面对带隙结构的影响.计算结果在退化为正方格情况下时，与其他方法的计算结果取得很好的一致. 关键词： 光子晶体 金属光子带隙 传输矩阵法 微波真空电子器件     

荷移分光光度法测定氨酪酸

氨酪酸(am inobutyric acid),化学名称为γ-氨基丁酸。本品在体内与血氨结合生成尿毒排出体外,有降低血氨及促进大脑新陈代谢的作用,能够增强葡萄糖磷酸酯化酶的活性,利于脑细胞功能的恢复。主要用于脑卒中后遗症、脑动脉硬化症、头部外伤后以及一氧化碳中毒所致昏迷的辅助治疗     

磁电子学器件应用原理

本文介绍几种重要的磁电子器件的基本结构和工作原理,包括巨磁电阻与隧穿磁电阻传感器、巨磁电阻隔离器、巨磁电阻与隧穿磁电阻硬盘读出磁头、磁电阻随机存取存储器、自旋转移磁化反转与微波振荡器。自旋晶体管作为未来磁电子学或自旋电子学时代的基本元素,目前大都还处在概念型阶段,本文也将对几种自旋晶体管的大致原理作简要介绍。