Application of a Near-Optimal Reinforcement Learning Controller to a Robotics Problem in Manufacturing: A Hybrid Approach

Optimization theory provides a framework for determining the best decisions or actions with respect to some mathematical model of a process. This paper focuses on learning to act in a near-optimal manner through reinforcement learning for problems that either have no model or the model is too complex. One approach to solving this class of problems is via approximate dynamic programming. The application of these methods are established primarily for the case of discrete state and action spaces. In this paper we develop efficient methods of learning which act in complex systems with continuous state and action spaces. Monte-Carlo approaches are employed to estimate function values in an iterative, incremental procedure. Derivative-free line search methods are used to obtain a near-optimal action in the continuous action space for a discrete subset of the state space. This near-optimal control policy is then extended to the entire continuous state space via a fuzzy additive model. To compensate for approximation errors, a modified procedure for perturbing the generated control policy is developed. Convergence results under moderate assumptions and stopping criteria are established.     

Inhibition of the reduction of Cr(VI) at the magnetite–water interface by calcium carbonate coatings

The effect of calcium carbonate coatings on the reduction of aqueous chromate on the magnetite(1 1 1) surface has been investigated using a combination of synchrotron based X-ray photoemission spectroscopy (PES) and X-ray absorption near edge structure (XANES) spectroscopy, along with laboratory-based powder X-ray diffraction (XRD) and scanning electron microscopy (SEM). CaCO₃ coatings (dominantly calcite with minor quantities of aragonite and vaterite) of thicknesses ranging from 10 Å to 20 m were grown on magnetite(1 1 1) surfaces by exposure to supersaturated aqueous solutions followed by evaporation of the solution—a process that mimics pore-water evaporation in vadose zones leading to the formation of caliche and calcium carbonate coatings on mineral grains. Coating thicknesses were determined from attenuation of the Fe 2p photoemission signal by the carbonate coating. For coatings less than 15 Å thick, Cr 2p photoemission and Cr L_II, L_III-edge XANES spectra show that chromate is reduced by the underlying magnetite surface; however, as the minimum coating thickness increases beyond 15 Å, the magnetite surface becomes passivated and further chromate reduction ceases. Our findings suggest that carbonate coatings on natural magnetite grains can significantly reduce or eliminate their ability to reduce Cr(VI), which is a toxic and highly mobile environmental contaminant.     

Combined gas chromatography/tandem mass spectrometry using laser photodissociation

Two fibre-optic interfaces were constructed for laser photodissociation/tandem mass spectrometry using an ion trap spectrometer. These interfaces allow sample introduction from a liquid reservoir, solids probe and gas chromatograph. The laser/ion trap configuration was applied to the analysis of a mixture of alkylbenenes separated by gas chromatography using 573 nm photons.     

Synthesis and polymerization of o-hexylaniline: Characterization of the corresponding polyaniline

In the field of research on soluble conducting polymers, the poly(o-alkylanilines) are very interesting because we can expect them to give more soluble polymers and new properties. Like poly(o-propylaniline) (POP), which is more soluble than polyaniline (PANi), poly(o-hexylaniline) (POH) appears to be more soluble in organic solvents than POP because of the longer alkyl groups in the 2-position. The higher solubility confers better processability on this new polymer, and because of this solubility, an NMR study in solution became possible.The nitration of hexylbenzene and the reduction of the resulting product to o-hexylaniline were performed according to the literature. The chemical polymerization was easy and it is possible to produce this polymer in large quantities.The polymerization carried out in anhydrous NH₄F, 2.35 HF medium and in 5 M perchloric acid gave a polymer with almost quantitative yield. The electrochemical behaviour of POH displayed faster electron transfers than PANi in organic solvents, depending on the acido-basicity level of the aqueous solutions. Unlike PANi, fractal growth was not observed.     

Nonstandard analysis and compact Abelian groups

Gor'kii. Translated fromSibirskii Matematicheskii Zhurnal, Vol. 32, No. 2, pp. 26–40, March–April 1991.     

Application of a separating transformation to estimates of inner radii of open sets

We obtain solutions of new extremal problems of the geometric theory of functions of a complex variable related to estimates for the inner radii of nonoverlapping domains. Some known results are generalized to the case of open sets. __________ Translated from Ukrains’kyi Matematychnyi Zhurnal, Vol. 59, No. 10, pp. 1313–1321, October, 2007.     

Asymptotic stability of semigroups associated with linear weak dissipative systems with memory

We study the asymptotic behavior of the solutions of a class of linear dissipative integral differential equations. We show in the abstract setting a necessary and sufficient condition to get an exponential decay of the solution. In the case of the lack of exponential decay, we find the polynomial rate of decay of the solution. Some examples are given.     

Singular numbers of the monodromy operator and sufficient conditions of the asymptotic stability of periodic system of differential equations with fixed delay

To obtain sufficient conditions for the asymptotic stability of linear periodic systems with fixed delay commensurable with the period of coefficients, singular numbers of the monodromy operator are used. To find these numbers, a self-adjoint boundary value problem for ordinary differential equations is applied. We study the motion of eigenvalues of this boundary value problem under a variation of a parameter. Obtaining sufficient conditions for the asymptotic stability is reduced to finding the bifurcation value of the parameter for the boundary value problem.