Application of a Near-Optimal Reinforcement Learning Controller to a Robotics Problem in Manufacturing: A Hybrid Approach

Optimization theory provides a framework for determining the best decisions or actions with respect to some mathematical model of a process. This paper focuses on learning to act in a near-optimal manner through reinforcement learning for problems that either have no model or the model is too complex. One approach to solving this class of problems is via approximate dynamic programming. The application of these methods are established primarily for the case of discrete state and action spaces. In this paper we develop efficient methods of learning which act in complex systems with continuous state and action spaces. Monte-Carlo approaches are employed to estimate function values in an iterative, incremental procedure. Derivative-free line search methods are used to obtain a near-optimal action in the continuous action space for a discrete subset of the state space. This near-optimal control policy is then extended to the entire continuous state space via a fuzzy additive model. To compensate for approximation errors, a modified procedure for perturbing the generated control policy is developed. Convergence results under moderate assumptions and stopping criteria are established.     

Photothermal techniques applied to the determination of the water vapor diffusion coefficient and thermal diffusivity of edible films.

Water vapor diffusion coefficient (WVDC) and thermal diffusivity (alpha) were determined in gelatin-starch films through photothermal techniques. The effect of different variables in the elaboration of these films, such as starch and glycerol concentrations and pH, were evaluated through the response surface methodology. The results indicated that an increase in the glycerol concentration and pH favored the WVDC of the films. On the other hand, alpha was influenced principally by the starch content and pH of the film-forming solution. The minimum alpha value was 4.5 x 10(-4) cm2/s, which is compared with alpha values reported for commercial synthetic polymers.     

Ethylene/1‐hexene copolymerization with tetramethyldisiloxane‐bridged bis(indenyl) metallocenes

Ethylene/1‐hexene copolymerizations with disiloxane‐bridged metallocenes, rac‐ and meso‐1,1,3,3‐tetramethyldisiloxanediyl‐bis(1‐indenyl)zirconium dichloride (rac‐ 1 , meso‐ 1 ) activated by modified methylaluminoxane were performed to investigate the influence of conformational dynamics on comonomer selectivity. Although ¹H NOESY (nuclear Overhauser and exchange spectroscopy) analysis indicated that the most stable conformation for the meso isomer in solution was that in which both indenes project over the metal coordination site, this isomer showed higher 1‐hexene selectivity in copolymerization (r_e = 140 ± 30, r_h = 0.024 ± 0.004) than the rac isomer with only one indene over the coordination site (r_e = 240 ± 20, r_h = 0.005 ± 0.001). The meso isomer showed high 1‐hexene selectivity, a high product of reactivity ratios (r_er_h = 3.3 ± 0.5) and produced copolymers that could be separated into fractions with different ethylene content suggesting that the active species exhibited multisite behavior and populated conformations with different comonomer selectivities during the copolymerization. © 2004 Wiley Periodicals, Inc. J Polym Sci Part A: Polym Chem 42: 3323–3331, 2004     

Pointwise and L2 bounds in some nonstandard problems for the heat equation

We consider some initial-boundary value problems for the linear and nonlinear heat equation where the gradient of the solution is prescribed on the boundary. Assuming that a solution exists, we obtain bounds for the solution and its gradient by maximum principle arguments or by means of differential and integral inequalities.