A Learning Algorithm for the Control of Continuous Action Set-Point Regulator Systems

The convergence properties for reinforcement learning approaches, such as temporal differences and Q-learning, have been established under moderate assumptions for discrete state and action spaces. In practice, however, many systems have either continuous action spaces or a large number of discrete elements. This paper presents an approximate dynamic programming approach to reinforcement learning for continuous action set-point regulator problems, which learns near-optimal control policies based on scalar performance measures. The continuous-action space (CAS) algorithm uses derivative-free line search methods to obtain the optimal action in the continuous space. The theoretical convergence properties of the algorithm are presented. Several heuristic stopping criteria are investigated and practical application is illustrated by two example problems—the inverted pendulum balancing problem and the power system stabilization problem.     

Defect Formation,T-Atom Substitution and Adsorption of Guest Molecules in MSE-Type Zeolite Framework—DFT Modeling

We used computational modeling, based on Density Functional Theory, to help understand the preference for the formation of silanol nests and the substitution of Si by Ti or Al in different crystallographic positions of the MSE-type framework. All these processes were found to be energetically favorable by more than 100 kJ/mol. We suggested an approach for experimental identification of the T atom position in Ti-MCM-68 zeolite via simulation of infrared spectra of pyridine and acetonitrile adsorption at Ti. The modeling of adsorption of hydrogen peroxide at Ti center in the framework has shown that the molecular adsorption was preferred over the dissociative adsorption by 20 to 40 kJ/mol in the presence or absence of neighboring T-atom vacancy, respectively.     

Tryptammonium (2S,3S)-hydrogentartrate monohydrate [Struct Chem (2008) 19:147–154]: redetermination at 110 K and re-refinement against room temperature data

The crystal structure of the title compound was incorrectly reported as a methanol solvate [Koleva et al. in Struct Chem 19:147–154, 2008] on the basis of room temperature X-ray diffraction data. We have now established the presence of a disordered water molecule rather than the methanol molecule using a low temperature data set collected at 110 K.     

A proposition on memes and meta-memes in computing for higher-order learning

In computational intelligence, the term ‘memetic algorithm’ has come to be associated with the algorithmic pairing of a global search method with a local search method. In a sociological context, a ‘meme’ has been loosely defined as a unit of cultural information, the social analog of genes for individuals. Both of these definitions are inadequate, as ‘memetic algorithm’ is too specific, and ultimately a misnomer, as much as a ‘meme’ is defined too generally to be of scientific use. In this paper, we extend the notion of memes from a computational viewpoint and explore the purpose, definitions, design guidelines and architecture for effective memetic computing. Utilizing two conceptual case studies, we illustrate the power of high-order meme-based learning. With applications ranging from cognitive science to machine learning, memetic computing has the potential to provide much-needed stimulation to the field of computational intelligence by providing a framework for higher order learning.     

Complex Classical Fields: A Framework for Reflection Positivity

We explore a framework for complex classical fields, appropriate for describing quantum field theories. Our fields are linear transformations on a Hilbert space, so they are more general than random variables for a probability measure. Our method generalizes Osterwalder and Schrader’s construction of Euclidean fields. We allow complex-valued classical fields in the case of quantum field theories that describe neutral particles. From an analytic point-of-view, the key to using our method is reflection positivity. We investigate conditions on the Fourier representation of the fields to ensure that reflection positivity holds. We also show how reflection positivity is preserved by various spacetime compactifications of ${\mathbb{R}^{d}}$ in different coordinate directions.     

Transients and Evolution in NiTi

Many of the applications that seek to utilize shape memory alloys for their unique set of properties inevitably must deal, on some level, with the dimensional instability that is inherent to these materials under cyclic thermomechanical loading conditions. As a result, a better understanding of the transient and evolutionary behavior of a shape memory alloy is critical to both the successful design of useful actuation systems and development of accurate material models that can adequately capture the types of dimensional instability that can arise during component design. To this end, a set of experiments were conducted wherein the temperature cycling excursion was held fixed while the applied stress was varied. The results indicated that the extent of strain evolution produced under the initially applied stress has a significant impact on both the amount of transient that is observed as well as the rate of evolution observed under subsequent stress levels. In particular, lowering the applied stress to 50 MPa after cycling under an initial stress of 75 MPa did not stabilize the strain. However, lowering the applied stress to 50 MPa after cycling under an initial stress of 150 MPa produced a nearly saturated strain/temperature response. The thermomechanical observations are discussed in terms of the nature of strain evolution and its connection to the concept of a local/global minimization of the energy of the system, however, the exact mechanisms associated with these strain evolutions were not determined.     

Probing the QCD critical point with relativistic heavy-ion collisions

We utilize an event-by-event relativistic hydrodynamic calculation performed at a number of different incident beam energies to investigate the creation of hot and dense QCD matter near the critical point. Using state-of-the-art analysis and visualization tools we demonstrate that each collision event probes QCD matter characterized by a wide range of temperatures and baryo-chemical potentials, making a dynamical response of the system to the vicinity of the critical point very difficult to isolate above the background.     

An Experimental Design Approach to Quantitative Expression for Quality Control of a Multicomponent Antidiabetic Formulation by the HILIC Method

A rapid and reproducible hydrophilic liquid chromatography (HILIC) process was established for concomitant determination of remogliflozin etabonate (RE), vildagliptin (VD), and metformin (MF) in a formulation. A face-centered central composite experimental design was employed to optimize and predict the chromatographic condition by statistically studying the surface response model and design space with desirability close to one. A HILIC column with a simple mobile phase of acetonitrile (65% v/v) and 20 mM phosphate buffer (35% v/v, pH 6, controlled with orthophosphoric acid) was used to separate RE, VD, and MF. RE, VD, and MF were separated in 3.6 min using an isocratic mode mobile phase flow at a flow rate of 1.4 mL at room temperature, and the analytes were examined by recording the absorption at 210 nm. The developed HILIC method was thoroughly validated for all parameters recommended by ICH, and linearity was observed in the ranges 20–150 µg/mL, 10–75 µg/mL, and 50–750 µg/mL for RE, VD, and MF, respectively, along with excellent regression coefficients (r² > 0.999). The calculated percentage relative deviation and relative error ascertained the precision and accuracy of the method. The selectivity and accuracy were further confirmed by the high percentage recovery of added standard drugs to the formulation using the standard addition technique. The robustness of the HILIC processes was confirmed by developing a half-normal probability plot and Pareto chart, as the slight variation of a single factor had no significant influence on the assay outcomes. Utilization of the optimized HILIC procedure for concurrent quantification of RE, VD, and MF in solid dosage forms showed accurate and reproducible results. Hence, the fast HILIC method can be regularly employed for the quality assurance of pharmaceutical preparations comprising RE, VD, and MF.     

A reversible safety-catch method for the hydrogenolysis of N-benzyl moieties

The benzyl groups of β-hydroxy-N-benzyl sulfonamides are labile toward hydrogenolysis-unlike N-benzyl sulfonamides lacking the β-hydroxy moiety. We find that N-acyl-N-benzyl sulfonamides undergo hydrogenolysis under very mild conditions. Based upon these observations, we developed a reversible safety-catch method using tert-butoxycarbonyl moieties to activate N-benzyl sulfonamides toward hydrogenolysis. We also explored the utility of the safety-catch activation method for other nitrogen-based functionality such as N-benzyl carboxamides, imides, and related functionality.