Application of a Near-Optimal Reinforcement Learning Controller to a Robotics Problem in Manufacturing: A Hybrid Approach

Optimization theory provides a framework for determining the best decisions or actions with respect to some mathematical model of a process. This paper focuses on learning to act in a near-optimal manner through reinforcement learning for problems that either have no model or the model is too complex. One approach to solving this class of problems is via approximate dynamic programming. The application of these methods are established primarily for the case of discrete state and action spaces. In this paper we develop efficient methods of learning which act in complex systems with continuous state and action spaces. Monte-Carlo approaches are employed to estimate function values in an iterative, incremental procedure. Derivative-free line search methods are used to obtain a near-optimal action in the continuous action space for a discrete subset of the state space. This near-optimal control policy is then extended to the entire continuous state space via a fuzzy additive model. To compensate for approximation errors, a modified procedure for perturbing the generated control policy is developed. Convergence results under moderate assumptions and stopping criteria are established.     

A new type SVM-projected SVM

Support vector machine (SVM), developed by Vapnik et al., is a new and promising technique for classification and regression and has been proved to be competitive with the best available learning machines in many applications. However, the classification speed of SVM is substantially slower than that of other techniques with similar generalization ability. A new type SVM named projected SVM (PSVM), which is a combination of feature vector selection (FVS) method and linear SVM (LSVM), is proposed in present paper. In PSVM, the FVS method is first used to select a relevant subset (feature vectors, FVs) from the training data, and then both the training data and the test data are projected into the subspace constructed by FVs, and finally linear SVM(LSVM) is applied to classify the projected data. The time required by PSVM to calculate the class of new samples is proportional to the count of FVs. In most cases, the count of FVs is smaller than that of support vectors (SVs), and therefore PSVM is faster than SVM in running. Compared with other speeding-up techniques of SVM, PSVM is proved to possess not only speeding-up ability but also de-noising ability for high-noised data, and is found to be of potential use in mechanical fault pattern recognition.     

Sum-fuzzy implementation of a choice function using artificial learning procedure with fixed fraction

In one if his paper Luo transformed the problem of sum-fuzzy rationality into artificial learning procedure and gave an algorithm which used the learning rule of perception. This paper extends the Luo method for finding a sum-fuzzy implementation of a choice function and offers an algorithm based on the artificial learning procedure with fixed fraction. We also present a concrete example which uses this algorithm.     

Optimization technique based on learning automata

Optimization techniques are finding increasingly numerous applications in process design, in parallel to the increase of computer sophistication. The process synthesis problem can be stated as a largescale constrained optimization problem involving numerous local optima and presenting a nonlinear and nonconvex character. To solve this kind of problem, the classical optimization methods can lead to analytical and numerical difficulties. This paper describes the feasibility of an optimization technique based on learning systems which can take into consideration all the prior information concerning the process to be optimized and improve their behavior with time. This information generally occurs in a very complex analytical, empirical, or know-how form. Computer simulations related to chemical engineering problems (benzene chlorination, distillation sequence) and numerical examples are presented. The results illustrate both the performance and the implementation simplicity of this method.Nomenclature c _i penalty probability - cp precision parameter on constraints - D variation domain of the variablex - f(·) objective function - g(·) constraints - i,j indexes - k iteration number - N number of actions - P probability distribution vector - p _i ith component of the vectorP as iterationk - r number of reactors in the flowsheet - u(k) discrete value or action chosen by the algorithm at iterationk - u _i discrete value of the optimization variable in [u _min,u _max] - u _min lowest value of the optimization variable - u _max largest value of the optimization variable - Z random number - x variable for the criterion function - xp precision parameter on criterion function - W(k) performance index unit output at iterationk - ₀, ₁ reinforcement scheme parameters - _p sum of the probability distribution vector components     

Schedule-an expert-like system for machine scheduling

The construction of an expert-like system for machine scheduling called SCHEDULE is presented. Essential parts of SCHEDULE were developed by students in a laboratory course Operations Research on Microcomputers at the University of Karlsruhe, Germany. SCHEDULE consists of the components data base, knowledge base, inference engine, explanation facility, dialog component, and knowledge acquisition component. The knowledge base contains an algorithm base for solving different types of scheduling problems. To establish the rules of the knowledge base the well-known three-field classification of deterministic machine scheduling problems and the concept of the reduction digraph are exploited. Experiences gained during building and demonstrating SCHEDULE are reported.     

Properties of a genetic algorithm extended by a random self-learning operator and asymmetric mutations: A convergence study for a task of powder-pattern indexing

Genetic algorithms represent a powerful global-optimisation tool applicable in solving tasks of high complexity in science, technology, medicine, communication, etc. The usual genetic-algorithm calculation scheme is extended here by introduction of a quadratic self-learning operator, which performs a partial local search for randomly selected representatives of the population. This operator is aimed as a minor deterministic contribution to the (stochastic) genetic search. The population representing the trial solutions is split into two equal subpopulations allowed to exhibit different mutation rates (so called asymmetric mutation). The convergence is studied in detail exploiting a crystallographic-test example of indexing of powder diffraction data of orthorhombic lithium copper oxide, varying such parameters as mutation rates and the learning rate. It is shown through the averaged (over the subpopulation) fitness behaviour, how the genetic diversity in the population depends on the mutation rate of the given subpopulation. Conditions and algorithm parameter values favourable for convergence in the framework of proposed approach are discussed using the results for the mentioned example. Further data are studied with a somewhat modified algorithm using periodically varying mutation rates and a problem-specific operator. The chance of finding the global optimum and the convergence speed are observed to be strongly influenced by the effective mutation level and on the self-learning level. The optimal values of these two parameters are about 6 and 5%, respectively. The periodic changes of mutation rate are found to improve the explorative abilities of the algorithm. The results of the study confirm that the applied methodology leads to improvement of the classical genetic algorithm and, therefore, it is expected to be helpful in constructing of algorithms permitting to solve similar tasks of higher complexity.     

基于超星学习通平台的有机化学录播课教学实践

由于新型冠状病毒肺炎疫情,高校延期开学。为了实现"停课不停教、停课不停学",需要积极开展在线教学。本文基于超星学习通平台,开展了有机化学课程录播课教学实践,为高校教师网络教学提供一定的参考。     

THPep: A machine learning-based approach for predicting tumor homing peptides

In the present era, a major drawback of current anti-cancer drugs is the lack of satisfactory specificity towards tumor cells. Despite the presence of several therapies against cancer, tumor homing peptides are gaining importance as therapeutic agents. In this regard, the huge number of therapeutic peptides generated in recent years, demands the need to develop an effective and interpretable computational model for rapidly, effectively and automatically predicting tumor homing peptides. Therefore, a sequence-based approach referred herein as THPep has been developed to predict and analyze tumor homing peptides by using an interpretable random forest classifier in concomitant with amino acid composition, dipeptide composition and pseudo amino acid composition. An overall accuracy and Matthews correlation coefficient of 90.13% and 0.76, respectively, were achieved from the independent test set on an objective benchmark dataset. Upon comparison, it was found that THPep was superior to the existing method and holds high potential as a useful tool for predicting tumor homing peptides. For the convenience of experimental scientists, a web server for this proposed method is provided publicly at http://codes.bio/thpep/.     

基于课程标准的教学——以常见阴、阳离子的检验为例

以常见的阴、阳离子的检验为例，研究了如何实施基于标准的教学。校本教材的开发为标准、教材、教学、评价的一致性提供保障；以学生应知的和能做的驱动课堂活动；根据达成标准应有怎样的质量表现，试卷编制先于教学设计。课堂上，“教”“学”双方都明确学习目标，教师提供多种策略来满足学生多样的学习需要，如提供工具，搭建脚手架，并以“微”研究性学习的方式展开教学，给学生提供了充分的进步空间。     

核心素养为本的单元教学设计与实施——以“探究水的组成”为例

以“探究水的组成”教学为例，通过课标、教材及学习者分析，从认识角度、探究水平、认识水平等3个维度整体规划“身边的化学物质”主题单元目标学习进阶，明确“探究水的组成”课时目标，通过温故建模、据模探究、探究推理等3个阶段的教学实施，建立具体物质的研究思路模型，运用模型研究陌生物质（氢气），初步形成定量研究物质组成的能力。