Application of a Near-Optimal Reinforcement Learning Controller to a Robotics Problem in Manufacturing: A Hybrid Approach

Optimization theory provides a framework for determining the best decisions or actions with respect to some mathematical model of a process. This paper focuses on learning to act in a near-optimal manner through reinforcement learning for problems that either have no model or the model is too complex. One approach to solving this class of problems is via approximate dynamic programming. The application of these methods are established primarily for the case of discrete state and action spaces. In this paper we develop efficient methods of learning which act in complex systems with continuous state and action spaces. Monte-Carlo approaches are employed to estimate function values in an iterative, incremental procedure. Derivative-free line search methods are used to obtain a near-optimal action in the continuous action space for a discrete subset of the state space. This near-optimal control policy is then extended to the entire continuous state space via a fuzzy additive model. To compensate for approximation errors, a modified procedure for perturbing the generated control policy is developed. Convergence results under moderate assumptions and stopping criteria are established.     

On the properties of solutions to the Goursat-Darboux problem with boundary and distributed controls

We consider a control system described by the Goursat-Darboux equation. The system is controlled by distributed and boundary controls. The controls are subject to the constraints given as multivalued mappings with closed, possibly nonconvex, values depending on the phase variable. Alongside the initial constraints, we consider the convexified constraints and the constraints whose values are the extreme points of the convexified constraints. We study the questions of existence of solutions and establish connections between the solutions under various constraints.     

Vehicle density and velocity estimation on highways for on-ramp metering control

A scheme for vehicle density and velocity estimation in a stretch of highway based on a modified cell transmission model [C. F. Daganzo, Transportation Research, Part B, 28B(4),269–287, 1994. Elsevier is presented. The scheme is intended for use with on-ramp metering control algorithms, providing local knowledge of densities and velocities that is helpful to improve on-ramp metering control performance. Estimation of density is obtained by nonlinear estimators, while velocity estimation is obtained by gradient algorithms. There is one density–velocity estimator for free traffic flow and other for congested traffic flow. Both estimator schemes work in parallel. The final estimation of density and velocity results from a convex combination of the predictions of the two estimators. This combination depends on occupancy or density measurements at the boundaries of the stretch and is produced by a fuzzy inference system. Stability and convergence of the density and velocity estimation scheme is proved by Lyapunov based techniques. Simulation results comparing measured and estimated traffic data are presented. They confirm good performance of the estimators. Research sponsored by grants UNAM PAPIIT IN110403 and CONACYT 47583.     

正定矩阵标准型的复合矩阵的一般形式

实对称正定矩阵的复合矩阵正定性的研究已有结论,但对于一般意义下的正定矩阵的复合矩阵是否仍然是正定的研究需要利用一般的正定矩阵的标准形的复合矩阵进行讨论,给出了一般公式及具体算法,为讨论其复合矩阵的正定性提供了基础条件.     

Duality theorems for convex problems with time delay

In this paper, we examine a class of convex problems of Bolza type, involving a time delay in the state. It encompasses a variety of time-delay problems arising in the calculus of variations and optimal control. A duality analysis is carried out which, among other things, leads to a characterization of minimizers in terms of the Euler-Lagrange inclusion. The results obtained improve in significant respects on what is achievable by techniques previously employed, based on elimination of the time delay by introduction of an infinite-dimensional state space or on the method of steps.     

Generic existence of Lipschitzian solutions of optimal control problems without convexity assumptions

The Tonelli existence theorem in the calculus of variations and its subsequent modifications were established for integrands f which satisfy convexity and growth conditions. In this paper we consider a large class of optimal control problems which is identified with a complete metric space of integrands without convexity assumptions and show that for a generic integrand the corresponding optimal control problem possesses a unique solution and this solution is Lipschitzian.     

Semideterministic Global Optimization Method: Application to a Control Problem of the Burgers Equation

This paper has two objectives. We introduce a new global optimization algorithm reformulating optimization problems in terms of boundary-value problems. Then, we apply this algorithm to a pointwise control problem of the viscous Burgers equation, where the control weight coefficient is progressively decreased. The results are compared with those obtained with a genetic algorithm and an LM-BFGS algorithm in order to check the efficiency of our method and the necessity of using global optimization techniques.     

Some results on controllability of a nonlinear degenerate parabolic system by bilinear control

In this paper, we discuss the controllability of a nonlinear degenerate parabolic system with bilinear control. Based on the shrinking property of the solutions, we prove that the system is not globally approximately controllable. Furthermore, we give an approximate null controllability result. We also prove that the system is not globally exactly null controllable by a comparison principle.     

挖坑机钻头主轴纵向振动系统模型的建立与控制

首先利用弹性杆理论和H am ilton变分原理建立了挖坑机钻头主轴及钻尖在工作过程中由于外激励的作用使钻尖与土壤互相作用而产生纵向振动的动力学模型,同时给出边界条件和初始条件.其次通过把系统外激励函数当作控制变量,利用Banach空间几何性质证明了此系统存在唯一最优控制元.     

年龄相关的非线性时变种群扩散系统最优分布控制的存在性

讨论了一类非线性时变种群扩散系统的最优分布控制问题,利用LionsJL的偏微控制理论和先验估计,证明了系统最优分布控制的存在性.所得结果可为非线性种群扩散系统中的最优控制问题的实际研究提供必要的理论基础.