时不变分数阶系统反周期解的存在性

反周期解问题是非线性微分系统动力学的重要特征.近年来,非线性整数阶微分系统的反周期解问题得到了广泛的研究,非线性分数阶微分系统的反周期解问题也得到了初步的讨论.不同于已有的工作,该文研究时不变分数阶系统反周期解的存在性问题.证明了时不变分数阶系统在有限时间区间内不存在反周期解,而当分数阶导数的下限趋近于无穷大时,时不变分数阶系统却存在反周期解.     

分数阶微分方程不存在周期解:综述（英文）

周期解的存在性是分数阶动力系统领域中的一个热门话题.分数阶导数不同于经典的整数阶倒数,它们之间的基本区别在于是否存在遗传特性,即分数阶导数是具有弱奇异核的非局部算子,而整数阶导数则为局部算子.简要概述了带分数阶导数(如Grunwald-Letnikov导数、Reimann-Liouville导数、Caputo导数)的分数阶微分方程是否存在周期性解的近期结果.周期函数的经典整数阶导数与该周期函数具有相同周期这一结果已被证明.然而,周期函数的分数阶导数却不是具有相同的周期的周期函数.与此同时,本文也对非常数周期函数分数阶动力系统的周期不存在性以及分数阶动力系统的长时解的周期存在性进行了回顾.     

分数阶尺度函数的分解与重构算法

引入分数阶多分辨分析与分数阶尺度函数的概念.运用时频分析方法与分数阶小波变换,研究了分数阶正交小波的构造方法,得到分数阶正交小波存在的充要条件.给出分数阶尺度函数与小波的分解与重构算法,算法比经典的尺度函数与小波的分解与重构算法更具有一般性.     

计算Riemann-Liouville分数阶积分和导数的数值算法

研究计算Riemann-Liouville (RL)分数阶积分和导数的数值算法.首先,分析了RL分数阶积分和导数的定义式,由于定义式中包含一个积分瑕点,使RL分数阶积分和导数难于计算.然后,给出了一种去掉积分瑕点的方法,在此基础上设计出计算RL分数阶积分和导数的数值算法,并证明了此数值算法具有一阶精度.最后,给出了计算实例,计算结果说明提出的算法是有效的.     

分数阶光滑函数线性和二次插值公式余项估计

本文在局部分数阶导数定义的基础上给出了高阶局部分数阶导数定义,并据此得到了一般形式的分数阶Taylor公式.用该公式给出了分数阶光滑函数线性和二次插值公式余项的表达式,并进一步导出了分段线性插值的收敛阶估计.针对分数阶导数临界阶计算困难的问题,本文利用线性插值余项设计了一种外推算法,能够比较准确地求出函数在某点的局部分数阶导数的临界阶.最后通过编写算法的Mathematica程序,验证了理论分析的正确性,并用实例说明了算法的有效性.     

一类分数阶Langevin方程block-by-block算法的数值分析

分数阶Langevin方程有重要的科学意义和工程应用价值,基于经典block-by-block算法,求解了一类含有Caputo导数的分数阶Langevin方程的数值解.Block-by-block算法通过引入二次Lagrange基函数插值,构造出逐块收敛的非线性方程组,通过在每一块耦合求得分数阶Langevin方程的数值解.在0<α<1条件下,应用随机Taylor展开证明block-by-block算法是3+α阶收敛的,数值试验表明在不同α和时间步长h取值下,block-by-block算法具有稳定性和收敛性,克服了现有方法求解分数阶Langevin方程速度慢精度低的缺点,表明block-by-block算法求解分数阶Langevin方程是高效的.     

一类具有p-Laplacian算子的分数阶微分方程反周期边值问题解的存在唯一性

该文研究了一类具有p-Laplacian算子的非线性Caputo分数阶微分方程反周期边值问题解的存在唯一性.首先,利用分数阶微分方程和反周期边值条件给出了该边值问题的Green函数,然后利用p-Laplacian算子的性质和Banach压缩映射原理得到该边值问题解的存在唯一性结论,最后给出两个例子验证结论的合理性.值得一提的是此文研究的微分方程的反周期边值条件是带有Caputo分数阶微分.     

基于分数间隔的四二阶归一化累积量盲均衡算法

针对目前分数间隔盲均衡算法存在的缺陷,提出了基于分数间隔的四二阶归一化累积量盲均衡算法.先对接收信号进行分数间隔采样,然后利用四二阶归一化累积量将盲均衡算法归结为无约束的极值问题,从而简化了算法,加快了收敛速度,降低了误码率.仿真验证了算法的有效性.     

Conformable分数阶单机无穷大电力系统分岔与混沌研究

基于Conformable分数阶微分定义和Adomian分解算法,设计了Conformable分数阶非线性系统半解析解算法和Lyapunov指数谱算法.采用Lyapunov指数谱、分岔图和吸引子相图分析了Conformable分数阶单机无穷大电力系统中的分岔与混沌现象,揭示了系统状态随参数和微分阶数变化时的规律以及系统走向混沌的道路.Matlab仿真数值模拟结果表明:Conformable分数阶单机无穷大电力系统的动力学特征丰富,系统产生混沌的最小阶数为0.41,系统初值的改变直接影响系统状态,并发现了多涡卷混沌吸引子和共存吸引子,功角失稳是产生多涡卷吸引子的根本原因.研究结果表明了求解算法的有效性与Conformable分数阶单机无穷大电力系统动力学特性的丰富性.     

一个含参量积分例题的研究性学习和启示——引入Riemann-Liouville分数阶积分和导数的一种方法

本文将一个关于含参量积分的例题拓展成了针对学生的研究性学习课题.通过对这一例题的研究性学习,对整数阶微积分进行了推广,给出了Riemann-Liouville分数阶积分和导数的定义,并通过例题强化了学生对这一定义的认识,扩大了学生的知识视野.最后强调了研究性学习在数学分析教学中的重要性.     

Autonomous vehicle navigation using evolutionary reinforcement learning

Reinforcement learning schemes perform direct on-line search in control space. This makes them appropriate for modifying control rules to obtain improvements in the performance of a system. The effectiveness of a reinforcement learning strategy is studied here through the training of a learning classifier system (LCS) that controls the movement of an autonomous vehicle in simulated paths including left and right turns. The LCS comprises a set of condition-action rules (classifiers) that compete to control the system and evolve by means of a genetic algorithm (GA). Evolution and operation of classifiers depend upon an appropriate credit assignment mechanism based on reinforcement learning. Different design options and the role of various parameters have been investigated experimentally. The performance of vehicle movement under the proposed evolutionary approach is superior compared with that of other (neural) approaches based on reinforcement learning that have been applied previously to the same benchmark problem.     

Neuroevolution strategies for episodic reinforcement learning

Because of their convincing performance, there is a growing interest in using evolutionary algorithms for reinforcement learning. We propose learning of neural network policies by the covariance matrix adaptation evolution strategy (CMA-ES), a randomized variable-metric search algorithm for continuous optimization. We argue that this approach, which we refer to as CMA Neuroevolution Strategy (CMA-NeuroES), is ideally suited for reinforcement learning, in particular because it is based on ranking policies (and therefore robust against noise), efficiently detects correlations between parameters, and infers a search direction from scalar reinforcement signals. We evaluate the CMA-NeuroES on five different (Markovian and non-Markovian) variants of the common pole balancing problem. The results are compared to those described in a recent study covering several RL algorithms, and the CMA-NeuroES shows the overall best performance.     

SOLVING DYNAMIC WILDLIFE RESOURCE OPTIMIZATION PROBLEMS USING REINFORCEMENT LEARNING

ABSTRACT. An important technical component of natural resource management, particularly in an adaptive management context, is optimization. This is used to select the most appropriate management strategy, given a model of the system and all relevant available information. For dynamic resource systems, dynamic programming has been the de facto standard for deriving optimal state‐specific management strategies. Though effective for small‐dimension problems, dynamic programming is incapable of providing solutions to larger problems, even with modern microcomputing technology. Reinforcement learning is an alternative, related procedure for deriving optimal management strategies, based on stochastic approximation. It is an iterative process that improves estimates of the value of state‐specific actions based in interactions with a system, or model thereof. Applications of reinforcement learning in the field of artificial intelligence have illustrated its ability to yield near‐optimal strategies for very complex model systems, highlighting the potential utility of this method for ecological and natural resource management problems, which tend to be of high dimension. I describe the concept of reinforcement learning and its approach of estimating optimal strategies by temporal difference learning. I then illustrate the application of this method using a simple, well‐known case study of Anderson [1975], and compare the reinforcement learning results with those of dynamic programming. Though a globally‐optimal strategy is not discovered, it performs very well relative to the dynamic programming strategy, based on simulated cumulative objective return. I suggest that reinforcement learning be applied to relatively complex problems where an approximate solution to a realistic model is preferable to an exact answer to an oversimplified model.     

考虑框架协议的动态报童模型强化学习建模研究

企业为了稳定货源和供货关系,常与供应商签订一定时期的框架性协议。为了解决零售商在框架协议下采购报童产品的问题,本文运用强化学习建立库存决策模型并使用Q学习算法求取较优订货策略。通过生成样本随机数来模拟需求量,对比研究Q学习算法订货和传统方法订货的差别。通过多次数值实验,发现使用强化学习方法订货相比于传统订货方法(定量订货法、移动平均预测、指数平滑法)平均利润提高约7%～22%,且多次实验下强化学习方法订货相比于理想状态的平均利润相差约8%。这些发现验证了强化学习解决库存问题的有效性和可行性。本文还研究了相关参数变化对总利润的影响,发现利润随着贪婪率(ε)增加而降低、随着学习率(α)的增加而增加。该结论能够为解决相关库存问题提供新的思路。     

Adaptive aggregation for reinforcement learning in average reward Markov decision processes

We present an algorithm which aggregates online when learning to behave optimally in an average reward Markov decision process. The algorithm is based on the reinforcement learning algorithm UCRL and uses confidence intervals for aggregating the state space. We derive bounds on the regret our algorithm suffers with respect to an optimal policy. These bounds are only slightly worse than the original bounds for UCRL.     

Comparison of strategy learning methods in Farmer–Pest problem for various complexity environments without delays

In this paper effectiveness of several agent strategy learning algorithms is compared in a new multi-agent Farmer–Pest learning environment. Learning is often utilized by multi-agent systems which can deal with complex problems by means of their decentralized approach. With a number of learning methods available, a need for their comparison arises. This is why we designed and implemented new multi-dimensional Farmer–Pest problem domain, which is suitable for benchmarking learning algorithms. This paper presents comparison results for reinforcement learning (SARSA) and supervised learning (Naïve Bayes, C4.5 and Ripper). These algorithms are tested on configurations with various complexity with not delayed rewards. The results show that algorithm performances depend highly on the environment configuration and various conditions favor different learning algorithms.     

企业财务危机预警Rough-Fuzzy-ANN模型的建立及应用

本文提出了一种基于粗糙一模糊神经网络（Rough-Fuzzy-ANN）的企业财务危机建模和预测新方法，并给出了相应的算法，在通过以我国上市公司财务数据为基础进行的实证分析之后，结果表明，Fuzzy-Rough-ANN模型具有预测精度高。学习和泛化能力强，适应性广的优点；同时有效、可行，为企业财务危机的动态预警提供了一条新的途径。     

A Learning Algorithm for the Control of Continuous Action Set-Point Regulator Systems

The convergence properties for reinforcement learning approaches, such as temporal differences and Q-learning, have been established under moderate assumptions for discrete state and action spaces. In practice, however, many systems have either continuous action spaces or a large number of discrete elements. This paper presents an approximate dynamic programming approach to reinforcement learning for continuous action set-point regulator problems, which learns near-optimal control policies based on scalar performance measures. The continuous-action space (CAS) algorithm uses derivative-free line search methods to obtain the optimal action in the continuous space. The theoretical convergence properties of the algorithm are presented. Several heuristic stopping criteria are investigated and practical application is illustrated by two example problems—the inverted pendulum balancing problem and the power system stabilization problem.     

我国上市公司被特别处理的组合预测模型研究

针对组合预测比单项预测具有更高的预测精度，本提出了一种基于模糊神经网络的上市公司被ST的非线性组合建模与预测新方法，并给出了相应的混合学习算法。通过与多元线性回归模型、Fisher模型和Logistc回归模型的预测结果对比表明，该方法具有预测精度高，学习与泛化能力强，适应性广的优点。     

带时间窗的车间搬运机器人路径优化建模及算法研究

本文以车间搬运机器人为研究对象,在考虑时间窗的前提下,求解机器人进行物料配送和成品回收场景下的路径优化问题。提出一种强化学习遗传蚁群算法,首先利用扫描法求解初始搬运机器人的数量,并将子路径节点的几何中心设置为虚拟节点,利用嵌入遗传算子的蚁群算法求解连接虚拟节点的最优路径,再利用强化学习算法求解子路径的最优结果;最后将基本成本、运输成本和时间惩罚成本的加权和作为目标解,并最终求出满足约束条件的最优解。通过与基准问题求解结果对比,验证了强化学习遗传蚁群算法的优越性。