卷积神经网络用于近红外光谱预测土壤含水率

近红外光谱分析技术在土壤含水率预测方面具有独特的优势,是一种便捷且有效的方法。卷积神经网络作为高性能的深度学习模型,能够从复杂光谱数据中自主提取有效特征结构进行学习,与传统的浅层学习模型相比具有更强的模型表达能力。将卷积神经网络用于近红外光谱预测土壤含水率,并提出了有效的卷积神经网络光谱回归建模方法,简化了光谱数据的预处理要求,且具有更高的光谱预测精度。首先对不同含水率下土壤样品的光谱反射率数据进行简单的预处理,通过主成分分析减少光谱数据量,并将处理后的光谱数据变换为二维光谱信息矩阵,以适应卷积神经网络特殊的学习结构。然后基于卷积神经网络算法,设置双层卷积和池化结构逐层提取光谱数据的内部特征信息,并采用局部连接和权值共享减少网络参数、提高泛化性能。通过试验优化网络结构和各项参数,最终获得针对土壤光谱数据的卷积神经网络土壤含水率预测模型,并与传统的BP,PLSR和LSSVM模型进行对比实验。结果表明在训练样本达到一定数量时,卷积神经网络的预测精度和回归拟合度均高于三种传统模型。在少量训练样本参与建模的情况下,模型预测表现高于BP神经网络,但略低于PLSR和LSSVM模型。随着参与训练样本量的增加,卷积神经网络的预测精度和回归拟合度也随之稳定提升,达到并显著优于传统模型水平。因此,卷积神经网络能够利用近红外光谱数据对土壤含水率做出有效预测,且在较多样本参与建模时取得更好效果。     

基于卷积神经网络与光谱特征的夏威夷果品质鉴定研究

夏威夷果含油量高,在开缝之后容易发生变质,现有关于夏威夷果品质鉴定的方法多为传统的破坏性检验,很难满足无损检测的需求。卷积神经网络(CNN)作为应用最广泛的深度学习网络模型之一,具有比浅层学习方法更强的特征提取与模型表达能力,在光谱数据方面的应用拥有很大潜力。基于夏威夷果在可见-近红外的光谱特征分析,研究用于提取夏威夷果光谱特征的卷积神经网络模型,并提出一种高效无损鉴定夏威夷果品质的方法。首先以三种不同品质的夏威夷果(好籽、哈籽及霉籽)为研究对象,分析样本在500~2 100 nm的光谱信息;在光谱数据预处理中引入白化处理方法,用以增强数据的相关性差异;然后在模型训练过程中,将样本随机分为训练集和预测集,探讨不同CNN结构、卷积层数、卷积核大小及个数、池化层类型、全连接层神经元个数以及激活函数对分类结果的影响,并采用激活函数ReLU和Dropout方法,预防样本数据过少引起的过拟合现象;最后通过分析模型分类准确率和计算效率,确定了一个6层结构的CNN模型： 输入层-卷积层-池化层-全连接层(200神经元)-全连接层(100神经元)-输出层。实验结果表明： 上述网络模型对校正集和预测集的分类准确率均达到100％。因此,改进后的卷积神经网络模型可充分学习夏威夷果的光谱特征并有效分类,将深度学习理论与光谱分析相结合的方法能够实现对夏威夷果品质的准确鉴定,同时为夏威夷果等坚果类食品的高效、无损、实时在线检测提供了新思路。     

Recognizing pedestrian’s unsafe behaviors in far-infrared imagery at night

Pedestrian behavior recognition is important work for early accident prevention in advanced driver assistance system (ADAS). In particular, because most pedestrian-vehicle crashes are occurred from late of night to early of dawn, our study focus on recognizing unsafe behavior of pedestrians using thermal image captured from moving vehicle at night. For recognizing unsafe behavior, this study uses convolutional neural network (CNN) which shows high quality of recognition performance. However, because traditional CNN requires the very expensive training time and memory, we design the light CNN consisted of two convolutional layers and two subsampling layers for real-time processing of vehicle applications. In addition, we combine light CNN with boosted random forest (Boosted RF) classifier so that the output of CNN is not fully connected with the classifier but randomly connected with Boosted random forest. We named this CNN as randomly connected CNN (RC-CNN). The proposed method was successfully applied to the pedestrian unsafe behavior (PUB) dataset captured from far-infrared camera at night and its behavior recognition accuracy is confirmed to be higher than that of some algorithms related to CNNs, with a shorter processing time.     

基于GADF变换和多尺度CNN的哈密瓜表面农药残留可见-近红外光谱判别方法

针对哈密瓜表面农药残留化学检测方法成本高且具有破坏性等问题,探索了可见-近红外（Vis-NIR）光谱技术对农药残留定性判别的可行性。以哈密瓜为载体,百菌清和吡虫啉农药为研究对象,采集哈密瓜表面无残留、百菌清和吡虫啉残留的可见-近红外漫反射光谱,利用格拉姆角场（GAF）将一维光谱数据转换为二维彩色图像,构建GAF图像数据集。设计一种包含Inception结构的多尺度卷积神经网络模型用于哈密瓜表面农药残留种类判别,包括1层输入层、3层卷积层、1层融合层、1层平坦层、2层全连接层和1层输出层。模型测试混淆矩阵结果表明,格拉姆角差场（GADF）变换对哈密瓜表面农药残留的可见-近红外光谱表达能力较强。此外,构建AlexNet、VGG-16卷积神经网络（CNN）模型和支持向量机（SVM）、极限学习机（ELM）机器学习模型与提出的多尺度CNN模型进行性能对比。结果表明,3种CNN模型对哈密瓜表面有无农药残留的判别效果较好,综合判别准确率均高于SVM和ELM模型。对比3种CNN模型性能,多尺度CNN模型的性能最佳,训练耗时为14 s,综合判别准确率为98.33%。多尺度CNN模型结构利用多种小尺寸滤波器组合（1×1,3×3和5×5）和并行卷积模块,能够捕获不同层次和尺度的特征,通过级联融合模式进行深度特征融合,提高了模型的特征提取能力。与传统深度CNN模型相比,在保证计算复杂度不变的情况下,多尺度CNN模型的精度得到了有效提高。实验结果表明,GADF变换结合多尺度CNN模型可以有效进行光谱数据解析,利用可见-近红外光谱技术可以实现哈密瓜表面农药残留的定性判别。研究结果为大型瓜果表面农药残留的快速无损检测技术的研发提供了理论参考。     

基于卷积神经网络的恒星光谱自动分类方法

恒星光谱自动分类是研究恒星光谱的基础内容,快速、准确自动识别、分类恒星光谱可提高搜寻特殊天体速度,对天文学研究有重大意义。目前我国大型巡天项目LAMOST每年发布数百万条光谱数据,对海量恒星光谱进行快速、准确自动识别与分类研究已成为天文学大数据分析与处理领域的研究热点之一。针对恒星光谱自动分类问题,提出一种基于卷积神经网络(CNN)的K和F型恒星光谱分类方法,并与支持向量机(SVM)、误差反向传播算法(BP)对比,采用交叉验证方法验证分类器性能。与传统方法相比CNN具有权值共享,减少模型学习参数;可直接对训练数据自动进行特征提取等优点。实验采用Tensorflow深度学习框架,Python3.5编程环境。K和F恒星光谱数据集采用国家天文台提供的LAMOST DR3数据。截取每条光谱波长范围为3 500~7 500 部分,对光谱均匀采样生成数据集样本,采用min-max归一化方法对数据集样本进行归一化处理。CNN结构包括：输入层,卷积层C1,池化层S1,卷积层C2,池化层S2,卷积层C3,池化层S3,全连接层,输出层。输入层为一批K和F型恒星光谱相同的3 700个波长点处流量值。C1层设有10个大小为1×3步长为1的卷积核。S1层采用最大池化方法,采样窗口大小为1×2,无重叠采样,生成10张特征图,与C1层特征图数量相同,大小为C1层特征图的二分之一。C2层设有20个大小为1×2步长为1的卷积核,输出20张特征图。S2层对C2层20张特征图下采样输出20张特征图。C3层设有30个大小为1×3步长为1的卷积核,输出30张特征图。S3层对C3层30张特征图下采样输出30张特征图。全连接层神经元个数设置为50,每个神经元都与S3层的所有神经元连接。输出层神经元个数设置为2,输出分类结果。卷积层激活函数采用ReLU函数,输出层激活函数采用softmax函数。对比算法SVM类型为C-SVC,核函数采用径向基函数,BP算法设有3个隐藏层,每个隐藏层设有20,40和20个神经元。数据集分为训练数据和测试数据,将训练数据的40%,60%,80%和100%作为5个训练集,测试数据作为测试集。分别将5个训练集放入模型中训练,共迭代8 000次,每次训练好的模型用测试集进行验证。对比实验采用100%的训练数据作为训练集,测试数据作为测试集。采用精确率、召回率、F-score、准确率四个评价指标评价模型性能,对实验结果进行详细分析。分析结果表明CNN算法可对K和F型恒星光谱快速自动分类和筛选,训练集数据量越大,模型泛化能力越强,分类准确率越高。对比实验结果表明采用CNN算法对K和F型恒星光谱自动分类较传统机器学习SVM和BP算法自动分类准确率更高。     

State generators and complex neural memories

The mechanism of self-indexing for feedback neural networks that generates memories from short subsequences is generalized so that a single bit together with an appropriate update order suffices for each memory. This mechanism explains how stimulating an appropriate neuron can recall a memory. Although information is distributed in this model, yet our self-indexing mechanism makes it appear localized. Also a new complex valued neuron model is presented to generalize McCulloch-Pitts neurons.     

The Multi-Focus-Image-Fusion Method Based on Convolutional Neural Network and Sparse Representation

Multi-focus-image-fusion is a crucial embranchment of image processing. Many methods have been developed from different perspectives to solve this problem. Among them, the sparse representation (SR)-based and convolutional neural network (CNN)-based fusion methods have been widely used. Fusing the source image patches, the SR-based model is essentially a local method with a nonlinear fusion rule. On the other hand, the direct mapping between the source images follows the decision map which is learned via CNN. The fusion is a global one with a linear fusion rule. Combining the advantages of the above two methods, a novel fusion method that applies CNN to assist SR is proposed for the purpose of gaining a fused image with more precise and abundant information. In the proposed method, source image patches were fused based on SR and the new weight obtained by CNN. Experimental results demonstrate that the proposed method clearly outperforms existing state-of-the-art methods in addition to SR and CNN in terms of both visual perception and objective evaluation metrics, and the computational complexity is greatly reduced. Experimental results demonstrate that the proposed method not only clearly outperforms the SR and CNN methods in terms of visual perception and objective evaluation indicators, but is also significantly better than other state-of-the-art methods since our computational complexity is greatly reduced.     

基于Cholesky分解的增量式RELM及其在时间序列预测中的应用

针对应用于混沌时间序列预测的正则极端学习机(RELM)网络结构设计问题,提出一种基于Cholesky分解的增量式RELM训练算法.该算法通过逐次增加隐层神经元的方式自动确定最佳的RELM网络结构,并以Cholesky分解方式计算其输出权值,有效减小了隐层神经元递增过程的计算代价.混沌时间序列预测实例表明,该算法可有效实现最佳RELM网络结构的自动确定,且计算效率高.利用该算法训练后的RELM预测模型具有预测精度高的优点,适用于混沌时间序列预测. 关键词： 神经网络 极端学习机 混沌时间序列 时间序列预测     

Partial state feedback control of chaotic neural network and its application

The chaos control in the chaotic neural network is studied using the partial state feedback with a control signal from a few control neurons. The controlled CNN converges to one of the stored patterns with a period which depends on the initial conditions, i.e., the set of control neurons and other control parameters. We show that the controlled CNN can distinguish between two initial patterns even if they have a small difference. This implies that such a controlled CNN can be feasibly applied to information processing such as pattern recognition.     

基于改进卷积神经网络算法的语音识别

为了解决传统卷积神经网络识别连续语音数据时识别性能较差的问题,提出一种改进的卷积神经网络算法。该方法引入Fisher准则以及L2正则化约束,在反向传播调整参数阶段,既保证参数误差的最小化,又确保分类以后的样本类间分布较分散,类内分布较集中,同时保证网络权值具有合适的数量级以有效缓解过拟合问题;采用一种更符合生物神经元激活特性的新型log激活函数进行卷积神经网络的优化,进一步提高语音识别的正确率。在语音识别库TIMIT以及THCHS30上的实验结果表明,相较于传统卷积神经网络算法,本文提出的改进算法能较好的提高语音识别率,且泛化能力更强。     

Cyclotron Autoresonance with TE and TM Guided Waves

The electron cyclotron autoresonance is of both theoretical and practical interest for acceleration of charged particles and for generation of microwave radiation in various gyro-devices. In this paper a unified first-principles approach is applied to the analysis of the interaction of gyrating electrons with electromagnetic waves with different characteristic impedance. In contrast to the previous works that consider the underlying autoresonance integral only for the case of plane transverse waves and TE modes our analysis is applied also to TM modes. An approximate constant of motion is derived for the latter case from which it follows that the requirements for autoresonance can be satisfied easily in comparison to the TE modes. Some fundamental relations which follow from the analysis are discussed as well.     

Improvement of signal transmission through spike-timing-dependent plasticity in neural networks

We explore the effects of spike-timing-dependent plasticity (STDP) on weak signal transmission in a noisy neural network. We first consider the network where an ensemble of independent neurons, which are subjected to a common weak signal, are connected in parallel to a single postsynaptic neuron via excitatory synapses. STDP can make the signal transmission more efficient, and this effect is more prominent when the presynaptic activities exhibit some correlations. We further consider a two-layer network where there are only couplings between two layers and find that postsynaptic neurons can fire synchronously under suitable conditions. Both the reliability and timing precision of neuronal firing in the output layer are remarkably improved with STDP. These results indicate that STDP can play crucial roles in information processing in nervous systems.Received: 23 March 2004, Published online: 12 July 2004PACS: 87.18.Sn Neural networks - 87.17.Aa Theory and modeling; computer simulation     

识别Z玻色子喷注的卷积神经网络方法

高能物理中喷注识别任务是从背景中识别出感兴趣的特定信号,这些信号对于在大型强子对撞机上发现新的粒子,或者新的过程都有着非常重要的意义.量能器中产生的能量沉积可以看做是对喷注的一种拍照,分析这样产生的数据在机器学习领域中属于一个典型的视觉识别任务.基于喷注图片,本文探索了利用卷积神经网络(convolutional neural networks,CNNs)识别量子色动力学背景下的Z玻色子喷注,并与传统的增强决策树(boosted decision trees,BDTs)方法进行了对比.在本文利用的输入前提下,三种相关的性能参数表明,CNN比BDT带来了约1.5倍的效果提升.除此之外,通过最优与最差的喷注图与混淆矩阵,说明了CNN通过训练学习到的内容与整体识别能力.     

Efficient Network Architecture Search Using Hybrid Optimizer

Manually designing a convolutional neural network (CNN) is an important deep learning method for solving the problem of image classification. However, most of the existing CNN structure designs consume a significant amount of time and computing resources. Over the years, the demand for neural architecture search (NAS) methods has been on the rise. Therefore, we propose a novel deep architecture generation model based on Aquila optimization (AO) and a genetic algorithm (GA). The main contributions of this paper are as follows: Firstly, a new encoding strategy representing the CNN coding structure is proposed, so that the evolutionary computing algorithm can be combined with CNN. Secondly, a new mechanism for updating location is proposed, which incorporates three typical operators from GA cleverly into the model we have designed so that the model can find the optimal solution in the limited search space. Thirdly, the proposed method can deal with the variable-length CNN structure by adding skip connections. Fourthly, combining traditional CNN layers and residual blocks and introducing a grouping strategy provides greater possibilities for searching for the optimal CNN structure. Additionally, we use two notable datasets, consisting of the MNIST and CIFAR-10 datasets for model evaluation. The experimental results show that our proposed model has good results in terms of search accuracy and time.     

A polarization splitter based on slow light effect in a left-handed material waveguide

We propose a polarization splitter (PS) based on the slow light effect in a three-layer waveguide that consists of left-handed material (LHM) and dielectric layers with no birefringence. Transverse electric (TE) and transverse magnetic (TM) waves can be split by the intrinsic configuration of permittivity and permeability of LHM and dielectric layers. Simulation results are given to discuss the influence of electromagnetic parameters on the properties of our PS, and the capture and release of TE or TM waves are also realized in our PS which provides a flexible method to modulate the switching of different polarization waves.     

Statistical physics on cellular neural network computers

The computational paradigm represented by Cellular Neural/nonlinear Networks (CNN) and the CNN Universal Machine (CNN-UM) as a Cellular Wave Computer, gives new perspectives also for computational statistical physics. Thousands of locally interconnected cells working in parallel, analog signals giving the possibility of generating truly random numbers, continuity in time and the optical sensors included on the chip are just a few important advantages of such computers. Although CNN computers are mainly used and designed for image processing, here we argue that they are also suitable for solving complex problems in computational statistical physics. This study presents two examples of stochastic simulations on CNN: the site-percolation problem and the two-dimensional Ising model. Promising results are obtained using an ACE16K chip with 128×128 cells. In the second part of the work we discuss the possibility of using the CNN architecture in studying problems related to spin-glasses. A CNN with locally variant parameters is used for developing an optimization algorithm on spin-glass type models. Speed of the algorithms and further trends in developing the CNN chips are discussed.     

A New Pooling Approach Based on Zeckendorf’s Theorem for Texture Transfer Information

The pooling layer is at the heart of every convolutional neural network (CNN) contributing to the invariance of data variation. This paper proposes a pooling method based on Zeckendorf’s number series. The maximum pooling layers are replaced with Z pooling layer, which capture texels from input images, convolution layers, etc. It is shown that Z pooling properties are better adapted to segmentation tasks than other pooling functions. The method was evaluated on a traditional image segmentation task and on a dense labeling task carried out with a series of deep learning architectures in which the usual maximum pooling layers were altered to use the proposed pooling mechanism. Not only does it arbitrarily increase the receptive field in a parameterless fashion but it can better tolerate rotations since the pooling layers are independent of the geometric arrangement or sizes of the image regions. Different combinations of pooling operations produce images capable of emphasizing low/high frequencies, extract ultrametric contours, etc.     

一类自突触作用下神经元电路的设计和模拟

神经元在自突触作用下可以诱发各类放电活动的迁移, 神经元动作电位对电自突触的响应比较敏感. 通常用包含延迟因子和增益的反馈回路电流来刻画自突触对神经元动作电位的影响. 基于Pspice软件, 设计了包含自突触效应的神经元电路, 用以延迟反馈电路来模拟电自突触对电位的调制作用. 研究结果发现: 1)在外界刺激和电自突触回路协同作用下, 神经元电路输出信号可以呈现静息态, 尖峰放电, 簇放电状态. 2)在时变增大的外界刺激下和自突触回路驱动下, 神经元电路的输出电位序列在多种电活动模式之间(静息, 尖峰放电, 簇放电)交替出现, 其机理在于自突触回路具有记忆特性, 神经元对于不同的外界刺激可以做出不同模式的响应. 3)在给定比较大外界刺激下, 改变反馈回路的增益, 发现电路输出的序列也可以呈现不同模式交替, 即神经元对于相同的刺激可以通过自我调节自突触增益来产生不同模式的响应, 其机理可能在于回路的有效反馈, 这有助于理解突触的可塑性.     

A novel CNN algorithm for pathological complete response prediction using an I-SPY TRIAL breast MRI database

PurposeTo apply our convolutional neural network (CNN) algorithm to predict neoadjuvant chemotherapy (NAC) response using the I-SPY TRIAL breast MRI dataset.MethodsFrom the I-SPY TRIAL breast MRI database, 131 patients from 9 institutions were successfully downloaded for analysis. First post-contrast MRI images were used for 3D segmentation using 3D slicer. Our CNN was implemented entirely of 3 × 3 convolutional kernels and linear layers. The convolutional kernels consisted of 6 residual layers, totaling 12 convolutional layers. Dropout with a 0.5 keep probability and L2 normalization was utilized. Training was implemented by using the Adam optimizer. A 5-fold cross validation was used for performance evaluation. Software code was written in Python using the TensorFlow module on a Linux workstation with one NVidia Titan X GPU.ResultsOf 131 patients, 40 patients achieved pCR following NAC (group 1) and 91 patients did not achieve pCR following NAC (group 2). Diagnostic accuracy of our CNN two classification model distinguishing patients with pCR vs non-pCR was 72.5 (SD ± 8.4), with sensitivity 65.5% (SD ± 28.1) and specificity of 78.9% (SD ± 15.2). The area under a ROC Curve (AUC) was 0.72 (SD ± 0.08).ConclusionIt is feasible to use our CNN algorithm to predict NAC response in patients using a multi-institution dataset.