Beyond convolutional neural networks computing: New trends on ISSCC 2023 machine learning chips

<正>Machine learning (ML) domain specific architectures(DSA) and chips have been prevailed in the past few years.These custom DSA designs outperform conventional general purposed architectures in terms of energy efficiency, processing latency and performance scalability. The data intensive nature of ML requires large amounts of processing power and memory access. Data flow architectures,     

Projection learning for self-organizing neural networks

A new learning scheme, called projection learning (PL), for self-organizing neural networks is presented. By iteratively subtracting out the projection of the “twinning” neuron onto the null space of the input vector, the neuron is made more similar to the input. By subtracting the projection onto the null space as opposed to making the weight vector directly aligned to the input, we attempt to reduce the bias of the weight vectors. This reduced bias will improve the generalizing abilities of the network. Such a feature is important in problems where the in-class variance is very high, such as, traffic sign recognition problems. Comparisons of PL with standard Kohonen learning indicate that projection learning is faster. Projection learning is implemented on a new self-organizing neural network model called the reconfigurable neural network (RNN). The RNN is designed to incorporate new patterns online without retraining the network. The RNN is used to recognize traffic signs for a mobile robot navigation system     

Novel learning method for analogue neural networks

An analogue-neural-network learning method based on hardware is proposed. All the network parameters are oscillated with slightly different frequencies, and the spectra appearing in the error signal are used to change the parameters. Learning can be carried out without knowing the internal conditions of neural networks.<>     

二进神经网络的模式匹配学习

二进神经网络的知识提取需要了解每个神经元的逻辑意义。一般来说,对二进神经网络学习结果的分析是困难的。该文提出了一种基于线性可分结构系结构分析的学习算法,采用这种方法对布尔空间的样本集合进行学习,得到的二进神经网络隐层神经元都归属于一类或几类线性可分结构系,只要这几类线性可分结构系的逻辑意义是清晰的,就可以分析整个学习结果的知识内涵。     

Modified backpropagation algorithm for fast learning in neural networks

A fast learning rule for artificial neural systems which is based on modifications to a backpropagation algorithm is described. The rule minimises the error function along the direction of the gradient and backpropagates the error pattern according to a constant error energy approach.<>     

Detecting Byzantine attack in cognitive radio networks using machine learning

Wireless Networks - One primary function in a cognitive radio network (CRN) is spectrum sensing. In an infrastructure-based CRN, instead of individual nodes independently sensing the presence of...     

Induction machine fault detection using SOM-based RBF neural networks

A radial-basis-function (RBF) neural-network-based fault detection system is developed for performing induction machine fault detection and analysis. Four feature vectors are extracted from power spectra of machine vibration signals. The extracted features are inputs of an RBF-type neural network for fault identification and classification. The optimal network architecture of the RBF network is determined automatically by our proposed cell-splitting grid algorithm. This facilitates the conventional laborious trial-and-error procedure in establishing an optimal architecture. In this paper, the proposed RBF machine fault diagnostic system has been intensively tested with unbalanced electrical faults and mechanical faults operating at different rotating speeds. The proposed system is not only able to detect electrical and mechanical faults, but the system is also able to estimate the extent of faults.     

Harmonic neural networks for on-line learning vector quantisation

A self-creating harmonic neural network (HNN) trained with a competitive algorithm effective for on-line learning vector quantisation is presented. It is shown that by employing dual resource counters to record the activity of each node during the training process, the equi-error and equi-probable criteria can be harmonised. Training in HNNs is smooth and incremental, and it not only achieves the biologically plausible on-line learning property, but it can also avoid the stability-plasticity dilemma, the dead-node problem, and the deficiency of the local minimum. Characterising HNNs reveals the great controllability of HNNs in favouring one criterion over the other, when faced with a must-choose situation between equi-error and equi-probable. Comparison studies on learning vector quantisation involving stationary and non-stationary, structured and non-structured inputs demonstrate that the HNN outperforms other competitive networks in terms of quantisation error, learning speed and codeword search efficiency     

机器学习算法在数据挖掘中的应用

针对数据挖掘算法中常用的机器学习型算法进行研究。机器学习型算法特色是运用了人工智能技术,能在大量样本集训练和学习后自动找出运算需要的参数和模式。以机器学习型算法中的人工神经网络为例研究数据挖掘技术,针对学习速度慢、抗干扰能力弱以及容易陷入局部最小值等缺点和传统的遗传算法存在算法早熟以及局部寻优能力弱等问题,提出一种通过改进常规遗传算法的染色体结构和遗传算子,并且通过引入自适应交叉和变异概率来对BP神经网络结构参数进行优化的改进型遗传优化BP神经网络模型。最后通过煤矿空压机故障诊断系统这一实例来研究改进型算法的数据挖掘技术的性能。研究结果表明,改进后的算法建立的诊断模型相比常规神经网络的诊断模型诊断准确率更好,诊断效率更快。     

A fixed-point algorithm to minimax learning with neural networks

In some real applications, such as medical diagnosis or remote sensing, available training data do not often reflect the true a priori probabilities of the underlying data distribution. The classifier designed from these data may be suboptimal. Building classifiers that are robust against changes in prior probabilities is possible by applying a minimax learning strategy. In this paper, we propose a simple fixed-point algorithm that is able to train a neural minimax classifier [i.e., a classifier minimizing the worst (maximum) possible risk]. Moreover, we present a new parametric family of loss functions that is able to provide the most accurate estimates for the posterior class probabilities near the decision regions, and we also discuss the application of these functions together with a minimax learning strategy. The results of the experiments carried out on different real databases point out the ability of the proposed algorithm to find the minimax solution and produce a robust classifier when the real a priori probabilities differ from the estimated ones.     

基于蚁群算法的二进神经网络学习算法

本文提出一种实现任意布尔函数的二进神经网络学习算法,该算法首先借助蚁群算法优化选择核心节点及节点访问顺序;其次,根据优化的节点访问顺序给出扩张分类超平面的步骤,减少了隐层神经元的数目,同时给出隐层神经元及输出元的表达形式;并进一步通过理论分析了该算法的收敛性。该算法成功地改进了已有学习算法的不足,并通过典型实例验证了该算法的有效性。     

神经网络在GDP预测中的应用

GDP预测因其影响因素众多,且各影响因素之间又存在着非常复杂的非线性关系,传统的线性预测方法对其进行预测时结果并不理想。基于提高GDP预测精度的考虑,运用人工神经网络的相关理论,建立了基于BP神经网络的黑龙江省GDP预测模型。结果表明,将神经网络应用于GDP预测可获得较高的预测精度,具有一定的实用价值。     

Conditional distribution learning with neural networks and itsapplication to channel equalization

We present a conditional distribution learning formulation for real-time signal processing with neural networks based on an extension of maximum likelihood theory-partial likelihood (PL) estimation-which allows for (i) dependent observations and (ii) sequential processing. For a general neural network conditional distribution model, we establish a fundamental information-theoretic connection, the equivalence of maximum PL estimation, and accumulated relative entropy (ARE) minimization, and obtain large sample properties of PL for the general case of dependent observations. As an example, the binary case with the sigmoidal perceptron as the probability model is presented. It is shown that the single and multilayer perceptron (MLP) models satisfy conditions for the equivalence of the two cost functions: ARE and negative log partial likelihood. The practical issue of their gradient descent minimization is then studied within the well-formed cost functions framework. It is shown that these are well-formed cost functions for networks without hidden units; hence, their gradient descent minimization is guaranteed to converge to a solution if one exists on such networks. The formulation is applied to adaptive channel equalization, and simulation results are presented to show the ability of the least relative entropy equalizer to realize complex decision boundaries and to recover during training from convergence at the wrong extreme in cases where the mean square error-based MLP equalizer cannot     

混沌神经网络在短期电力负荷预测中的应用

在分析混沌时间序列的基础上,综合混沌与神经网络方法建立了相应的混沌神经网络模型,并利用该模型对电力月负荷进行了预测,取得了令人满意的结果.     

Virtualized controller placement for multi-domain optical transport networks using machine learning

Photonic Network Communications - Optical multi-domain transport networks are often controlled by a hierarchical distributed architecture of controllers. Optimal placement of these controllers is...     

Fast learning process of multilayer neural networks using recursiveleast squares method

A new approach for the learning process of multilayer perceptron neural networks using the recursive least squares (RLS) type algorithm is proposed. This method minimizes the global sum of the square of the errors between the actual and the desired output values iteratively. The weights in the network are updated upon the arrival of a new training sample and by solving a system of normal equations recursively. To determine the desired target in the hidden layers an analog of the back-propagation strategy used in the conventional learning algorithms is developed. This permits the application of the learning procedure to all the layers. Simulation results on the 4-b parity checker and multiplexer networks were obtained which indicate significant reduction in the total number of iterations when compared with those of the conventional and accelerated back-propagation algorithms     

仿真假体视觉下神经网络算法的应用研究

尽管已有多种图像处理策略被应用到视觉假体的仿真研究中并提高了被试的识别表现,但在植入电极数量有限的情况下,如何保证盲人获得足够的拓扑信息是视觉假体仍需解决的问题。在此背景下,本文将两种神经网络算法应用到仿真假体视觉中对图像进行前景目标提取和像素化处理,首先利用图像分割数据集训练一个U-net网络得到前景提取后的结果,将其像素化之后与提取前的原图配对,再利用配对后的数据集训练一个Pix2Pix网络从而实现了将彩色图像"翻译"为像素化图像的目标。实验结果表明,与传统图像处理算法相比U-net网络具有更准确的目标提取效果,且经Pix2pix网络"翻译"后的图像也与标签图像更相似,有助于提高假体佩戴者的识别准确率。     

Protocol misbehavior detection framework using machine learning classification in vehicular Ad Hoc networks

Wireless Networks - A novel approach is proposed to detect protocol misbehavior using state-of-the-art machine learning frameworks and entropy. Nodes in Vehicular Ad Hoc Networks (VANETs) use...     

神经计算技术在ATM光纤高速智能管理网中的应用(续)

4　呼叫管理级呼叫管理级的主要功能是呼叫连接接纳控制。呼叫连接接纳控制意味着 ,仅当网络节判断得知新的连接将不会降低已接入老用户要求的服务质量时 ,才能允许接纳一个新用户的连接。例如 ,当用户 USE1要求与另一个用户 USE2通话时 ,首先 USE1用户向ATM网络节呼叫级控制器发送一个建立连接请求的信号。表征此连接请求的参数将包括目的用户、请求连接的承诺信息速率 (Commited InformationRate)、要求的服务质量等级 (GOS)以及生成信元信息特征参数等。网络节控制器由传输信息量参数评价各连接的服务质量水平 ,仅当评价表明对其…