Analog implementation of neural networks

Analog silicon-based neural hardware, which represents a large category among special-purpose analog and digital neurocomputers, and neural processing algorithms are reviewed. Artificial neural networks usually contain a large number of synaptic connections and many fewer processing neurons. The central problem in implementing artificial neural networks-making weights that are continuously adjustable, preferably in response to an analog control signal-is discussed. A simple integrated-circuit analog multiplier built from all-MOS components for use in electrically tunable synapses is described     

Parallel system design for time-delay neural networks

The authors develop a parallel structure for the time-delay neural network used in some speech recognition applications. The effectiveness of the design is illustrated by: (1) extracting a window computing model from the time-delay neural systems; (2) building its pipelined architecture with parallel or serial processing stages; and (3) applying this parallel window computing to some typical speech recognition systems. An analysis of the complexity of the proposed design shows a greatly reduced complexity while maintaining a high throughput rate     

Parallel algorithms/architectures for neural networks

This paper advocates digital VLSI architectures for implementing a wide variety of artificial neural networks (ANNs). A programmable systolic array is proposed, which maximizes the strength of VLSI in terms of intensive and pipelined computing and yet circumvents the limitation on communication. The array is meant to be more general purpose than most other ANN architectures proposed. It may be used for a variety of algorithms in both the retrieving and learning phases of ANNs: e.g., single layer feedback networks, competitive learning networks, and multilayer feed-forward networks. A unified approach to modeling of existing neural networks is proposed. This unified formulation leads to a basic structure for a universal simulation tool and neurocomputer architecture. Fault-tolerance approach and partitioning scheme for large or non-homogeneous networks are also proposed. Finally, the implementations based on commercially available VLSI chips (e.g., Inmos T800) and custom VLSI technology are discussed in great detail.     

Novel CMOS sampled-data VLSI implementation of artificial neural networks

A mixed-mode VLSI implementation of artificial neural networks offers a tradeoff solution for speed, area saving, and flexibility. A novel CMOS sampled-data programmable synapse and a simple CMOS analogue neuron have been developed. Using a 1.2 mu m CMOS technology, the synapse consumed 120*120 mu m/sup 2/ and the neuron consumed 120*260 mu m/sup 2/.<>     

Parallel, self-organizing, hierarchical neural networks. II

For pt.I see IEEE Trans. Neural Networks, vol.1, p.167-78 (1990). Parallel, self-organizing, hierarchical neural networks (PSHNNs) involve a number of stages with error detection at the end of each stage, i.e., rejection of error-causing vectors, which are then fed into the next stage after a nonlinear transformation. The stages operate in parallel during testing. Statistical properties and the mechanisms of vector rejection of the PSHNN are discussed in comparison to the maximum likelihood method and the backpropagation network. The PSHNN is highly fault tolerant and robust against errors in the weight values due to the adjustment of the error detection bounds to compensate errors in the weight values. These properties are exploited to develop architectures for programmable implementations in which the programmable parts are reduced to on-off or bipolar switching operations for bulk computations and attenuators for pointwise operations     

Parallel,self-organizing,hierarchical neural networks with forward-backward training

A forward-backward training algorithm for parallel, self-organizing hierarchical neural networks (PSHNNs) is described. Using linear algebra, it is shown that the forward-backward training of ann-stage PSHNN until convergence is equivalent to the pseudo-inverse solution for a single, total network designed in the least-squares sense with the total input vector consisting of the actual input vector and its additional nonlinear transformations. These results are also valid when a single long input vector is partitioned into smaller length vectors. A number of advantages achieved are: small modules for easy and fast learning, parallel implementation of small modules during testing, faster convergence rate, better numerical error-reduction, and suitability for learning input nonlinear transformations by other neural networks. The backpropagation (BP) algorithm is proposed for learning input nonlinearitics. Better performance in terms of deeper minimum of the error function and faster convergence rate is achieved when a single BP network is replaced by a PSHNN of equal complexity in which each stage is a BP network of smaller complexity than the single BP network.     

CMOS current-mode implementation of spatiotemporal probabilistic neural networks for speech recognition

In this paper, a Spatiotemporal Probabilistic Neural Network (SPNN) is proposed for spatiotemporal pattern recognition. This new model is developed by applying the concept of Gaussian density function to the network structure of the SPR (Spatiotemporal Pattern Recognition). The main advantages of this model include faster training and recalling process for patterns. In addition, the overall architecture is also simple, modular, regular, locally connected, and suitable for VLSI implementation. One set of independent speaker isolated (Mandarin digit) speech database is used as an example to demonstrate the superiority of the neural networks for spatiotemporal pattern recognition. The testing result with a reduced error rate of 7% shows that the SPNN is very attractive and effective for practical applications. p ]The CMOS current-mode IC technology is used to implement the SPNN to achieve the objective of minimum classification error in a more direct manner. In this design, neural computation is performed in analog circuits while template information is stored in digital circuits. The prototyping speech recognition processor for the 12th LPC calculation is designed by 1.2μm CMOS technology. The HSPICE simulation results are also presented, which verifies the function of the designed neural system.     

Low complexity VLSI implementation of CORDIC-based exponent calculation for neural networks

This article presents a low hardware complexity for exponent calculations based on CORDIC. The proposed CORDIC algorithm is designed to overcome major drawbacks (scale-factor compensation, low range of convergence and optimal selection of micro-rotations) of the conventional CORDIC in hyperbolic mode of operation. The micro-rotations are identified using leading-one bit detection with uni-direction rotations to eliminate redundant iterations and improve throughput. The efficiency and performance of the processor are independent of the probability of rotation angles being known prior to implementation. The eight-staged pipelined architecture implementation requires an 8?×?N ROM in the pre-processing unit for storing the initial coordinate values; it no longer requires the ROM for storing the elementary angles. It provides an area-time efficient design for VLSI implementation for calculating exponents in activation functions and Gaussain Potential Functions (GPF) in neural networks. The proposed CORDIC processor requires 32.68% less adders and 72.23% less registers compared to that of the conventional design. The proposed design when implemented on Virtex 2P (2vp50ff1148-6) device, dissipates 55.58% less power and has 45.09% less total gate count and 16.91% less delay as compared to Xilinx CORDIC Core. The detailed algorithm design along with FPGA implementation and area and time complexities is presented.     

采用脉冲宽度调制技术实现人工神经网络

该文提出了一种采用脉冲宽度调制(PWM)技术实现的人工神经网络(ANN),神经网络的权值采用数字存储器存储,突触电路由开关组成,电容积分电路和电流采样保持电路完成输入信号和权值信号的乘加运算,输出采用双电流镜。电路结构简单,易于集成,可用于构成前馈网络,如实现异或运算的人工神经网络。     

GENERALIZED INVERSE GROUP OF SIGNAL AND ITS IMPLEMENTATION WITH NEURAL NETWORKS

A new concept, the generalized inverse group (GIG) of signal, is firstly proposed and its properties, leaking coefficients and implementation with neural networks are presented. Theoretical analysis and computational simulation have shown that (1) there is a group of finite length of generalized inverse signals for any given finite signal, which forms the GIG; (2) each inverse group has different leaking coefficients, thus different abnormal states; (3) each GIG can be implemented by a grouped and improved single-layer perceptron which appears with fast convergence. When used in deconvolution, the proposed GIG can form a new parallel finite length of filtering deconvolution method. On off-line processing, the computational time is reduced to O(N) from O(N2). And the less the leaking coefficient is, the more reliable the deconvolution will be.     

基于嵌入式的神经网络滤波的研究与实现

在电力系统中谐波检测的实时性和精确度是一个重要部分,目前谐波检测中多以低通滤波器进行滤波,但是由于其要采集大量数据才可以实现稳定输出的固有特性,限制了其实时性和精确度的提高。人工神经网络可以进行滤波分析,但现有神经网络滤波几乎都基于MATLAB仿真,硬件测试的较少。在Linux系统中用C++在Qt软件编程实现三角基函数神经网络滤波,并把文中算法在X86和ARM两个不同芯片下的运行效果与在MATLAB中低通滤波器滤波做对比,发现在实际测试中该算法可以提高精确度,加快响应速度,其次还介绍了算法在基于X86和ARM两个不同芯片下环境配置,编译,实现等技术细节。     

A NEW METHOD FOR EIGENSTRUCTURES EXTRACTION AND ITS NEURAL NETWORKS IMPLEMENTATION

The cost function for eigenstructures extraction is discussed in detail in this paper, one can obtain the largest eigenvector by minimizing the cost function. In order to obtain other eigenvectors, a covariance matrix series is constructed. If one compares the cost function with the energy function of a neural networks, the neural networks can be easily introduced to extract the eigenvectors. Theoretical analysis and computer simulations show that the proposed method is reasonable and feasible.     

Parallel counter implementation

An (n, m) parallel counter is a circuit withn inputs that produces anm-bit binary count of the number of its inputs that are ONEs. This article reports on the design of large parallel counters with up to 1023 inputs. Design trade-offs are examined regarding the use of counter cells of size ranging from (3,2) to (31,5) as building blocks.     

基于神经网络的ATM网络多媒体流拥塞控制

提出了一种在用户-网络接口（UNI）处利用神经网络方法实现ATM网络多媒体流拥塞控制的新方法。在该方法中，控制器输出为信源编码率及其对应的用户百分比，即根据信源编码率及对应的用户百分比调整进入复用缓冲器多媒体流速率，从而克服了以往拥塞控制方法中仅仅调整编码率带来的对所有信源进行整体调整的缺陷，使摔制系统在信元丢失率最小情况下保证了多媒体流的质量，从而有效地利用了网络资源。本文还给出了两种实现方式，方式1中，神经网络输出层变量包括编码率及对应用户百分比，由连续编码率量化成离散值；方式2中，神经网络输出层变量只有连续的编码率，然后通过一定的换算公式计算出离散的编码率和对应的用户数。这两种实现方式中，方式1较为直观，但比方式2复杂。对话音流、视频流的仿真表明方法的有效性。     

Parallel spatial-temporal convolutional neural networks for anomaly detection and location in crowded scenes

Anomaly detection and location in crowded scenes have attracted a lot of attention in computer vision research community recently due to the increased applications of intelligent surveillance improve security in public. We propose a novel parallel spatial-temporal convolution neural networks model to detect and localize the abnormal behavior in video surveillance. Our approach contains two main steps. Firstly, considering the typical position of camera and the large number of background information, we introduce a novel spatial-temporal cuboid of interest detection method with varied-size cell structure and optical flow algorithm. Then, we use the parallel 3D convolution neural networks to describe the same behavior in different temporal-lengths. That step ensures that the most of behavior information in cuboids could be captured, also insures the reduction of information unrelated to the major behavior. The evaluation results on benchmark datasets show the superiority of our method compared to the state-of-the-art methods.     

Analog VLSI neural networks: implementation issues and examples inoptimization and supervised learning

Time-critical neural network applications that require fully parallel hardware implementations for maximal throughput are considered. The rich array of technologies that are being pursued is surveyed, and the analog CMOS VLSI medium approach is focused on. This medium is messy in that limited dynamic range, offset voltages, and noise sources all reduce precision. The authors examine how neural networks can be directly implemented in analog VLSI, giving examples of approaches that have been pursued to date. Two important application areas are highlighted: optimization, because neural hardware may offer a speed advantage of orders of magnitude over other methods; and supervised learning, because of the widespread use and generality of gradient-descent learning algorithms as applied to feedforward networks     

Hopfield neural networks for state estimation: parameters, efficient implementation and results

State estimation processes measurements and other information to find the network state vector. In this paper, state estimation is considered as an optimization problem to be solved with a Hopfield neural network. Several activation models for this network are simulated and compared. A new method is proposed that calculates the integration step parameter for this network in an autonomous way, eliminating the need for determining it in a manual way for each particular problem. This algorithm has been successfully tested for a wide range of electrical nets. Neural and classic analytical methods are compared.     

聚类分析的并行化实现技术研究

文中基于使传统聚类算法能够满足当前大数据分析的对计算效率的需求,采用将传统聚类算法分布式化的方法提高传统聚类算法效率。结合近年来广泛使用的Map Reduce分布式处理模型,对K-means、PAM、CLARA 3种算法进行了分布式化实验,并从数据规模和节点数量两个方面考察、讨论了一些影响并行算法性能的因素。实验分析表明,该方法能够有效地将聚类方法并行化,并可以应用在分布式系统当中。     

Parallel design of convolutional neural networks for remotesensing images object recognition based ondata-driven array processor

Object recognition in very high-resolution remote sensing images is a basic problem in the field of aerial and satellite image analysis. With the development of sensor technology and aerospace remote sensing technology, thequality and quantity of remote sensing images are improved. Traditional recognition methods have a certainlimitation in describing higher-level features, but object recognition method based on convolutional neural network(CNN) can not only deal with large scale images, but also train features automatically with high efficiency. It ismainly used on object recognition for remote sensing images. In this paper, an AlexNet CNN model is trained using2 100 remote sensing images, and correction rate can reach 97.6% after 2 000 iterations. Then based on trainedmodel, a parallel design of CNN for remote sensing images object recognition based on data-driven array processor(DDAP) is proposed. The consuming cycles are counted. Simultaneously, the proposed architecture is realized onXilinx V6 development board, and synthesized based on SMIC 130 nm complementary metal oxid semiconductor(CMOS) technology. The experimental results show that the proposed architecture has a certain degree ofparallelism to achieve the purpose of accelerating calculations.     

Optical neural networks

Classical optical information processing and classical neural networks can be adapted and combined to create optical neural networks which offer significant and fundamental advantages over electronic neural networks in various well-defined cases. A systematic morphology of optical neural networks is presented. Special problems they create are discussed. The state of the art of their implementation is indicated, and some supportable speculations on their future are given