Stochastic Neural Computation

A. Design of Spiking Recurrent Neural Network System on a Chip with Learning Capability

Introduction : Following the development of deep-learning neural networks (DNN) able to extract the features of raw data automatically, artificial intelligence (AI) becomes widely applied to data mining and reasoning, as well as to automatic control on systems like self-driving vehicles or robots. In many system-control applications, it is crucial to respond to environmental changes or stimuli in real time. This demand makes essential the implementation of AI algorithms as customized, low-power microsystems. However, most integrated-circuit designs for a neural network focus only on accelerating the network’s feedforward inference. Efficient implementation of learning algorithms for adapting network parameters “on-line” is seldom addressed. On the other hand, the recurrent neural network (RNN) is inherently more suitable for modelling time-variant data (e.g. language translation, video tracking, etc.), while novel hardware implementation of RNNs is seldom proposed. This is mainly because the recurrent structure makes both learning algorithms and hardware implementation more complicated. Nevertheless, this project investigates the feasibility of realizing probabilistic RNNs, such as the Boltzmann Machine and its variants, in VLSI based on spiking neuron circuits. Representing RNNs as networks of spiking neurons provides the advantage that the learning algorithm is hardware-friendly, because adaptation of each synaptic weight only depends on the spiking events generated the two neurons connected by the synapse. Moreover, signals between neurons and synapses are all transmitted as digital pulses(spikes), which are inherently robust against interference. Therefore, the novelty of this project lies in : (a) Exploring the feasibility of implementing a spiking, recurrent neural networks in VLSI with on-chip learning capability (b) The probabilistic behavior required by spiking neurons is induced by incorporating the intrinsic noise of transistors, and the noise is easily-adaptable (c) The synaptic weight is stored on non-volatile analog memory arrays not only adaptable on-line but also compatible with the standard CMOS technology. (d)Exploiting the multi-chip connections to realize deep-learning RNNs. The microsystem design in this project is expected to be useful for embedding artificial intelligence in robots, self-driving vehicles, or many IoT sensors/actuators

Characteristics : Matlab or C programming will be used to conclude the specifications of modular circuits for a scalable recurrent Spiking Neural Network (RSNN) system, and to simulate the behaviour of a large-scale probabilistic system. A large-scale RSNN system will be finally implemented and tested.

Pre-requisites:Experience in Matlab or C programming, Knowledge in Electronics and mixed-mode VLSI design

Professional skills obtained:Knowledge in probabilistic models, Skills in translating algorithms into VLSI implementation, Methodology in designing VLSI system-on-a-chip. 

B. Stochastic Neural VLSI for Learning Stochastic Equilibriums

Introduction :Probabilistic neural computation refers to brain-inspired algorithms that are based on computing units (called neurons) whose inputs merely decide their "output probabilities". Such probabilistic behaviour enables probabilistic models to generalise natural variability of real data by the use of its internal stochasticity. Probabilistic models thus have been shown promising for modelling noisy biomedical signals. This project investigates the feasibility of implementing an intelligent system-on-a-chip based on the Continuous Restricted Boltzmann Machine(CRBM), a probabilistic model that has been shown both useful for modeling biomedical data as its stochastic equilibriums and hardware-friendly. A programmable, scalable CRBM system will be implemented for practical biomedical applications. The robustness of a large-scale CRBM system against substrate noise and computational errors will also be investigated.

Characteristics : Matlab or C programming will be used to conclude the specifications of modular circuits for a scalable CRBM system, and to simulate the behaviour of a large-scale probabilistic system. A large-scale CRBM system will be finally implemented and tested.

Pre-requisites:Experience in Matlab or C programming, Knowledge in Electronics and mixed-mode VLSI design

Professional skills obtained:Knowledge in probabilistic models, Skills in translating algorithms into VLSI implementation, Methodology in designing VLSI system-on-a-chip. 

Fig. 1 : (a)The modular CRBM (mCRBM) microsystem enabling a large CRBM network to be formed by interconnecting multiple chips (b) The mCRBM microsystem can learn to regenerate the three types of noisy spikes and then (c) project the data into three separate clusters in three-dimensional space (d) The Diffusion Network (DN) microsystem (e) The handwritten rregenerated by the DN (f)After learning the QRS segments of normal (blue inlet) and abnormal (red inlet), the DN generate distinctly dynamics for normal and abnormal ECGs to facilitate data classification.

C. Stochastic Neural VLSI for Learning Stochastic Dynamics

Introduction :Many biomedical signals are time-variant and noisy, while the capability to recognise abnormality in such biomedical signals in real-time is crucial for improving the treatmeant on diseases such as epilepsy. The Diffusion Network (DN) has been shown as a probabilistic model capable of modeling noisy, time-variant data as its stochastic dynamics. The analogy between the Diffusion Network and the CRBM further indicates the hardware amenability of the Diffusion Network. This project investigates the capability of the Diffusion Network to model real biomedical data, and its amenability to VLSI implementation.Afterwards, a scalable DN system whose network size can be expanded by interconnecting multiple chips will be designed for general biomedical applications (e.g. data regression at brain-machine interfaces).

Characteristics : Matlab or C programming will be used to examine the capability of the Diffusion Network, to simplify the algorithm for VLSI implementation, and to simulate the behaviour of a probabilistic VLSI system based on the Diffusion Network. A prototype of the DN system will then be implemented in VLSI and tested.

Pre-requisites:Experience in Matlab or C programming, Knowledge in Electronics and mixed-mode VLSI design.

Professional skills obtained:Knowledge in probabilistic models, Skills in translating algorithms into VLSI implementation, Methodology in designing VLSI system-on-a-chip.