SPUR Project

2011 winner (10 of 20)

Learning temporal dependencies using neural networks: a comparison of different neural networks for predicting chaotic time series

Supervisors: Dr Jonathan Tepper, Dr Heather Powell (both Computing and Technology)

Student: Steven Batchelor-Manning

Learning to capture temporal dependencies within a linear input sequence is essential for many tasks such as weather forecasting, stock market prediction, and speech recognition. It is also important for modelling cognitive processes such as sentence comprehension. Artificial neural networks (ANNs) represent a class of computational models that have been widely applied to the problem of learning temporal dependencies. ANNs are loosely based on the functioning of biological neurons in animal brains. There are two ANN connection topologies, feedforward neural networks (e.g. FF-MLPs) and recurrent neural networks (RNNs) and both have proved popular for temporal applications due to their strength in handling non-linear functional dependencies. Recent advances in RNN research has seen the advent of a new class of RNNs referred to as the reservoir computing (RC) paradigm which have proved to be very effective for addressing temporal problems. The project sought to discern when an RNN can achieve better results than an FF-MLP, at what point the gradient descent RNNs begin to fail and whether the benefits in performance offered by the RC models are statistically significant. Student tasks included conducting experiments on chaotic financial time series data and investigating the efficacy of a number of popular feedforward and recurrent neural network architectures to learn the task of forecasting US inflation. The student also assisted with analysis and reporting.

.