General Regression Neural Networks

General Regression Neural Networks (GRNN) is one of the type of neural networks with a one pass learning algorithm. The simplicity of algorithm for GRNN is one of the advantage of this neural network and many researchers use it. In one of recent paper was proposed algorithm that is using an ensemble of several General Regression Neural Networks to improve the accuracy of forecasting. [1] In another paper MOPSO algorithm was applied for improvement radial basis function (RDF) which is used also in GRNN [2]

A simple example can explain how GRNN is working. Let’s say several people decided to establish some fund. One contributed $2, another $3 and yet another $100. Now we calculate 2 numbers – one is a sum of dollar amount times the person number :
and another number is just the sum of $ amount:
Now divide 306 / 105 and we get ~ 3
Thus our algorithm calculated top contributor – person number 3. This is exactly what last 2 layers of GRNN are doing.
The output of GRNN is result of division of 2 numbers from 2 nodes of previous layer. Now you probably noticed that in the example above the top contributor ($100 contribution ) was a way ahead of others. The hidden layer in GRNN is responsible for this.

The pattern that is very close to training data will get big value while the other will get very small values. This is what RBF is doing. And the sigma parameter is used to adjust the RBF function output.

This is how GRNN is working. And at minimum it only requires one parameter sigma. Some researchers describe how to apply particle swarm optimization to find optimal number of hidden neurons. And we can use a several network to get even more computing power or better accuracy. Ensemble learning is very well described in [3].

You can find many different ways how to combine 2 or more neural networks. Feel free to grab perl code for GRNN so you can play with it. You can also look at few more data examples and some math. General Regression Neural Networks Thus the forecasting with GRNN is easy and new papers appear that do research on using RGNN

[1] Iffat A. Gheyas, Leslie S. Smith. A Neural Network Approach to Time Series Forecasting, Proceedings of the World Congress on Engineering 2009 Vol II , July 1 – 3, London, U.K.
[2] S.N. Qasem and S.M. Shamsuddin Generalization Improvement of Radial Basis Function Network Based on Multi-Objective Particle Swarm Optimization, Journal of Artificial Intelligence, 2009

Leave a Comment