next up previous
Next: Parallel Neural Networks Up: Time Series(TS) Previous: Evaluation with the neurobasic



Neural Networks is a field with multiple applications, which has not by any means been exhausted by the studies and research that have been done until now. This model of problem representation is interesting especially because of the fact that it tries to approach the human way of perceiving the surrounding world. The artificial neuron is a simplified expression, reduced to its essence and to what were considered to be its basic features, of the biological neuron.

This part researches a possible applications of the neural network and intends to be a study of the problem from both a theoretical and a technical point of view.

I described the development and usage of an energy function that reduces the sensitivity of the network in respect to minor errors, based on the Lyapunov infinite norm concept. From this model, a ANN was designed and a SE forecasting tool was constructed, using the TS behaviour of SE.

I compared the network constructed based on this energy function with a network using the classical Backpropagation method and showed the differences between the two.

The proposed energy function is less sensitive to minor errors when compared with the standard LMS-function of the backpropagation method.

This can be seen from the theoretical deductions, due to the usage of the maximum error sphere and the resulting dead-zone delimiters, that have the effect that the system does not react when within these dead-zones, meaning while errors are small. Therefore from their very definition, these movement equations lead to the expectation that the weight changing will have less oscillations and will converge quicker.

From the technical point of view, I did some experiments to validate this theory and to compare backpropagation and my L-method, with the help of two indicators, correlation and random walk, and showed on a number of test sets that my method is better with regard to these coefficients, so it is correlated closer to the real output and further away from the random walk.

Also, comparison of MSE development in time for different problems and different sizes of the same problem were done. With more tuning parameters than classical BP, the convergence can be adjusted better to the particular problem case.

Thus, I proved from theoretical and practical point of view that my method is superior to the traditional BP for SE forecasting.

For further work I intend to extend the system and find out what other problems it is applicable to. In constructing the new function I exploited the TS structure of SE events, therefore I expect that it can be useful for other problems which deal with TS forecasting.

The choice of the neural network is the most important step that influences the total amount of program execution time. For some problems, the best-fit network is relatively easy to find out, only by translating the requirements of the problem into a network. Still, there are some problems for which the right choice of an optimum neural net is quite difficult. Nevertheless, the programmer can guide himself by following some of the common features of the problem solving with neural networks: all start from the optimization of an energy function, which, simplified, can show the way to a particular optimization network.

The complexity of a problem is determined by the number of iterations (computational steps) that have to be followed in order to find the solution of the problem. The complexity of a problem is strongly connected to the implementation time of the problem. The complexity of a parallel network is given by the maximum number of computations done on the "busiest" neuron, i.e., of the one that has the maximum amount of computations to perform, if one considers that all the neurons in the network are running completely independently and in parallel. In the case in which the neurons are not executed all in parallel, for instance, because of the fact that they are interdependent, and there exist neurons that can only start computing after a first group of neurons have finished their work, these neuron groups are then grouped in structures called layers. The complexity of a network with N layers is given by the sum of the maximum computational times of the neurons in each of the layers.

The UNIX operating system, which was used for implementing the problem, permits only the simulation of computational parallelism. The proposed simulation solution implements a multitude of parallel mechanisms, parallelization of the processing jobs (the execution of the neurons on virtual parallel processors, the job division between different processes through the master-slave mechanism, the implementation of various communication mechanisms for the processes that share a workstation). Still, the UNIX system does not offer a real type of parallelism, so that a study of the real execution time of the implemented method could not be applied. Nevertheless the convergence time, for a reasonable dimensioned problem, was quite acceptable for real time requirements. The next part will detail these thoughts and show some experiments.

For the user interface, the MOTIF tool was chosen, which can be used on a number of different work stations, such as HP (Hewlett Packard), DEC, IBM. This interface offers a set of display windows for both graphics and text display, as well as a set of buttons and other different display options, compatible with the UNIX programming.

Motif is built of a nucleus of libraries and graphical functions, that are attached to the set of C-libraries and C-functions under UNIX. With the help of MOTIF, a user friendly interface was introduced between the computational neural network background and the user, with a multitude of help buttons that provide information about the options that can be accessed at a given moment of time.

The program has a project structure, of program modules, compiled till they reach the object stage. These objects have the connections with the specific libraries included and are linked with the link editor under C++ (the CC program, under UNIX). For the clarity of the presentation, the functions in the ANNEX have not been archived into libraries, although a former version of the program used this method of code reduction.

next up previous
Next: Parallel Neural Networks Up: Time Series(TS) Previous: Evaluation with the neurobasic

Alexandra Cristea
Tue Feb 9 20:20:27 JST 1999