The 1-Norm Problem

To compare the performance of the backpropagation and the combined algorithm on large and highly nonlinear problems, both algorithms were used to train a network for the 1-Norm of a 7-dimensional binary vector to an error of 0.001. 4 runs of the sequential versions of both algorithms have been performed.

Backpropagation

All 4 programs used online learning at a learn rate of 1. The impulse constants 0 and 0.8 were used on two programs each. The simulations were started by the following commands.

cntback -N7 -O1 -e0.001 -f1 -g250000 -o1
cntback -N7 -O1 -e0.001 -f1 -g250000 -o1 -a0.8

Both simulations with $\alpha=0.8$ failed, while the other two programs succeeded after 55097 and 45974 iterations (11661 s and 9800 s)

Genetic Backpropagation

The combined algorithm used a population size of 50 with 5 backpropagation steps per generation and was started by the following command

cntseq -N7 -O1 -e0.001 -f1 -g1000 -p50 -b5

All 4 programs succeeded after an average of 240 generations and an average computation time of 18797.5 s. The following weight matrices $W^{(1)}$ and $W^{(2)}$ were found by one of the programs:

$$W^{(1)}=\left(\begin{array}{r r r r r r r r} 10.00& 9.06& -6.... ...00& -4.27& -2.94& -3.57& -3.73& 2.86& 9.45 \end{array}\right)$$

$$W^{(2)}=\left(\begin{array}{r r r r r r r r} 5.22& 10.00& 8.67& 6.16& -6.16& -9.29& -10.00& -2.31 \end{array}\right)$$

Since the encoding range of the weights is by default defined as $[-10,10]$, the occurrences of $10$ and $-10$ indicate range overflows caused by the backpropagation steps.

Considering that the parallel version at this problem size ($t=256$) would run with an efficiency of effectively 100%, the combined algorithm would definitely be the better choice for this problem.