Training Error

Monitoring the error histories is the quickest way to determine the training progress of a network. Along with the visual readout of both training and test set errors, complete error histories for the run can be obtained with NetGraph. This provides the modeler with valuable information about the progress and the relative state of a network’s convergence. NetGraph’s AutoZoom feature can be particularly helpful when viewing error histories. AutoZoom allows the modeler to zoom in on convergence trends that might be obscured by scaling.

By monitoring the training set’s RMS error, the modeler can determine the pace of network learning, the frequency of instabilities and the general state of convergence (or divergence). Qnet’s training algorithms attempt to drive the training set error to a minimum value. As a network reaches this converged or steady state, the error value will approach a minimum value.

By monitoring the test set’s RMS error, the modeler can determine the overtraining status of the network and how well the network responds to cases not contained in the training set. For some networks, the test set’s error will simply decline along with the training error to some minimum value. Another possibility is that it will decline, reach a minimum, and then increase indefinitely thereafter, even though the training set’s error continues to decrease. For this case, the model should only be trained to the point of the test set’s minimum error. When the test set error begins to increase it can be assumed that memorization is predominating and overtraining has begun. Unfortunately, determining this point is not that simple. False or local minimums may occur in the test set error. These local minimums indicate that some mix of the learning is taking place. Some networks may exhibit long periods of training where the test set error increases before declining again. The figure depicts some possible scenarios. (Note: To minimize the effect of test set error computations on training speed, Qnet computes the error value during screen update iterations only. Setting large screen update intervals will limit your ability to monitor the test set error.)

If a test set’s error begins to increase, training should be continued to determine whether the minimum is local or global in nature. The AutoSave feature of Qnet allows you to return to a point at or near the minimum if it is global in nature. AutoSave will store the network at selected intervals during training. This interval can be specified during setup or training. To retrieve an AutoSave network, select “File/Save AutoSave File...” from the training menu. You will be asked to specify a network file name and an iteration value (from a list of iterations at which the network was saved). Select the iteration that is nearest, but still prior to the start of overtraining. If additional training is required from this iteration, exit the current training session and open the newly saved network file for training.

When either the training set error or test set error begins to increase while using the FAST-Prop training algorithm, return to standard backpropagation by setting the FAST-Prop coefficient to zero. While the FAST-Prop method of training can accelerate the learning process, this training method can at times decrease stable learning qualities. The training and/or test set errors may start to increase or fluctuate. Standard backpropagation does not normally exhibit this behavior.