Starting New Network Training

A new network is initialized by setting all processing node weights to random values. Qnet does this automatically for new networks. The reset weights option can also be selected for any network during setup or training. When initiating the training process for a new network, decisions must be made regarding training parameter values.

First, a seed value for the learning rate must be chosen during training setup. A learn rate value in the 0.005 to 0.1 range usually works well for new networks. If the initial guess turns out to be too high and the network diverges, simply reset the network by selecting “Options/Initialize-Reset Weights” from the training window’s menu bar. Select "Options/Set Learn Rate" to try a lower learning rate value.

A second consideration for new networks concerns the iteration number at which LRC should be activated. When training begins with a new network, the training error can oscillate wildly. This is normal behavior for new networks. If Qnet’s LRC option is active during these initial oscillations, the learn rate will be lowered in an attempt to eliminate them. This can slow training by driving eta to an artificially low value. To prevent this from occurring, set the “Learn Rate Control Start Iteration” item in the training parameters setup dialog window to at least 50 or 100 iterations. The LRC option can also be turned on and off interactively during the training process.

Other Qnet training parameters can normally be kept at their default values. The parameters include: the FAST-Prop Coefficient, the minimum and maximum learn rate settings for LRC (LRC will adjust these from their defaults as needed), the momentum factor (alpha), the patterns per weight update, screen update rate, and the AutoSave rate. All parameters can be adjusted during the training process if required.