Learn Rate Control

Toggle Learn Rate Control (LRC) on and off. The learn rate coefficient can be controlled manually during training or Qnet can control it automatically using the LRC feature. LRC will drive eta higher or lower in a systematic fashion depending on the current learning activity. If the network appears to be learning at a relatively slow rate, eta is driven up quickly. Conversely, if the network is learning at a fast pace, Qnet will hold eta constant or even lower it to avoid instabilities. If at any time the network shows signs of instability (seen as oscillations in the training error), eta is lowered to damp the instabilities. Damping instabilities is critical to preventing training divergence. The LRC feature can be turned on and off interactively during the training process, and it can be activated at setup time by specifying the iteration number that LRC will start (for new networks should wait several hundred iterations prior to turning learn rate control on.)

Note: For the many network designs and data models, LRC is an effective tool that accelerates learning and prevents divergence. If a model exhibits poor learning characteristics with LRC active (i.e. training divergences,, instabilities), simply turn LRC off (Options menu of the training window). Take manual control of eta and set it to a value low enough for stable, sustained learning. In addition, when all training cases are not used in the weight update cycle (Patterns per Weight Update Cycle), LRC should be off. Error descent in this case can be rather noisy and training characteristics can be adversely affected by varying the learn rate.

See Learn Rates and Learn Rate Control for more information