4.2.6 Processes

Since everything has been properly initialized, and the various displays are all laid out for our use, we are ready to try to teach the network the XOR function. We train networks, and test them, by running Process objects that operate on them.

Process objects are described at length in section 12 Processes and Statistics. We will only say enough about them here to give you a general sense of what they are and what they do. You can think of processes as objects that consist of several parts: an initialization part, a loop part, and a finishing-up part. When a process is run, it first carries out its initialization process; then it loops for a specified number of iterations or until some performance criterion is met, and then it finishes up and exits. For many processes, the loop consists of simply calling the process's sub-process over and over again. At the bottom of the process tree, we call routines that actually interact with the network. Processes also spawn sub-processes that calculate statistics, such as the sum of squares statistic that measures how well the network has solved the learning problem that has been posed to it. Some statistics, called loop_stats, are calculated at the end of each step in the process loop; others are calculated only when finishing up.

In the case of backpropagation, we have a three-level process hierarchy. At the bottom of the hierarchy is the TrialProcess. This process takes an event, consisting of an input pattern and a target pattern, from the Environment, and then carries out the three phase process described previously (see section 4.1 Backpropagation and XOR). The processing is actually implemented by looping through the Layers in ascending numerical order; then looping, within each layer, through the Units, and calling functions associated with each unit to compute the quantities as previously described. All of the variables mentioned there are explicitly stored in each unit, and can be inspected once they have been computed by selecting the appropriate button in the NetView. At the end of the trial, the sum of squares statistic is calculated; in this case it isn't much of a sum since there is just one output unit.

The Trial process is actually called as a sub-process of the EpochProcess. All the Epoch process does is loop through the set of pattern pairs in the Environment, calling the trial process to process each pattern pair. Before it starts, it initializes its statistic, the sum of the trial-wise sum-squared-error (sum_sum_SE_Stat), to 0. As it loops, it computes the sum_sum_SE_Stat by aggregating the sum_SE_Stat statistic over each of the training trails, so that at the end of the epoch, it reflects the sum over all of the trials in the epoch. At the end of the epoch, it passes the sum_sum_SE_Stat up to the training process.

The Epoch process is called by the TrainProcess. This process initializes the epoch counter before it starts. It stops when the counter reaches a specified maximum value, or when the latest (or last) sum-squared-error value (lst_sum_SE_Stat) falls below its stopping criterion. The stopping criterion is actually a property of the Train process's SE_Stat statistic, lst_sum_SE_Stat. This statistic simply copies the last sum_sum_SE_Stat from the epoch process, and compares it to the criterion, passing a special return value to the Train process when the criterion is met. If so, the process exits from the loop at that point, and training is stopped.