12.4.3 Iterating over Trials: EpochProcess

The EpochProcess loops over the set of Events in the Environment (see section 11 Environments, Events, and Patterns). Each presentation of an event is known as a trial, and this process typically has a TrialProcess as its sub_proc, although the situation is different when the environment contains sequences of events (see section 12.6.1 Processes for Sequences of Events).

The epoch process is responsible for ordering the presentation of events to the network. Thus, at the beginning of the epoch (when the process is initialized), it tells the environment to initialize itself (using InitEvents()), and then obtains the total number of events in the environment (using EventCount(), see section 11.1 Environments). The epoch process then makes a list of event indexes, which represents the order in which events will be presented. Depending on the state of the order variable, this list will either remain sequential or be randomized.

The epoch process is also responsible for determining when to update the weights in the network, since this can usually be done either after each event or at the end of the epoch (depending on the state of the wt_update variable). The epoch process itself calls the UpdateWeights function on the network, even when it is doing updates after each event. Thus, lower-level processes should never call this function themselves.

The following variables are on the EpochProcess:

12.5 Variables

Counter trial
The number of the current trial being executed. This is the counter for the epoch process. It is automatically initialized to be the number of events in the environment.
Event* cur_event
This is a pointer to the current event being processed. After each trial, the EpochProcess updates this variable to point to the next event based on its list of event indexes. It gets the event from the environment using the GetEvent function of the environment (see section 11.1 Environments).
Order order
Controls the order in which Events are presented to the network. The values for this are:
SEQUENTIAL
Present events in sequential order (i.e. in the order they currently are in the Environment events group).
PERMUTED
Present events in permuted order. This ensures that each event is only presented once per epoch, but the order is randomized.
RANDOM
This picks an event at random (with replacement) from the list of events. This does not use the epoch process's list of events, and it allows the same event to be presented multiple times in an epoch, while other events might not be presented at all.
WtUpdate wt_update
Determines when the network's weights are updated (if at all). The possible values are:
TEST
Don't update weights at all (for testing the network). This also causes the training process to not increment the epoch counter of the network after each epoch, since the epoch counter is supposed to reflect the extent of training experience the network has had.
ON_LINE
Update the weights on-line (after every event).
BATCH
Update the weights after every epoch (batch mode).
SMALL_BATCH
Update the weights after every batch_n events. This allows an intermediate level of batch mode learning which can be parameterized independent of the number of events in the epoch.
int batch_n
Specifies the number of events between weight updates if wt_update is SMALL_BATCH.