10.4.1 Implementational Details About Units

The base UnitSpec defines a standardized way of splitting up the computations that take place on a unit. This allows there to be a single function at the level of the Network which iterates through all of the layers and they iterate through all of their units and call these functions. This makes writing process code easier, and provides a conceptual skeleton on which to implement different algorithms. Note that the unit has simple "stub" versions of these functions which simply call the corresponding one on the spec. This also simplifies programming.

InitState(Unit* u)
Initializes the unit's state variables, including activations, net input, etc.
InitWtDelta(Unit* u)
Initializes the stored connection weight changes (i.e., changes that have not yet been applied to the weights).
InitWtState(Unit* u)
Initializes the connection weight values for all of the receiving connections of the unit.
Compute_Net(Unit* u)
Iterates over the receiving connections of the unit and sets the unit's net field to the summed product of the sending unit's activation value times the weight.
Send_Net(Unit* u)
Iterates over the sending connections of the unit and increments the net field of the unit's it sends to. This way of computing net input is useful when not all units send activation (i.e., if there is a threshold for sending activation or "firing"). A given algorithm will either use Compute_Net or Send_Net, but not both.
Compute_Act(Unit* u)
Turns the net input value into an activation value, typically by applying a sigmoidal activation function, but this varies depending on the particular algorithm used. The version in the base UnitSpec just copies the net input into the activation (i.e., it is linear).
Compute_dWt(Unit* u)
Iterates over the receiving connections on the unit and calls the Compute_dWt function on them, which should compute the amount that the weights should be changed based on the current state of the network, and a learning rule which translates this into weight changes. It should always add an increment the current weight change value, so that learning can occur either pattern-by-pattern ("online" mode) or over multiple patterns ("batch" mode).
UpdateWeights(Unit* u)
Actually updates the weights by adding the changes computed by Compute_dWt to the weights, applying learning rates, etc. This function should always reset the weight change variable after it updates the weights, so that Compute_dWt can always increment weight changes. Note that this function is called by the EpochProcess, which decides whether to perform online or batch-mode learning (see section 12.4.3 Iterating over Trials: EpochProcess).