|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectclarion.system.AbstractImplicitModule
clarion.system.AbstractTrainableImplicitModule
clarion.system.AbstractNeuralNet
clarion.system.BPNet
clarion.system.AbstractRuntimeTrainableBPNet
clarion.system.SimplifiedQBPNet
public class SimplifiedQBPNet
This class implements a simplified Q-learning backpropagating neural network within CLARION. It extends the AbstractRuntimeTrainableBPNet class.
Usage:
A simplified Q-learning backpropagating neural network can be taken advantage of when reinforcement to train the network is recieved at every time step. This class is capable of performing learning during runtime.
Note that the weight vectors and thresholds for the network can be hardcoded if they were recorded from a previous training session (just like a standard backpropagating neural network).
The Input for the neural network is a collection of dimension-values where each value represents one node.
If the network is being used as an action network in the ACS and you are using goals or specialized working memory chunks, remember that the input space must also contain all dimension-value pairs within those chunks that differ from the sensory information space.
Once the input space has been defined it cannot be changed. At any given forward pass through the network, the network can accept an arbitrary collection of dimension-values as input, but it will only adjust the activations of the inputs that were specified during initialization.
The nodes in the output layer are represented as output chunks.
The general procedure when using this class is:
Note: The current implementation of CLARION does not allow for multiple action dimensions on the bottom level as stipulated in the CLARION tutorial. However, actions that contain multiple action dimensions are still possible by simply specifying action chunks on the top level that contain multiple activated action dimensions. This is the case because the output of the bottom level only specifies actions and not action dimension-values.
Field Summary |
---|
Fields inherited from class clarion.system.AbstractRuntimeTrainableBPNet |
---|
Feedback, GLOBAL_POSITIVE_MATCH_THRESHOLD, NM, PM, POSITIVE_MATCH_THRESHOLD |
Fields inherited from class clarion.system.BPNet |
---|
GLOBAL_LEARNING_RATE, GLOBAL_MOMENTUM, GLOBAL_RZERO, LEARNING_RATE, MOMENTUM, RZERO |
Fields inherited from class clarion.system.AbstractNeuralNet |
---|
GLOBAL_LOWER_INIT_THRESHOLD, GLOBAL_LOWER_INIT_WEIGHT, GLOBAL_UPPER_INIT_THRESHOLD, GLOBAL_UPPER_INIT_WEIGHT, Hidden, HiddenThresholds, HiddenToOutputWeights, InputToHiddenWeights, LOWER_INIT_THRESHOLD, LOWER_INIT_WEIGHT, OutputThresholds, UPPER_INIT_THRESHOLD, UPPER_INIT_WEIGHT |
Fields inherited from class clarion.system.AbstractTrainableImplicitModule |
---|
DesiredOutput |
Fields inherited from class clarion.system.AbstractImplicitModule |
---|
ACTUATION_TIME, ChosenOutput, DECISION_TIME, GLOBAL_ACTUATION_TIME, GLOBAL_DECISION_TIME, GLOBAL_PERCEPTION_TIME, InputAsCollection, Output, PERCEPTION_TIME |
Constructor Summary | |
---|---|
SimplifiedQBPNet(java.util.Collection<Dimension> InputSpace,
int NumHidden,
AbstractOutputChunkCollection<? extends AbstractOutputChunk> Outputs)
Initializes a backpropagating neural network that uses simplified Q-Learning for training the network. |
Method Summary | |
---|---|
void |
backwardPass()
Updates the neural network using simplified Q-Learning. |
boolean |
checkMatchCriterion()
Checks to see if the positive match criterion is satisfied given the immediate feedback received. |
Methods inherited from class clarion.system.AbstractRuntimeTrainableBPNet |
---|
getFeedback, getNM, getPM, incrementNM, incrementPM, resetMatchStatistics, setFeedback, setNM, setPM, updateMatchStatistics |
Methods inherited from class clarion.system.BPNet |
---|
computeHiddenActivation, computeOutputActivation, modifyHiddenToOutput, modifyInputToHidden |
Methods inherited from class clarion.system.AbstractNeuralNet |
---|
forwardPass, getHiddenThresholds, getHtoOWeightMatrix, getItoHWeightMatrix, getNumHidden, getOutputThresholds, hardcodeWeights |
Methods inherited from class clarion.system.AbstractTrainableImplicitModule |
---|
getSumSqErrors, setDesiredOutput, setDesiredOutput |
Methods inherited from class clarion.system.AbstractImplicitModule |
---|
getChosenOutput, getInput, getNumInput, getNumOutput, getOutput, getOutput, getResponseTime, setChosenOutput, setInput, setInput, setInput |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Methods inherited from interface clarion.system.InterfaceTrainable |
---|
getSumSqErrors, setDesiredOutput, setDesiredOutput |
Constructor Detail |
---|
public SimplifiedQBPNet(java.util.Collection<Dimension> InputSpace, int NumHidden, AbstractOutputChunkCollection<? extends AbstractOutputChunk> Outputs)
The Input for the neural network is a collection of dimension-values where each value represents one node.
If the network is being used as an action network in the ACS and you are using goals or specialized working memory chunks, remember that the input space must also contain all dimension-value pairs within those chunks that differ from the sensory information space.
Once input has been set it cannot be changed. At any given forward pass through the network, the network can accept an arbitrary collection of dimension-values as input, but it will only adjust the activations of the inputs that were specified during initialization.
The nodes in the output layer are represented as output chunks.
InputSpace
- A collection of dimension-value pairs to set as the input nodes.NumHidden
- The number of hidden nodes.Outputs
- The chunks to associate with the output layer.Method Detail |
---|
public void backwardPass()
backwardPass
in interface InterfaceTrainable
backwardPass
in class BPNet
public boolean checkMatchCriterion()
This check is usually performed before the backwardPass method is called but after the setFeedback and setChosenAction methods have been called.
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |