|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectclarion.system.AbstractSubsystem
clarion.system.ACS
public final class ACS
This class implements the Action Centered Subsystem (ACS) within CLARION. It extends the AbstractSubsystem class.
Usage: The ACS is the subsystem that is minimally required in order for a CLARION agent to run. It MUST be fully setup within a CLARION agent before the CLARION agent can perform within a task environment.
Note that simply initializing the ACS is not sufficient enough to setup the ACS. At the very least you MUST also add at least one implicit module to the bottom level of the ACS along with all of the actions associated with the output layer of that module.
The PRIMARY methods for the ACS are the "learn" and "chooseAction" methods and they do precisely what their names suggest.
Note that the ACS is initialized with already built in "default" external, working memory, and goal actions:
This class contains MANY parameters for handling different aspects of the ACS from level selection options/methods to triggers for turning on and off various "automatic" aspects of the ACS (such as extraction and refinement). If you are having trouble determining where to locate a parameter you need to set, this is not a bad place to start looking (if the parameter relates to the ACS, of course).
This class contains both global (static) and local constants. The default is to use the local constants. If you want to change any of the global constants, you need to do so before any instances of this class are initialized.
Nested Class Summary | |
---|---|
static class |
ACS.LevelSelectionMethods
The various methods that can be used for level selection (such as stochastic or combined) |
static class |
ACS.LevelSelectionOptions
The options for level selection (such as fixed or variable) |
private static class |
ACS.ModuleTypes
The various different types of modules used for action selection (BL,RER,IRL,FR) |
Field Summary | |
---|---|
private ImplicitModuleCollection |
BLImplicitModuleStore
The bottom level implicit modules. |
private AbstractAction |
ChosenAction
The chosen action. |
private DimensionValueCollection |
CurrentInput
The current input (including current sensory information, current goal, chunks in working memory, and drive stimulus). |
double |
EXTERNAL_PROBABILITY
The probability of choosing an external action. |
RuleExtractor |
Extractor
The rule extractor for rule extraction. |
double |
FIXED_BL_COMB_WEIGHT
The fixed bottom level weight when level combination is used. |
double |
FIXED_BL_PROBABILITY
The fixed probability of using the bottom level. |
double |
FIXED_FR_COMB_WEIGHT
The fixed weight for a fixed rule when level combination is used. |
double |
FIXED_FR_PROBABILITY
The fixed probability of using a fixed rule. |
double |
FIXED_IRL_COMB_WEIGHT
The fixed IRL rule weight when level combination is used. |
double |
FIXED_IRL_PROBABILITY
The fixed probability of using an IRL rule. |
double |
FIXED_RER_COMB_WEIGHT
The fixed RER rule weight when level combination is used. |
double |
FIXED_RER_PROBABILITY
The fixed probability of using an RER rule. |
private RuleCollection |
FRRuleStore
The FR rule store. |
static double |
GLOBAL_EXTERNAL_PROBABILITY
The probability of choosing an external action. |
static double |
GLOBAL_FIXED_BL_COMB_WEIGHT
The fixed bottom level weight when level combination is used. |
static double |
GLOBAL_FIXED_BL_PROBABILITY
The fixed probability of using the bottom level. |
static double |
GLOBAL_FIXED_FR_COMB_WEIGHT
The fixed weight for a fixed rule when level combination is used. |
static double |
GLOBAL_FIXED_FR_PROBABILITY
The fixed probability of using a fixed rule. |
static double |
GLOBAL_FIXED_IRL_COMB_WEIGHT
The fixed IRL rule weight when level combination is used. |
static double |
GLOBAL_FIXED_IRL_PROBABILITY
The fixed probability of using an IRL rule. |
static double |
GLOBAL_FIXED_RER_COMB_WEIGHT
The fixed RER rule weight when level combination is used. |
static double |
GLOBAL_FIXED_RER_PROBABILITY
The fixed probability of using an RER rule. |
static double |
GLOBAL_GOAL_PROBABILITY
The probability of choosing a goal action. |
static ACS.LevelSelectionMethods |
GLOBAL_LEVEL_SELECTION_METHOD
Specifies the method to use for level selection. |
static ACS.LevelSelectionOptions |
GLOBAL_LEVEL_SELECTION_OPTION
Specifies the option to use for level selection. |
static double |
GLOBAL_MATCH_DISCOUNT
The match discount factor. |
static double |
GLOBAL_MCS_A
The constant A used to factor in the MCS measure when calculating the probability or weight of for the bottom level during level selection. |
static double |
GLOBAL_MCS_B
The constant B used to factor in the MCS measure when calculating the probability or weight of for the bottom level during level selection. |
static boolean |
GLOBAL_PERFORM_BL_LEARNING
Specifies whether or not bottom level learning should be performed. |
static boolean |
GLOBAL_PERFORM_EXTRACTION
Specifies whether or not extraction should be used. |
static boolean |
GLOBAL_PERFORM_REFINEMENT_IRL
Specifies whether or not refinement should be used for IRL rules. |
static boolean |
GLOBAL_PERFORM_REFINEMENT_RER
Specifies whether or not refinement should be used for RER rules. |
static double |
GLOBAL_VARIABLE_BL_BETA
The BL beta when using variable stochastic level selection. |
static double |
GLOBAL_VARIABLE_BL_COMB_WEIGHT_BETA
The BL beta when using the variable weighted combination. |
static double |
GLOBAL_VARIABLE_C3
The C3 constant used for variable level selection. |
static double |
GLOBAL_VARIABLE_C4
The C4 constant used for variable level selection. |
static double |
GLOBAL_VARIABLE_FR_BETA
The FR beta when using variable stochastic level selection. |
static double |
GLOBAL_VARIABLE_FR_COMB_WEIGHT_BETA
The FR beta when using the variable weighted combination. |
static double |
GLOBAL_VARIABLE_IRL_BETA
The IRL beta when using variable stochastic level selection. |
static double |
GLOBAL_VARIABLE_IRL_COMB_WEIGHT_BETA
The IRL beta when using the variable weighted combination. |
static double |
GLOBAL_VARIABLE_RER_BETA
The RER beta when using variable stochastic level selection. |
static double |
GLOBAL_VARIABLE_RER_COMB_WEIGHT_BETA
The RER beta when using the variable weighted combination. |
static double |
GLOBAL_WM_PROBABILITY
The probability of choosing a working memory action. |
double |
GOAL_PROBABILITY
The probability of choosing a goal action. |
private RuleCollection |
IRLRuleStore
The IRL rule store. |
ACS.LevelSelectionMethods |
LEVEL_SELECTION_METHOD
Specifies the method to use for level selection. |
ACS.LevelSelectionOptions |
LEVEL_SELECTION_OPTION
Specifies the option to use for level selection. |
static double |
MATCH_DISCOUNT
The match discount factor. |
private java.util.HashMap<java.lang.Object,AbstractRule> |
MatchAllRules
Contains the hidden match all rules for all actions associated with the rules in the rule collections. |
AbstractMatchCalculator |
MatchCalculator
The match calculator to use for updating match statistics for ALL components located within the ACS. |
double |
MCS_A
The constant A used to factor in the MCS measure when calculating the probability or weight of for the bottom level during level selection. |
double |
MCS_B
The constant B used to factor in the MCS measure when calculating the probability or weight of for the bottom level during level selection. |
protected double |
MCS_BL_PROBABILITY
The probability measure specified by the probability setting module in the MCS for the bottom level of the ACS when stochastic level selection is used. |
protected double |
MCS_BL_WEIGHT
The weight measure specified by the probability setting module in the MCS for the bottom level of the ACS when weighted combination is used. |
protected double |
MCS_TL_PROBABILITY
The probability measure specified by the probability setting module in the MCS for the top level of the ACS when stochastic level selection is used. |
protected double |
MCS_TL_WEIGHT
The weight measure specified by the probability setting module in the MCS for the top level of the ACS when weighted combination is used. |
private ACS.ModuleTypes |
ModuleUsed
The module type that was last used to choose the action. |
boolean |
PERFORM_BL_LEARNING
Specifies whether or not bottom level learning should be performed. |
boolean |
PERFORM_EXTRACTION
Specifies whether or not extraction should be used. |
boolean |
PERFORM_REFINEMENT_IRL
Specifies whether or not refinement should be used for IRL rules. |
boolean |
PERFORM_REFINEMENT_RER
Specifies whether or not refinement should be used for RER rules. |
private ActionCollection |
PossibleActions
The collection of possible actions |
RuleRefiner |
Refiner
The rule refiner for rule refinement. |
private RuleCollection |
RERRuleStore
The RER rule store. |
StochasticSelector |
Selector
The stochastic selector for rule selection. |
private java.util.Collection<AbstractAction> |
shortlist
The subset of possible actions that were involved in the last action decision making stochastic selection. |
double |
VARIABLE_BL_BETA
The BL beta when using variable stochastic level selection. |
double |
VARIABLE_BL_COMB_WEIGHT_BETA
The BL beta when using the variable weighted combination. |
double |
VARIABLE_C3
The C3 constant used for variable level selection. |
double |
VARIABLE_C4
The C4 constant used for variable level selection. |
double |
VARIABLE_FR_BETA
The FR beta when using variable stochastic level selection. |
double |
VARIABLE_FR_COMB_WEIGHT_BETA
The FR beta when using the variable weighted combination. |
double |
VARIABLE_IRL_BETA
The IRL beta when using variable stochastic level selection. |
double |
VARIABLE_IRL_COMB_WEIGHT_BETA
The IRL beta when using the variable weighted combination. |
double |
VARIABLE_RER_BETA
The RER beta when using variable stochastic level selection. |
double |
VARIABLE_RER_COMB_WEIGHT_BETA
The RER beta when using the variable weighted combination. |
protected WorkingMemory |
WM
Points to the working memory from the instance of the CLARION class to which this instance of the ACS is attached. |
double |
WM_PROBABILITY
The probability of choosing a working memory action. |
Fields inherited from class clarion.system.AbstractSubsystem |
---|
InputSpace |
Constructor Summary | |
---|---|
ACS(CLARION Agent)
Minimally initializes the ACS. |
Method Summary | |
---|---|
void |
addAction(AbstractAction act)
Adds an action to the list of possible actions and generates a match all rule for the specified action. |
void |
addActions(java.util.Collection<? extends AbstractAction> acts)
Adds a collection of actions to the list of possible actions and generates a match all rule for all the actions that were added. |
void |
addBLModule(AbstractImplicitModule im)
Adds an implicit module to the bottom level of the ACS. |
void |
addBLModules(java.util.Collection<? extends AbstractImplicitModule> ims)
Adds a collection of implicit modules to the bottom level of the ACS. |
void |
addRule(AbstractRule R)
Adds a rule to the rule store. |
void |
addRules(java.util.Collection<? extends AbstractRule> R)
Adds a collection of rules to the rule store. |
protected void |
attachSelfToAgent(CLARION Agent)
Attaches the ACS to the specified CLARION agent. |
protected AbstractAction |
chooseAction(DimensionValueCollection CurrInput,
long TimeStamp)
Chooses an action given the current input (which includes sensory information, the current goal, and any chunks in working memory) and the previously chosen action. |
protected void |
endEpisode()
Performs the appropriate end of episode instructions for the ACS. |
private void |
generateMatchAllRule(AbstractAction act)
Generates a match all rule associated with the specified action and adds it to the list of match all rules. |
java.util.Collection<AbstractImplicitModule> |
getBLImplicitModules()
Gets all of the bottom level implicit modules. |
ImplicitModuleCollection |
getBLImplicitModuleStore()
Gets the collection holding the bottom level implicit modules. |
protected AbstractAction |
getChosenAction()
Get the last action chosen by the ACS. |
java.util.Collection<AbstractRule> |
getFixedRules()
Gets all of the fixed rules in the rule store. |
RuleCollection |
getFixedRuleStore()
Gets the subset of the rule store that holds the FR rules. |
java.util.Collection<AbstractRule> |
getIRLRules()
Gets all of the IRL rules in the rule store. |
RuleCollection |
getIRLRuleStore()
Gets the subset of the rule store that holds the IRL rules. |
private AbstractRule |
getMatchAllRule(AbstractAction act)
Gets the match all rule associated with the specified action. |
java.util.Collection<AbstractAction> |
getPossibleActions()
Gets all of the possible actions. |
java.util.Collection<AbstractRule> |
getRERRules()
Gets all of the RER rules in the rule store. |
RuleCollection |
getRERRuleStore()
Gets the subset of the rule store that holds the RER rules. |
protected void |
learn(DimensionValueCollection NewInput,
double feedback,
long TimeStamp)
Performs bottom level learning, rule refinement, and match statistic updates based on the specified new input. |
protected void |
learn(DimensionValueCollection NewInput,
long TimeStamp)
Performs bottom level learning, rule refinement, and match statistic updates based on the specified new input. |
private void |
performCombination(java.lang.Long TimeStamp)
Combines the outcomes of the top and bottom levels when level combination is being used. |
protected void |
resetChosenAction()
Resets the chosen action. |
private RuleCollection |
selectType()
Selects a rule type to use on the top level when performing action decision making with stochastic level selection. |
private void |
updateBottomLevel(DimensionValueCollection NewInput,
double feedback)
Updates the bottom level implicit representations (if they support online learning) given the new sensory information and feedback (if any was received). |
private void |
updateMatchAllStats(AbstractAction ChosenAction,
double feedback,
AbstractMatchCalculator MatchCalculator)
Updates the positive or negative match statistics for the match all rule corresponding to the chosen action. |
private void |
updateRules(double feedback,
long TimeStamp)
Updates the match statistics for the rules in the rule store (if its type was used) and tries to refine those rules in the rule store whose type (i.e. |
Methods inherited from class clarion.system.AbstractSubsystem |
---|
updateInputSpace |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
private RuleCollection RERRuleStore
private RuleCollection IRLRuleStore
private RuleCollection FRRuleStore
private ImplicitModuleCollection BLImplicitModuleStore
private ActionCollection PossibleActions
private java.util.HashMap<java.lang.Object,AbstractRule> MatchAllRules
protected WorkingMemory WM
private AbstractAction ChosenAction
private ACS.ModuleTypes ModuleUsed
private DimensionValueCollection CurrentInput
public StochasticSelector Selector
public RuleExtractor Extractor
public RuleRefiner Refiner
public AbstractMatchCalculator MatchCalculator
public static ACS.LevelSelectionMethods GLOBAL_LEVEL_SELECTION_METHOD
public static ACS.LevelSelectionOptions GLOBAL_LEVEL_SELECTION_OPTION
public ACS.LevelSelectionMethods LEVEL_SELECTION_METHOD
public ACS.LevelSelectionOptions LEVEL_SELECTION_OPTION
public static double GLOBAL_VARIABLE_C3
public double VARIABLE_C3
public static double GLOBAL_VARIABLE_C4
public double VARIABLE_C4
protected double MCS_BL_PROBABILITY
protected double MCS_BL_WEIGHT
protected double MCS_TL_PROBABILITY
protected double MCS_TL_WEIGHT
public static double GLOBAL_MCS_A
public static double GLOBAL_MCS_B
public double MCS_A
public double MCS_B
public static double GLOBAL_FIXED_BL_PROBABILITY
public static double GLOBAL_FIXED_BL_COMB_WEIGHT
public double FIXED_BL_PROBABILITY
public double FIXED_BL_COMB_WEIGHT
public static double GLOBAL_VARIABLE_BL_BETA
public static double GLOBAL_VARIABLE_BL_COMB_WEIGHT_BETA
public double VARIABLE_BL_BETA
public double VARIABLE_BL_COMB_WEIGHT_BETA
public static double GLOBAL_FIXED_RER_PROBABILITY
public static double GLOBAL_FIXED_RER_COMB_WEIGHT
public double FIXED_RER_PROBABILITY
public double FIXED_RER_COMB_WEIGHT
public static double GLOBAL_VARIABLE_RER_BETA
public static double GLOBAL_VARIABLE_RER_COMB_WEIGHT_BETA
public double VARIABLE_RER_BETA
public double VARIABLE_RER_COMB_WEIGHT_BETA
public static double GLOBAL_FIXED_IRL_PROBABILITY
public static double GLOBAL_FIXED_IRL_COMB_WEIGHT
public double FIXED_IRL_PROBABILITY
public double FIXED_IRL_COMB_WEIGHT
public static double GLOBAL_VARIABLE_IRL_BETA
public static double GLOBAL_VARIABLE_IRL_COMB_WEIGHT_BETA
public double VARIABLE_IRL_BETA
public double VARIABLE_IRL_COMB_WEIGHT_BETA
public static double GLOBAL_FIXED_FR_PROBABILITY
public static double GLOBAL_FIXED_FR_COMB_WEIGHT
public double FIXED_FR_PROBABILITY
public double FIXED_FR_COMB_WEIGHT
public static double GLOBAL_VARIABLE_FR_BETA
public static double GLOBAL_VARIABLE_FR_COMB_WEIGHT_BETA
public double VARIABLE_FR_BETA
public double VARIABLE_FR_COMB_WEIGHT_BETA
public static double GLOBAL_EXTERNAL_PROBABILITY
public static double GLOBAL_GOAL_PROBABILITY
public static double GLOBAL_WM_PROBABILITY
public double EXTERNAL_PROBABILITY
public double GOAL_PROBABILITY
public double WM_PROBABILITY
public static boolean GLOBAL_PERFORM_EXTRACTION
public boolean PERFORM_EXTRACTION
public static boolean GLOBAL_PERFORM_BL_LEARNING
public boolean PERFORM_BL_LEARNING
public static boolean GLOBAL_PERFORM_REFINEMENT_RER
public boolean PERFORM_REFINEMENT_RER
public static boolean GLOBAL_PERFORM_REFINEMENT_IRL
public boolean PERFORM_REFINEMENT_IRL
public static double GLOBAL_MATCH_DISCOUNT
public static double MATCH_DISCOUNT
private java.util.Collection<AbstractAction> shortlist
Constructor Detail |
---|
public ACS(CLARION Agent)
Note that simply calling this constructor is not sufficient enough to setup the ACS. At the very least you MUST also add at least one neural net to the bottom level of the ACS along with all of the actions associated with the output layer of that net.
Agent
- The agent to which the ACS is being attached.Method Detail |
---|
protected AbstractAction chooseAction(DimensionValueCollection CurrInput, long TimeStamp)
CurrInput
- The current input to the ACS.TimeStamp
- The current time stamp.
private RuleCollection selectType()
private void performCombination(java.lang.Long TimeStamp)
TimeStamp
- The current time stamp.protected void learn(DimensionValueCollection NewInput, long TimeStamp)
NewInput
- The new input that was perceived.TimeStamp
- The current time stamp.protected void learn(DimensionValueCollection NewInput, double feedback, long TimeStamp)
NewInput
- The new input that was perceived.feedback
- The feedback received.TimeStamp
- The current time stamp.private void updateBottomLevel(DimensionValueCollection NewInput, double feedback)
If any external rules can be extracted from the bottom level, this method extracts them.
NewInput
- The new input.feedback
- The feedback.private void updateRules(double feedback, long TimeStamp)
feedback
- The feedback.TimeStamp
- The current time stamp.protected AbstractAction getChosenAction()
protected void resetChosenAction()
public java.util.Collection<AbstractImplicitModule> getBLImplicitModules()
public ImplicitModuleCollection getBLImplicitModuleStore()
Note: This method returns an ACTUAL object located within the ACS. Therefore, any changes made to it COULD have an effect on how the ACS handles the implicit modules. Therefore, it is advised that you be VERY careful when using the implicit module collection returned by this method.
public java.util.Collection<AbstractAction> getPossibleActions()
public java.util.Collection<AbstractRule> getRERRules()
public RuleCollection getRERRuleStore()
Note: This method returns an ACTUAL object located within the ACS. Therefore, any changes made to it COULD have an effect on how the ACS handles the RER rules. Therefore, it is advised that you be VERY careful when using the rule collection returned by this method.
public java.util.Collection<AbstractRule> getIRLRules()
public RuleCollection getIRLRuleStore()
Note: This method returns an ACTUAL object located within the ACS. Therefore, any changes made to it COULD have an effect on how the ACS handles the IRL rules. Therefore, it is advised that you be VERY careful when using the rule collection returned by this method.
public java.util.Collection<AbstractRule> getFixedRules()
public RuleCollection getFixedRuleStore()
Note: This method returns an ACTUAL object located within the ACS. Therefore, any changes made to it COULD have an effect on how the ACS handles the FR rules. Therefore, it is advised that you be VERY careful when using the rule collection returned by this method.
public void addAction(AbstractAction act)
This method should ONLY be called during initialization of the ACS.
act
- The external action to add.public void addActions(java.util.Collection<? extends AbstractAction> acts)
This method should ONLY be called during initialization of the ACS.
acts
- The collection of actions to add.public void addBLModule(AbstractImplicitModule im) throws java.lang.IllegalArgumentException
Note that if the specified implicit module is a neural network, it should be pre-trained (if necessary) before it is added to the ACS.
If you are using goals or specialized working memory chunks, remember that the input layer of the implicit module must contain nodes for any dimension-value pairs within those chunks that differ from the sensory information space.
If any of the actions in the output are not specified within the collection of possible actions, this method will add those actions to the collection of possible actions. In addition, if the input layer of the implicit module contains dimension-value pairs that are new (i.e. they have not been specified within the input space), this method will add those dimension-value pairs to the input space.
im
- The implicit module to add to the bottom level.
java.lang.IllegalArgumentException
- If the specified implicit module contains and output chunk on the output
layer that extends from something other than AbstractAction.public void addBLModules(java.util.Collection<? extends AbstractImplicitModule> ims)
Note that if any of the specified implicit modules is a neural network, it should be pre-trained (if necessary) before it is added to the ACS.
If you are using goals or specialized working memory chunks, remember that the input layer of the implicit modules must contain nodes for any dimension-value pairs within those chunks that differ from the sensory information space.
If any of the actions in the output are not specified within the collection of possible actions, this method will add those actions to the collection of possible actions. In addition, if the input layer of any of the implicit modules contain dimension-value pairs that are new (i.e. they have not been specified within the input space), this method will add those dimension-value pairs to the input space.
ims
- The implicit modules to add to the bottom level.public void addRule(AbstractRule R)
If the specified rule is covered by any rules currently in the rule store, then the rule will be added to the collection of child rules for that rule. If the specified rule is equal to any rule in the rule store (or any of the children of the rules in the rule store) the rule will not be added.
If the action of the rule is not specified within the collection of possible actions, this method will add that action to the collection of possible actions. In addition, if the rule contains dimension-value pairs in the condition of the rule that are new (i.e. they have not been specified within the input space), this method will add those dimension-value pairs to the input space.
If you are using goals or specialized working memory chunks, remember that the condition of a rule that you want to address those chunks must contain nodes for any dimension-value pairs within those chunks that differ from the sensory information space.
R
- The rule to add.public void addRules(java.util.Collection<? extends AbstractRule> R)
If the action of a rule is not specified within the collection of possible actions, this method will add that action to the collection of possible actions and generate a match all rule for the specified action. In addition, if a rule contains dimension-value pairs in the condition of the rule that are new (i.e. they have not been specified within the input space for the ACS), this method will add those dimension-value pairs to the input space of the ACS.
If you are using goals or specialized working memory chunks, remember that the condition of a rule that you want to address those chunks must contain nodes for any dimension-value pairs within those chunks that differ from the sensory information space.
R
- The collection of rules to add.private void generateMatchAllRule(AbstractAction act)
act
- The action for which a match all rule is to be generated.private AbstractRule getMatchAllRule(AbstractAction act)
act
- The action associated with the match all rule you wish to get.
private void updateMatchAllStats(AbstractAction ChosenAction, double feedback, AbstractMatchCalculator MatchCalculator)
ChosenAction
- The chosen action.feedback
- The feedback.protected void endEpisode()
endEpisode
in class AbstractSubsystem
protected void attachSelfToAgent(CLARION Agent)
attachSelfToAgent
in class AbstractSubsystem
Agent
- The agent to wish this ACS will be attached.
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |