clarion.system
Class ACS

java.lang.Object
  extended by clarion.system.AbstractSubsystem
      extended by clarion.system.ACS

public final class ACS
extends AbstractSubsystem

This class implements the Action Centered Subsystem (ACS) within CLARION. It extends the AbstractSubsystem class.

Usage: The ACS is the subsystem that is minimally required in order for a CLARION agent to run. It MUST be fully setup within a CLARION agent before the CLARION agent can perform within a task environment.

Note that simply initializing the ACS is not sufficient enough to setup the ACS. At the very least you MUST also add at least one implicit module to the bottom level of the ACS along with all of the actions associated with the output layer of that module.

The PRIMARY methods for the ACS are the "learn" and "chooseAction" methods and they do precisely what their names suggest.

Note that the ACS is initialized with already built in "default" external, working memory, and goal actions:

It is completely optional if you wish to implement the "DO_NOTHING" actions, however if you do, then you must specify those actions in the output layer of your implicit modules and/or rules in order for them to be used.

This class contains MANY parameters for handling different aspects of the ACS from level selection options/methods to triggers for turning on and off various "automatic" aspects of the ACS (such as extraction and refinement). If you are having trouble determining where to locate a parameter you need to set, this is not a bad place to start looking (if the parameter relates to the ACS, of course).

This class contains both global (static) and local constants. The default is to use the local constants. If you want to change any of the global constants, you need to do so before any instances of this class are initialized.

Version:
6.0.4.5
Author:
Nick Wilson

Nested Class Summary
static class ACS.LevelSelectionMethods
          The various methods that can be used for level selection (such as stochastic or combined)
static class ACS.LevelSelectionOptions
          The options for level selection (such as fixed or variable)
private static class ACS.ModuleTypes
          The various different types of modules used for action selection (BL,RER,IRL,FR)
 
Field Summary
private  ImplicitModuleCollection BLImplicitModuleStore
          The bottom level implicit modules.
private  AbstractAction ChosenAction
          The chosen action.
private  DimensionValueCollection CurrentInput
          The current input (including current sensory information, current goal, chunks in working memory, and drive stimulus).
 double EXTERNAL_PROBABILITY
          The probability of choosing an external action.
 RuleExtractor Extractor
          The rule extractor for rule extraction.
 double FIXED_BL_COMB_WEIGHT
          The fixed bottom level weight when level combination is used.
 double FIXED_BL_PROBABILITY
          The fixed probability of using the bottom level.
 double FIXED_FR_COMB_WEIGHT
          The fixed weight for a fixed rule when level combination is used.
 double FIXED_FR_PROBABILITY
          The fixed probability of using a fixed rule.
 double FIXED_IRL_COMB_WEIGHT
          The fixed IRL rule weight when level combination is used.
 double FIXED_IRL_PROBABILITY
          The fixed probability of using an IRL rule.
 double FIXED_RER_COMB_WEIGHT
          The fixed RER rule weight when level combination is used.
 double FIXED_RER_PROBABILITY
          The fixed probability of using an RER rule.
private  RuleCollection FRRuleStore
          The FR rule store.
static double GLOBAL_EXTERNAL_PROBABILITY
          The probability of choosing an external action.
static double GLOBAL_FIXED_BL_COMB_WEIGHT
          The fixed bottom level weight when level combination is used.
static double GLOBAL_FIXED_BL_PROBABILITY
          The fixed probability of using the bottom level.
static double GLOBAL_FIXED_FR_COMB_WEIGHT
          The fixed weight for a fixed rule when level combination is used.
static double GLOBAL_FIXED_FR_PROBABILITY
          The fixed probability of using a fixed rule.
static double GLOBAL_FIXED_IRL_COMB_WEIGHT
          The fixed IRL rule weight when level combination is used.
static double GLOBAL_FIXED_IRL_PROBABILITY
          The fixed probability of using an IRL rule.
static double GLOBAL_FIXED_RER_COMB_WEIGHT
          The fixed RER rule weight when level combination is used.
static double GLOBAL_FIXED_RER_PROBABILITY
          The fixed probability of using an RER rule.
static double GLOBAL_GOAL_PROBABILITY
          The probability of choosing a goal action.
static ACS.LevelSelectionMethods GLOBAL_LEVEL_SELECTION_METHOD
          Specifies the method to use for level selection.
static ACS.LevelSelectionOptions GLOBAL_LEVEL_SELECTION_OPTION
          Specifies the option to use for level selection.
static double GLOBAL_MATCH_DISCOUNT
          The match discount factor.
static double GLOBAL_MCS_A
          The constant A used to factor in the MCS measure when calculating the probability or weight of for the bottom level during level selection.
static double GLOBAL_MCS_B
          The constant B used to factor in the MCS measure when calculating the probability or weight of for the bottom level during level selection.
static boolean GLOBAL_PERFORM_BL_LEARNING
          Specifies whether or not bottom level learning should be performed.
static boolean GLOBAL_PERFORM_EXTRACTION
          Specifies whether or not extraction should be used.
static boolean GLOBAL_PERFORM_REFINEMENT_IRL
          Specifies whether or not refinement should be used for IRL rules.
static boolean GLOBAL_PERFORM_REFINEMENT_RER
          Specifies whether or not refinement should be used for RER rules.
static double GLOBAL_VARIABLE_BL_BETA
          The BL beta when using variable stochastic level selection.
static double GLOBAL_VARIABLE_BL_COMB_WEIGHT_BETA
          The BL beta when using the variable weighted combination.
static double GLOBAL_VARIABLE_C3
          The C3 constant used for variable level selection.
static double GLOBAL_VARIABLE_C4
          The C4 constant used for variable level selection.
static double GLOBAL_VARIABLE_FR_BETA
          The FR beta when using variable stochastic level selection.
static double GLOBAL_VARIABLE_FR_COMB_WEIGHT_BETA
          The FR beta when using the variable weighted combination.
static double GLOBAL_VARIABLE_IRL_BETA
          The IRL beta when using variable stochastic level selection.
static double GLOBAL_VARIABLE_IRL_COMB_WEIGHT_BETA
          The IRL beta when using the variable weighted combination.
static double GLOBAL_VARIABLE_RER_BETA
          The RER beta when using variable stochastic level selection.
static double GLOBAL_VARIABLE_RER_COMB_WEIGHT_BETA
          The RER beta when using the variable weighted combination.
static double GLOBAL_WM_PROBABILITY
          The probability of choosing a working memory action.
 double GOAL_PROBABILITY
          The probability of choosing a goal action.
private  RuleCollection IRLRuleStore
          The IRL rule store.
 ACS.LevelSelectionMethods LEVEL_SELECTION_METHOD
          Specifies the method to use for level selection.
 ACS.LevelSelectionOptions LEVEL_SELECTION_OPTION
          Specifies the option to use for level selection.
static double MATCH_DISCOUNT
          The match discount factor.
private  java.util.HashMap<java.lang.Object,AbstractRule> MatchAllRules
          Contains the hidden match all rules for all actions associated with the rules in the rule collections.
 AbstractMatchCalculator MatchCalculator
          The match calculator to use for updating match statistics for ALL components located within the ACS.
 double MCS_A
          The constant A used to factor in the MCS measure when calculating the probability or weight of for the bottom level during level selection.
 double MCS_B
          The constant B used to factor in the MCS measure when calculating the probability or weight of for the bottom level during level selection.
protected  double MCS_BL_PROBABILITY
          The probability measure specified by the probability setting module in the MCS for the bottom level of the ACS when stochastic level selection is used.
protected  double MCS_BL_WEIGHT
          The weight measure specified by the probability setting module in the MCS for the bottom level of the ACS when weighted combination is used.
protected  double MCS_TL_PROBABILITY
          The probability measure specified by the probability setting module in the MCS for the top level of the ACS when stochastic level selection is used.
protected  double MCS_TL_WEIGHT
          The weight measure specified by the probability setting module in the MCS for the top level of the ACS when weighted combination is used.
private  ACS.ModuleTypes ModuleUsed
          The module type that was last used to choose the action.
 boolean PERFORM_BL_LEARNING
          Specifies whether or not bottom level learning should be performed.
 boolean PERFORM_EXTRACTION
          Specifies whether or not extraction should be used.
 boolean PERFORM_REFINEMENT_IRL
          Specifies whether or not refinement should be used for IRL rules.
 boolean PERFORM_REFINEMENT_RER
          Specifies whether or not refinement should be used for RER rules.
private  ActionCollection PossibleActions
          The collection of possible actions
 RuleRefiner Refiner
          The rule refiner for rule refinement.
private  RuleCollection RERRuleStore
          The RER rule store.
 StochasticSelector Selector
          The stochastic selector for rule selection.
private  java.util.Collection<AbstractAction> shortlist
          The subset of possible actions that were involved in the last action decision making stochastic selection.
 double VARIABLE_BL_BETA
          The BL beta when using variable stochastic level selection.
 double VARIABLE_BL_COMB_WEIGHT_BETA
          The BL beta when using the variable weighted combination.
 double VARIABLE_C3
          The C3 constant used for variable level selection.
 double VARIABLE_C4
          The C4 constant used for variable level selection.
 double VARIABLE_FR_BETA
          The FR beta when using variable stochastic level selection.
 double VARIABLE_FR_COMB_WEIGHT_BETA
          The FR beta when using the variable weighted combination.
 double VARIABLE_IRL_BETA
          The IRL beta when using variable stochastic level selection.
 double VARIABLE_IRL_COMB_WEIGHT_BETA
          The IRL beta when using the variable weighted combination.
 double VARIABLE_RER_BETA
          The RER beta when using variable stochastic level selection.
 double VARIABLE_RER_COMB_WEIGHT_BETA
          The RER beta when using the variable weighted combination.
protected  WorkingMemory WM
          Points to the working memory from the instance of the CLARION class to which this instance of the ACS is attached.
 double WM_PROBABILITY
          The probability of choosing a working memory action.
 
Fields inherited from class clarion.system.AbstractSubsystem
InputSpace
 
Constructor Summary
ACS(CLARION Agent)
          Minimally initializes the ACS.
 
Method Summary
 void addAction(AbstractAction act)
          Adds an action to the list of possible actions and generates a match all rule for the specified action.
 void addActions(java.util.Collection<? extends AbstractAction> acts)
          Adds a collection of actions to the list of possible actions and generates a match all rule for all the actions that were added.
 void addBLModule(AbstractImplicitModule im)
          Adds an implicit module to the bottom level of the ACS.
 void addBLModules(java.util.Collection<? extends AbstractImplicitModule> ims)
          Adds a collection of implicit modules to the bottom level of the ACS.
 void addRule(AbstractRule R)
          Adds a rule to the rule store.
 void addRules(java.util.Collection<? extends AbstractRule> R)
          Adds a collection of rules to the rule store.
protected  void attachSelfToAgent(CLARION Agent)
          Attaches the ACS to the specified CLARION agent.
protected  AbstractAction chooseAction(DimensionValueCollection CurrInput, long TimeStamp)
          Chooses an action given the current input (which includes sensory information, the current goal, and any chunks in working memory) and the previously chosen action.
protected  void endEpisode()
          Performs the appropriate end of episode instructions for the ACS.
private  void generateMatchAllRule(AbstractAction act)
          Generates a match all rule associated with the specified action and adds it to the list of match all rules.
 java.util.Collection<AbstractImplicitModule> getBLImplicitModules()
          Gets all of the bottom level implicit modules.
 ImplicitModuleCollection getBLImplicitModuleStore()
          Gets the collection holding the bottom level implicit modules.
protected  AbstractAction getChosenAction()
          Get the last action chosen by the ACS.
 java.util.Collection<AbstractRule> getFixedRules()
          Gets all of the fixed rules in the rule store.
 RuleCollection getFixedRuleStore()
          Gets the subset of the rule store that holds the FR rules.
 java.util.Collection<AbstractRule> getIRLRules()
          Gets all of the IRL rules in the rule store.
 RuleCollection getIRLRuleStore()
          Gets the subset of the rule store that holds the IRL rules.
private  AbstractRule getMatchAllRule(AbstractAction act)
          Gets the match all rule associated with the specified action.
 java.util.Collection<AbstractAction> getPossibleActions()
          Gets all of the possible actions.
 java.util.Collection<AbstractRule> getRERRules()
          Gets all of the RER rules in the rule store.
 RuleCollection getRERRuleStore()
          Gets the subset of the rule store that holds the RER rules.
protected  void learn(DimensionValueCollection NewInput, double feedback, long TimeStamp)
          Performs bottom level learning, rule refinement, and match statistic updates based on the specified new input.
protected  void learn(DimensionValueCollection NewInput, long TimeStamp)
          Performs bottom level learning, rule refinement, and match statistic updates based on the specified new input.
private  void performCombination(java.lang.Long TimeStamp)
          Combines the outcomes of the top and bottom levels when level combination is being used.
protected  void resetChosenAction()
          Resets the chosen action.
private  RuleCollection selectType()
          Selects a rule type to use on the top level when performing action decision making with stochastic level selection.
private  void updateBottomLevel(DimensionValueCollection NewInput, double feedback)
          Updates the bottom level implicit representations (if they support online learning) given the new sensory information and feedback (if any was received).
private  void updateMatchAllStats(AbstractAction ChosenAction, double feedback, AbstractMatchCalculator MatchCalculator)
          Updates the positive or negative match statistics for the match all rule corresponding to the chosen action.
private  void updateRules(double feedback, long TimeStamp)
          Updates the match statistics for the rules in the rule store (if its type was used) and tries to refine those rules in the rule store whose type (i.e.
 
Methods inherited from class clarion.system.AbstractSubsystem
updateInputSpace
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

RERRuleStore

private RuleCollection RERRuleStore
The RER rule store.


IRLRuleStore

private RuleCollection IRLRuleStore
The IRL rule store.


FRRuleStore

private RuleCollection FRRuleStore
The FR rule store.


BLImplicitModuleStore

private ImplicitModuleCollection BLImplicitModuleStore
The bottom level implicit modules.


PossibleActions

private ActionCollection PossibleActions
The collection of possible actions


MatchAllRules

private java.util.HashMap<java.lang.Object,AbstractRule> MatchAllRules
Contains the hidden match all rules for all actions associated with the rules in the rule collections.


WM

protected WorkingMemory WM
Points to the working memory from the instance of the CLARION class to which this instance of the ACS is attached.


ChosenAction

private AbstractAction ChosenAction
The chosen action.


ModuleUsed

private ACS.ModuleTypes ModuleUsed
The module type that was last used to choose the action.


CurrentInput

private DimensionValueCollection CurrentInput
The current input (including current sensory information, current goal, chunks in working memory, and drive stimulus).


Selector

public StochasticSelector Selector
The stochastic selector for rule selection.


Extractor

public RuleExtractor Extractor
The rule extractor for rule extraction.


Refiner

public RuleRefiner Refiner
The rule refiner for rule refinement.


MatchCalculator

public AbstractMatchCalculator MatchCalculator
The match calculator to use for updating match statistics for ALL components located within the ACS.


GLOBAL_LEVEL_SELECTION_METHOD

public static ACS.LevelSelectionMethods GLOBAL_LEVEL_SELECTION_METHOD
Specifies the method to use for level selection.


GLOBAL_LEVEL_SELECTION_OPTION

public static ACS.LevelSelectionOptions GLOBAL_LEVEL_SELECTION_OPTION
Specifies the option to use for level selection.


LEVEL_SELECTION_METHOD

public ACS.LevelSelectionMethods LEVEL_SELECTION_METHOD
Specifies the method to use for level selection.


LEVEL_SELECTION_OPTION

public ACS.LevelSelectionOptions LEVEL_SELECTION_OPTION
Specifies the option to use for level selection.


GLOBAL_VARIABLE_C3

public static double GLOBAL_VARIABLE_C3
The C3 constant used for variable level selection.


VARIABLE_C3

public double VARIABLE_C3
The C3 constant used for variable level selection.


GLOBAL_VARIABLE_C4

public static double GLOBAL_VARIABLE_C4
The C4 constant used for variable level selection.


VARIABLE_C4

public double VARIABLE_C4
The C4 constant used for variable level selection.


MCS_BL_PROBABILITY

protected double MCS_BL_PROBABILITY
The probability measure specified by the probability setting module in the MCS for the bottom level of the ACS when stochastic level selection is used.


MCS_BL_WEIGHT

protected double MCS_BL_WEIGHT
The weight measure specified by the probability setting module in the MCS for the bottom level of the ACS when weighted combination is used.


MCS_TL_PROBABILITY

protected double MCS_TL_PROBABILITY
The probability measure specified by the probability setting module in the MCS for the top level of the ACS when stochastic level selection is used.


MCS_TL_WEIGHT

protected double MCS_TL_WEIGHT
The weight measure specified by the probability setting module in the MCS for the top level of the ACS when weighted combination is used.


GLOBAL_MCS_A

public static double GLOBAL_MCS_A
The constant A used to factor in the MCS measure when calculating the probability or weight of for the bottom level during level selection.


GLOBAL_MCS_B

public static double GLOBAL_MCS_B
The constant B used to factor in the MCS measure when calculating the probability or weight of for the bottom level during level selection.


MCS_A

public double MCS_A
The constant A used to factor in the MCS measure when calculating the probability or weight of for the bottom level during level selection.


MCS_B

public double MCS_B
The constant B used to factor in the MCS measure when calculating the probability or weight of for the bottom level during level selection.


GLOBAL_FIXED_BL_PROBABILITY

public static double GLOBAL_FIXED_BL_PROBABILITY
The fixed probability of using the bottom level. This is used if the fixed level selection method is used.


GLOBAL_FIXED_BL_COMB_WEIGHT

public static double GLOBAL_FIXED_BL_COMB_WEIGHT
The fixed bottom level weight when level combination is used.


FIXED_BL_PROBABILITY

public double FIXED_BL_PROBABILITY
The fixed probability of using the bottom level. This is used if the fixed level selection method is used.


FIXED_BL_COMB_WEIGHT

public double FIXED_BL_COMB_WEIGHT
The fixed bottom level weight when level combination is used.


GLOBAL_VARIABLE_BL_BETA

public static double GLOBAL_VARIABLE_BL_BETA
The BL beta when using variable stochastic level selection.


GLOBAL_VARIABLE_BL_COMB_WEIGHT_BETA

public static double GLOBAL_VARIABLE_BL_COMB_WEIGHT_BETA
The BL beta when using the variable weighted combination.


VARIABLE_BL_BETA

public double VARIABLE_BL_BETA
The BL beta when using variable stochastic level selection.


VARIABLE_BL_COMB_WEIGHT_BETA

public double VARIABLE_BL_COMB_WEIGHT_BETA
The BL beta when using the variable weighted combination.


GLOBAL_FIXED_RER_PROBABILITY

public static double GLOBAL_FIXED_RER_PROBABILITY
The fixed probability of using an RER rule. This is used if the fixed level selection method is used.


GLOBAL_FIXED_RER_COMB_WEIGHT

public static double GLOBAL_FIXED_RER_COMB_WEIGHT
The fixed RER rule weight when level combination is used.


FIXED_RER_PROBABILITY

public double FIXED_RER_PROBABILITY
The fixed probability of using an RER rule. This is used if the fixed level selection method is used.


FIXED_RER_COMB_WEIGHT

public double FIXED_RER_COMB_WEIGHT
The fixed RER rule weight when level combination is used.


GLOBAL_VARIABLE_RER_BETA

public static double GLOBAL_VARIABLE_RER_BETA
The RER beta when using variable stochastic level selection.


GLOBAL_VARIABLE_RER_COMB_WEIGHT_BETA

public static double GLOBAL_VARIABLE_RER_COMB_WEIGHT_BETA
The RER beta when using the variable weighted combination.


VARIABLE_RER_BETA

public double VARIABLE_RER_BETA
The RER beta when using variable stochastic level selection.


VARIABLE_RER_COMB_WEIGHT_BETA

public double VARIABLE_RER_COMB_WEIGHT_BETA
The RER beta when using the variable weighted combination.


GLOBAL_FIXED_IRL_PROBABILITY

public static double GLOBAL_FIXED_IRL_PROBABILITY
The fixed probability of using an IRL rule. This is used if the fixed level selection method is used.


GLOBAL_FIXED_IRL_COMB_WEIGHT

public static double GLOBAL_FIXED_IRL_COMB_WEIGHT
The fixed IRL rule weight when level combination is used.


FIXED_IRL_PROBABILITY

public double FIXED_IRL_PROBABILITY
The fixed probability of using an IRL rule. This is used if the fixed level selection method is used.


FIXED_IRL_COMB_WEIGHT

public double FIXED_IRL_COMB_WEIGHT
The fixed IRL rule weight when level combination is used.


GLOBAL_VARIABLE_IRL_BETA

public static double GLOBAL_VARIABLE_IRL_BETA
The IRL beta when using variable stochastic level selection.


GLOBAL_VARIABLE_IRL_COMB_WEIGHT_BETA

public static double GLOBAL_VARIABLE_IRL_COMB_WEIGHT_BETA
The IRL beta when using the variable weighted combination.


VARIABLE_IRL_BETA

public double VARIABLE_IRL_BETA
The IRL beta when using variable stochastic level selection.


VARIABLE_IRL_COMB_WEIGHT_BETA

public double VARIABLE_IRL_COMB_WEIGHT_BETA
The IRL beta when using the variable weighted combination.


GLOBAL_FIXED_FR_PROBABILITY

public static double GLOBAL_FIXED_FR_PROBABILITY
The fixed probability of using a fixed rule. This is used if the fixed level selection method is used.


GLOBAL_FIXED_FR_COMB_WEIGHT

public static double GLOBAL_FIXED_FR_COMB_WEIGHT
The fixed weight for a fixed rule when level combination is used.


FIXED_FR_PROBABILITY

public double FIXED_FR_PROBABILITY
The fixed probability of using a fixed rule. This is used if the fixed level selection method is used.


FIXED_FR_COMB_WEIGHT

public double FIXED_FR_COMB_WEIGHT
The fixed weight for a fixed rule when level combination is used.


GLOBAL_VARIABLE_FR_BETA

public static double GLOBAL_VARIABLE_FR_BETA
The FR beta when using variable stochastic level selection.


GLOBAL_VARIABLE_FR_COMB_WEIGHT_BETA

public static double GLOBAL_VARIABLE_FR_COMB_WEIGHT_BETA
The FR beta when using the variable weighted combination.


VARIABLE_FR_BETA

public double VARIABLE_FR_BETA
The FR beta when using variable stochastic level selection.


VARIABLE_FR_COMB_WEIGHT_BETA

public double VARIABLE_FR_COMB_WEIGHT_BETA
The FR beta when using the variable weighted combination.


GLOBAL_EXTERNAL_PROBABILITY

public static double GLOBAL_EXTERNAL_PROBABILITY
The probability of choosing an external action.


GLOBAL_GOAL_PROBABILITY

public static double GLOBAL_GOAL_PROBABILITY
The probability of choosing a goal action.


GLOBAL_WM_PROBABILITY

public static double GLOBAL_WM_PROBABILITY
The probability of choosing a working memory action.


EXTERNAL_PROBABILITY

public double EXTERNAL_PROBABILITY
The probability of choosing an external action.


GOAL_PROBABILITY

public double GOAL_PROBABILITY
The probability of choosing a goal action.


WM_PROBABILITY

public double WM_PROBABILITY
The probability of choosing a working memory action.


GLOBAL_PERFORM_EXTRACTION

public static boolean GLOBAL_PERFORM_EXTRACTION
Specifies whether or not extraction should be used. Extraction is performed by default.


PERFORM_EXTRACTION

public boolean PERFORM_EXTRACTION
Specifies whether or not extraction should be used. Extraction is performed by default.


GLOBAL_PERFORM_BL_LEARNING

public static boolean GLOBAL_PERFORM_BL_LEARNING
Specifies whether or not bottom level learning should be performed. Learning is performed by default.


PERFORM_BL_LEARNING

public boolean PERFORM_BL_LEARNING
Specifies whether or not bottom level learning should be performed. Learning is performed by default.


GLOBAL_PERFORM_REFINEMENT_RER

public static boolean GLOBAL_PERFORM_REFINEMENT_RER
Specifies whether or not refinement should be used for RER rules. Refinement is performed by default.


PERFORM_REFINEMENT_RER

public boolean PERFORM_REFINEMENT_RER
Specifies whether or not refinement should be used for RER rules. Refinement is performed by default.


GLOBAL_PERFORM_REFINEMENT_IRL

public static boolean GLOBAL_PERFORM_REFINEMENT_IRL
Specifies whether or not refinement should be used for IRL rules. Refinement is performed by default.


PERFORM_REFINEMENT_IRL

public boolean PERFORM_REFINEMENT_IRL
Specifies whether or not refinement should be used for IRL rules. Refinement is performed by default.


GLOBAL_MATCH_DISCOUNT

public static double GLOBAL_MATCH_DISCOUNT
The match discount factor.


MATCH_DISCOUNT

public static double MATCH_DISCOUNT
The match discount factor.


shortlist

private java.util.Collection<AbstractAction> shortlist
The subset of possible actions that were involved in the last action decision making stochastic selection.

Constructor Detail

ACS

public ACS(CLARION Agent)
Minimally initializes the ACS. This constructor builds a "frame" for this instance of the ACS from which all desired components can be attached. During initialization this instance of the ACS will attach itself to the CLARION agent you specify.

Note that simply calling this constructor is not sufficient enough to setup the ACS. At the very least you MUST also add at least one neural net to the bottom level of the ACS along with all of the actions associated with the output layer of that net.

Parameters:
Agent - The agent to which the ACS is being attached.
Method Detail

chooseAction

protected AbstractAction chooseAction(DimensionValueCollection CurrInput,
                                      long TimeStamp)
Chooses an action given the current input (which includes sensory information, the current goal, and any chunks in working memory) and the previously chosen action.

Parameters:
CurrInput - The current input to the ACS.
TimeStamp - The current time stamp.
Returns:
The chosen action.

selectType

private RuleCollection selectType()
Selects a rule type to use on the top level when performing action decision making with stochastic level selection. If the top level is not to be used, this method returns null.

Returns:
The rule collection to use on the top level or null if the top level is not to be used.

performCombination

private void performCombination(java.lang.Long TimeStamp)
Combines the outcomes of the top and bottom levels when level combination is being used.

Parameters:
TimeStamp - The current time stamp.

learn

protected void learn(DimensionValueCollection NewInput,
                     long TimeStamp)
Performs bottom level learning, rule refinement, and match statistic updates based on the specified new input. If this method is used, learning is performed without any feedback.

Parameters:
NewInput - The new input that was perceived.
TimeStamp - The current time stamp.

learn

protected void learn(DimensionValueCollection NewInput,
                     double feedback,
                     long TimeStamp)
Performs bottom level learning, rule refinement, and match statistic updates based on the specified new input.

Parameters:
NewInput - The new input that was perceived.
feedback - The feedback received.
TimeStamp - The current time stamp.

updateBottomLevel

private void updateBottomLevel(DimensionValueCollection NewInput,
                               double feedback)
Updates the bottom level implicit representations (if they support online learning) given the new sensory information and feedback (if any was received). This method also updates the match statistics for the bottom level components and the bottom level module itself.

If any external rules can be extracted from the bottom level, this method extracts them.

Parameters:
NewInput - The new input.
feedback - The feedback.

updateRules

private void updateRules(double feedback,
                         long TimeStamp)
Updates the match statistics for the rules in the rule store (if its type was used) and tries to refine those rules in the rule store whose type (i.e. RER/IRL/FR) was used.

Parameters:
feedback - The feedback.
TimeStamp - The current time stamp.

getChosenAction

protected AbstractAction getChosenAction()
Get the last action chosen by the ACS. This method is protected and meant only for use inside the CLARION Library.

Returns:
The last action chosen.

resetChosenAction

protected void resetChosenAction()
Resets the chosen action.


getBLImplicitModules

public java.util.Collection<AbstractImplicitModule> getBLImplicitModules()
Gets all of the bottom level implicit modules. The collection returned is unmodifiable and is meant for reporting the internal state only.

Returns:
An unmodifiable collection of all of the bottom level implicit modules.

getBLImplicitModuleStore

public ImplicitModuleCollection getBLImplicitModuleStore()
Gets the collection holding the bottom level implicit modules.

Note: This method returns an ACTUAL object located within the ACS. Therefore, any changes made to it COULD have an effect on how the ACS handles the implicit modules. Therefore, it is advised that you be VERY careful when using the implicit module collection returned by this method.

Returns:
The implicit module collection.

getPossibleActions

public java.util.Collection<AbstractAction> getPossibleActions()
Gets all of the possible actions. The collection returned is unmodifiable and is meant for reporting the internal state only.

Returns:
An unmodifiable collection of all of the possible actions.

getRERRules

public java.util.Collection<AbstractRule> getRERRules()
Gets all of the RER rules in the rule store.

Returns:
A collection of all of the RER rules in the ACS.

getRERRuleStore

public RuleCollection getRERRuleStore()
Gets the subset of the rule store that holds the RER rules.

Note: This method returns an ACTUAL object located within the ACS. Therefore, any changes made to it COULD have an effect on how the ACS handles the RER rules. Therefore, it is advised that you be VERY careful when using the rule collection returned by this method.

Returns:
The RER rule collection.

getIRLRules

public java.util.Collection<AbstractRule> getIRLRules()
Gets all of the IRL rules in the rule store.

Returns:
A collection of all of the IRL rules in the ACS.

getIRLRuleStore

public RuleCollection getIRLRuleStore()
Gets the subset of the rule store that holds the IRL rules.

Note: This method returns an ACTUAL object located within the ACS. Therefore, any changes made to it COULD have an effect on how the ACS handles the IRL rules. Therefore, it is advised that you be VERY careful when using the rule collection returned by this method.

Returns:
The IRL rule collection.

getFixedRules

public java.util.Collection<AbstractRule> getFixedRules()
Gets all of the fixed rules in the rule store.

Returns:
A collection of all of the fixed rules in the ACS.

getFixedRuleStore

public RuleCollection getFixedRuleStore()
Gets the subset of the rule store that holds the FR rules.

Note: This method returns an ACTUAL object located within the ACS. Therefore, any changes made to it COULD have an effect on how the ACS handles the FR rules. Therefore, it is advised that you be VERY careful when using the rule collection returned by this method.

Returns:
The FR rule collection.

addAction

public void addAction(AbstractAction act)
Adds an action to the list of possible actions and generates a match all rule for the specified action. If the action is already in the list of possible actions, it will not be added.

This method should ONLY be called during initialization of the ACS.

Parameters:
act - The external action to add.

addActions

public void addActions(java.util.Collection<? extends AbstractAction> acts)
Adds a collection of actions to the list of possible actions and generates a match all rule for all the actions that were added. If any of the actions in the collection are already in the list of possible actions, they will not be added.

This method should ONLY be called during initialization of the ACS.

Parameters:
acts - The collection of actions to add.

addBLModule

public void addBLModule(AbstractImplicitModule im)
                 throws java.lang.IllegalArgumentException
Adds an implicit module to the bottom level of the ACS. This method should ONLY be called during initialization of the ACS.

Note that if the specified implicit module is a neural network, it should be pre-trained (if necessary) before it is added to the ACS.

If you are using goals or specialized working memory chunks, remember that the input layer of the implicit module must contain nodes for any dimension-value pairs within those chunks that differ from the sensory information space.

If any of the actions in the output are not specified within the collection of possible actions, this method will add those actions to the collection of possible actions. In addition, if the input layer of the implicit module contains dimension-value pairs that are new (i.e. they have not been specified within the input space), this method will add those dimension-value pairs to the input space.

Parameters:
im - The implicit module to add to the bottom level.
Throws:
java.lang.IllegalArgumentException - If the specified implicit module contains and output chunk on the output layer that extends from something other than AbstractAction.

addBLModules

public void addBLModules(java.util.Collection<? extends AbstractImplicitModule> ims)
Adds a collection of implicit modules to the bottom level of the ACS. This method should ONLY be called during initialization of the ACS.

Note that if any of the specified implicit modules is a neural network, it should be pre-trained (if necessary) before it is added to the ACS.

If you are using goals or specialized working memory chunks, remember that the input layer of the implicit modules must contain nodes for any dimension-value pairs within those chunks that differ from the sensory information space.

If any of the actions in the output are not specified within the collection of possible actions, this method will add those actions to the collection of possible actions. In addition, if the input layer of any of the implicit modules contain dimension-value pairs that are new (i.e. they have not been specified within the input space), this method will add those dimension-value pairs to the input space.

Parameters:
ims - The implicit modules to add to the bottom level.

addRule

public void addRule(AbstractRule R)
Adds a rule to the rule store. The rule will be placed in the appropriate rule store given its type (RER,IRL, or FR). If the rule is not of a recognized type, the rule will not be added.

If the specified rule is covered by any rules currently in the rule store, then the rule will be added to the collection of child rules for that rule. If the specified rule is equal to any rule in the rule store (or any of the children of the rules in the rule store) the rule will not be added.

If the action of the rule is not specified within the collection of possible actions, this method will add that action to the collection of possible actions. In addition, if the rule contains dimension-value pairs in the condition of the rule that are new (i.e. they have not been specified within the input space), this method will add those dimension-value pairs to the input space.

If you are using goals or specialized working memory chunks, remember that the condition of a rule that you want to address those chunks must contain nodes for any dimension-value pairs within those chunks that differ from the sensory information space.

Parameters:
R - The rule to add.

addRules

public void addRules(java.util.Collection<? extends AbstractRule> R)
Adds a collection of rules to the rule store. If any of the rules in the collection are covered by a rule currently in the rule store, that rule will be added to the collection of child rules for that rule. If the specified rule is equal to any rule in the rule store (or any of the children of the rules in the rule store) the rule will not be added.

If the action of a rule is not specified within the collection of possible actions, this method will add that action to the collection of possible actions and generate a match all rule for the specified action. In addition, if a rule contains dimension-value pairs in the condition of the rule that are new (i.e. they have not been specified within the input space for the ACS), this method will add those dimension-value pairs to the input space of the ACS.

If you are using goals or specialized working memory chunks, remember that the condition of a rule that you want to address those chunks must contain nodes for any dimension-value pairs within those chunks that differ from the sensory information space.

Parameters:
R - The collection of rules to add.

generateMatchAllRule

private void generateMatchAllRule(AbstractAction act)
Generates a match all rule associated with the specified action and adds it to the list of match all rules.

Parameters:
act - The action for which a match all rule is to be generated.

getMatchAllRule

private AbstractRule getMatchAllRule(AbstractAction act)
Gets the match all rule associated with the specified action. If the action does not have a match all rule, this method returns null. While it MAY be possible for this method to return null, it SHOULD never happen.

Parameters:
act - The action associated with the match all rule you wish to get.
Returns:
The match all rule for the specified action.

updateMatchAllStats

private void updateMatchAllStats(AbstractAction ChosenAction,
                                 double feedback,
                                 AbstractMatchCalculator MatchCalculator)
Updates the positive or negative match statistics for the match all rule corresponding to the chosen action.

Parameters:
ChosenAction - The chosen action.
feedback - The feedback.

endEpisode

protected void endEpisode()
Performs the appropriate end of episode instructions for the ACS. This method is called by the CLARION class when its endEpisode method is called.

Specified by:
endEpisode in class AbstractSubsystem

attachSelfToAgent

protected void attachSelfToAgent(CLARION Agent)
Attaches the ACS to the specified CLARION agent.

Specified by:
attachSelfToAgent in class AbstractSubsystem
Parameters:
Agent - The agent to wish this ACS will be attached.