Object.Exploration (object v0.1.2)

Object-Oriented Exploration strategies for OORL framework.

Implements novelty-based and uncertainty-based exploration as specified in AAOS section 10, enabling objects to discover novel interactions and configurations in their environment.

Key exploration strategies:

  • Novelty-based exploration using state visitation counts
  • Uncertainty-based exploration using prediction confidence
  • Curiosity-driven exploration with information gain
  • Social exploration through interaction dyads
  • Meta-exploration for learning strategy optimization

Summary

Types

collaboration_record()
curiosity_model()
exploration_params()
exploration_record()
novelty_tracker()
object_id()
social_exploration_state()
state_key()
strategy_type()
t()
uncertainty_estimator()

Functions

adapt_exploration_parameters(explorer, performance_metrics)

Adapts exploration parameters based on performance feedback.

compute_exploration_bonus(explorer, state, action)

Computes exploration bonus for a given state-action pair based on the exploration strategy.

evaluate_exploration_effectiveness(explorer)

Evaluates the effectiveness of the current exploration strategy.

identify_novel_interactions(explorer, available_partners)

Identifies novel interaction patterns and opportunities with other objects.

new(object_id, strategy \\ :hybrid, opts \\ [])

Creates a new exploration system for an object.

select_exploration_action(explorer, available_actions, value_estimates)

Selects an action based on exploration strategy and value estimates.

update_exploration_state(explorer, state, action, outcome)

Updates exploration state based on observed outcome.

Types

collaboration_record()

@type collaboration_record() :: %{
  partner_id: object_id(),
  exploration_outcome: float(),
  timestamp: DateTime.t()
}

curiosity_model()

@type curiosity_model() :: %{
  information_gain_estimates: %{required(state_key()) => float()},
  surprise_threshold: float(),
  curiosity_bonus_scale: float(),
  learning_progress: float()
}

exploration_params()

@type exploration_params() :: %{
  exploration_rate: float(),
  novelty_weight: float(),
  uncertainty_weight: float(),
  curiosity_weight: float(),
  social_weight: float(),
  decay_rate: float()
}

exploration_record()

@type exploration_record() :: %{
  timestamp: DateTime.t(),
  state: any(),
  action: any(),
  novelty_score: float(),
  uncertainty_score: float(),
  curiosity_score: float(),
  total_exploration_bonus: float(),
  outcome: any()
}

novelty_tracker()

@type novelty_tracker() :: %{
  state_visitation_counts: %{required(state_key()) => integer()},
  novelty_threshold: float(),
  decay_factor: float(),
  novelty_bonus_scale: float()
}

object_id()

@type object_id() :: String.t()

social_exploration_state()

@type social_exploration_state() :: %{
  interaction_novelty: %{required(object_id()) => float()},
  dyad_exploration_targets: [object_id()],
  social_curiosity_scores: %{required(object_id()) => float()},
  collaboration_history: [collaboration_record()]
}

state_key()

@type state_key() :: term()

strategy_type()

@type strategy_type() ::
  :novelty_based | :uncertainty_based | :curiosity_driven | :hybrid | :social

t()

@type t() :: %Object.Exploration{
  curiosity_model: curiosity_model(),
  exploration_history: [exploration_record()],
  exploration_parameters: exploration_params(),
  exploration_strategy: strategy_type(),
  novelty_tracker: novelty_tracker(),
  object_id: String.t(),
  social_exploration_state: social_exploration_state(),
  uncertainty_estimator: uncertainty_estimator()
}

uncertainty_estimator()

@type uncertainty_estimator() :: %{
  prediction_errors: [float()],
  confidence_threshold: float(),
  uncertainty_bonus_scale: float(),
  model_uncertainty: float()
}

Functions

adapt_exploration_parameters(explorer, performance_metrics)

Adapts exploration parameters based on performance feedback.

Parameters

  • explorer: Exploration system struct
  • performance_metrics: Recent performance data

Returns

Updated exploration system with adapted parameters

compute_exploration_bonus(explorer, state, action)

Computes exploration bonus for a given state-action pair based on the exploration strategy.

Parameters

  • explorer: Exploration system struct
  • state: Current state
  • action: Action being considered

Returns

Exploration bonus value (higher values encourage exploration)

evaluate_exploration_effectiveness(explorer)

Evaluates the effectiveness of the current exploration strategy.

Parameters

  • explorer: Exploration system struct

Returns

Map with overall effectiveness score, detailed metrics, and recommendations

identify_novel_interactions(explorer, available_partners)

Identifies novel interaction patterns and opportunities with other objects.

Parameters

  • explorer: Exploration system struct
  • available_partners: List of potential interaction partners

Returns

Map with novel partners, recommendations, and expected information gain

new(object_id, strategy \\ :hybrid, opts \\ [])

Creates a new exploration system for an object.

Parameters

  • object_id: ID of the object
  • strategy: Exploration strategy (:novelty_based, :uncertainty_based, :curiosity_driven, :hybrid, :social)
  • opts: Optional configuration parameters

Returns

New exploration system struct

Examples

iex> Object.Exploration.new("agent1", :hybrid)
%Object.Exploration{object_id: "agent1", exploration_strategy: :hybrid, ...}

select_exploration_action(explorer, available_actions, value_estimates)

Selects an action based on exploration strategy and value estimates.

Parameters

  • explorer: Exploration system struct
  • available_actions: List of possible actions
  • value_estimates: Map of action -> estimated value

Returns

Selected action that balances exploration and exploitation

update_exploration_state(explorer, state, action, outcome)

Updates exploration state based on observed outcome.

Parameters

  • explorer: Exploration system struct
  • state: State where action was taken
  • action: Action that was executed
  • outcome: Observed outcome/result

Returns

Updated exploration system with recorded experience