Extensions

This module is a set of extensions. You can either choose one of our built-in extensions or implement your extension with the help of the Custom extensions guide.

BaseExt

class BaseExt

Container for domain-specific knowledge and functions for a given environment. Provides the transformation from the raw observations to the agent update and sample spaces. Stores the default argument values for agent initialization.

abstract property observation_space: Space: Basic observations of the environment in Gymnasium format.

get_agent_params(agent_type: type = None, agent_parameter_space: Dict = None, user_parameters: dict[str, any] = None) → dict[str, any]

Composes agent initialization arguments from values passed by the user and default values stored in the parameter functions. Returns a dictionary with the parameters matching the agent parameters space.

Parameters:

agent_type (type, optional) – Type of the selected agent.
agent_parameter_space (gym.spaces.Dict, optional) – Parameters required by the agents’ constructor in Gymnasium format.
user_parameters (dict, optional) – Parameters provided by the user.

Returns:

Dictionary with the initialization parameters for the agent.

Return type:

dict

setup_transformations(agent_update_space: Space = None, agent_sample_space: Space = None) → None

Creates functions that transform raw observations and values provided by the observation functions to the agent update and sample spaces.

Parameters:

agent_update_space (gym.spaces.Space, optional) – Observations required by the agent update function in Gymnasium format.
agent_sample_space (gym.spaces.Space, optional) – Observations required by the agent sample function in Gymnasium format.

transform(*args, action: any = None, **kwargs) → tuple[any, any]

Transforms raw observations and values provided by the observation functions to the agent observation and sample spaces. Provides the last action selected by the agent if it is required by the agent.

Parameters:

*args (tuple) – Environment observations.
action (any) – The last action selected by the agent.
**kwargs (dict) – Environment observations.

Returns:

Agent update and sample observations.

Return type:

tuple[any, any]

BasicMab

class BasicMab(n_arms: int)

Bases: BaseExt

Basic multi-armed bandit (MAB) extension for Reinforced-lib. This extension can be used with MAB algorithms which do not require any additional information about the environment apart from the number of arms.

Gymnasium

class Gymnasium(env_id: str)

Bases: BaseExt

Gymnasium [1] extension. Simplifies interaction of RL agents with the Gymnasium environments by providing the environment state, reward, terminal flag, and shapes of the observation and action spaces.

Parameters:: env_id (str) – Name of the Gymnasium environment.

References

GymnasiumVectorized

class GymnasiumVectorized(env_id: str, num_envs: int)

Bases: BaseExt

Vectorized Gymnasium [1] extension. Simplifies interaction of RL agents with the vectorized Gymnasium environments by providing the environment state, reward, terminal flag, and shapes of the observation and action spaces.

Parameters:

env_id (str) – Name of the Gymnasium environment.
num_envs (int) – Number of parallel environments.

Extension utils

class ObservationInfo(name: str, type: Space)

Description of the observation function that provides one of the values from the agent observation space.

name

Name of the provided observation.

Type:: str

type

Type of the provided value in Gymnasium format.

Type:: gym.spaces.Space

name: str: Alias for field number 0

type: Space: Alias for field number 1

class ParameterInfo(name: str, type: Space)

Description of the parameter function that provides one of the parameters of the agent constructor.

name

Name of the provided parameter.

Type:: str

type

Type of the provided parameter in Gymnasium format.

Type:: gym.spaces.Space

name: str: Alias for field number 0

type: Space: Alias for field number 1

observation(observation_name: str = None, observation_type: Space = None) → Callable

Decorator used to annotate the observation functions.

Parameters:

observation_name (str, optional) – Name of the provided observation.
observation_type (gym.spaces.Space, optional) – Type of the provided value in Gymnasium format.

Returns:

Function that returns the appropriate observation.

Return type:

Callable

parameter(parameter_name: str = None, parameter_type: Space = None) → Callable

Decorator used to annotate the parameter functions.

Parameters:

parameter_name (str, optional) – Name of the provided parameter.
parameter_type (gym.spaces.Space, optional) – Type of the provided parameter in Gymnasium format.

Returns:

Function that returns the appropriate parameter.

Return type:

Callable

test_box(a: Space, b: Box) → bool

Tests if the space a is identical to the gym.space.Box space b.

Parameters:

a (gym.spaces.Space) – Space a.
b (gym.spaces.Box) – Box space b.

Returns:

Result of the comparison.

Return type:

bool

test_discrete(a: Space, b: Discrete) → bool

Tests if the space a is identical to the gym.space.Discrete space b.

Parameters:

a (gym.spaces.Space) – Space a.
b (gym.spaces.Discrete) – Discrete space b.

Returns:

Result of the comparison.

Return type:

bool

test_multi_binary(a: Space, b: MultiBinary) → bool

Tests if the space a is identical to the gym.space.MultiBinary space b.

Parameters:

a (gym.spaces.Space) – Space a.
b (gym.spaces.MultiBinary) – MultiBinary space b.

Returns:

Result of the comparison.

Return type:

bool

test_multi_discrete(a: Space, b: MultiDiscrete) → bool

Tests if the space a is identical to the gym.space.MultiDiscrete space b.

Parameters:

a (gym.spaces.Space) – Space a.
b (gym.spaces.MultiDiscrete) – MultiDiscrete space b.

Returns:

Result of the comparison.

Return type:

bool

test_sequence(a: Space, b: Sequence) → bool

Tests if the space a is identical to the gym.space.Sequence space b.

Parameters:

a (gym.spaces.Space) – Space a.
b (gym.spaces.Sequence) – Sequence space b.

Returns:

Result of the comparison.

Return type:

bool

test_space(a: Space, b: Space) → bool

Tests if the space a is identical to the space b.

Parameters:

a (gym.spaces.Space) – Space a.
b (gym.spaces.Space) – Space b.

Returns:

Result of the comparison.

Return type:

bool