Introduction

irl-gym is a custom Farama Gymansium derivative to be used for primarily for research purposes. This package is intended to be used primarily with our Decision Making toolbox (to be made public pending minor revisions).

Install

From irl-gym/ enter pip install -e .

To uninstall

pip uninstall irl_gym

Assumptions and Specifications

Our enviroments assumes the following:

Input

To declare an algorithm, use env = gym.make("irl_gym/GridTunnel-v0", max_episode_steps=<desired max steps>, seed=<seed>, params=param)

As seen above, each algorithm should accepts as input a dictionary named params containing at a minimum (this is not enforced but necessary to work with our algorithms).

param r_range:

(tuple) of the reward min and reward max

For rendering, include

param render:

(str) render mode (see metadata for options), default: “none”

param prefix:

(string) where to save images, default: “<cwd>/plot”

param save_frames:

(bool) save images for gif, default: False

For logging,

param log_level:

(str) Level of logging to use. For more info see logging levels, default: “WARNING”

Environment Members

Variables

action_space (spaces.<type>)

This is consistent with Gym standard for spaces.

We generally use int for actions but this is not required.

_log (Logger)

Logger for printing and logging.

metadata (dict)

Contains “render_modes” for setting render mode.

observation_space (spaces.<type>)

This is consistent with Gym standard for spaces.

We generally use dict for observations but this is not required.

_params (dict)

For storing parameter values from params.

_state (<observation type>)

Member for representing state.

Functions

Gym Standard More info on Gym

__init__(seedint, paramsdict)

Initializes environment (we lump most of this functionality into reset).

INPUT Seed for RNG, params for environment

reset(seedint, optionsdict)

Resets the environment state, seed, and parameters.

Gym assumes seed is only set in __init__ , however, we do not make this assumption as it gives greater flexibility to the planning tools.

INPUT Seed for RNG, params for environment

RETURNS Observation, Info

step(a<action type>)

Performs desired action and increments the environment by one timestep.

INPUT Desired action

RETURNS Observation, Reward, Is Done, Is Truncated, Info

_get_obs()

Private member that retrieves an observation.

RETURNS Observation

_get_info()

Private member that retrieves desired information.

RETURNS Info

reward(s,a,sp)

Calculates reward for desired state transition.

INPUT State, Action, Resulting State

RETURNS Reward

render()

Renders environment as specified by render_mode.

IRL Standard

get_actions(s)

Retrieves available actions from given state.

May not return all subsquent states if they are too large, in this case it may help to factor states

INPUT State

RETURNS list of actions, list of resulting states

__init__.py

To add your env to the compiler, be sure your file is located in the envs/ folder and insert the following code to your __init__.py:

register(
    id='irl_gym/<EnvName>-v0',
    entry_point='irl_gym.envs:<EnvClass>',
    max_episode_steps=100,
    reward_threshold = None,
    disable_env_checker=False,
    nondeterministic = True,
    order_enforce = True,
    autoreset = False,
    kwargs =
    {
        "params":
        {
        }
    }
)

Citation

If you are using this in your work, please cite as:

@misc{beard2022irl_gym,
    author = {Beard, Jared J., Butts, Ronald M. , Gu, Yu},
    title = {IRL-Gym: Custom Gym environments for academic research},
    year = {2022},
    publisher = {GitHub},
    journal = {GitHub repository},
    howpublished = {\url{https://github.com/wvu-irl/irl-gym}},
}