Back to Repositories

Validating OrderEnforcing Wrapper Behavior in OpenAI Gym

This test suite validates the OrderEnforcing wrapper functionality in OpenAI Gym environments, ensuring proper initialization and method call ordering. It verifies that environments are correctly wrapped and that operations are executed in the expected sequence.

Test Coverage Overview

The test suite provides comprehensive coverage of the OrderEnforcing wrapper implementation.

Key areas tested include:

Automatic wrapper application through gym.make()
Step and render method restrictions before reset
Proper behavior after environment reset
Optional render order enforcement disabling

Implementation Analysis

The testing approach utilizes pytest’s parametrize feature to validate OrderEnforcing across multiple environment specifications. The tests employ explicit wrapper verification and exception handling to ensure proper order enforcement.

Technical patterns include:

Parametrized testing across environment specs
Exception-based validation
State tracking verification

Technical Details

Testing infrastructure includes:

pytest framework for test organization
OpenAI Gym environment utilities
CartPoleEnv as reference implementation
Custom wrapper detection utilities
ResetNeeded error handling

Best Practices Demonstrated

The test suite exemplifies several testing best practices:

Isolation of test cases
Comprehensive edge case coverage
Clear test case organization
Explicit state validation
Proper setup and teardown patterns

openai/gym

tests/wrappers/test_order_enforcing.py

            
import pytest

import gym
from gym.envs.classic_control import CartPoleEnv
from gym.error import ResetNeeded
from gym.wrappers import OrderEnforcing
from tests.envs.utils import all_testing_env_specs
from tests.wrappers.utils import has_wrapper


@pytest.mark.parametrize(
    "spec", all_testing_env_specs, ids=[spec.id for spec in all_testing_env_specs]
)
def test_gym_make_order_enforcing(spec):
    """Checks that gym.make wrappers the environment with the OrderEnforcing wrapper."""
    env = gym.make(spec.id, disable_env_checker=True)

    assert has_wrapper(env, OrderEnforcing)


def test_order_enforcing():
    """Checks that the order enforcing works as expected, raising an error before reset is called and not after."""
    # The reason for not using gym.make is that all environments are by default wrapped in the order enforcing wrapper
    env = CartPoleEnv(render_mode="rgb_array_list")
    assert not has_wrapper(env, OrderEnforcing)

    # Assert that the order enforcing works for step and render before reset
    order_enforced_env = OrderEnforcing(env)
    assert order_enforced_env.has_reset is False
    with pytest.raises(ResetNeeded):
        order_enforced_env.step(0)
    with pytest.raises(ResetNeeded):
        order_enforced_env.render()
    assert order_enforced_env.has_reset is False

    # Assert that the Assertion errors are not raised after reset
    order_enforced_env.reset()
    assert order_enforced_env.has_reset is True
    order_enforced_env.step(0)
    order_enforced_env.render()

    # Assert that with disable_render_order_enforcing works, the environment has already been reset
    env = CartPoleEnv(render_mode="rgb_array_list")
    env = OrderEnforcing(env, disable_render_order_enforcing=True)
    env.render()  # no assertion error