Back to Repositories

Testing Container Runtime Language Compatibility in OpenHands

This test suite validates container runtime environments and language version compatibility in the OpenHands EventStreamRuntime. It ensures proper language installations and version checking for Python, Node.js, and Go environments within containerized test environments.

Test Coverage Overview

The test suite provides comprehensive coverage of container image validation and language version verification.

Key areas tested include:

Python 3.12 environment validation including pip availability
Node.js v22 version verification
Go 1.23 runtime environment checking
Command execution and exit code validation

Integration points include container runtime initialization and command execution through CmdRunAction.

Implementation Analysis

The testing approach uses pytest fixtures for runtime initialization and cleanup, with selective test execution based on container image types.

Key patterns include:

Fixture-based test environment setup
Image-specific test skipping logic
Command execution validation
Version string verification

Leverages pytest.skip() for conditional test execution and fixture-based resource management.

Technical Details

Testing tools and configuration:

pytest for test execution and assertions
Custom runtime_cls fixture for container management
EventStreamRuntime for command execution
CmdRunAction for command invocation
Logger integration for action and observation tracking

Best Practices Demonstrated

The test suite exemplifies strong testing practices through isolated test cases and thorough environment validation.

Notable practices include:

Proper test isolation and cleanup
Conditional test execution
Explicit version checking
Comprehensive error handling
Clear test case organization

all-hands-ai/openhands

tests/runtime/test_images.py

            
"""Image-related tests for the EventStreamRuntime, which connects to the ActionExecutor running in the sandbox."""

import pytest
from conftest import _close_test_runtime, _load_runtime

from openhands.core.logger import openhands_logger as logger
from openhands.events.action import CmdRunAction

# ============================================================================================================================
# Image-specific tests
# ============================================================================================================================


def test_bash_python_version(temp_dir, runtime_cls, base_container_image):
    """Make sure Python is available in bash."""
    if base_container_image not in [
        'python:3.12-bookworm',
    ]:
        pytest.skip('This test is only for python-related images')

    runtime = _load_runtime(
        temp_dir, runtime_cls, base_container_image=base_container_image
    )

    action = CmdRunAction(command='which python')
    logger.info(action, extra={'msg_type': 'ACTION'})
    obs = runtime.run_action(action)
    logger.info(obs, extra={'msg_type': 'OBSERVATION'})
    assert obs.exit_code == 0

    action = CmdRunAction(command='python --version')
    logger.info(action, extra={'msg_type': 'ACTION'})
    obs = runtime.run_action(action)
    logger.info(obs, extra={'msg_type': 'OBSERVATION'})
    assert obs.exit_code == 0
    assert 'Python 3.12' in obs.content  # Check for specific version

    action = CmdRunAction(command='pip --version')
    logger.info(action, extra={'msg_type': 'ACTION'})
    obs = runtime.run_action(action)
    logger.info(obs, extra={'msg_type': 'OBSERVATION'})
    assert obs.exit_code == 0
    assert 'pip' in obs.content  # Check that pip is available

    _close_test_runtime(runtime)


def test_nodejs_22_version(temp_dir, runtime_cls, base_container_image):
    """Make sure Node.js is available in bash."""
    if base_container_image not in [
        'node:22-bookworm',
    ]:
        pytest.skip('This test is only for nodejs-related images')

    runtime = _load_runtime(
        temp_dir, runtime_cls, base_container_image=base_container_image
    )

    action = CmdRunAction(command='node --version')
    logger.info(action, extra={'msg_type': 'ACTION'})
    obs = runtime.run_action(action)
    logger.info(obs, extra={'msg_type': 'OBSERVATION'})
    assert obs.exit_code == 0
    assert 'v22' in obs.content  # Check for specific version

    _close_test_runtime(runtime)


def test_go_version(temp_dir, runtime_cls, base_container_image):
    """Make sure Go is available in bash."""
    if base_container_image not in [
        'golang:1.23-bookworm',
    ]:
        pytest.skip('This test is only for go-related images')

    runtime = _load_runtime(
        temp_dir, runtime_cls, base_container_image=base_container_image
    )

    action = CmdRunAction(command='go version')
    logger.info(action, extra={'msg_type': 'ACTION'})
    obs = runtime.run_action(action)
    logger.info(obs, extra={'msg_type': 'OBSERVATION'})
    assert obs.exit_code == 0
    assert 'go1.23' in obs.content  # Check for specific version

    _close_test_runtime(runtime)