Back to Repositories

Testing AI Model Identifier Parsing in OpenHands

This test suite validates the extractModelAndProvider utility function in OpenHands, which parses model identifiers from various AI providers into standardized components. The tests verify correct handling of different provider formats and special cases for popular AI models.

Test Coverage Overview

The test suite provides comprehensive coverage of model identifier parsing across multiple AI providers including Azure, Vertex AI, Cohere, Cloudflare, and popular models from OpenAI and Anthropic. Key test cases include:

Basic provider/model separation
Complex nested path handling
Different separator characters (‘/’ and ‘.’)
Auto-provider detection for known models
Edge cases with empty providers

Implementation Analysis

The testing approach uses Jest’s describe/it structure with multiple expect assertions to validate the extractModelAndProvider function. The implementation leverages TypeScript’s type system to ensure correct object structure returns, with each test case verifying the provider, model, and separator properties.

Pattern matching and string parsing are thoroughly tested across different format variations.

Technical Details

Testing Framework: Jest/Vitest
Language: TypeScript
Key Components:

describe/it blocks for test organization
expect assertions for validation
Type-safe return object verification
Multiple test cases in single it blocks

Best Practices Demonstrated

The test suite exemplifies several testing best practices including comprehensive edge case coverage, clear test case organization, and efficient test grouping. Notable practices include:

Logical grouping of related test cases
Clear test descriptions
Comprehensive provider format coverage
Proper handling of special cases

all-hands-ai/openhands

frontend/__tests__/utils/extract-model-and-provider.test.ts

            
import { describe, it, expect } from "vitest";
import { extractModelAndProvider } from "../../src/utils/extract-model-and-provider";

describe("extractModelAndProvider", () => {
  it("should work", () => {
    expect(extractModelAndProvider("azure/ada")).toEqual({
      provider: "azure",
      model: "ada",
      separator: "/",
    });

    expect(
      extractModelAndProvider("azure/standard/1024-x-1024/dall-e-2"),
    ).toEqual({
      provider: "azure",
      model: "standard/1024-x-1024/dall-e-2",
      separator: "/",
    });

    expect(extractModelAndProvider("vertex_ai_beta/chat-bison")).toEqual({
      provider: "vertex_ai_beta",
      model: "chat-bison",
      separator: "/",
    });

    expect(extractModelAndProvider("cohere.command-r-v1:0")).toEqual({
      provider: "cohere",
      model: "command-r-v1:0",
      separator: ".",
    });

    expect(
      extractModelAndProvider(
        "cloudflare/@cf/mistral/mistral-7b-instruct-v0.1",
      ),
    ).toEqual({
      provider: "cloudflare",
      model: "@cf/mistral/mistral-7b-instruct-v0.1",
      separator: "/",
    });

    expect(extractModelAndProvider("together-ai-21.1b-41b")).toEqual({
      provider: "",
      model: "together-ai-21.1b-41b",
      separator: "",
    });
  });

  it("should add provider for popular models", () => {
    expect(extractModelAndProvider("gpt-4o-mini")).toEqual({
      provider: "openai",
      model: "gpt-4o-mini",
      separator: "/",
    });

    expect(extractModelAndProvider("gpt-4o")).toEqual({
      provider: "openai",
      model: "gpt-4o",
      separator: "/",
    });

    expect(extractModelAndProvider("claude-3-5-sonnet-20240620")).toEqual({
      provider: "anthropic",
      model: "claude-3-5-sonnet-20240620",
      separator: "/",
    });

    expect(extractModelAndProvider("claude-3-haiku-20240307")).toEqual({
      provider: "anthropic",
      model: "claude-3-haiku-20240307",
      separator: "/",
    });

    expect(extractModelAndProvider("claude-2.1")).toEqual({
      provider: "anthropic",
      model: "claude-2.1",
      separator: "/",
    });
  });
});