Back to Repositories

Testing Workflow Extension Capabilities in TaskMatrix

This test suite validates the workflow extension capabilities of the lowCodeLLM module, focusing on the ability to dynamically extend workflows based on task prompts and existing workflow states. The tests ensure proper JSON handling and workflow step generation.

Test Coverage Overview

The test suite provides comprehensive coverage of the extend_workflow functionality in lowCodeLLM.

Key areas tested include:

JSON test case loading and parsing
Workflow extension with various task prompts
Step-based workflow modifications
Result validation for generated workflow steps

Integration points focus on JSON file handling and LLM response processing.

Implementation Analysis

The testing approach implements a systematic verification of workflow extension capabilities using parameterized test cases stored in JSON format. The implementation utilizes Python’s native JSON handling and assertion mechanisms to validate the LLM’s output structure and content.

Technical patterns include:

File-based test case management
Controlled LLM temperature settings
Rate-limited testing with time delays

Technical Details

Testing tools and configuration:

Python’s built-in json module for data handling
System path manipulation for module access
Time-based execution control
LowCodeLLM initialization with specific parameters (0.5, 0)
External test case file: extend_workflow_test_cases.json

Best Practices Demonstrated

The test implementation showcases several testing best practices for AI-driven workflow systems. Notable practices include:

Externalized test data management
Controlled AI model parameters
Rate limiting for API stability
Structured assertion patterns
Modular test organization

chenfei-wu/taskmatrix

LowCodeLLM/src/test/test_extend_workflow.py

            
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT License.

import json
import sys
import os
import time
sys.path.append(os.getcwd())

def test_extend_workflow():
    from lowCodeLLM import lowCodeLLM
    cases = json.load(open("./test/testcases/extend_workflow_test_cases.json", "r"))
    llm = lowCodeLLM(0.5, 0)
    for c in cases:
        task_prompt = c["task_prompt"]
        current_workflow = c["current_workflow"]
        step = c["step"]
        result = llm.extend_workflow(task_prompt, current_workflow, step)
        time.sleep(5)
        assert len(json.loads(result)) >= 1