Back to Repositories

Testing LowCodeLLM Workflow Execution in TaskMatrix

This test suite validates the workflow execution functionality in the LowCodeLLM framework, focusing on the execute method’s ability to process tasks with various inputs and workflows. The tests ensure proper handling of task prompts, workflow confirmation, and input processing.

Test Coverage Overview

The test suite provides comprehensive coverage of the LowCodeLLM execute functionality.

Key areas tested include:

Task prompt processing and validation
Workflow confirmation handling
Historical data integration
Input processing and validation

Edge cases are handled through multiple test cases loaded from a JSON file, ensuring robust execution paths.

Implementation Analysis

The testing approach utilizes a data-driven methodology, loading test cases from an external JSON file to validate the execute method’s functionality. The implementation employs systematic verification of string outputs and incorporates delay mechanisms to ensure proper API handling.

Technical patterns include:

JSON-based test case management
Systematic type checking and validation
Rate limiting through time delays

Technical Details

Testing tools and configuration:

Python’s built-in assert mechanism
JSON file-based test case management
System path manipulation for module access
Time management for API rate limiting
LowCodeLLM framework with configurable parameters

Best Practices Demonstrated

The test implementation showcases several testing best practices for workflow validation.

Notable practices include:

External test case management
Systematic output validation
Proper error handling
Rate limit consideration
Modular test structure

chenfei-wu/taskmatrix

LowCodeLLM/src/test/test_execute.py

            
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT License.

import json
import sys
import os
import time
sys.path.append(os.getcwd())

def test_extend_workflow():
    from lowCodeLLM import lowCodeLLM
    cases = json.load(open("./test/testcases/execute_test_cases.json", "r"))
    llm = lowCodeLLM(0.5, 0)
    for c in cases:
        task_prompt = c["task_prompt"]
        confirmed_workflow = c["confirmed_workflow"]
        history = c["history"]
        curr_input = c["curr_input"]
        result = llm.execute(task_prompt, confirmed_workflow, history, curr_input)
        time.sleep(5)
        assert type(result) == str
        assert len(result) > 0