Back to Repositories

Testing LangChain-FastChat Integration Workflow in lm-sys/FastChat

This test suite validates the integration between FastChat and LangChain, focusing on document processing, embedding generation, and question-answering capabilities using OpenAI-compatible APIs. It demonstrates how to use FastChat as a drop-in replacement for OpenAI services in LangChain applications.

Test Coverage Overview

The test suite covers essential LangChain-FastChat integration points including:

Document loading and processing capabilities
Vector embeddings generation using OpenAI-compatible endpoints
Question-answering chain functionality
Multiple query handling and response generation

Implementation Analysis

The testing approach utilizes a straightforward chain pattern that combines document loading, embedding creation, and LLM-based querying. It leverages FastChat’s API compatibility with OpenAI interfaces, demonstrating how to configure environment variables and initialize key components like ChatOpenAI and OpenAIEmbeddings.

The test implements a VectorstoreIndexCreator for document processing and uses a predefined set of diverse questions to validate response generation.

Technical Details

Testing Components:

LangChain’s ChatOpenAI and OpenAIEmbeddings classes
TextLoader for document processing
VectorstoreIndexCreator for index creation
FastChat server running locally with vicuna-7b-v1.5 model
Environment configuration for API endpoint and key

Best Practices Demonstrated

The test exemplifies several testing best practices:

Clear separation of configuration and test logic
Comprehensive coverage of different question types
Proper environment variable management
Modular component initialization
Structured query-response validation approach

lm-sys/fastchat

tests/test_openai_langchain.py

            
# Usage:
# python3 -m fastchat.serve.model_worker --model-path lmsys/vicuna-7b-v1.5 --model-names gpt-3.5-turbo,text-davinci-003,text-embedding-ada-002
# export OPENAI_API_BASE=http://localhost:8000/v1
# export OPENAI_API_KEY=EMPTY
# wget https://raw.githubusercontent.com/hwchase17/langchain/v0.0.200/docs/modules/state_of_the_union.txt

import os

from langchain.chat_models import ChatOpenAI
from langchain.document_loaders import TextLoader
from langchain.embeddings import OpenAIEmbeddings
from langchain.indexes import VectorstoreIndexCreator


def test_chain():
    embedding = OpenAIEmbeddings(model="text-embedding-ada-002")
    loader = TextLoader("state_of_the_union.txt")
    index = VectorstoreIndexCreator(embedding=embedding).from_loaders([loader])

    llm = ChatOpenAI(model="gpt-3.5-turbo")

    questions = [
        "Who is the speaker",
        "What did the president say about Ketanji Brown Jackson",
        "What are the threats to America",
        "Who are mentioned in the speech",
        "Who is the vice president",
        "How many projects were announced",
    ]

    for query in questions:
        print("Query:", query)
        print("Answer:", index.query(query, llm=llm))


if __name__ == "__main__":
    os.environ["OPENAI_API_BASE"] = "http://localhost:8000/v1"
    os.environ["OPENAI_API_KEY"] = "empty"
    test_chain()