Back to Repositories

Testing GlowTTS D-Vector Speaker Embedding Integration in Coqui-AI TTS

This test suite validates the GlowTTS model training and inference with d-vectors in the Coqui-AI TTS framework. It covers model configuration, training initialization, checkpoint management, and inference capabilities with speaker embeddings.

Test Coverage Overview

The test suite provides comprehensive coverage of GlowTTS model functionality with d-vector speaker embeddings. Key areas tested include:

Model configuration and initialization with speaker embedding support
Training pipeline with LJSpeech dataset integration
Checkpoint saving and restoration verification
Inference pipeline with speaker ID handling
Config integrity validation after model restoration

Implementation Analysis

The testing approach implements a full training-inference cycle using the Coqui-AI TTS CLI interface. It employs a modular structure to test:

GlowTTSConfig setup with d-vector specifications
Training initialization with custom parameters
Checkpoint management and model restoration
CLI-based inference with speaker embeddings

Technical Details

Testing infrastructure includes:

CUDA device management for GPU testing
Dynamic path handling for outputs and checkpoints
JSON configuration validation
LJSpeech test dataset integration
Speaker embedding dimension verification (256-dim)
Phoneme cache management

Best Practices Demonstrated

The test implementation showcases several testing best practices:

Isolated test environment with cleanup
Comprehensive config validation
Sequential training-inference pipeline verification
Resource cleanup after test completion
GPU device handling abstraction

coqui-ai/tts

tests/tts_tests2/test_glow_tts_d-vectors_train.py

            
import glob
import json
import os
import shutil

from trainer import get_last_checkpoint

from tests import get_device_id, get_tests_output_path, run_cli
from TTS.tts.configs.glow_tts_config import GlowTTSConfig

config_path = os.path.join(get_tests_output_path(), "test_model_config.json")
output_path = os.path.join(get_tests_output_path(), "train_outputs")


config = GlowTTSConfig(
    batch_size=2,
    eval_batch_size=8,
    num_loader_workers=0,
    num_eval_loader_workers=0,
    text_cleaner="english_cleaners",
    use_phonemes=True,
    phoneme_language="en-us",
    phoneme_cache_path="tests/data/ljspeech/phoneme_cache/",
    run_eval=True,
    test_delay_epochs=-1,
    epochs=1,
    print_step=1,
    print_eval=True,
    test_sentences=[
        "Be a voice, not an echo.",
    ],
    data_dep_init_steps=1.0,
    use_speaker_embedding=False,
    use_d_vector_file=True,
    d_vector_file="tests/data/ljspeech/speakers.json",
    d_vector_dim=256,
)
config.audio.do_trim_silence = True
config.audio.trim_db = 60
config.save_json(config_path)

# train the model for one epoch
command_train = (
    f"CUDA_VISIBLE_DEVICES='{get_device_id()}' python TTS/bin/train_tts.py --config_path {config_path} "
    f"--coqpit.output_path {output_path} "
    "--coqpit.datasets.0.formatter ljspeech_test "
    "--coqpit.datasets.0.meta_file_train metadata.csv "
    "--coqpit.datasets.0.meta_file_val metadata.csv "
    "--coqpit.datasets.0.path tests/data/ljspeech "
    "--coqpit.datasets.0.meta_file_attn_mask tests/data/ljspeech/metadata_attn_mask.txt "
    "--coqpit.test_delay_epochs 0"
)
run_cli(command_train)

# Find latest folder
continue_path = max(glob.glob(os.path.join(output_path, "*/")), key=os.path.getmtime)

# Inference using TTS API
continue_config_path = os.path.join(continue_path, "config.json")
continue_restore_path, _ = get_last_checkpoint(continue_path)
out_wav_path = os.path.join(get_tests_output_path(), "output.wav")
speaker_id = "ljspeech-1"
continue_speakers_path = config.d_vector_file

# Check integrity of the config
with open(continue_config_path, "r", encoding="utf-8") as f:
    config_loaded = json.load(f)
assert config_loaded["characters"] is not None
assert config_loaded["output_path"] in continue_path
assert config_loaded["test_delay_epochs"] == 0

# Load the model and run inference
inference_command = f"CUDA_VISIBLE_DEVICES='{get_device_id()}' tts --text 'This is an example.' --speaker_idx {speaker_id} --speakers_file_path {continue_speakers_path} --config_path {continue_config_path} --model_path {continue_restore_path} --out_path {out_wav_path}"
run_cli(inference_command)

# restore the model and continue training for one more epoch
command_train = f"CUDA_VISIBLE_DEVICES='{get_device_id()}' python TTS/bin/train_tts.py --continue_path {continue_path} "
run_cli(command_train)
shutil.rmtree(continue_path)