Back to Repositories

Testing OpenSearch REST DAO Integration in conductor-oss/conductor

This test suite provides the base testing infrastructure for OpenSearch REST DAO operations in Conductor. It establishes a foundation for testing OpenSearch integration by managing client connections, index operations, and cleanup procedures.

Test Coverage Overview

The test suite covers essential OpenSearch REST client operations and database indexing functionality.

Key areas tested include:

REST client initialization and configuration
Index management operations
Connection handling and cleanup
Error handling and recovery scenarios

Integration points focus on OpenSearch client connectivity and index manipulation through REST APIs.

Implementation Analysis

The testing approach implements a base test class pattern using JUnit for OpenSearch DAO testing. It utilizes Spring’s RetryTemplate for resilient operations and implements proper test lifecycle management through @Before and @After annotations.

Technical patterns include:

Container-based test infrastructure
Dynamic host/port configuration
Systematic index cleanup
Resource management for client connections

Technical Details

Testing tools and configuration:

JUnit test framework
OpenSearch REST client
Spring RetryTemplate
Docker container for OpenSearch instance
Custom properties configuration
BufferedReader for index management

Best Practices Demonstrated

The test implementation showcases several testing best practices for database integration testing.

Notable practices include:

Proper test isolation through setup/teardown
Resource cleanup after test execution
Abstraction of common test functionality
Configurable test environment
Structured error handling

conductor-oss/conductor

os-persistence/src/test/java/com/netflix/conductor/os/dao/index/OpenSearchRestDaoBaseTest.java

            
/*
 * Copyright 2023 Conductor Authors.
 * <p>
 * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
 * the License. You may obtain a copy of the License at
 * <p>
 * http://www.apache.org/licenses/LICENSE-2.0
 * <p>
 * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
 * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
 * specific language governing permissions and limitations under the License.
 */
package com.netflix.conductor.os.dao.index;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.Reader;

import org.apache.http.HttpHost;
import org.junit.After;
import org.junit.Before;
import org.opensearch.client.Request;
import org.opensearch.client.Response;
import org.opensearch.client.RestClient;
import org.opensearch.client.RestClientBuilder;
import org.springframework.retry.support.RetryTemplate;

public abstract class OpenSearchRestDaoBaseTest extends OpenSearchTest {

    protected RestClient restClient;
    protected OpenSearchRestDAO indexDAO;

    @Before
    public void setup() throws Exception {
        String httpHostAddress = container.getHttpHostAddress();
        String host = httpHostAddress.split(":")[1].replace("//", "");
        int port = Integer.parseInt(httpHostAddress.split(":")[2]);

        properties.setUrl(httpHostAddress);

        RestClientBuilder restClientBuilder = RestClient.builder(new HttpHost(host, port, "http"));
        restClient = restClientBuilder.build();

        indexDAO =
                new OpenSearchRestDAO(
                        restClientBuilder, new RetryTemplate(), properties, objectMapper);
        indexDAO.setup();
    }

    @After
    public void tearDown() throws Exception {
        deleteAllIndices();

        if (restClient != null) {
            restClient.close();
        }
    }

    private void deleteAllIndices() throws IOException {
        Response beforeResponse = restClient.performRequest(new Request("GET", "/_cat/indices"));
        Reader streamReader = new InputStreamReader(beforeResponse.getEntity().getContent());
        BufferedReader bufferedReader = new BufferedReader(streamReader);

        String line;
        while ((line = bufferedReader.readLine()) != null) {
            System.out.println("Deleting line: " + line);
            String[] fields = line.split("(\\s+)");
            String endpoint = String.format("/%s", fields[2]);
            System.out.println("Deleting index: " + endpoint);
            restClient.performRequest(new Request("DELETE", endpoint));
        }
    }
}