Back to Repositories

Testing File-Based Metadata Management Operations in Alibaba Canal

A comprehensive test suite for FileMixedMetaManager in Alibaba Canal, focusing on metadata management and persistence operations. This test class validates subscription handling, batch processing, and cursor management functionalities using file-based storage.

Test Coverage Overview

The test suite provides extensive coverage of FileMixedMetaManager’s core functionalities.

Key areas tested include:

Subscription management and persistence
Batch operation handling and cleanup
Cursor position tracking and recovery
Data directory management and cleanup

Edge cases covered include system restarts and data persistence verification across manager instances.

Implementation Analysis

The testing approach employs JUnit 4 framework with systematic validation of metadata operations. The implementation uses a temporary file system storage pattern for test isolation.

Notable patterns include:

Before/After test lifecycle management
Manager instance recreation for persistence validation
Periodic synchronization testing with configurable intervals
Clean state verification through directory cleanup

Technical Details

Testing infrastructure includes:

JUnit 4 testing framework
Apache Commons IO for file operations
System temporary directory for test data storage
Custom assertion methods from AbstractMetaManagerTest
Configurable synchronization periods
ClientIdentity and Position objects for state management

Best Practices Demonstrated

The test suite exemplifies high-quality testing practices through careful test isolation and comprehensive validation.

Notable practices include:

Proper test cleanup and setup
Verification of data persistence across instances
Systematic testing of all manager operations
Clear separation of test scenarios
Effective use of inheritance for common test functionality

alibaba/canal

meta/src/test/java/com/alibaba/otter/canal/meta/FileMixedMetaManagerTest.java

            
package com.alibaba.otter.canal.meta;

import java.io.File;
import java.io.IOException;
import java.util.List;
import java.util.Map;

import org.apache.commons.io.FileUtils;
import org.junit.Assert;
import org.junit.Before;
import org.junit.Ignore;
import org.junit.Test;

import com.alibaba.otter.canal.protocol.ClientIdentity;
import com.alibaba.otter.canal.protocol.position.Position;
import com.alibaba.otter.canal.protocol.position.PositionRange;
@Ignore
public class FileMixedMetaManagerTest extends AbstractMetaManagerTest {

    private static final String tmp     = System.getProperty("java.io.tmpdir", "/tmp");
    private static final File   dataDir = new File(tmp, "canal");

    @Before
    public void setUp() {
        try {
            FileUtils.deleteDirectory(dataDir);
        } catch (IOException e) {
            Assert.fail(e.getMessage());
        }
    }

    @Test
    public void testSubscribeAll() {
        FileMixedMetaManager metaManager = new FileMixedMetaManager();
        metaManager.setDataDirByFile(dataDir);
        metaManager.setPeriod(100);

        metaManager.start();
        doSubscribeTest(metaManager);

        sleep(2000L);
        // 重新构建一次，能获得上一次zk上的记录
        FileMixedMetaManager metaManager2 = new FileMixedMetaManager();
        metaManager2.setDataDirByFile(dataDir);
        metaManager2.setPeriod(100);
        metaManager2.start();

        List<ClientIdentity> clients = metaManager2.listAllSubscribeInfo(destination);
        Assert.assertEquals(2, clients.size());
        metaManager.stop();
    }

    @Test
    public void testBatchAll() {
        FileMixedMetaManager metaManager = new FileMixedMetaManager();
        metaManager.setDataDirByFile(dataDir);
        metaManager.setPeriod(100);

        metaManager.start();
        doBatchTest(metaManager);

        metaManager.clearAllBatchs(clientIdentity);
        Map<Long, PositionRange> ranges = metaManager.listAllBatchs(clientIdentity);
        Assert.assertEquals(0, ranges.size());
        metaManager.stop();
    }

    @Test
    public void testCursorAll() {
        FileMixedMetaManager metaManager = new FileMixedMetaManager();
        metaManager.setDataDirByFile(dataDir);
        metaManager.setPeriod(100);
        metaManager.start();

        Position lastPosition = doCursorTest(metaManager);

        sleep(1000L);
        // 重新构建一次，能获得上一次zk上的记录
        FileMixedMetaManager metaManager2 = new FileMixedMetaManager();
        metaManager2.setDataDirByFile(dataDir);
        metaManager2.setPeriod(100);
        metaManager2.start();

        Position position = metaManager2.getCursor(clientIdentity);
        Assert.assertEquals(position, lastPosition);
        metaManager.stop();
    }
}