Back to Repositories

Testing Apache2 Log Parser Implementation in Fluentd

This test suite validates the Apache2 log parser functionality in Fluentd, ensuring accurate parsing of Apache access log entries. The tests verify proper extraction of fields like IP address, timestamp, HTTP method, and user agent from log lines.

Test Coverage Overview

The test suite provides comprehensive coverage of Apache2 log parsing scenarios.

Key areas tested include:

Standard Apache log format parsing
Timestamp conversion accuracy
HTTP request components extraction
Handling of missing HTTP version
Proper escape sequence processing

Implementation Analysis

The testing approach uses Fluent’s Test::Driver::Parser framework to validate the Apache2Parser plugin implementation. Tests verify correct field extraction and data type conversion, with particular attention to timestamp parsing and special character handling.

The suite employs fixture-based testing with predefined expected outcomes for various log entry formats.

Technical Details

Testing components include:

Fluent::Test::Driver::Parser for parser testing
Test::Unit framework for assertions
Custom time format validation
Regular expression pattern matching
Escape sequence handling verification

Best Practices Demonstrated

The test suite exemplifies several testing best practices including isolated test cases, comprehensive setup methods, and thorough edge case coverage.

Notable practices include:

Proper test initialization and cleanup
Consistent assertion patterns
Clear test case naming
Modular test organization

fluent/fluentd

test/plugin/test_parser_apache2.rb

            
require_relative '../helper'
require 'fluent/test/driver/parser'
require 'fluent/plugin/parser'

class Apache2ParserTest < ::Test::Unit::TestCase
  def setup
    Fluent::Test.setup
    @parser = Fluent::Test::Driver::Parser.new(Fluent::Plugin::Apache2Parser)
    @expected = {
      'user'    => nil,
      'method'  => 'GET',
      'code'    => 200,
      'size'    => 777,
      'host'    => '192.168.0.1',
      'path'    => '/',
      'referer' => nil,
      'agent'   => 'Opera/12.0'
    }
    @parser.configure({})
  end

  def test_parse
    @parser.instance.parse('192.168.0.1 - - [28/Feb/2013:12:00:00 +0900] "GET / HTTP/1.1" 200 777 "-" "Opera/12.0"') { |time, record|
      assert_equal(event_time('28/Feb/2013:12:00:00 +0900', format: '%d/%b/%Y:%H:%M:%S %z'), time)
      assert_equal(@expected, record)
    }
    assert_equal(Fluent::Plugin::Apache2Parser::REGEXP,
                 @parser.instance.patterns['format'])
    assert_equal(Fluent::Plugin::Apache2Parser::TIME_FORMAT,
                 @parser.instance.patterns['time_format'])
  end

  def test_parse_without_http_version
    @parser.instance.parse('192.168.0.1 - - [28/Feb/2013:12:00:00 +0900] "GET /" 200 777 "-" "Opera/12.0"') { |time, record|
      assert_equal(event_time('28/Feb/2013:12:00:00 +0900', format: '%d/%b/%Y:%H:%M:%S %z'), time)
      assert_equal(@expected, record)
    }
  end

  def test_parse_with_escape_sequence
    @parser.instance.parse('192.168.0.1 - - [28/Feb/2013:12:00:00 +0900] "GET /\" HTTP/1.1" 200 777 "referer \\\ \"" "user agent \\\ \""') { |_, record|
      assert_equal('/\"', record['path'])
      assert_equal('referer \\\ \"', record['referer'])
      assert_equal('user agent \\\ \"', record['agent'])
    }
  end
end