Back to Repositories

Testing PascaLIGO Syntax Tokenization in monaco-editor

This test suite validates the tokenization functionality for PascaLIGO syntax highlighting in the Monaco Editor. It ensures proper parsing and coloring of language elements like comments, keywords, and numeric values.

Test Coverage Overview

The test suite provides comprehensive coverage of PascaLIGO syntax tokenization.

Key areas tested include:
  • Single-line and multi-line comments
  • Keyword recognition
  • Number formats (integers, floats, hexadecimal)
  • White space handling
  • Delimiter parsing

Implementation Analysis

The testing approach uses a structured pattern of test cases organized by language feature. Each test case defines an input line and expected token segmentation with type information. The implementation leverages the Monaco Editor’s testTokenization helper for consistent verification.

Technical Details

Testing tools and configuration:
  • testTokenization utility from Monaco test runner
  • Token type definitions for PascaLIGO syntax
  • Structured test cases with line/token pairs
  • Precise token index tracking

Best Practices Demonstrated

The test suite exhibits strong testing practices through systematic organization and thorough coverage.

Notable practices include:
  • Isolated test cases for each syntax feature
  • Explicit token boundary checking
  • Comprehensive comment syntax validation
  • Edge case handling for different number formats

microsoft/monaco-editor

src/basic-languages/pascaligo/pascaligo.test.ts

            
/*---------------------------------------------------------------------------------------------
 *  Copyright (c) Microsoft Corporation. All rights reserved.
 *  Licensed under the MIT License. See License.txt in the project root for license information.
 *--------------------------------------------------------------------------------------------*/

import { testTokenization } from '../test/testRunner';

testTokenization('pascaligo', [
	// Comments - single line
	[
		{
			line: '//',
			tokens: [{ startIndex: 0, type: 'comment.pascaligo' }]
		}
	],

	[
		{
			line: '    // a comment',
			tokens: [
				{ startIndex: 0, type: 'white.pascaligo' },
				{ startIndex: 4, type: 'comment.pascaligo' }
			]
		}
	],

	[
		{
			line: '// a comment',
			tokens: [{ startIndex: 0, type: 'comment.pascaligo' }]
		}
	],

	[
		{
			line: '//sticky comment',
			tokens: [{ startIndex: 0, type: 'comment.pascaligo' }]
		}
	],

	// Comments - multi line (single line)
	[
		{
			line: '(**)',
			tokens: [{ startIndex: 0, type: 'comment.pascaligo' }]
		}
	],

	[
		{
			line: '    (* a comment *)',
			tokens: [
				{ startIndex: 0, type: 'white.pascaligo' },
				{ startIndex: 4, type: 'comment.pascaligo' }
			]
		}
	],

	[
		{
			line: '(* a comment *)',
			tokens: [{ startIndex: 0, type: 'comment.pascaligo' }]
		}
	],

	[
		{
			line: '(*sticky comment*)',
			tokens: [{ startIndex: 0, type: 'comment.pascaligo' }]
		}
	],

	// Comments - multi line (multi line)
	[
		{
			line: '(* start of multiline comment ',
			tokens: [{ startIndex: 0, type: 'comment.pascaligo' }]
		},
		{
			line: 'a comment between curly',
			tokens: [{ startIndex: 0, type: 'comment.pascaligo' }]
		},
		{
			line: 'end of multiline comment*)',
			tokens: [{ startIndex: 0, type: 'comment.pascaligo' }]
		}
	],

	// Keywords
	[
		{
			line: 'function add (const a',
			tokens: [
				{ startIndex: 0, type: 'keyword.function.pascaligo' },
				{ startIndex: 8, type: 'white.pascaligo' },
				{ startIndex: 9, type: 'identifier.pascaligo' },
				{ startIndex: 12, type: 'white.pascaligo' },
				{ startIndex: 13, type: 'delimiter.parenthesis.pascaligo' },
				{ startIndex: 14, type: 'keyword.const.pascaligo' },
				{ startIndex: 19, type: 'white.pascaligo' },
				{ startIndex: 20, type: 'identifier.pascaligo' }
			]
		}
	],

	// Numbers
	[
		{
			line: '0',
			tokens: [{ startIndex: 0, type: 'number.pascaligo' }]
		}
	],
	[
		{
			line: '0;',
			tokens: [
				{ startIndex: 0, type: 'number.pascaligo' },
				{ startIndex: 1, type: 'delimiter.pascaligo' }
			]
		}
	],
	[
		{
			line: '2.4',
			tokens: [{ startIndex: 0, type: 'number.float.pascaligo' }]
		}
	],
	[
		{
			line: '2.4;',
			tokens: [
				{ startIndex: 0, type: 'number.float.pascaligo' },
				{ startIndex: 3, type: 'delimiter.pascaligo' }
			]
		}
	],
	[
		{
			line: '$123FF',
			tokens: [{ startIndex: 0, type: 'number.hex.pascaligo' }]
		}
	]
]);