如何在Antlr G4语法中添加正则规则并解决语法错误？

数据老陈

2026-7-4

解决Antlr G4正则表达式字面量语法错误问题

问题描述

添加regex_Literal规则后出现以下语法错误：

error(50): Expression.g4:149:15: syntax error: '/' came as a complete surprise to me while looking for rule element
error(50): Expression.g4:149:27: syntax error: '/' came as a complete surprise to me while looking for rule element

目标是让函数第一个参数支持正则表达式字面量，原语法代码如下：

grammar Expression
  ;

options {
  caseInsensitive = true;
}

// Parser rules.

parse_all: expressionSequence EOF;

expressionSequence: expression (COMMA expression)*;

expression
  : OPEN_PARENS expression CLOSE_PARENS                                                      # Parenthesis
  | MINUS right = expression                                                                 # UnaryMinus
  | NOT right = expression                                                                   # Not
  | functionDeclaration                                                                      # Function
  | left = expression op = (ASTERISK | SLASH) right = expression                             # Arithmetic
  | left = expression op = (PLUS | MINUS) right = expression                                 # Arithmetic
  | left = expression op = (EQ | NE | GT | GTE | LT | LTE) right = expression                # Comparison
  | left = expression op = TILDE right = expression                                          # Contains
  | left = expression IN right = expression_array                                            # In
  | left = expression op = (AND | OR) right = expression                                     # AndOr
  | condition = expression QUESTIONMARK true_expr = expression COLON false_expr = expression # Ternary
  | ifStatement                                                                              # IfElse
  | expression_array                                                                         # Array
  | field = IDENTIFIER DOT attribute = IDENTIFIER                                            # AttributeAccessor
  | STRING_LITERAL                                                                           # StringLiteral
  | boolean_literal                                                                          # BooleanLiteral
  | IDENTIFIER                                                                               # Identifier
  | numeric_literal                                                                          # NumericLiteral
  | regex_Literal                                                                            # RegexLiteral
  ;

functionDeclaration
  : DOLLAR function_name = IDENTIFIER OPEN_PARENS expressionSequence? CLOSE_PARENS
  ;

conditionalStatement
  : condition = expression QUESTIONMARK true_expr = expression COLON false_expr = expression
  ;

ifStatement
  : IF condition_block (ELSEIF condition_block)* ELSE statement_block
  ;

condition_block: expression THEN statement_block;

statement_block: expression;

expression_array: OPEN_BRACKET expressionSequence CLOSE_BRACKET;

boolean_literal: TRUE # True | FALSE # False;

numeric_literal: NUMBER # Numeric | INTEGER # Integer;

// Lexer rules.

OPEN_BRACKET: '[';
CLOSE_BRACKET: ']';
OPEN_PARENS: '(';
CLOSE_PARENS: ')';

COLON: ':';
COMMA: ',';
DOLLAR: '$';
DOT: '.';
DOUBLE_QUOTE: '&quot;';
QUESTIONMARK: '?';
SEMICOLON: ';';
SINGLE_QUOTE: '\'';
TILDE: '~';

EQ: '=';
NE: '!=';
GT: '&gt;';
GTE: '&gt;=';
LT: '&lt;';
LTE: '&lt;=';

ASTERISK: '*';
MINUS: '-';
PLUS: '+';
SLASH: '/';

AND: 'AND';
OR: 'OR';
NOT: 'NOT';
IN: 'IN';

IF: 'IF';
THEN: 'THEN';
ELSE: 'ELSE';
ELSEIF: 'ELSEIF';

TRUE: 'TRUE';
FALSE: 'FALSE';

NUMBER: ( DIGIT+ DOT DIGIT+ | DIGIT+ DOT | DOT DIGIT+);
INTEGER: DIGIT+;

IDENTIFIER: LETTER+ (LETTER | DIGIT)*;
STRING_LITERAL: DOUBLE_QUOTE (~[\r\n\\&quot;])*? DOUBLE_QUOTE;

fragment LETTER: [A-Z] | '_';
fragment DIGIT: [0-9];

fragment NewLine
  : '\r\n'
  | '\r'
  | '\n'
  | '\u0085' // &lt;Next Line CHARACTER (U+0085)&gt;'
  | '\u2028' //'&lt;Line Separator CHARACTER (U+2028)&gt;'
  | '\u2029' //'&lt;Paragraph Separator CHARACTER (U+2029)&gt;'
  ;

fragment Whitespace
  : UnicodeClassZS //'&lt;Any Character With Unicode Class Zs&gt;'
  | '\u0009'       //'&lt;Horizontal Tab Character (U+0009)&gt;'
  | '\u000B'       //'&lt;Vertical Tab Character (U+000B)&gt;'
  | '\u000C'       //'&lt;Form Feed Character (U+000C)&gt;'
  ;

fragment UnicodeClassZS
  : '\u0020' // SPACE
  | '\u00A0' // NO_BREAK SPACE
  | '\u1680' // OGHAM SPACE MARK
  | '\u180E' // MONGOLIAN VOWEL SEPARATOR
  | '\u2000' // EN QUAD
  | '\u2001' // EM QUAD
  | '\u2002' // EN SPACE
  | '\u2003' // EM SPACE
  | '\u2004' // THREE_PER_EM SPACE
  | '\u2005' // FOUR_PER_EM SPACE
  | '\u2006' // SIX_PER_EM SPACE
  | '\u2008' // PUNCTUATION SPACE
  | '\u2009' // THIN SPACE
  | '\u200A' // HAIR SPACE
  | '\u202F' // NARROW NO_BREAK SPACE
  | '\u3000' // IDEOGRAPHIC SPACE
  | '\u205F' // MEDIUM MATHEMATICAL SPACE
  ;

WHITESPACES: (Whitespace | NewLine)+ -&gt; skip;

regex_Literal: '/' regexContent? '/';

regexContent: ~[/]+ | '/' ~[/]+;

ANY: .;

错误原因

规则位置错误：regex_Literal和regexContent是小写开头的Parser规则，却被放在了Lexer规则区域。Antlr对Lexer（大写开头）和Parser规则的语法要求不同，Lexer规则中不能直接使用/这类裸字符作为规则元素，必须引用已定义的Lexer Token。
直接使用裸字符：规则中直接写/，但/已经被定义为SLASH这个Lexer Token，Parser规则中必须使用Token名称而非裸字符。

解决方案

步骤1：调整规则位置

将regex_Literal和regexContent移到Parser规则区域，放在numeric_literal规则之后、// Lexer rules.注释之前。

步骤2：替换裸字符为Token名称

把规则中的/替换为已定义的SLASH Token，并优化regexContent规则以支持正则内的转义斜杠（符合常规正则写法）。

修改后的完整语法代码

grammar Expression
  ;

options {
  caseInsensitive = true;
}

// Parser rules.

parse_all: expressionSequence EOF;

expressionSequence: expression (COMMA expression)*;

expression
  : OPEN_PARENS expression CLOSE_PARENS                                                      # Parenthesis
  | MINUS right = expression                                                                 # UnaryMinus
  | NOT right = expression                                                                   # Not
  | functionDeclaration                                                                      # Function
  | left = expression op = (ASTERISK | SLASH) right = expression                             # Arithmetic
  | left = expression op = (PLUS | MINUS) right = expression                                 # Arithmetic
  | left = expression op = (EQ | NE | GT | GTE | LT | LTE) right = expression                # Comparison
  | left = expression op = TILDE right = expression                                          # Contains
  | left = expression IN right = expression_array                                            # In
  | left = expression op = (AND | OR) right = expression                                     # AndOr
  | condition = expression QUESTIONMARK true_expr = expression COLON false_expr = expression # Ternary
  | ifStatement                                                                              # IfElse
  | expression_array                                                                         # Array
  | field = IDENTIFIER DOT attribute = IDENTIFIER                                            # AttributeAccessor
  | STRING_LITERAL                                                                           # StringLiteral
  | boolean_literal                                                                          # BooleanLiteral
  | IDENTIFIER                                                                               # Identifier
  | numeric_literal                                                                          # NumericLiteral
  | regex_Literal                                                                            # RegexLiteral
  ;

functionDeclaration
  : DOLLAR function_name = IDENTIFIER OPEN_PARENS expressionSequence? CLOSE_PARENS
  ;

conditionalStatement
  : condition = expression QUESTIONMARK true_expr = expression COLON false_expr = expression
  ;

ifStatement
  : IF condition_block (ELSEIF condition_block)* ELSE statement_block
  ;

condition_block: expression THEN statement_block;

statement_block: expression;

expression_array: OPEN_BRACKET expressionSequence CLOSE_BRACKET;

boolean_literal: TRUE # True | FALSE # False;

numeric_literal: NUMBER # Numeric | INTEGER # Integer;

// 移动到Parser区域的正则规则
regex_Literal: SLASH regexContent? SLASH;

regexContent: ( ~SLASH | ESCAPED_SLASH )+;

// 定义转义斜杠的辅助规则（支持正则内的\/写法）
fragment ESCAPED_SLASH: '\\' SLASH;

// Lexer rules.

OPEN_BRACKET: '[';
CLOSE_BRACKET: ']';
OPEN_PARENS: '(';
CLOSE_PARENS: ')';

COLON: ':';
COMMA: ',';
DOLLAR: '$';
DOT: '.';
DOUBLE_QUOTE: '&quot;';
QUESTIONMARK: '?';
SEMICOLON: ';';
SINGLE_QUOTE: '\'';
TILDE: '~';
BACKSLASH: '\\'; // 添加反斜杠的Lexer Token

EQ: '=';
NE: '!=';
GT: '&gt;';
GTE: '&gt;=';
LT: '&lt;';
LTE: '&lt;=';

ASTERISK: '*';
MINUS: '-';
PLUS: '+';
SLASH: '/';

AND: 'AND';
OR: 'OR';
NOT: 'NOT';
IN: 'IN';

IF: 'IF';
THEN: 'THEN';
ELSE: 'ELSE';
ELSEIF: 'ELSEIF';

TRUE: 'TRUE';
FALSE: 'FALSE';

NUMBER: ( DIGIT+ DOT DIGIT+ | DIGIT+ DOT | DOT DIGIT+);
INTEGER: DIGIT+;

IDENTIFIER: LETTER+ (LETTER | DIGIT)*;
STRING_LITERAL: DOUBLE_QUOTE (~[\r\n\\&quot;])*? DOUBLE_QUOTE;

fragment LETTER: [A-Z] | '_';
fragment DIGIT: [0-9];

fragment NewLine
  : '\r\n'
  | '\r'
  | '\n'
  | '\u0085' // &lt;Next Line CHARACTER (U+0085)&gt;'
  | '\u2028' //'&lt;Line Separator CHARACTER (U+2028)&gt;'
  | '\u2029' //'&lt;Paragraph Separator CHARACTER (U+2029)&gt;'
  ;

fragment Whitespace
  : UnicodeClassZS //'&lt;Any Character With Unicode Class Zs&gt;'
  | '\u0009'       //'&lt;Horizontal Tab Character (U+0009)&gt;'
  | '\u000B'       //'&lt;Vertical Tab Character (U+000B)&gt;'
  | '\u000C'       //'&lt;Form Feed Character (U+000C)&gt;'
  ;

fragment UnicodeClassZS
  : '\u0020' // SPACE
  | '\u00A0' // NO_BREAK SPACE
  | '\u1680' // OGHAM SPACE MARK
  | '\u180E' // MONGOLIAN VOWEL SEPARATOR
  | '\u2000' // EN QUAD
  | '\u2001' // EM QUAD
  | '\u2002' // EN SPACE
  | '\u2003' // EM SPACE
  | '\u2004' // THREE_PER_EM SPACE
  | '\u2005' // FOUR_PER_EM SPACE
  | '\u2006' // SIX_PER_EM SPACE
  | '\u2008' // PUNCTUATION SPACE
  | '\u2009' // THIN SPACE
  | '\u200A' // HAIR SPACE
  | '\u202F' // NARROW NO_BREAK SPACE
  | '\u3000' // IDEOGRAPHIC SPACE
  | '\u205F' // MEDIUM MATHEMATICAL SPACE
  ;

WHITESPACES: (Whitespace | NewLine)+ -&gt; skip;

ANY: .;

额外说明

添加了BACKSLASH Token和ESCAPED_SLASH片段规则，支持正则表达式中用\/表示字面量斜杠，更符合正则的常规使用习惯。
调整后的regexContent规则允许任意非斜杠字符，或者转义后的斜杠，确保正则内容可以正确解析。

内容的提问来源于stack exchange，提问作者tRuEsAtM

如何在Antlr G4语法中添加正则规则并解决语法错误？

问题描述 #

错误原因 #

解决方案 #

步骤1：调整规则位置 #

步骤2：替换裸字符为Token名称 #

修改后的完整语法代码 #

额外说明 #