ANTLR4为何触发Full Context检查？空格相关问题排查

Lexer代码 #

fragment WS: [ \r\n\f\t]*;

ESCAPE: '\\' -> pushMode(ESCAPING);
OBRACK: '[';
CBRACK: ']';
OBRACE: '{' WS;
CBRACE: '}';
CPAREN: ')';
CCARET: '>';
COMMAWS: ',' WS;
EQUALS: '=' WS;
PERCENT: '%' -> pushMode(SUBSTITUTION);
SEMICOLON: WS ';' WS;
OANSI: '\u001B' -> pushMode(ANSI);
RSPACE: ' ';
FUNCHAR:
    [a-zA-Z0-9]+ '(' WS
; // 偷懒的可打印字符定义方式，实际可打印字符更多！
OTHER: ~('\\' | '[' | ']' | '{' | '}' | '(' | ')' | '<' | '>' | ',' | '%' | '$' | ';' | ':' | '\u001B' | ' ' | '=')+;
ANY_AT_ALL: .+?;

// --------------- SUBSTITUTION模式 -------------
mode SUBSTITUTION;
COLON: ':' -> popMode;
REG_STARTCARET: [qQ]'<' -> popMode;
REG_NUM: [qQ][0-9] -> popMode;
VWX: [vwxVWX][a-zA-Z] -> popMode;
ARG_NUM: [0-9] -> popMode;
SPACE: [bB] -> popMode;
BLANKLINE: [rR] -> popMode;
TAB: [tT] -> popMode;
DBREF: '#' -> popMode;
ENACTOR_NAME: 'n' -> popMode;
CAP_ENACTOR_NAME: 'N' -> popMode;
ACCENT_NAME: '~' -> popMode;
MONIKER_NAME: [kK] -> popMode;
SUB_PRONOUN: [sS] -> popMode;
OBJ_PRONOUN: [oO] -> popMode;
POS_PRONOUN: [pP] -> popMode;
ABS_POS_PRONOUN: [aA] -> popMode;
CALLED_DBREF: '@' -> popMode;
EXECUTOR_DBREF: '!' -> popMode;
LOCATION_DBREF: [lL] -> popMode;
LASTCOMMAND_BEFORE_EVAL: [cC] -> popMode;
LASTCOMMAND_AFTER_EVAL: [uU] -> popMode;
INVOCATION_DEPTH: '?' -> popMode;
CURRENT_ARG_COUNT: '+' -> popMode;
ITEXT_NUM: [iI][0-9]+ -> popMode;
ITEXT_LAST: [iI] 'L' -> popMode;
STEXT_NUM: '$' [0-9]+ -> popMode;
OTHER_SUB: . -> popMode;

// --------------- ESCAPING模式 -----------------
mode ESCAPING;
ANY: . -> popMode;

// --------------- ANSI模式 ---------------------
mode ANSI;
CANSI: 'm' -> popMode;
ANSICHARACTER: ~'m'+;

Parser核心代码 #

startPlainString: evaluationString EOF;

evaluationString:
    function explicitEvaluationString?
    | explicitEvaluationString
;

explicitEvaluationString:
    bracePattern explicitEvaluationStringConcatenatedRepeat*
    | bracketPattern explicitEvaluationStringConcatenatedRepeat*
    | PERCENT validSubstitution explicitEvaluationStringConcatenatedRepeat*
    | beginGenericText explicitEvaluationStringConcatenatedRepeat*
;

explicitEvaluationStringConcatenatedRepeat:
    bracePattern
    | bracketPattern
    | PERCENT validSubstitution
    | genericText
;

bracePattern:
    OBRACE { ++inBraceDepth; } explicitEvaluationString*? CBRACE { --inBraceDepth; }
;

bracketPattern:
    OBRACK evaluationString CBRACK
;

funName:
    FUNCHAR {++inFunction;}
; // TODO: 替换可嵌入函数名以生成函数名，[]调用同理。

function: funName funArguments? CPAREN {--inFunction;} ;

funArguments: funArgument ({inBraceDepth == 0}? COMMAWS funArgument)*?;

funArgument: evaluationString;

validSubstitution:
    complexSubstitutionSymbol
    | substitutionSymbol
;

complexSubstitutionSymbol: (
        REG_STARTCARET {lookingForRegisterCaret = true;} explicitEvaluationString*? CCARET {lookingForRegisterCaret = false;
            }
        | REG_NUM
        | ITEXT_NUM
        | ITEXT_LAST
        | STEXT_NUM
        | VWX
    )
;

substitutionSymbol: (
        SPACE
        | BLANKLINE
        | TAB
        | COLON
        | DBREF
        | ENACTOR_NAME
        | CAP_ENACTOR_NAME
        | ACCENT_NAME
        | MONIKER_NAME
        | PERCENT
        | SUB_PRONOUN
        | OBJ_PRONOUN
        | POS_PRONOUN
        | ABS_POS_PRONOUN
        | ARG_NUM
        | CALLED_DBREF
        | EXECUTOR_DBREF
        | LOCATION_DBREF
        | LASTCOMMAND_BEFORE_EVAL
        | LASTCOMMAND_AFTER_EVAL
        | INVOCATION_DEPTH
        | EQUALS
        | CURRENT_ARG_COUNT
        | OTHER_SUB
    )
;

genericText: beginGenericText | FUNCHAR;

beginGenericText:
    escapedText
    | ansi
    | { inFunction == 0 }? CPAREN
    | { !inCommandMatch }? RSPACE
    | { !inCommandList || inBraceDepth > 0 }? SEMICOLON
    | { (!lookingForCommandArgCommas && inFunction == 0) || inBraceDepth > 0 }? COMMAWS
    | { !lookingForCommandArgEquals }? EQUALS
    | { !lookingForRegisterCaret }? CCARET
    | (OTHER|ANY_AT_ALL) 
;

escapedText: ESCAPE ANY;

ansi: OANSI ANSICHARACTER? CANSI;

问题现象 #

测试输入为abc %i0，解析日志中出现以下关键提示：

line 1:7 reportAttemptingFullContext d=11 (explicitEvaluationString), input=' %i0'
line 1:3 reportContextSensitivity d=11 (explicitEvaluationString), input=' '

已确认空格被Lexer识别为RSPACE，未触发ANY_AT_ALL规则。

根本原因分析 #

触发Full Context检查的核心是规则歧义，具体路径如下：

1. 当解析到空格（RSPACE）时，存在两条匹配路径：
   • 路径1：explicitEvaluationString → beginGenericText（匹配RSPACE） → 后续继续匹配explicitEvaluationStringConcatenatedRepeat*
   • 路径2：explicitEvaluationString → 已匹配的beginGenericText（前面的abc） → explicitEvaluationStringConcatenatedRepeat* → genericText → beginGenericText（匹配RSPACE）
2. ANTLR4的LL(*)解析器无法通过有限前瞻判断哪条路径正确，因此触发Full Context检查来尝试所有可能路径，导致日志提示出现。
3. 规则设计上的递归歧义是根源：explicitEvaluationString的最后一个分支是beginGenericText explicitEvaluationStringConcatenatedRepeat*，而explicitEvaluationStringConcatenatedRepeat又包含genericText（可匹配beginGenericText），同一个beginGenericText既可以作为explicitEvaluationString的起始部分，也可以作为后续重复项的一部分，当前瞻到RSPACE时无法区分归属。

另外，beginGenericText中RSPACE的匹配依赖语义谓词{ !inCommandMatch }?，该谓词的状态在解析到空格时未明确，进一步加剧了解析器的判断难度，只能通过Full Context验证。

内容的提问来源于stack exchange，提问作者Mercutio