基于Antlr4的C语言多行#define预处理指令解析问题求助

Got it, let's fix your multi-line #define parsing issue with ANTLR4. The problem with your current MultiDefine rule is that it doesn't properly handle whitespace before the line-continuation backslash, and the recursive structure doesn't correctly capture all content across escaped newlines. Here's a revised approach that works reliably for both single and multi-line #define directives:

Step 1: Refine Lexer Rules with Modes #

Using ANTLR's lexer modes is cleaner for preprocessor directives since they have distinct syntax from regular C code. Update your lexer rules like this:

// Main mode tokens (keep your existing C tokens here)
Whitespace : [ \t]+ -> channel(HIDDEN) ;
Newline : ('\r' '\n'? | '\n') -> channel(HIDDEN) ;
BlockComment : '/*' .*? '*/' ;
LineComment : '//' ~[\r\n]* ;

// Switch to preprocessor mode when we see a '#'
PreprocessorHash : '#' -> pushMode(PREPROCESSOR_MODE);

mode PREPROCESSOR_MODE;

// Whitespace in preprocessor directives is ignored (except for separating tokens)
PrepWhitespace : [ \t]+ -> channel(HIDDEN);
// Exit preprocessor mode on an unescaped newline
PrepNewline : ('\r' '\n'? | '\n') -> popMode, channel(HIDDEN);
// Match line-continuation backslash (with optional leading whitespace) followed by newline
EscapedNewline : [ \t]* '\\' ('\r' '\n'? | '\n') -> channel(HIDDEN);

// #define directive: switch to define content mode
DefineDirective : 'define' -> pushMode(DEFINE_CONTENT_MODE);
// #include directive: match everything until newline
IncludeDirective : 'include' ~[\r\n]* -> popMode;
// Catch-all for other preprocessor directives (optional, adjust as needed)
OtherPrepDirective : ~[\r\n]+ -> popMode;

mode DEFINE_CONTENT_MODE;

// Whitespace in define content is hidden by default; remove -> channel(HIDDEN) to preserve it
DefineWhitespace : [ \t]+ -> channel(HIDDEN);
// Match all content (including escaped newlines) until an unescaped newline
DefineContent : (EscapedNewline | ~[\r\n])+ -> popMode;

Step 2: Update Parser Rules #

Simplify your parser rules to leverage the new lexer tokens:

preprocessorDeclaration : includeDeclaration | defineDeclaration ;

includeDeclaration : PreprocessorHash IncludeDirective ;

defineDeclaration : PreprocessorHash DefineDirective DefineContent ;

// Keep your existing declaration rule, ensuring preprocessorDeclaration is included
declaration : 
    declarationSpecifiers initDeclaratorList ';' 
    | declarationSpecifiers ';' 
    | staticAssertDeclaration 
    | preprocessorDeclaration 
    | comment ;

Why This Works #

• Lexer Modes: Isolates preprocessing syntax from regular C code, avoiding token conflicts that can happen when mixing preprocessor and core C rules.
• EscapedNewline: Properly handles optional whitespace before the line-continuation backslash (a common pattern in real-world macros) and hides the escape sequence so the parser sees the macro body as a single logical block.
• DefineContent: Captures all characters across escaped newlines, treating the entire multi-line macro as a single piece of content for the parser to process.

Testing Your Example #

This setup will correctly parse your sample multi-line #define:

#define MACRO(num, str) {\\
    printf("%d", num);\\
    printf(" is");\\
    printf(" %s number", str);\\
    printf("\n");\\
}

The EscapedNewline tokens will be hidden, and DefineContent will include the entire macro body (braces, printf calls, etc.) as a single token ready for further processing.

Optional Adjustments #

• If you need to parse the macro body's structure (like parameters, braces, or literals) instead of just capturing it as a string, extend the DEFINE_CONTENT_MODE with more granular rules (e.g., tokens for parentheses, braces, and string literals).
• If you want to preserve whitespace in the macro body (for accurate reconstruction), remove the -> channel(HIDDEN) from DefineWhitespace.

内容的提问来源于stack exchange，提问作者Ritwik Bhar