ANTLR4严格顺序语法中零散自由文本元素的优化处理问询

我正在使用ANTLR 4.10.1 Maven插件生成Java解析器，已定义一个元素顺序严格的数据格式语法。但该格式允许在任意两个完整元素之间插入freetext元素，目前通过词法规则匹配自由文本，再借助自定义错误处理维持解析。现希望找到无需在语法中为所有其他元素包裹可选自由文本的更优方案，同时需保留自由文本与其前序元素的关联关系。

原始语法示例 #

grammar POC;

FIELD_SEPERATOR : '-';
FIELD_VALUE : ~[-
]+;
EOL : '\r\n';
FREETEXT : 'FREETEXT' FIELD_SEPERATOR .*? '\n';


poc : data1 data2 data3? data4 EOF;
data1 : 'DATA1' FIELD_SEPERATOR FIELD_VALUE EOL;
data2 : 'DATA2' FIELD_SEPERATOR FIELD_VALUE EOL;
data3 : 'DATA3' FIELD_SEPERATOR FIELD_VALUE EOL;
data4 : 'DATA4' FIELD_SEPERATOR FIELD_VALUE EOL;

测试数据示例 #

DATA1-this is a value
DATA2-this is a value
FREETEXT-This is a freetext
DATA3-this is a value
DATA4-this is a value

尝试的修改方案（将FREETEXT放入HIDDEN通道） #

修改后的词法规则：

FREETEXT : 'FREETEXT' FIELD_SEPERATOR .*? '\n' -> channel(HIDDEN);

对应Java实现代码 #

public class POC extends POCBaseListener {

    private static BufferedTokenStream freeTextBuffer = null;

    public static void main(String[] args) throws IOException {

        var lexer = new POCLexer(CharStreams.fromStream(POC.class.getResourceAsStream("/poc.txt")));
        var lexer2 = new POCLexer(CharStreams.fromStream(POC.class.getResourceAsStream("/poc.txt")));

        var parser = new POCParser(new CommonTokenStream(lexer));

        freeTextBuffer = new BufferedTokenStream(lexer2);
        freeTextBuffer.fill();

        ParseTree tree = parser.poc();
        ParseTreeWalker walker = new ParseTreeWalker();
        walker.walk(new POC(), tree);
    }

    @Override
    public void exitEveryRule(ParserRuleContext ctx) {
        int tokenIndex = ctx.stop.getTokenIndex();
        var freetext = freeTextBuffer.getHiddenTokensToRight(tokenIndex);
        if (freetext != null) {
            System.out.println(String.format("Found freetext for: %s", ctx.getStart().getText()));
            var i = freetext.iterator();
            while (i.hasNext()) {
                var text = i.next().getText().replace("\n", "");
                System.out.println(String.format(" -> %s", text));
            }
        }
    }
}

控制台输出结果 #

Found freetext for: DATA2
 -> FREETEXT-This is a freetext

内容的提问来源于stack exchange，提问作者blackened