JavaScript表达式解析器开发:REx语法问题与Nearley实现方案
我正尝试为微软相关文档中描述的EBNF文法生成JavaScript解析器。文档中给出的EBNF无法直接运行,因此我尝试对其做简化,使其适配REx解析器生成器。
我的目标是在JavaScript中可以解析如下表达式并返回True或False结果:
AttributeA > 2- AttributeA的值大于2HasCategory(Assembly)- 节点拥有Assembly分类Assembly- 节点拥有Assembly分类HasValue(AttributeA)- 属性AttributeA存在值,非未定义状态AttributeA < AttributeB- 属性AttributeA的值小于属性AttributeB的值IsReference- 属性IsReference的值为TrueAttributeA + 2 > 5 and AttributeB - 5 != 7AttributeA * 1.25 >= 500
我正在使用的在线REx解析器生成器地址为:https://www.bottlecaps.de/rex/。如果大家有其他可生成JavaScript代码的解析器生成器推荐,也希望能分享相关链接。
我目前遇到的核心问题是MethodCall的定义,尝试了多种不同的定义方式全部失败。当我移除MethodCall和MethodArgs定义后,REx解析器生成器可以正常生成解析器。因此非常希望能获得相关帮助解决这个问题。
目前我已完成的文法如下:
Expression ::= BinaryExpression | MethodCall | "(" Expression ")" | Number BinaryExpression ::= RelationalExpression ( ( '=' | '!=' ) RelationalExpression )* RelationalExpression ::= AdditiveExpression ( ( '<' | '>' | '<=' | '>=' | 'and' | 'or' ) AdditiveExpression )* AdditiveExpression ::= MultiplicativeExpression ( ( '+' | '-' ) MultiplicativeExpression )* MultiplicativeExpression ::= UnaryExpression ( ( '*' | '/' | '%' ) UnaryExpression )* UnaryExpression ::= "!" Identifier | "+" Identifier | "-" Identifier | Identifier MethodCall ::= Identifier "(" MethodArgs* ")" MethodArgs ::= Expression | Expression "," MethodArgs Identifier ::= Letter ( Letter | Digit | "_" )* Number ::= Digit ('.' Digit) | (Digit)* <?TOKENS?> Letter ::= [A-Za-z] Digit ::= [0-9]
以下是我尝试过的部分MethodCall定义版本,全部失败:
MethodCall ::= Identifier "(" MethodArgs? ")" MethodArgs ::= Expression ("," MethodArgs)* MethodCall ::= Identifier "(" MethodArgs ")" MethodArgs ::= ( Expression ("," MethodArgs)* )? MethodCall ::= MethodName "(" MethodArgs? ")" MethodName ::= Identifier MethodArgs ::= Expression ("," MethodArgs)* MethodCall ::= Identifier "(" MethodArgs ")" MethodArgs ::= Expression ("," MethodArgs)* | "" MethodCall ::= Identifier MethodArgs MethodArgs ::= "(" (Expression ("," MethodArgs)* | "") ")" MethodCall ::= Identifier "(" MethodArgs ")" MethodArgs ::= Expression | Expression "," MethodArgs | "" MethodCall ::= Identifier "(" Expression ")"
我还参考了多门其他语言的文法实现寻找灵感,但目前仍未找到解决方案:
- EcmaScript文法参考:
https://www.bottlecaps.de/rex/EcmaScript.ebnf - Java文法参考:
https://www.bottlecaps.de/rex/Java.ebnf - JavaBNF参考:
https://cs.au.dk/~amoeller/RegAut/JavaBNF.html - Python函数定义参考:
https://docs.python.org/3/reference/compound_stmts.html#function-definitions
更新
在此同步问题的最终解决结果:我始终无法调整EBNF文法满足需求,因此采纳@rici的建议调研了Nearley,将文法转换为Nearley支持的语法后成功跑通了工具链。对于我的项目而言Nearley是更合适的选择:它的文档完善、工具链成熟、报错信息友好,非常感谢@rici推荐Nearley。
以下是我最终实现的文法,我已用如下输入做过测试:
'2 + 4', '2 + 4 - 6', '(2 + 4)', '!true', '!(true)', 'hasCategory(test)', 'hasCategory(test,test2)', 'hasCategory( test , test2 )', 'hasCategory(test,test2, test3)', 'IsReference', 'IsReference()', '2 * 4', '(2 / 4)', '2 * 4 + 2', '(2 / 4) + 2', '2 > 4', '2 >= 2', '2 = 4', '2 == 2', '2 != 4', '2 !== 2', '(2 * 4 + 2) > 4', '(2 * 4 + 2) > (4 + 10)', 'true', 'true or false', 'true || false', 'true and false', 'true && false', '(true or false)', '!(true or false)', '2 != 1+1', '2 != (1+1)', '2 != (1+2)', '(2 > 2 or (2 != 1+1))',
@builtin "whitespace.ne" # `_` 表示任意数量的空白字符 @builtin "number.ne" # `int`, `decimal`, 和 `percentage` 数值原语 @builtin "postprocessors.ne" @{% function methodCall(nameId, argsId = -1) { return function(data) { return { type: 'methodCall', name: data[nameId], args: argsId == -1 ? [] : data[argsId] }; } } function value() { return function(data) { return { type: 'value', value: data[0] }; } } %} expression -> methodCall {% id %} | relationalExpression {% value() %} | booleanExpression {% value() %} | _ identifier _ {% methodCall(1) %} booleanExpression -> parentheses {% id %} | parentheses _ "and"i _ parentheses {% d => d[0] && d[4] %} | parentheses _ "&&" _ parentheses {% d => d[0] && d[4] %} | parentheses _ "or"i _ parentheses {% d => d[0] || d[4] %} | parentheses _ "||" _ parentheses {% d => d[0] || d[4] %} parentheses -> _ "(" relationalExpression ")" _ {% nth(2) %} | _ "(" booleanExpression ")" _ {% nth(2) %} | unaryExpression {% id %} relationalExpression -> _ additiveExpression _ {% nth(1) %} | relationalExpression _ "=" _ additiveExpression {% d => d[0] == d[4] %} | relationalExpression _ "==" _ additiveExpression {% d => d[0] == d[4] %} | relationalExpression _ "!=" _ additiveExpression {% d => d[0] != d[4] %} | relationalExpression _ "!==" _ additiveExpression {% d => d[0] != d[4] %} | relationalExpression _ "<" _ additiveExpression {% d => d[0] < d[4] %} | relationalExpression _ ">" _ additiveExpression {% d => d[0] > d[4] %} | relationalExpression _ "<=" _ additiveExpression {% d => d[0] <= d[4] %} | relationalExpression _ ">=" _ additiveExpression {% d => d[0] >= d[4] %} additiveExpression -> _ multiplicativeExpression _ {% nth(1) %} | additiveExpression _ "+" _ multiplicativeExpression {% d => d[0] + d[4] %} | additiveExpression _ "-" _ multiplicativeExpression {% d => d[0] - d[4] %} multiplicativeExpression -> _ parentheses _ {% nth(1) %} | parentheses _ "*" _ parentheses {% d => d[0] * d[4] %} | parentheses _ "/" _ parentheses {% d => d[0] / d[4] %} | parentheses _ "%" _ parentheses {% d => d[0] % d[4] %} unaryExpression -> _ "!" _ expression _ {% d => !d[3] %} | _ decimal _ {% nth(1) %} | _ unsigned_int _ {% nth(1) %} | _ boolean _ {% nth(1) %} | _ identifier _ {% nth(1) %} methodCall -> identifier "(" methodArgs ")" {% methodCall(0, 2) %} | identifier "(" _ ")" {% methodCall(0) %} methodArgs -> _ identifier _ {% d => [d[1]] %} | _ identifier _ "," _ methodArgs _ {% d => [d[1]].concat(d[5]) %} boolean -> "true"i {% () => true %} | "false"i {% () => false %} identifier -> [A-Za-z0-9_]:+ {% (data, l, reject) => { var ident = data[0].join(''); if (ident.toLowerCase() === 'true' || ident.toLowerCase() === 'false') { return reject; } else { return ident; } } %}
内容的提问来源于stack exchange,提问作者Allan Simonsen

