如何在Rascal中禁用SQL DDL关键字连写的非法语法？

我在Rascal中实现SQL的DDL语法，目标仅支持如下合法的CREATE TABLE语句：

CREATE EXTERNAL TABLE page_view(viewTime INT, userid BIGINT,
     page_url STRING, referrer_url STRING,
     ip STRING COMMENT 'IP Address of the User',
     country STRING COMMENT 'country of origination')
 COMMENT 'This is the staging page view table'
 ROW FORMAT DELIMITED FIELDS TERMINATED BY '\054'
 STORED AS TEXTFILE
 LOCATION '<hdfs_location>';

但目前发现，像下面这种将关键字连写（CREATEEXTERNALTABLE）的非法语句也会被判定为有效：

CREATEEXTERNALTABLE page_view(viewTime INT, userid BIGINT,
     page_url STRING, referrer_url STRING,
     ip STRING COMMENT 'IP Address of the User',
     country STRING COMMENT 'country of origination')
 COMMENT 'This is the staging page view table'
 ROW FORMAT DELIMITED FIELDS TERMINATED BY '\054'
 STORED AS TEXTFILE
 LOCATION '<hdfs_location>';

当前使用的产生式规则如下：

syntax CreateTable
  = withColumns: 'CREATE' TemporaryTable? ExternalTable? 'TABLE' IfNotExists? TableName 
                        Columns?
                        Comment?
                        PartitionedByClause?
                        ClusteredByClause?
                        RowFormatClause?
                        StorageClause? 
                        LocationClause?
                        TablePropertiesClause?
  | withQuery: 'CREATE' TemporaryTable? ExternalTable? 'TABLE' IfNotExists? TableName 
                    RowFormatClause? 
                    StorageClause?
                    CreateTableQuery
  | withLike: 'CREATE' TemporaryTable? ExternalTable? 'TABLE' IfNotExists?  TableName Like TableName 
                    TablePropertiesClause?
  ;

问题出在词法分析阶段没有将SQL关键字与普通标识符严格区分，导致连写的关键字被误判为合法标识符并匹配到语法规则。可以通过以下步骤修复：

1. 将关键字标记为保留字 #

在Rascal中，使用reserved()修饰符标记所有SQL关键字，强制词法分析器将它们识别为独立的词法单元，且必须用空白分隔。修改你的产生式规则如下：

syntax CreateTable
  = withColumns: reserved("CREATE") TemporaryTable? ExternalTable? reserved("TABLE") IfNotExists? TableName 
                        Columns?
                        Comment?
                        PartitionedByClause?
                        ClusteredByClause?
                        RowFormatClause?
                        StorageClause? 
                        LocationClause?
                        TablePropertiesClause?
  | withQuery: reserved("CREATE") TemporaryTable? ExternalTable? reserved("TABLE") IfNotExists? TableName 
                    RowFormatClause? 
                    StorageClause?
                    CreateTableQuery
  | withLike: reserved("CREATE") TemporaryTable? ExternalTable? reserved("TABLE") IfNotExists?  TableName Like TableName 
                    TablePropertiesClause?
  ;

// 同时将其他关键字子句也标记为保留字
syntax TemporaryTable = reserved("TEMPORARY");
syntax ExternalTable = reserved("EXTERNAL");
syntax IfNotExists = reserved("IF NOT EXISTS");
syntax Like = reserved("LIKE");
// 类似地，COMMENT、ROW FORMAT等关键字所在的子句也需要做同样处理

2. 定义标识符规则排除关键字 #

明确标识符的词法规则，确保标识符不会与关键字冲突：

// 定义空白分隔符
layout Whitespace = [\t\n\r\ ]*;

// 枚举所有需要的SQL关键字
lexical Keyword = reserved("CREATE") | reserved("EXTERNAL") | reserved("TABLE") | reserved("TEMPORARY") | reserved("IF NOT EXISTS") | reserved("LIKE") | reserved("COMMENT") | ...;

// 定义标识符，排除所有关键字
lexical Identifier = [a-zA-Z_][a-zA-Z0-9_]* !>> Keyword;

3. 效果验证 #

修改后，词法分析器会将CREATE、EXTERNAL、TABLE识别为独立的词法单元，必须用空白分隔。当遇到CREATEEXTERNALTABLE这类连写字符串时，它会被当成普通标识符，无法匹配CREATE + EXTERNAL + TABLE的语法规则，从而被判定为非法语句。

内容的提问来源于stack exchange，提问作者Pius Arhanbhunde