Python自制语言解释器报错：仅输出两字符串后触发KeyboardInterrupt

问题描述 #

我用Python开发了一款自定义语言解释器，运行时出现异常：仅输出两个字符串后就触发KeyboardInterrupt报错。此前该解释器可正常输出所有NUM和EXPR类型内容，问题疑似出在parse函数中。曾尝试修改parse函数，修改后能输出566755664645454，但无法输出5 + 6。

相关代码 #

from sys import *

tokens = []

def open_file(filename):
    data = open(filename, 'r').read()
    data += "<EOF>"
    return data

def lex(filecontents):
    tok = ""
    state = 0
    string = ""
    expr = ""
    n = ""
    isexpr = 0
    for char in filecontents:
        tok += char
        if tok == " ":
            if state == 0:
                tok = ""
            else:
                tok = " "
        elif tok == "\n" or tok =="<EOF>":
            if expr != "" and isexpr == 1:
                tokens.append("EXPR:" + expr)
                expr = ""
            elif expr != "" and isexpr == 0:
                tokens.append("NUM:" + expr)
                expr = ""
            tok = ""
        elif tok == "PRINT" or tok == "print":
            tokens.append("PRINT")
            tok = ""
        elif tok == "0" or tok == "1" or tok == "2" or tok == "3" or tok == "4" or tok == "5" or tok == "6" or tok == "7" or tok == "8" or tok == "9":
            expr += tok
            tok = ""
        elif tok == "+":
            isexpr = 1
            expr += tok
            tok = ""
        elif tok == '"':
            if state == 0:
                state = 1
            elif state == 1:
                tokens.append("STRING:" + string + '"')
                string = ""
                state = 0
                tok = ""
        elif state == 1:
            string += tok
            tok = ""
    #print(tokens)
    return tokens

def parse(toks):
    i = 0
    while(i < len(toks)):
        if toks[i] + " " + toks[i+1][0:6] == "PRINT STRING" or toks[i] + " " + toks[i+1][0:3] == "PRINT NUM" or toks[i] + " " + toks[i+1][0:4] == "PRINT EXPR":
            if toks[i+1][0:6] == "STRING":
                print(toks[i+1][7:])
            elif toks[i+1][0:3] == "NUM":
                print(toks[i+1][4:])
            elif toks[i+1][0:4] == "EXPR":
                print(toks[i+1][5:])
            i+= 2

def run():
    data = open_file(argv[1])
    toks = lex(data)
    parse(toks)

run()

测试用例（test.lang） #

PRINT "HELLO WORLD"
print "string"
566755664645454
5 + 6
print 55
print 5 + 8

报错信息 #

PS C:\Users\essam\Desktop\spl> python basic.py test.lang
"HELLO WORLD"
"string"
Traceback (most recent call last):
  File "C:\Users\essam\Desktop\spl\basic.py", line 73, in <module>
    run()
  File "C:\Users\essam\Desktop\spl\basic.py", line 71, in run
    parse(toks)
  File "C:\Users\essam\Desktop\spl\basic.py", line 58, in parse
    while(i < len(toks)):
              ^^^^^^^^^
KeyboardInterrupt

问题分析 #

1. 死循环导致KeyboardInterrupt：parse函数中，只有当匹配到PRINT + 类型的组合时，才会执行i += 2。遇到单独的NUM/EXPR类型token时，if条件不成立，i值永远不递增，直接进入死循环，最终触发中断。
2. 未处理单独的NUM/EXPR：parse函数没有针对非PRINT开头的token做处理逻辑，导致这类内容无法输出。
3. 表达式空格丢失：lex函数处理空格时，会丢弃表达式中的空格，导致5 + 6被解析成5+6，不符合预期格式。

修复方案 #

1. 修复parse函数的死循环与内容输出问题 #

修改parse函数，增加对单独NUM/EXPR的处理，确保无论是否匹配PRINT，i值都能递增：

def parse(toks):
    i = 0
    while(i < len(toks)):
        if toks[i] == "PRINT":
            # 处理PRINT命令，先判断是否有后续token
            if i+1 >= len(toks):
                break
            token = toks[i+1]
            if token.startswith("STRING:"):
                print(token[7:])
            elif token.startswith("NUM:"):
                print(token[4:])
            elif token.startswith("EXPR:"):
                print(token[5:])
            i += 2
        else:
            # 处理单独的NUM或EXPR
            token = toks[i]
            if token.startswith("NUM:"):
                print(token[4:])
            elif token.startswith("EXPR:"):
                print(token[5:])
            i += 1

2. 修复lex函数的表达式空格丢失问题 #

修改lex函数中的空格处理逻辑，保留表达式中的空格：

if tok == " ":
    if state == 0:
        if isexpr == 1:
            expr += tok  # 表达式中的空格需要保留
        tok = ""
    else:
        tok = " "

修复后效果 #

运行脚本后将输出所有内容：

"HELLO WORLD"
"string"
566755664645454
5 + 6
55
5 + 8

内容的提问来源于stack exchange，提问作者s.s.scriptties