如何解析Python REST API中含EQUAL/AND/OR等自定义运算符的查询参数？

Great question! Parsing those custom query operators like EQUAL, AND, and OR from a URL parameter is totally achievable—here's a structured, practical approach to make it work:

Your query strings follow a function-style (S-expression) syntax, where each operator acts as a function that takes arguments (either other operators or raw values like product_id and "56789"). For example:

• EQUAL(product_id,"56789") is a function call with two arguments: a field name and a quoted value
• AND(EQUAL(...), EQUAL(...)) is a function call with two nested EQUAL expressions as arguments

This recursive structure is key—we’ll use it to build a parsable tree of operations.

Break the problem into three manageable stages:

Tokenization (Lexing) #

First, split the raw query string into discrete "tokens"—the building blocks of your syntax. Tokens include:

• Operator names (AND, OR, EQUAL)
• Punctuation ((, ), ,)
• Field names (product_id)
• Quoted values ("56789")

Use regex to reliably extract these tokens, making sure to treat quoted strings as single tokens (even if they contain characters like commas, though your examples don’t have this).

Parsing to an Abstract Syntax Tree (AST) #

Convert the flat list of tokens into a nested AST—a tree structure that represents the logic of the query. For example, AND(EQUAL(product_id,"56789"),EQUAL(product_id,"2236")) becomes:

{
  "type": "AND",
  "left": {
    "type": "EQUAL",
    "field": "product_id",
    "value": "56789"
  },
  "right": {
    "type": "EQUAL",
    "field": "product_id",
    "value": "2236"
  }
}

Recursive descent parsing works perfectly here—since operators can nest inside each other, we can write recursive functions to handle each operator type.

Converting the AST to Database Queries #

Once you have the AST, translate it into SQL (or your ORM’s query syntax). Each node in the map corresponds to a database operation:

• EQUAL(field, value) → field = 'value'
• AND(left, right) → (left_query AND right_query)
• OR(left, right) → (left_query OR right_query)

Here’s a concrete example to tie it all together:

Tokenizer #

import re

def tokenize(query_str):
    # Regex pattern to match all token types
    pattern = r'([A-Z]+)|(\()|(\))|(,)|(".*?")|([a-zA-Z0-9_]+)'
    tokens = re.findall(pattern, query_str)
    # Flatten results and filter out empty strings
    return [token for group in tokens for token in group if token]

Recursive Descent Parser #

class QueryParser:
    def __init__(self, tokens):
        self.tokens = tokens
        self.current_pos = 0

    def _peek(self):
        return self.tokens[self.current_pos] if self.current_pos < len(self.tokens) else None

    def _consume(self):
        self.current_pos += 1

    def parse(self):
        token = self._peek()
        if token in ("AND", "OR"):
            # Parse logical operators with two nested arguments
            self._consume()
            assert self._peek() == "(", f"Expected '(' after {token}"
            self._consume()
            
            left_expr = self.parse()
            
            assert self._peek() == ",", f"Expected ',' after left argument of {token}"
            self._consume()
            
            right_expr = self.parse()
            
            assert self._peek() == ")", f"Expected ')' after {token} arguments"
            self._consume()
            
            return {"type": token, "left": left_expr, "right": right_expr}
        
        elif token == "EQUAL":
            # Parse equality operator with field and value
            self._consume()
            assert self._peek() == "(", "Expected '(' after EQUAL"
            self._consume()
            
            field = self._peek()
            self._consume()
            
            assert self._peek() == ",", "Expected ',' after field in EQUAL"
            self._consume()
            
            value = self._peek().strip('"')  # Remove surrounding quotes
            self._consume()
            
            assert self._peek() == ")", "Expected ')' after EQUAL arguments"
            self._consume()
            
            return {"type": token, "field": field, "value": value}
        
        else:
            raise ValueError(f"Unexpected token: {token} at position {self.current_pos}")

AST to SQL Converter #

def ast_to_sql(ast):
    if ast["type"] == "EQUAL":
        # Use parameterized queries in production to avoid SQL injection!
        return f"{ast['field']} = %s"  # %s is a placeholder for DBAPI
    elif ast["type"] == "AND":
        left_sql = ast_to_sql(ast["left"])
        right_sql = ast_to_sql(ast["right"])
        return f"({left_sql} AND {right_sql})"
    elif ast["type"] == "OR":
        left_sql = ast_to_sql(ast["left"])
        right_sql = ast_to_sql(ast["right"])
        return f"({left_sql} OR {right_sql})"
    else:
        raise ValueError(f"Unsupported AST type: {ast['type']}")

Test It Out #

# Test a nested AND query
query_str = "AND(EQUAL(product_id,'56789'),EQUAL(product_id,'2236'))"
tokens = tokenize(query_str)
parser = QueryParser(tokens)
ast = parser.parse()
sql_clause = ast_to_sql(ast)

print(sql_clause)
# Output: ((product_id = %s AND product_id = %s))

• Error Handling: Add robust validation for invalid queries (e.g., mismatched parentheses, missing arguments, unknown operators) and return clear HTTP 400 Bad Request responses.
• Security: Never directly concatenate user input into SQL—use parameterized queries (like the %s placeholder above) or an ORM (e.g., SQLAlchemy) to prevent SQL injection.
• Extensibility: Add new operators (like LIKE, GT, LT) by extending the parser and converter functions. You could even use a dictionary to map operator types to handler functions for cleaner code.
• Nested Queries: The recursive parser handles arbitrarily deep nesting (e.g., AND(OR(EQUAL(a,"b"),EQUAL(c,"d")),EQUAL(e,"f"))) out of the box.

内容的提问来源于stack exchange，提问作者siva