Python优化嵌套循环与索引查询：加速订单簿价格计算

Hey there, let's tackle this efficiency problem with your order book price calculation code. The core issue here is those repeated list1.index() calls—they're killing your performance, so let's break down the problem and fix it step by step.

背景与需求回顾 #

You're working with a flattened order book list list1, where each trading pair (like ethbtc) is followed by 40 sets of price-quantity data. Your goal is to calculate the highest execution price when going all-in on a trade: for example, if you want to buy 11 ETH and the first tier only has 10 ETH available, you'll take the remaining 1 ETH from the next tier, making the highest execution price the second tier's price.

现有代码的性能瓶颈 #

Your current code works, but it's slow because of two big issues:

• Repeated linear searches: Every call to list1.index(symbolnow) scans the entire list from the start to find the trading pair's position. With 100 symbols and 40 tiers each, that's 4000 unnecessary linear searches—way too inefficient for a large list.
• Redundant existence checks: The symbolnow in list1 check also scans the whole list every time, adding more unnecessary overhead.

最优优化方案：预构建交易对映射字典 #

The fix is simple: pre-process list1 into a dictionary once, where each key is a trading pair, and the value is a list of (price, quantity) tuples for that pair. This turns all your O(n) lookup operations into O(1), which will drastically speed up your code.

Step 1: Preprocess list1 into a structured dictionary #

First, we'll parse the flat list into a usable format once, so we never have to scan the full list again:

# Preprocess the flat list1 into a dictionary of trading pairs to their order tiers
order_book = {}
index = 0
list_length = len(list1)

while index < list_length:
    symbol = list1[index]
    # Each symbol is followed by 40 price-quantity pairs (80 total elements)
    tiers = []
    for tier_idx in range(40):
        # Calculate positions for price and quantity in the flat list
        price_pos = index + 1 + (2 * tier_idx)
        qty_pos = index + 2 + (2 * tier_idx)
        price = float(list1[price_pos])
        quantity = float(list1[qty_pos])
        tiers.append( (price, quantity) )
    
    order_book[symbol] = tiers
    # Move index past the current symbol and its 40 tiers
    index += 1 + (40 * 2)

Step 2: Optimized main logic #

Now, use the pre-built dictionary to calculate the highest execution price for each symbol in symbollist:

mycurrentbalance = 5.5  # Example balance (e.g., 5.5 ETH to sell)
max_price_results = {}

for symbol in symbollist:
    tiers = order_book.get(symbol, [])
    accumulated_qty = 0.0
    highest_price = -1
    
    for price, qty in tiers:
        accumulated_qty += qty
        if accumulated_qty >= mycurrentbalance:
            highest_price = price
            break
    
    max_price_results[symbol] = highest_price

• This code skips all the slow list searches—we just look up the symbol's tiers directly from the dictionary.
• If all tiers combined don't have enough quantity, highest_price stays at -1, matching your original logic.

额外优化细节 #

• Adjust for sell scenarios: If you're selling (like your example where you sell 5.5 ETH and need the highest execution price), make sure you're iterating from the highest price tier down. Depending on your order book structure, you might need to sort the tiers in descending price order during preprocessing, or reverse the loop.
• Avoid redundant work: If symbollist has duplicate symbols, the preprocessing step only runs once—you don't have to re-parse the order book for duplicates.

Why list comprehensions or regex aren't the answer here #

• List comprehensions: They're just syntactic sugar for loops—they won't fix the core problem of slow linear searches.
• Regex: list1 is a structured list, not a string. Using regex to parse it would be overly complex and even slower than your original code.

内容的提问来源于stack exchange，提问作者Thomas Bernhard