如何提升Python中BSC网络待处理区块分析循环的运行速度？

Hey there! I see you're struggling with slow processing when analyzing pending BSC transactions—let's fix that. The core issue here is that hash_analise() makes synchronous API calls to fetch transaction data, which blocks your main loop from handling new events. Using thread.join() just forces sequential execution, making things even worse. Here are practical, actionable fixes to get your program running in real-time:

1. Use a Thread Pool for Parallel IO Processing #

Since fetching transactions is an IO-bound task (most time is spent waiting for network responses), a thread pool lets you handle multiple requests at once without blocking the main event loop. Instead of manually managing threads and joining them, let the pool handle scheduling efficiently:

from concurrent.futures import ThreadPoolExecutor

def hash_analise(hash):
    try:
        tx = web3.eth.get_transaction(hash)
        # Skip costly JSON conversion for filtering (we'll optimize this next!)
        if tx.get('to') == TOKEN_LOWER_CORRIGIDO or TOKEN_LOWER_CORRIGIDO in tx.get('input', ''):
            print("\nCONTÉM A STRING ESCOLHIDA")
            print("IMPRIME HASH1:", Web3.toJSON(tx))
    except web3.exceptions.TransactionNotFound:
        print(f"TRANSAÇÃO NÃO LOCALIZADA: {hash}")
    except Exception as e:
        print(f"Erro ao processar hash {hash}: {str(e)}")

if __name__ == "__main__":
    tx_filter = web3.eth.filter('pending')
    count = 0
    # Adjust max_workers based on your node's rate limits (start with 5-10)
    with ThreadPoolExecutor(max_workers=8) as executor:
        while True:
            for event in tx_filter.get_new_entries():
                txnhash = Web3.toJSON(event)[1:67]
                # Submit the task to the pool—no need to wait for it to finish
                executor.submit(hash_analise, txnhash)
                count += 1
                print("Main", count)

2. Optimize Filtering (Ditch JSON Conversion!) #

Converting every transaction to JSON just to search for a string is slow and unnecessary. Directly check the transaction object's fields instead:

• If TOKEN_LOWER_CORRIGIDO is a contract address: check tx['to'] directly
• If it's a function signature or data in the input: check tx['input'] without serialization

This cuts out a costly step and makes your filtering way faster.

3. Batch Transaction Requests (Reduce Network Overhead) #

Instead of fetching one transaction at a time, collect a batch of hashes and fetch them all in a single request (if your BSC node supports batch calls). This reduces the number of network round-trips:

def process_batch(hash_list):
    try:
        # Batch fetch transactions (verify your node supports this!)
        transactions = web3.eth.get_transactions(hash_list)
        for tx in transactions:
            if tx and (tx.get('to') == TOKEN_LOWER_CORRIGIDO or TOKEN_LOWER_CORRIGIDO in tx.get('input', '')):
                print("\nCONTÉM A STRING ESCOLHIDA")
                print("IMPRIME HASH1:", Web3.toJSON(tx))
    except Exception as e:
        print(f"Erro no processamento do lote: {str(e)}")

if __name__ == "__main__":
    tx_filter = web3.eth.filter('pending')
    count = 0
    batch_size = 10
    tx_batch = []
    
    with ThreadPoolExecutor(max_workers=8) as executor:
        while True:
            for event in tx_filter.get_new_entries():
                txnhash = Web3.toJSON(event)[1:67]
                tx_batch.append(txnhash)
                count += 1
                print("Main", count)
                
                # Process batch when we hit the size limit
                if len(tx_batch) >= batch_size:
                    executor.submit(process_batch, tx_batch)
                    tx_batch = []
            
            # Process any remaining transactions in the batch
            if tx_batch:
                executor.submit(process_batch, tx_batch)
                tx_batch = []

Note: Not all public BSC nodes support batch get_transactions—if yours doesn't, stick with the thread pool approach.

4. Switch to Asynchronous Web3 (Advanced) #

For even better performance, use an asynchronous Web3 client (like web3.py's async support with aiohttp). This lets you make non-blocking API calls without relying on threads:

import asyncio
from web3 import AsyncWeb3, AsyncHTTPProvider

async def hash_analise(web3, hash):
    try:
        tx = await web3.eth.get_transaction(hash)
        if tx.get('to') == TOKEN_LOWER_CORRIGIDO or TOKEN_LOWER_CORRIGIDO in tx.get('input', ''):
            print("\nCONTÉM A STRING ESCOLHIDA")
            print("IMPRIME HASH1:", Web3.toJSON(tx))
    except web3.exceptions.TransactionNotFound:
        print(f"TRANSAÇÃO NÃO LOCALIZADA: {hash}")
    except Exception as e:
        print(f"Erro ao processar hash {hash}: {str(e)}")

async def main():
    web3 = AsyncWeb3(AsyncHTTPProvider('https://bsc-dataseed.binance.org/'))
    tx_filter = await web3.eth.filter('pending')
    count = 0
    
    while True:
        events = await tx_filter.get_new_entries()
        tasks = []
        for event in events:
            txnhash = Web3.toJSON(event)[1:67]
            tasks.append(hash_analise(web3, txnhash))
            count += 1
            print("Main", count)
        
        # Run all tasks concurrently
        await asyncio.gather(*tasks)

if __name__ == "__main__":
    asyncio.run(main())

Bonus Tips #

• Use a local BSC node: Public nodes have rate limits and higher latency. Running your own light node will drastically speed up requests.
• Limit console logging: Printing every failed transaction can slow things down—log to a file instead, or only log critical errors.
• Adjust concurrency limits: Don't overload your node with too many concurrent requests—start with 5-10 workers and tweak based on performance.

内容的提问来源于stack exchange，提问作者Michel Andrade Santos