如何利用列表推导式简化含多层for循环与if判断的Python代码并降低时间复杂度？

Great question! Let’s tackle this from two angles: whether list comprehensions are a fit here, and how to streamline the code while cutting down on time complexity.

Short answer: list comprehensions work great for simple, 1-2 layer loops with basic conditions, but for code this deeply nested (5-6 loops + layered conditional checks), using a single list comprehension would turn your code into an unreadable wall of text. No one—including future you—would want to debug that.

That said, you can use list comprehensions for isolated filtering steps (like pre-selecting valid items) to clean up parts of the code. The core complex logic, though, is better split into small, focused functions.

Let’s break down the key optimizations step by step:

1. Cache API Requests to Avoid Redundant Calls #

Right now, your code calls request.get() inside the whitelist loop, which could mean multiple identical requests for the same row. Instead, first find all matching whitelist entries for the row, then make the API call once:

# Preprocess whitelist once (outside the df loop) to add sets for faster lookups
for wl in whitelist:
    wl['port_set'] = set(wl['port'])
    wl['rail_set'] = set(wl['rail'])

for index, row in df.iterrows():
    # First find all whitelist entries that match the row's name
    matching_wl = [wl for wl in whitelist if wl['name'] in row['name'] and wl['name'] != 'global']
    if not matching_wl:
        continue  # Skip if no matching whitelist entries
    
    # Only make the API call once per matching row
    variable = request.get("link", headers=headers).json() if matching_wl else None
    if not variable or row['services'] == 0:
        continue

This cuts down on unnecessary network IO, which is usually the biggest time drain in code like this.

2. Split Complex Logic into Small, Reusable Functions #

Take the messy port-checking and count-updating logic and turn it into dedicated functions. This makes the code easier to read, test, and maintain:

def check_port_validity(port_range, allowed_ports):
    """Check if a port range matches the whitelist and return 0 (valid) or 1 (invalid)"""
    from_port, to_port = port_range['from'], port_range['to']
    if from_port == to_port:
        return 0 if from_port in allowed_ports else 1
    # Check if any whitelist port falls within the range
    return 0 if any(p for p in allowed_ports if from_port < p < to_port) else 1

def update_security_counts(var, wl_item):
    """Calculate n_private and n_protected based on the variable and whitelist entry"""
    n_private, n_protected = 1, 1  # Default to invalid
    
    if 'ports' not in var:
        return n_private, n_protected
    
    for v_port in var['ports']:
        if not v_port:  # Skip empty port entries
            continue
        
        # Update private count if applicable
        if 'private' in wl_item['rail_set'] and 'private' in var['rail']:
            n_private = check_port_validity(v_port, wl_item['port_set'])
        
        # Update protected count if applicable
        if 'protected' in wl_item['rail_set'] and 'protected' in var['rail']:
            n_protected = check_port_validity(v_port, wl_item['port_set'])
    
    return n_private, n_protected

3. Pre-Filter Valid Items to Reduce Loop Iterations #

Instead of looping through every item and checking conditions mid-loop, pre-filter your data to only keep what matters:

# Inside the df loop, after getting variable...
# Pre-filter valid variable entries: active status, matching service ID, valid rail type
valid_vars = [
    var for var in variable
    if var['status'] == 'active'
    and any(t_service['id'] == var['id'] for t_service in row['services'])
    and set(var['rail']) & {'private', 'protected'}
]

# Now loop only through matching whitelist entries and valid vars
for wl_item in matching_wl:
    for var in valid_vars:
        if wl_item['place'] not in var['place']:
            continue
        # Get counts using our helper function
        n_private, n_protected = update_security_counts(var, wl_item)
        # Assign counts to your DataFrame here
        # df.loc[index, 'n_private'] = n_private
        # df.loc[index, 'n_protected'] = n_protected

This reduces the number of iterations in inner loops, which directly lowers time complexity.

4. Use Sets for Faster Lookups #

We added port_set and rail_set to the whitelist preprocessing step. Checking membership in a set (item in set) is O(1) vs O(n) for a list, which makes a big difference when your whitelist has lots of ports or rail types.

• List comprehensions are great for small filtering tasks (like matching_wl or valid_vars), but avoid using them for the entire nested logic.
• The biggest time savings come from reducing API calls and pre-filtering data to avoid unnecessary loops.
• Splitting logic into functions makes the code easier to debug and modify later.

内容的提问来源于stack exchange，提问作者Kochu Polu