使用列表推导式生成元组列表时遇循环提前终止问题求助

问题描述 #

编写函数接收（列名、数据类型）元组列表，生成包含Excel每行数据的元组列表，但设置起始索引0、结束索引10时，循环仅生成第一个元组就终止。相关代码如下：

主函数代码 #

# Validate the index range if provided
if index_range is not None:
    if not isinstance(index_range, tuple):
        raise ValueError(f"index_range provided is of type {type(index_range)} not tuple")
    elif len(index_range) != 2:
        raise ValueError(f"index_range provided has length {len(index_range)} but should have length 2")
    elif type(index_range[0]) is not int:
        raise ValueError(f"Start index in index_range is type {type(index_range[0])} but should be int")
    elif type(index_range[1]) is not int:
        raise ValueError(f"End index in index_range is type {type(index_range[0])} but should be int")
    elif index_range[1] < index_range[0]:
        raise ValueError(f"End index in index_range is {index_range[1]} and start index is {index_range[0]}. "
                         f"End index cannot be lower than start index")

# Read the 'data' workbook from this file
excel_file = pd.ExcelFile(path)
df = pd.read_excel(excel_file, 'data')

# Tuples list instance
list_tuples = list()

# Determine the indexes
if index_range is not None:
    start_index = index_range[0]
    end_index = index_range[1] + 1
else:
    start_index = 0
    end_index = df[df.columns[0]].count()

# Iterate through the results and insert data tuples into the list
for i in range(start_index, end_index):
    try:
        list_tuples.append(tuple(
            [convert_to(df.loc[i, el[0]], el[1]) for el in clmns]
        ))
    except Exception as ex:
        log.log_error(f"[{df.loc[i, clmns[0]]}] {str(ex)}", True)
        if is_test:
            list_tuples.append((
                df.loc[i, clmns[0]], None, None, None, None, None, None, None, None, None, None, None, None, None
            ))

# Check if there were results and finish
if len(list_tuples) > 0:
    to_call(list_tuples, is_test)
else:
    log.log_info('There are no results from Excel')

convert_to函数代码 #

if pd.isna(obj):
        return None
    elif as_type is None or obj is None:
        return obj
    else:
        return as_type(obj)

原因分析 #

1. 索引匹配错误：使用df.loc[i]是按标签索引，若Excel读取后的DataFrame索引不是从0开始的连续整数，后续i值无法定位到对应行，触发异常。
2. 日志函数终止程序：log.log_error第二个参数传True时，大概率内部实现了抛出异常或终止程序的逻辑，导致循环在第一次异常后直接停止。
3. 行数计算错误：用df[df.columns[0]].count()获取行数仅统计第一列非空值数量，若第一列存在缺失值，会导致end_index小于实际行数；指定index_range时，end_index = index_range[1]+1可能超出DataFrame实际行数范围。

解决方案 #

1. 改用位置索引替代标签索引 #

将df.loc[i, el[0]]改为df.iloc[i][el[0]]，iloc基于行的位置（从0开始）定位，不受DataFrame标签索引影响：

for i in range(start_index, end_index):
    try:
        row = df.iloc[i]
        list_tuples.append(tuple(
            [convert_to(row[el[0]], el[1]) for el in clmns]
        ))
    except Exception as ex:
        log.log_error(f"[{row[clmns[0]]}] {str(ex)}", False)
        if is_test:
            list_tuples.append((
                row[clmns[0]], None, None, None, None, None, None, None, None, None, None, None, None, None
            ))

2. 避免日志函数终止程序 #

将log.log_error的第二个参数改为False，确保异常捕获后循环继续执行：

log.log_error(f"[{row[clmns[0]]}] {str(ex)}", False)

3. 修正行数计算逻辑 #

用len(df)获取DataFrame实际行数，替代count()统计，同时限制end_index不超过实际行数：

if index_range is not None:
    start_index = index_range[0]
    end_index = min(index_range[1] + 1, len(df))
else:
    start_index = 0
    end_index = len(df)

4. 优化循环逻辑（可选） #

用生成器表达式结合列表推导式简化代码，同时过滤无效结果：

def process_row(row_idx):
    try:
        row = df.iloc[row_idx]
        return tuple(convert_to(row[col], dtype) for col, dtype in clmns)
    except Exception as ex:
        log.log_error(f"[{row[clmns[0][0]]}] {str(ex)}", False)
        if is_test:
            return (row[clmns[0][0]],) + (None,) * (len(clmns)-1)
        return None

list_tuples = [res for res in (process_row(i) for i in range(start_index, end_index)) if res is not None]

内容的提问来源于stack exchange，提问作者prz.raf