基于Python通过Logic Apps运行长耗时Azure Functions遇阻求助
问题:Azure Functions + Logic Apps 处理大Excel文件时的超时与请求过大问题
我尝试解决这个问题已有一段时间,始终没找到可行方案。需求是通过Azure Functions在Logic Apps中运行一项长耗时任务:用Python代码对比并转换两个包含近2万行数据(后续还会新增)的Excel文件。本地运行Python代码耗时约8分钟,转为Azure Functions后本地运行需11分钟,这显然会触发Logic Apps的超时限制。
我采用了HTTP webhook方案,思路是创建一个接收请求的函数,收到请求后立即返回202 Accepted响应,同时启动线程执行长耗时任务,任务完成后调用传入的callbackUri。实现代码如下:
import logging import azure.functions as func import time import requests import threading app = func.FunctionApp(http_auth_level=func.AuthLevel.ANONYMOUS) @app.function_name("http_trigger1") @app.route("http_trigger1") def http_trigger1(req: func.HttpRequest) -> func.HttpResponse: logging.info('Webhook request from Logic Apps received.') callback_url = req.params.get('callback_url') data = req.params.get('data') if not callback_url: try: req_body = req.get_json() except ValueError: pass else: callback_url = req_body.get('callback_url') data = req_body.get('data') if not callback_url: return func.HttpResponse( "Enter a valid callback URL", status_code=400 ) try: # 立即返回202 Accepted threading.Thread(target=process_and_callback, args=(callback_url, data)).start() return func.HttpResponse( "Accepted for processing", status_code=202 ) except Exception as e: logging.error(f'An error occurred: {str(e)}') return func.HttpResponse( "Error occurred while invoking callback", status_code=500 ) def process_and_callback(callback_url: str, data: str) -> None: try: # 模拟长耗时任务 time.sleep(720) callback_data = { "Subject": data } # 发起回调请求 response = requests.post(callback_url, json=callback_data) response.raise_for_status() logging.info(f'Callback successful with status code: {response.status_code}') except Exception as e: logging.error(f'An error occurred while invoking callback: {str(e)}')
用12分钟睡眠模拟长耗时任务时,该方案可正常绕过Logic Apps的超时限制,但替换为实际Excel处理代码后出现两个问题:
- Logic Apps间歇性返回
RequestEntityTooLarge错误(请求体包含callbackUri及两个Base64编码的Excel文件); - 重新提交后任务陷入运行状态超1.5小时(本地仅需11分钟)。
测试小文件时方案可行,仅大文件出现问题。以下是实际长耗时任务的核心代码片段:
# 读取Excel文件'file1' wb_file1 = pd.read_excel(file1) column_A = [f"{wb_file1.iloc[i, 7]},{wb_file1.iloc[i, 20]},{wb_file1.iloc[i, 21]}" for i in range(1, len(wb_file1))] # 读取Excel文件'file2' wb_file2 = pd.read_excel(file2, sheet_name=sheet_name) column_a_out = {} duplicates = [] for i, a in enumerate(column_A): row_number = wb_file2.index[(wb_file2.iloc[:, 7].astype(str) + ',' + wb_file2.iloc[:, 20].astype(str) + ',' + wb_file2.iloc[:, 21].astype(str)) == a].tolist() if not len(row_number) > 1: row = int(row_number[0]) + 2 column_a = a column_f = f"{wb_file1.iloc[i+1, 5]}" column_ad = f"{wb_file1.iloc[i+1, 29]}" column_af = f"{wb_file1.iloc[i+1, 31]}" column_a_file1[i+1] = [row, column_a, column_f, column_ad, column_af] else: duplicates.append(a) # 加载工作簿到openpyxl进行操作 wb_file2 = openpyxl.load_workbook(file2) sheet_file2 = wb_file2[sheet_name] for items in column_a_out: row_num, a, f, ad, af = column_a_out[items] sheet_file2.cell(row=row_num, column=6, value="" if f == 'nan' else f) sheet_file2.cell(row=row_num, column=30, value="" if ad == 'nan' else ad) sheet_file2.cell(row=row_num, column=32, value="" if af == 'nan' else af)
希望有人能帮我解决该问题,如需更多信息可随时提供。
内容的提问来源于stack exchange,提问作者No Name
相关产品推荐
相关产品推荐

