Flask部署Heroku遇长任务超时崩溃，求加载页异步展示实现方案

问题背景 #

Heroku对Web请求设置了约30秒的超时限制，你的Flask应用中，yf.download下载股票数据的操作偶尔会超过这个时限，导致请求崩溃。需要将长耗时任务异步执行，先返回加载页面，待任务完成后再更新页面展示结果。

现有路由核心代码 #

df = yf.download(symbols, start=request.form.get("start"), end=request.form.get("end"), auto_adjust = False, prepost = False, threads = True, proxy = None)["Adj Close"].dropna(axis=1, how='all')

for col in df.columns:
    if col.endswith(".L"):
        df.loc[:,col] = df.loc[:,col]*GBPtoUSD()
latest_prices2 = df.iloc[-1]  # prices as of the day you are allocating
if float(request.form.get("funds")) <= 0 or float(request.form.get("funds")) == " ":
    flash("Amount need to be a positive number")
    return redirect("/build")
if float(request.form.get("funds")) < float(latest_prices.min()):
    flash("Amount is not high enough to cover the lowest priced stock")
    return redirect("/build")
da = DiscreteAllocation(weights, latest_prices, total_portfolio_value=float(request.form.get("funds")))
alloc2, leftover2 = da.lp_portfolio()
session['alloc2']=alloc2
session['latest_prices2']=latest_prices2
mc.delete("symbols")
return render_template ("built.html",alloc2=alloc2,leftover2=leftover2)

现有built.html代码 #

{% extends "layout.html" %}

{% block title %}
    Built
{% endblock %}

{% block main %}
<script src="https://cdn.plot.ly/plotly-latest.min.js"></script>
<script>
    function myFunction() {
        document.getElementById("btn1").disabled = true;
        document.getElementById("btn2").disabled = true;
        document.getElementById("btn3").disabled = true;
        document.getElementById("btn4").disabled = true;
    }
</script>

<div align="left";; class="child";; id="plotly-timeseries"></div>
<script>
var graph = {{ plot_json | safe }};
Plotly.plot('plotly-timeseries', graph, {});
</script>

<h4>Efficient semi-variance optimization</h4>
<p>Here we will minimise the portfolio semivariance (i.e downside volatility) subject to a return constraint ({{ ret }}%).</p>
<p>Expected annual return: {{ "%.2f" | format(perf2[0]*100) }}%</p>
<p>Annual semi-deviation: {{ "%.2f" | format(perf2[1]*100) }}%</p>
<p>Sortino Ratio: {{ "%.2f" | format(perf2[2]) }}%</p>
<div class="alert alert-success" role="alert">
  <h4>If you happy with the suggested portfolio, the following output will show how many stocks to buy and the leftover funds</h4>
  <p style="color:red;">Please note that the symbols prices may change between the time it takes to choose your portfolio</p>
  <P>{{ alloc2 }}</P>
  <p>{{ "$%.2f" | format(leftover2) }} leftover</p>
  <form action="/allocation2" method="post">
    <button class="btn btn-primary" onclick="this.form.submit(); myFunction()" id="btn3">Buy</button>
  </form>
</div>
{% endblock %}

最佳实践与具体实现 #

核心思路 #

1. 用异步任务队列分离长耗时操作（避免线程随dyno重启丢失，推荐用Flask-Executor或Celery+Redis）
2. 前端通过轮询检查任务状态，完成后加载结果
3. 任务结果存储在Redis中，确保跨dyno可访问

步骤1：配置异步任务环境 #

方案：Flask-Executor（轻量快速上手）

安装依赖：

pip install flask-executor redis

在Flask应用中初始化：

from flask_executor import Executor
import redis
import uuid
import json

app = Flask(__name__)
# 从Heroku环境变量获取Redis地址
app.config['REDIS_URL'] = os.environ.get('REDIS_URL')
executor = Executor(app)
r = redis.from_url(app.config['REDIS_URL'])

步骤2：修改路由逻辑，异步执行任务 #

@app.route('/build', methods=['POST'])
def build():
    # 提取表单/会话中的参数
    symbols = mc.get("symbols")  # 按你的原有逻辑获取
    start = request.form.get("start")
    end = request.form.get("end")
    funds = request.form.get("funds")
    weights = ...  # 按原有逻辑获取权重数据

    # 生成唯一任务ID，用于追踪状态
    task_id = str(uuid.uuid4())
    session['task_id'] = task_id

    # 提交异步任务
    executor.submit_stored(task_id, run_portfolio_task, symbols, start, end, funds, weights)

    # 立即返回加载页面
    return render_template('loading.html', task_id=task_id)

# 长耗时任务函数
def run_portfolio_task(task_id, symbols, start, end, funds, weights):
    try:
        # 下载股票数据
        df = yf.download(symbols, start=start, end=end, auto_adjust=False, prepost=False, threads=True, proxy=None)["Adj Close"].dropna(axis=1, how='all')
        # 转换英镑计价股票为美元
        for col in df.columns:
            if col.endswith(".L"):
                df.loc[:,col] = df.loc[:,col]*GBPtoUSD()
        latest_prices2 = df.iloc[-1]

        # 资金验证
        if not funds.strip() or float(funds) <= 0:
            result = {'status': 'error', 'message': 'Amount need to be a positive number'}
            r.setex(task_id, 3600, json.dumps(result))
            return
        if float(funds) < float(latest_prices2.min()):
            result = {'status': 'error', 'message': 'Amount is not high enough to cover the lowest priced stock'}
            r.setex(task_id, 3600, json.dumps(result))
            return

        # 计算分配结果
        da = DiscreteAllocation(weights, latest_prices2, total_portfolio_value=float(funds))
        alloc2, leftover2 = da.lp_portfolio()

        # 整理结果（包含原built.html需要的所有变量）
        result = {
            'status': 'success',
            'alloc2': alloc2,
            'leftover2': leftover2,
            'plot_json': ...,  # 按原有逻辑生成图表数据
            'ret': ...,
            'perf2': ...
        }
        # 存入Redis，1小时后过期释放空间
        r.setex(task_id, 3600, json.dumps(result))
    except Exception as e:
        # 捕获异常并返回错误信息
        result = {'status': 'error', 'message': str(e)}
        r.setex(task_id, 3600, json.dumps(result))

步骤3：创建加载页面loading.html #

页面显示加载状态，同时轮询任务状态接口：

{% extends "layout.html" %}

{% block title %}Loading...{% endblock %}

{% block main %}
<div class="text-center mt-5">
    <div class="spinner-border text-primary" role="status">
        <span class="visually-hidden">Loading...</span>
    </div>
    <p class="mt-3">正在获取股票数据，请稍候...</p>
</div>

<script>
const taskId = "{{ task_id }}";
// 每2秒轮询一次任务状态
const pollInterval = setInterval(() => {
    fetch(`/task_status/${taskId}`)
        .then(res => res.json())
        .then(data => {
            if (data.status === 'success') {
                clearInterval(pollInterval);
                // 跳转到结果页面
                window.location.href = `/built_result/${taskId}`;
            } else if (data.status === 'error') {
                clearInterval(pollInterval);
                alert(data.message);
                window.location.href = '/build';
            }
            // 任务pending时继续轮询
        })
        .catch(err => console.error('检查任务状态失败:', err));
}, 2000);
</script>
{% endblock %}

步骤4：添加任务状态接口和结果页面路由 #

@app.route('/task_status/<task_id>')
def task_status(task_id):
    result = r.get(task_id)
    if not result:
        return json.dumps({'status': 'pending'})
    return result

@app.route('/built_result/<task_id>')
def built_result(task_id):
    result = r.get(task_id)
    if not result:
        flash('任务已过期或不存在')
        return redirect('/build')
    
    result_data = json.loads(result)
    if result_data['status'] == 'error':
        flash(result_data['message'])
        return redirect('/build')
    
    # 删除Redis中的结果，避免内存占用
    r.delete(task_id)
    # 直接复用原built.html，传入所有需要的变量
    return render_template('built.html', **result_data)

关键注意事项 #

1. Heroku Redis配置：需要在Heroku控制台添加Redis add-on，自动注入REDIS_URL环境变量
2. 任务持久化：必须用Redis存储任务结果，避免dyno重启导致任务数据丢失
3. 超时清理：设置Redis键的过期时间，防止无效数据堆积
4. 异常捕获：异步任务中必须捕获所有异常，避免任务静默失败

内容的提问来源于stack exchange，提问作者originn