Flask Session UnpickleError问题：Heroku上RQ后台任务异常求助

Hey there, let's tackle this Flask Session UnpickleError you're hitting with RQ and Redis on Heroku—this is a super common gotcha when passing data between Flask's request context and background workers, so let's break down why it's happening and how to fix it.

Why You're Seeing This Error #

Flask's session object is tied to the request context and is encrypted using your app's SECRET_KEY. When you pass the entire session (or even parts of it that contain context-bound objects) to an RQ background job, three key issues arise:

1. The worker process doesn't have access to the same request context as your Flask web route, so it can't properly deserialize context-dependent data.
2. If the session data is encrypted, the worker needs the exact same SECRET_KEY as your web app to decrypt it—if it doesn't, unpickling fails immediately.
3. You might be accidentally passing non-picklable objects (like SQLAlchemy model instances, request objects, or database connections) stored in the session, which can't be serialized for the background job.

Step-by-Step Fixes #

1. Stop Passing the Entire Session—Extract Raw, Picklable Data

Instead of passing the session object directly to your background function, pull out only the specific data you need. Stick to simple, serializable types like strings, numbers, lists, or basic dictionaries.

Bad Practice (Causes UnpickleError):

from rq import Queue
import redis
import os

q = Queue(connection=redis.from_url(os.getenv("REDIS_URL")))

@app.route("/dashboard")
def dashboard():
    # Load data into session
    session["selected_info"] = some_complex_data
    # Don't do this!
    job = q.enqueue(long_running_function, session)
    return "Job started!"

Good Practice:

@app.route("/dashboard")
def dashboard():
    session["selected_info"] = {"user_id": 123, "report_type": "sales"}
    # Extract only the data your job needs
    selected_data = session.get("selected_info")
    # Pass just the raw data
    job = q.enqueue(long_running_function, selected_data["user_id"], selected_data["report_type"])
    return "Job started!"

2. Ensure RQ Workers Share Your Flask App's Configuration

Your RQ worker needs access to the same SECRET_KEY and Redis configuration as your web app. On Heroku, make sure these are set as environment variables (run heroku config:set SECRET_KEY=your-key-here REDIS_URL=your-redis-url) and initialize your worker with the same app configuration.

Create a dedicated worker script (e.g., worker.py) that loads your Flask app's config:

from flask import Flask
from rq import Worker, Queue, Connection
import redis
import os

# Initialize Flask app to load config
app = Flask(__name__)
app.config["SECRET_KEY"] = os.getenv("SECRET_KEY")
app.config["REDIS_URL"] = os.getenv("REDIS_URL")

# Connect to Redis
conn = redis.from_url(app.config["REDIS_URL"])

if __name__ == "__main__":
    with Connection(conn):
        # Use your queue name (default is 'default')
        worker = Worker([Queue("default")])
        worker.work()

Then update your Procfile to start the worker correctly:

web: gunicorn app:app
worker: python worker.py

3. Avoid Passing Non-Picklable Objects

If you're storing things like SQLAlchemy model instances in your session, those can't be serialized. Instead, store the object's ID in the session, then fetch the object again in the background job.

Example:

# In your dashboard route
@app.route("/dashboard")
def dashboard():
    user = User.query.get(current_user.id)
    # Don't store the model instance in session
    # session["user"] = user  # ❌
    # Store the ID instead
    session["user_id"] = user.id
    job = q.enqueue(long_running_function, session["user_id"])
    return "Job started!"

# In your background function
def long_running_function(user_id):
    # Re-initialize the app/db context in the worker
    from app import db, User
    user = User.query.get(user_id)
    # Do your long-running work here

4. Verify Redis Configuration on Heroku

Double-check that both your web process and worker are using the same Redis URL. Run heroku config:get REDIS_URL to confirm the value, and make sure your worker script is pulling this from the environment variable (not hardcoding it).

Quick Recap #

The core issue is mixing request-bound session data with stateless background workers. Stick to passing only simple, serializable data, ensure your worker has the same app config as your web process, and avoid non-picklable objects entirely.

内容的提问来源于stack exchange，提问作者user1592380