开启lazy-apps的uWSGI+Flask多进程下Timer全局变量同步问题求助

Hey there, you're totally right about the root cause: with lazy-apps = true, every uWSGI worker process loads your Flask app independently. That means each worker gets its own copy of global_var, and the @timer(60) runs separately in every process. That's why you see initial 1s and then diverging counts later on.

Since disabling lazy-apps isn't an option due to MongoDB and other exceptions, here are practical, worker-safe solutions to sync your counter without dropping to a single process:

1. Use uWSGI Shared Memory (Fast, Server-Local) #

uWSGI has built-in shared memory support that lets all processes access the same variable directly. It's the fastest option for single-server setups:

import uwsgi
from uwsgidecorators import timer

# Initialize shared memory area (4 bytes is enough for a standard integer)
if uwsgi.masterpid() == uwsgi.worker_id():
    # Only the master process creates the shared area to avoid duplicates
    uwsgi.sharedarea_create(0, 4)
    uwsgi.sharedarea_set(0, 0, b'\x00\x00\x00\x00')  # Start counter at 0

@timer(60)
def foo():
    # Read current value from shared memory
    current_bytes = uwsgi.sharedarea_get(0, 0, 4)
    current_val = int.from_bytes(current_bytes, byteorder='little')
    
    # Increment and write back
    current_val += 1
    uwsgi.sharedarea_set(0, 0, current_val.to_bytes(4, byteorder='little'))
    
    print(current_val)

Pros: Blazing fast, no external dependencies.
Cons: Only works if all workers run on the same server.

2. Use Redis (Scalable, Distributed-Friendly) #

If you might scale to multiple servers later, a centralized datastore like Redis handles atomic increments natively, eliminating race conditions entirely:

First, install the Redis client:

pip install redis

Then adjust your code:

import redis
from uwsgidecorators import timer

# Connect to your Redis instance (update host/port/password as needed)
redis_client = redis.Redis(host='localhost', port=6379, db=0)

@timer(60)
def foo():
    # Atomic increment operation guarantees no race conditions
    global_var = redis_client.incr('grace_counter')
    print(global_var)

Pros: Works across multiple servers, fully concurrency-safe.
Cons: Adds an external dependency (Redis server).

3. Offload the Timer to a uWSGI Mule Process #

uWSGI's mule system lets you run isolated processes dedicated to background tasks. You can restrict your timer to run only in one mule, so there's no duplicate counter:

First, add a mule to your uwsgi.ini:

[uwsgi]
# ... your existing config ...
mule = 1  # Start one dedicated mule process

Then modify your code to target the mule:

from uwsgidecorators import timer, mule

# Only run this timer in the mule process
@mule
@timer(60)
def foo():
    global global_var
    global_var += 1
    print(global_var)

Pros: No external dependencies, keeps the counter isolated to a single process.
Cons: The mule doesn't handle web traffic, so it's only useful for background tasks.

4. File Lock + Local File (Simple, Low-Overhead) #

If you want a no-frills solution without extra tools, a file with an exclusive lock works for small-scale apps:

import fcntl
from uwsgidecorators import timer

COUNTER_PATH = '/tmp/grace_counter.txt'

# Initialize counter file if it doesn't exist
try:
    with open(COUNTER_PATH, 'r'):
        pass
except FileNotFoundError:
    with open(COUNTER_PATH, 'w') as f:
        f.write('0')

@timer(60)
def foo():
    with open(COUNTER_PATH, 'r+') as f:
        # Acquire exclusive lock to prevent race conditions
        fcntl.flock(f, fcntl.LOCK_EX)
        
        # Read, increment, and write back
        current_val = int(f.read().strip())
        current_val += 1
        f.seek(0)
        f.write(str(current_val))
        f.truncate()
        
        print(current_val)

Pros: Dead simple, no extra dependencies.
Cons: Slower than shared memory/Redis, not ideal for high-frequency tasks.

All these solutions play nicely with lazy-apps = true—they either share state across processes or restrict the timer to a single process. Pick the one that fits your deployment size and performance needs!

内容的提问来源于stack exchange，提问作者fabbronet