Python线程实现：不同及相近线程数量下的行为差异

Hey there! Let’s dig into why your threading code behaves differently when you change the number of threads, and how to fix it.

The core issue here is that your code is accessing and modifying a shared, thread-unsafe resource (the list p) without any synchronization between threads. Let’s break down what happens when you run multiple threads:

1. When you start more than one thread, all of them execute the work() function at the same time.
2. Each thread loops over range(0, len(p))—but len(p) gets calculated once at the start of the loop, while other threads are actively removing elements with p.pop(i).
3. This leads to race conditions:
   • One thread might pop an element, making the list shorter, while another thread tries to access an index that no longer exists (causing an IndexError).
   • Multiple threads could end up processing the same element, or elements might be skipped entirely because the list’s order changes unexpectedly mid-loop.

With only 1 thread, there’s no concurrency to cause these conflicts, so the code runs as expected. But add more threads, and chaos ensues!

You have two solid approaches to fix this: using a thread lock to synchronize access to the list, or using a thread-safe queue to manage tasks.

Option 1: Use a Thread Lock #

A threading.Lock ensures that only one thread can access the shared list at a time. This eliminates race conditions:

import threading

p = [11,22,33,44,55,66,77,88,99,111,222,333,444,555,666,777,888,999,1111,22222,44444,566666,877889528]
# Define a global lock to protect access to p
list_lock = threading.Lock()

def create_workers(num_threads):
    for _ in range(num_threads):
        t = threading.Thread(target=work)
        t.daemon = True
        t.start()

def work():
    while True:
        # Use `with` to automatically acquire/release the lock
        with list_lock:
            if not p:  # Exit if the list is empty
                break
            # Pop from the start to avoid index issues from changing list length
            item = p.pop(0)
        # Print outside the lock to keep the critical section small
        print(threading.current_thread().name, item)

if __name__ == '__main__':
    create_workers(3)  # Try any number of threads here!

Key improvements:

• The lock wraps all operations that read or modify p, so only one thread can touch the list at a time.
• We switched to a while True loop instead of looping over len(p), which avoids issues with the list shrinking mid-loop.

Option 2: Use a Thread-Safe Queue #

For thread-based task processing, Python’s queue.Queue is designed to be thread-safe out of the box. It handles synchronization internally, so you don’t need to manage locks manually:

import threading
from queue import Queue

p = [11,22,33,44,55,66,77,88,99,111,222,333,444,555,666,777,888,999,1111,22222,44444,566666,877889528]
task_queue = Queue()

# Populate the queue with all items first
for item in p:
    task_queue.put(item)

def create_workers(num_threads):
    for _ in range(num_threads):
        t = threading.Thread(target=work)
        t.daemon = True
        t.start()

def work():
    while not task_queue.empty():
        try:
            # Get an item from the queue (thread-safe)
            item = task_queue.get(timeout=1)
            print(threading.current_thread().name, item)
            # Mark the task as completed
            task_queue.task_done()
        except:
            # Exit if the queue is empty or times out
            break

if __name__ == '__main__':
    create_workers(3)

This approach is cleaner for most threading tasks, as it separates task storage from processing and handles all concurrency safety under the hood.

内容的提问来源于stack exchange，提问作者user3814582