Python多线程运行函数实例时保存输出结果的技术咨询

Hey there! Let's sort out how to save those DataFrames your threaded functions are returning. The default threading.Thread doesn't give you a direct way to grab return values, but we've got a couple of straightforward approaches to fix this. Also, I noticed a tiny bug in your bigbangtheory1 function—you were using new instead of old when creating the DataFrame, so I'll fix that in the examples below.

Approach 1: Use a Thread-Safe Queue #

Queues are perfect for collecting results from threads because they handle synchronization automatically, so you don't have to worry about race conditions. Here's how to implement it:

import threading
import pandas as pd
import queue

def bigbangtheory():
    new = {'name': ['Sheldon', 'Penny', 'Amy', 'Bernadette', 'Raj', 'Howard '],
           'episodes': [31, 24, 31, 29, 37, 40],
           'gender': ['male', 'female', 'female', 'female', 'male', 'male']}
    return pd.DataFrame(new, columns=['name','episodes', 'gender'])

def bigbangtheory1():
    old = {'name': ['Sheldon', 'Penny', 'Amy', 'Bernadette', 'Raj', 'Howard'],
           'episodes': [12, 32, 31, 32, 37, 40],
           'gender': ['male', 'female', 'female', 'female', 'male', 'male']}
    # Fixed: Use 'old' instead of 'new' here
    return pd.DataFrame(old, columns=['name','episodes', 'gender'])

def worker(func, result_queue):
    # Run the function and put its result into the queue
    result = func()
    result_queue.put(result)

if __name__ == "__main__":
    # Create a thread-safe queue to hold results
    result_queue = queue.Queue()
    
    # Create threads, passing the target function and queue to the worker
    t1 = threading.Thread(target=worker, args=(bigbangtheory, result_queue))
    t2 = threading.Thread(target=worker, args=(bigbangtheory1, result_queue))
    
    # Start and join threads
    t1.start()
    t2.start()
    t1.join()
    t2.join()
    
    # Retrieve results from the queue
    df_new = result_queue.get()
    df_old = result_queue.get()
    
    # Now you can save or process the DataFrames
    print("New Big Bang Data:")
    print(df_new)
    print("\nOld Big Bang Data:")
    print(df_old)
    
    # Save to CSV files
    df_new.to_csv('bigbang_new.csv', index=False)
    df_old.to_csv('bigbang_old.csv', index=False)

How this works: #

• We define a worker function that runs your target function and pushes the result into the queue.
• The queue handles all thread safety, so multiple threads can add results without conflicts.
• After all threads finish, we pull the results from the queue and use them as needed.

Approach 2: Use concurrent.futures.ThreadPoolExecutor (Recommended) #

If you want a cleaner, higher-level API, ThreadPoolExecutor is the way to go. It manages threads for you and makes retrieving return values trivial:

import pandas as pd
from concurrent.futures import ThreadPoolExecutor

def bigbangtheory():
    new = {'name': ['Sheldon', 'Penny', 'Amy', 'Bernadette', 'Raj', 'Howard '],
           'episodes': [31, 24, 31, 29, 37, 40],
           'gender': ['male', 'female', 'female', 'female', 'male', 'male']}
    return pd.DataFrame(new, columns=['name','episodes', 'gender'])

def bigbangtheory1():
    old = {'name': ['Sheldon', 'Penny', 'Amy', 'Bernadette', 'Raj', 'Howard'],
           'episodes': [12, 32, 31, 32, 37, 40],
           'gender': ['male', 'female', 'female', 'female', 'male', 'male']}
    return pd.DataFrame(old, columns=['name','episodes', 'gender'])

if __name__ == "__main__":
    # Create a thread pool with 2 workers
    with ThreadPoolExecutor(max_workers=2) as executor:
        # Submit each function to the pool and get a Future object
        future_new = executor.submit(bigbangtheory)
        future_old = executor.submit(bigbangtheory1)
        
        # Retrieve the results (this blocks until the function finishes)
        df_new = future_new.result()
        df_old = future_old.result()
    
    # Process or save the DataFrames
    print("New Big Bang Data:")
    print(df_new)
    print("\nOld Big Bang Data:")
    print(df_old)
    
    df_new.to_csv('bigbang_new.csv', index=False)
    df_old.to_csv('bigbang_old.csv', index=False)

Why this is better: #

• No need to manually manage threads or queues—ThreadPoolExecutor handles all that under the hood.
• Future objects make it easy to check if a task is done or retrieve its result.
• The with statement automatically cleans up the thread pool when you're done.

内容的提问来源于stack exchange，提问作者Slartibartfast