Python 2.7中如何向并行化矩阵生成函数传递额外变量？

Hey there! I get that you're trying to parallelize matrix generation with multiprocess.Pool, and you need to pass an extra variable x to your fun function—great question! Let's go through a couple of straightforward ways to make this work in Python 2.7.

First, a quick critical note: In Python 2.7, when using multiprocessing, you must wrap your main code in an if __name__ == "__main__": block to avoid infinite spawning of child processes. I'll include that in all examples below.

Method 1: Use a Lambda to Wrap Arguments #

This is probably the most intuitive approach. Modify your fun to accept i, j, and x as separate parameters, then use a lambda function to bundle x with each index pair when passing to map().

Here's how to adjust your code:

import numpy as np
import multiprocess as mp
import itertools

def fun(i, j, x):
    print i, j
    prod = i * j * x  # Now you can use x here!
    return prod

if __name__ == "__main__":
    x = 10  # Define your extra variable here
    p = mp.Pool()
    
    # Generate all (i,j) index pairs (example: 3x3 matrix)
    indices = list(itertools.product(range(3), range(3)))
    
    # Use lambda to pass each index pair plus x to fun
    results = p.map(lambda idx: fun(idx[0], idx[1], x), indices)
    
    # Convert results back into a matrix
    matrix = np.array(results).reshape(3, 3)
    print("Generated matrix:\n", matrix)
    
    p.close()
    p.join()

Method 2: Use functools.partial to Bind the Variable #

If you prefer to keep the original structure of fun (accepting a tuple (i,j)), you can use functools.partial to "fix" the x parameter before passing the function to the pool.

Note: With partial, we need to put x as the first parameter in fun so it gets bound correctly:

import numpy as np
import multiprocess as mp
import itertools
from functools import partial

def fun(x, (i, j)):  # x comes first for partial binding
    print i, j
    prod = i * j * x
    return prod

if __name__ == "__main__":
    x = 10
    p = mp.Pool()
    indices = list(itertools.product(range(3), range(3)))
    
    # Create a new function where x is already bound
    fun_with_x = partial(fun, x)
    
    # Now we can pass the index pairs directly to map()
    results = p.map(fun_with_x, indices)
    
    matrix = np.array(results).reshape(3, 3)
    print("Generated matrix:\n", matrix)
    
    p.close()
    p.join()

Which Method Should You Choose? #

• Use the lambda approach if you want to keep fun's parameters clear and explicit.
• Use partial if you want to maintain the original tuple-based input for fun (or if you have more parameters to bind later).

Both methods will let you use your extra variable x in the parallelized function without any issues!

内容的提问来源于stack exchange，提问作者0_o