Python:是否存在适配字典列表的pool.starmap替代函数?
Great question! Let’s tackle this in two parts—first whether pool.starmap works with lists of dictionaries, then how to achieve the "starmap for dicts" behavior you’re looking for.
pool.starmap handle a list of dictionaries? Short answer: No, not directly. Here’s why: pool.starmap is designed to take an iterable of tuples, where each tuple’s elements are passed as positional arguments to your target function. If you pass a list of dictionaries instead, starmap will treat each entire dictionary as a single positional argument.
For example, if your function expects two parameters a and b, passing [{"a":1, "b":2}] to starmap would try to call func({"a":1, "b":2})—which will throw a TypeError because the function is missing the second required argument.
starmap-like function for dictionaries? There’s no built-in starmapdict in Python’s multiprocessing module, but you can easily replicate the behavior with a few straightforward approaches:
1. Use pool.map with a lambda (simple, watch for cross-platform quirks)
The easiest way is to wrap your function call in a lambda that unpacks the dictionary using ** (keyword argument unpacking):
from multiprocessing import Pool def calculate_sum(a, b): return a + b # Your list of dictionaries input_dicts = [{"a": 5, "b": 3}, {"a": 10, "b": 2}, {"a": 7, "b": 4}] if __name__ == "__main__": with Pool() as pool: results = pool.map(lambda kwargs: calculate_sum(**kwargs), input_dicts) print(results) # Output: [8, 12, 11]
⚠️ Note: On Windows, lambdas can cause serialization errors because the default spawn start method can’t pickle lambdas. For cross-platform compatibility, use the next approach.
2. Use a helper function + functools.partial (cross-platform safe)
Define a small helper function to handle dictionary unpacking, then use functools.partial to bind your target function to it:
from multiprocessing import Pool import functools def unpack_and_execute(func, kwargs): """Helper to unpack dict kwargs and call the target function""" return func(**kwargs) def calculate_sum(a, b): return a + b input_dicts = [{"a": 5, "b": 3}, {"a": 10, "b": 2}, {"a": 7, "b": 4}] if __name__ == "__main__": with Pool() as pool: # Bind calculate_sum to the helper function bound_func = functools.partial(unpack_and_execute, calculate_sum) results = pool.map(bound_func, input_dicts) print(results) # Output: [8, 12, 11]
3. Roll your own starmapdict function
If you want a clean, reusable interface similar to starmap, wrap the above logic into a custom function:
from multiprocessing import Pool import functools def starmapdict(pool, func, iterable_of_dicts): bound_func = functools.partial(lambda f, kw: f(**kw), func) return pool.map(bound_func, iterable_of_dicts) # Usage example def calculate_sum(a, b): return a + b input_dicts = [{"a": 5, "b": 3}, {"a": 10, "b": 2}, {"a": 7, "b": 4}] if __name__ == "__main__": with Pool() as pool: results = starmapdict(pool, calculate_sum, input_dicts) print(results)
pool.starmap doesn’t support dictionaries out of the box, but using pool.map with dictionary unpacking (**) gives you exactly the behavior you want—whether directly with a lambda (for Unix-like systems) or a helper function (for cross-platform safety).
内容的提问来源于stack exchange,提问作者Tiphaine Champetier

