多GPU环境下vLLM在for循环中出现无限等待问题求助

问题现象 #

使用vLLM开启tensor_parallel_size=4（4卡GPU）时，循环调用自定义的attempt_api_call()函数会陷入无限等待：仅第一个worker完成生成，其余3个worker一直挂起。

• 单次调用attempt_api_call()无异常
• 设置tensor_parallel_size=1时循环调用正常，但单GPU会触发CUDA OOM，必须使用多GPU方案

环境配置 #

vLLM版本：0.5.1
LLM初始化代码：

VLLM_TENSOR_PARALLEL_SIZE = 4
llm = vllm.LLM(
    model_name,
    download_dir=os.getenv("HF_HOME"),
    enforce_eager=True,
    tensor_parallel_size=VLLM_TENSOR_PARALLEL_SIZE,
    gpu_memory_utilization=VLLM_GPU_MEMORY_UTILIZATION,
    trust_remote_code=True,
    dtype= "half",
)

关键错误日志 #

Processed prompts: 100%|█████████████████████████████████████████████████| 1/1 [00:01<00:00,  1.89s/it, est. speed input: 15.88 toks/s, output: 26.47 toks/s]
["Hello! I'm delighted to introduce myself. My name is LLaMA, and I'm a friendly AI assistant here to help you with any questions or tasks you may have. I'm happy to chat with you and provide assistance whenever you need it"]
WARNING 08-08 05:38:54 config.py:1354] Casting torch.bfloat16 to torch.float16.
INFO 08-08 05:38:54 config.py:698] Defaulting to use mp for distributed inference
INFO 08-08 05:38:54 llm_engine.py:169] Initializing an LLM engine (v0.5.1) with config: model='meta-llama/Meta-Llama-3-8B-Instruct', speculative_config=None, tokenizer='meta-llama/Meta-Llama-3-8B-Instruct', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, rope_scaling=None, rope_theta=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.float16, max_seq_len=8192, download_dir=None, load_format=LoadFormat.AUTO, tensor_parallel_size=4, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, kv_cache_dtype=auto, quantization_param_path=None, device_config=cuda, decoding_config=DecodingConfig(guided_decoding_backend='outlines'), observability_config=ObservabilityConfig(otlp_traces_endpoint=None), seed=0, served_model_name=meta-llama/Meta-Llama-3-8B-Instruct, use_v2_block_manager=False, enable_prefix_caching=False)
(VllmWorkerProcess pid=1154012) INFO 08-08 05:38:58 multiproc_worker_utils.py:215] Worker ready; awaiting tasks
(VllmWorkerProcess pid=1154013) INFO 08-08 05:38:58 multiproc_worker_utils.py:215] Worker ready; awaiting tasks
[W socket.cpp:697] [c10d] The client socket cannot be initialized to connect to [localhost]:37041 (errno: 97 - Address family not supported by protocol).
(VllmWorkerProcess pid=1154011) INFO 08-08 05:38:58 multiproc_worker_utils.py:215] Worker ready; awaiting tasks
[W socket.cpp:697] [c10d] The client socket cannot be initialized to connect to [localhost]:37041 (errno: 97 - Address family not supported by protocol).
[W socket.cpp:697] [c10d] The client socket cannot be initialized to connect to [localhost]:37041 (errno: 97 - Address family not supported by protocol).

已尝试无效方案 #

• 设置环境变量export VLLM_WORKER_MULTIPROC_METHOD=spawn
• 降级vLLM至0.5.0版本

问题原因分析 #

1. IPv6/IPv4通信冲突：错误日志中的errno:97表示地址协议不支持，说明vLLM worker进程尝试用IPv6连接localhost，但系统环境不支持IPv6，导致进程间通信失败，worker挂起等待。
2. 循环内重复初始化LLM：如果attempt_api_call()函数内每次都重新创建LLM实例，多GPU下会重复启动worker进程，引发端口冲突和通信混乱，最终导致死锁。

解决方案 #

1. 强制使用IPv4通信 #

设置环境变量强制vLLM使用IPv4：

export VLLM_HOST=0.0.0.0
export CUDA_DEVICE_ORDER=PCI_BUS_ID

同时检查/etc/hosts文件，确保localhost指向IPv4地址（127.0.0.1），注释掉IPv6的::1条目。

2. 复用LLM实例，避免重复初始化 #

将LLM初始化代码移到循环外部，在attempt_api_call()中复用已创建的实例：

# 循环外仅初始化一次LLM
VLLM_TENSOR_PARALLEL_SIZE = 4
llm = vllm.LLM(
    model_name,
    download_dir=os.getenv("HF_HOME"),
    enforce_eager=True,
    tensor_parallel_size=VLLM_TENSOR_PARALLEL_SIZE,
    gpu_memory_utilization=VLLM_GPU_MEMORY_UTILIZATION,
    trust_remote_code=True,
    dtype= "half",
)

def attempt_api_call(message):
    # 复用已有的llm实例生成回答
    outputs = llm.generate(message)
    return [output.outputs[0].text for output in outputs]

# 循环调用
for i in range(3):
    print(attempt_api_call(message))

3. 升级vLLM至最新版本 #

vLLM 0.5.x版本存在多GPU通信的已知bug，升级到0.6.x及以上版本可修复部分进程通信问题：

pip install --upgrade vllm

4. 检查系统端口与防火墙 #

确保系统未限制本地端口通信，防火墙规则允许localhost的TCP端口（如日志中的37041）进行进程间通信。

内容的提问来源于stack exchange，提问作者Yezun Chung