SemanticKernel中GetStreamingChatMessageContentsAsync返回空内容问题
SemanticKernel本地LLM流式调用无内容返回问题
我在本地LLM上使用SemanticKernel时,非流式调用代码可以正常运行:
var chat = app.Services.GetRequiredService<IChatCompletionService>();
ChatMessageContent response = await chat.GetChatMessageContentAsync(chatHistory); var items = response.Items; var firstitem = items.FirstOrDefault(); var textContent = firstitem as TextContent; Console.WriteLine(textContent?.Text);
这段代码能如期返回回复:Hello! How can I assist you today? 😊
但尝试流式调用时:
await foreach (StreamingChatMessageContent stream in chat.GetStreamingChatMessageContentsAsync("Hi")) { // await Task.Yield(); // 试过这个但没用 Console.WriteLine(stream.Content); }
却返回12条空结果,序列化后的内容如下:
{"Content":null,"Role":{"Label":"Assistant"},"ChoiceIndex":0,"ModelId":"deepseek-r1-distill-llama-8b","Metadata":{"CompletionId":"chatcmpl-m086eaeve495763ls6arwj","CreatedAt":"2025-02-13T10:22:51+00:00","SystemFingerprint":"deepseek-r1-distill-llama-8b","RefusalUpdate":null,"Usage":null,"FinishReason":null}}
随后返回一条带“stop”标识的结果:
{"Content":null,"Role":null,"ChoiceIndex":0,"ModelId":"deepseek-r1-distill-llama-8b","Metadata":{"CompletionId":"chatcmpl-m086eaeve495763ls6arwj","CreatedAt":"2025-02-13T10:22:51+00:00","SystemFingerprint":"deepseek-r1-distill-llama-8b","RefusalUpdate":null,"Usage":null,"FinishReason":"Stop"}}
我确认服务器运行正常(非流式调用可正常工作),但流式调用无法返回内容。以下是非流式调用和流式调用的服务器日志对比:
非流式调用服务器日志
2025-02-13 12:28:42 [DEBUG] Received request: POST to /v1/chat/completions with body { "messages": [ { "role": "user", "content": "Hi" } ], "model": "deepseek-r1-distill-llama-8b" } 2025-02-13 12:28:42 [INFO] [LM STUDIO SERVER] Running chat completion on conversation with 1 messages. 2025-02-13 12:28:42 [DEBUG] Sampling params: repeat_last_n = 64, repeat_penalty = 1.100, frequency_penalty = 0.000, presence_penalty = 0.000 dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = -1 top_k = 40, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, temp = 0.800 mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000 2025-02-13 12:28:42 [DEBUG] sampling: logits -> logit-bias -> penalties -> dry -> top-k -> typical -> top-p -> min-p -> xtc -> temp-ext -> dist generate: n_ctx = 4096, n_batch = 512, n_predict = -1, n_keep = 12 BeginProcessingPrompt 2025-02-13 12:28:42 [DEBUG] FinishedProcessingPrompt. Progress: 100 2025-02-13 12:28:42 [INFO] [LM STUDIO SERVER] Accumulating tokens ... (stream = false) 2025-02-13 12:28:42 [DEBUG] [deepseek-r1-distill-llama-8b] Accumulated 1 tokens <think> 2025-02-13 12:28:42 [DEBUG] [deepseek-r1-distill-llama-8b] Accumulated 2 tokens <think>\n\n 2025-02-13 12:28:42 [DEBUG] [deepseek-r1-distill-llama-8b] Accumulated 3 tokens <think>\n\n</think> 2025-02-13 12:28:42 [DEBUG] [deepseek-r1-distill-llama-8b] Accumulated 4 tokens <think>\n\n</think>\n\n 2025-02-13 12:28:42 [DEBUG] [deepseek-r1-distill-llama-8b] Accumulated 5 tokens <think>\n\n</think>\n\nHello 2025-02-13 12:28:42 [DEBUG] [deepseek-r1-distill-llama-8b] Accumulated 6 tokens <think>\n\n</think>\n\nHello! 2025-02-13 12:28:42 [DEBUG] [deepseek-r1-distill-llama-8b] Accumulated 7 tokens <think>\n\n</think>\n\nHello! How 2025-02-13 12:28:42 [DEBUG] [deepseek-r1-distill-llama-8b] Accumulated 8 tokens <think>\n\n</think>\n\nHello! How can 2025-02-13 12:28:42 [DEBUG] [deepseek-r1-distill-llama-8b] Accumulated 9 tokens <think>\n\n</think>\n\nHello! How can I 2025-02-13 12:28:42 [DEBUG] [deepseek-r1-distill-llama-8b] Accumulated 10 tokens <think>\n\n</think>\n\nHello! How can I assist 2025-02-13 12:28:42 [DEBUG] [deepseek-r1-distill-llama-8b] Accumulated 11 tokens <think>\n\n</think>\n\nHello! How can I assist you 2025-02-13 12:28:42 [DEBUG] [deepseek-r1-distill-llama-8b] Accumulated 12 tokens <think>\n\n</think>\n\nHello! How can I assist you today 2025-02-13 12:28:42 [DEBUG] [deepseek-r1-distill-llama-8b] Accumulated 13 tokens <think>\n\n</think>\n\nHello! How can I assist you today? 2025-02-13 12:28:42 [DEBUG] Incomplete UTF-8 character. Waiting for next token (skip) 2025-02-13 12:28:42 [DEBUG] [deepseek-r1-distill-llama-8b] Accumulated 14 tokens <think>\n\n</think>\n\nHello! How can I assist you today? 😊 2025-02-13 12:28:42 [DEBUG] target model llama_perf stats: llama_perf_context_print: load time = 4657.69 ms llama_perf_context_print: prompt eval time = 0.00 ms / 1 tokens ( 0.00 ms per token, inf tokens per second) llama_perf_context_print: eval time = 350.76 ms / 16 runs ( 21.92 ms per token, 45.62 tokens per second) llama_perf_context_print: total time = 361.22 ms / 17 tokens 2025-02-13 12:28:42 [INFO] [LM STUDIO SERVER] [deepseek-r1-distill-llama-8b] Generated prediction: { "id": "chatcmpl-qv5tc01fntgw2bsm3091wk", "object": "chat.completion", "created": 1739442522, "model": "deepseek-r1-distill-llama-8b", "choices": [ { "index": 0, "logprobs": null, "finish_reason": "stop", "message": { "role": "assistant", "content": "<think>\n\n</think>\n\nHello! How can I assist you today? 😊" } } ], "usage": { "prompt_tokens": 4, "completion_tokens": 14, "total_tokens": 18 }, "system_fingerprint": "deepseek-r1-distill-llama-8b" }
流式调用服务器日志
2025-02-13 12:30:18 [DEBUG] Received request: POST to /v1/chat/completions with body { "messages": [ { "role": "user", "content": "Hi" } ], "model": "deepseek-r1-distill-llama-8b", "stream": true, "stream_options": { "include_usage": true } } 2025-02-13 12:30:18 [INFO] [LM STUDIO SERVER] Running chat completion on conversation with 1 messages. 2025-02-13 12:30:18 [INFO] [LM STUDIO SERVER] Streaming response... 2025-02-13 12:30:18 [DEBUG] Sampling params: repeat_last_n = 64, repeat_penalty = 1.100, frequency_penalty = 0.000, presence_penalty = 0.000 dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = -1 top_k = 40, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, temp = 0.800 mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000 sampling: logits -> logit-bias -> penalties -> dry -> top-k -> typical -> top-p -> min-p -> xtc -> temp-ext -> dist generate: n_ctx = 4096, n_batch = 512, n_predict = -1, n_keep = 12 BeginProcessingPrompt 2025-02-13 12:30:18 [DEBUG] FinishedProcessingPrompt. Progress: 100 2025-02-13 12:30:18 [INFO] [LM STUDIO SERVER] First token generated. Continuing to stream response.. 2025-02-13 12:30:18 [INFO] [LM STUDIO SERVER] Received <think> - START 2025-02-13 12:30:18 [DEBUG] Incomplete UTF-8 character. Waiting for next token (skip) 2025-02-13 12:30:18 [DEBUG] target model llama_perf stats: llama_perf_context_print: load time = 4657.69 ms llama_perf_context_print: prompt eval time = 0.00 ms / 1 tokens ( 0.00 ms per token, inf tokens per second) llama_perf_context_print: eval time = 354.33 ms / 16 runs ( 22.15 ms per token, 45.16 tokens per second) llama_perf_context_print: total time = 365.10 ms / 17 tokens 2025-02-13 12:30:18 [INFO] Finished streaming response
内容的提问来源于stack exchange,提问作者Pierre
相关产品推荐
相关产品推荐

