Lighttpd mod_wstunnel通过UNIX套接字通信时消息被合并问题
问题现象
使用lighttpd的mod_wstunnel模块时,后端通过UNIX域套接字与lighttpd通信,若连续执行两次send/write调用发送消息,lighttpd日志显示其实际接收到的是两条消息拼接后的单条消息。
测试观察确认:两次向UNIX套接字执行write操作之间哪怕仅增加1ms延迟,lighttpd就能正确接收两条独立消息,并按原样转发给终端客户端。
目前已搭建最小概念验证样例,可稳定复现问题,相关内容如下:
1. lighttpd配置文件
启动命令:lighttpd -D -f lighttpd-poc.conf
server.document-root = var.CWD server.bind = "0.0.0.0" server.port = 8042 server.username = "www" server.groupname = "www" mimetype.assign = ( ".html" => "text/html", ".txt" => "text/plain", ".jpg" => "image/jpeg", ".png" => "image/png", ".js" => "text/javascript", ".css" => "text/css", ".json" => "application/json", ) static-file.exclude-extensions = ( ".fcgi", ".php", ".rb", "~", ".inc" ) index-file.names = ( "index.html" ) server.modules += ("mod_wstunnel") # 匹配/ws开头的URL请求,使用文本帧类型 $HTTP["url"] =~ "^/ws" { wstunnel.server = ( "/ws/" => ( ( "socket" => "/tmp/ws.socket", ) ) ) server.stream-response-body = 2 wstunnel.ping-interval = 30 # 客户端连接存活期间保持长连接,不依赖客户端交互频率 wstunnel.frame-type = "text" wstunnel.debug = 5 } # 尝试关闭lighttpd侧缓冲,未达预期:后端发送12MB消息时lighttpd仍会分块传输 server.stream-response-body = 2 server.stream-request-body = 2 # server.chunkqueue-chunk-sz = 1
2. 前端测试页面
将文件命名为poc.html放在lighttpd静态文件目录下即可访问:
<!DOCTYPE html> <script> var ws = new WebSocket('ws://' + location.host + '/ws/'); ws.onopen = function () { ws.send(JSON.stringify({ "foo": "bar" })); }; ws.onmessage = function (event) { try{ console.log(JSON.parse(event.data)); } catch(SyntaxError) { console.log(`Parse problem: ${event.data}`); } }; </script>
3. 后端WebSocket处理复现代码
编译运行命令:g++ poc.cpp && ./a.out
#include <cstdio> #include <unistd.h> #include <sys/socket.h> #include <sys/un.h> #include <cstdlib> #include <iostream> #include <thread> #include <chrono> using namespace std::chrono_literals; int main() { std::cout << "Started...\n"; struct sockaddr_un address; int server_fd, client_sock; if ((server_fd = socket(AF_UNIX, SOCK_STREAM, 0)) == -1) { perror("socket init failure"); exit(EXIT_FAILURE); } unlink("/tmp/ws.socket"); memset(&address, 0, sizeof(address)); address.sun_family = AF_UNIX; strncpy(address.sun_path, "/tmp/ws.socket", sizeof(address.sun_path) - 1); if (bind(server_fd, (struct sockaddr*)&address, sizeof(address)) < 0) { perror("bind failure"); exit(EXIT_FAILURE); } if (listen(server_fd, 42) < 0) { perror("listen failure"); exit(EXIT_FAILURE); } while (true) { int addrlen = sizeof(address); if ((client_sock = accept(server_fd, (struct sockaddr*)&address, (socklen_t*)&addrlen)) < 0) { perror("accept failure"); exit(EXIT_FAILURE); } char buf[] = "{\"foo\": \"BAR\", \"I\": \"\", \"R\": { \"X\": { \"Z\": 1.2699546813964844, \"T\": 2 } }}"; char buf2[] = "{ \"foo\": \"BAR\", \"I\": \"\", \"R\": { \"Y\": { \"C\": 421035002, \"M\": 2147483647 }}}"; int rc; char buffer[1024] = {0}; if ((rc = read(client_sock, buffer, 1024)) > 0) { int num_sent = write(client_sock, buf, strlen(buf)); // std::this_thread::sleep_for(1ms); // 取消注释添加1ms延迟后,消息可正常分离 num_sent = write(client_sock, buf2, strlen(buf2)); } } return 0; }
关键复现点:代码中被注释的
std::this_thread::sleep_for(1ms)行直接决定问题是否出现:注释该行时可稳定复现双消息被合并为单条的问题;取消注释启用1ms休眠后,两条消息可被正常分离发送。
访问http://localhost:8042/poc.html测试时,注释休眠行的场景下浏览器会收到拼接后的双JSON内容,导致JSON.parse调用报错。
排查进展与诉求
若使用二进制协议,可通过协议设计明确帧的起止边界,自行完成消息解析。但当前场景为后端直接发送JSON对象、前端接收解析,初步推测是内核层面执行了消息缓冲——从lighttpd日志可见,其收到的内容就已经是合并后的单条大消息或分离的两条小消息,因此可排除lighttpd和业务代码主动做缓冲的可能。
目前可通过增加发送间隔、使用json-multi-parse类库处理拼接内容临时规避问题,但这类方案稳定性不足、不符合常规设计逻辑,需要定位问题根本原因,明确缓冲产生的层级与机制,获取更合理的解决方案。本次测试使用的lighttpd版本为1.4.55。
内容的提问来源于stack exchange,提问作者Paul

