基于epoll的C++98 WebServer中Keep-Alive事件处理异常问题

问题描述 #

我用C++98编写了一个简易WebServer，处理Keep-Alive请求时复用socket，但epoll_wait持续触发该socket的事件，几次recv读到0字节后服务器终止运行；发送带Connection:close头的请求则运行正常。目前不清楚处理Keep-Alive头的正确epoll事件处理方式，是否只需调用epoll_ctl(EPOLL_CTL_DEL)移除事件？

主循环代码 #

bool Webserver::_mainLoop() {
    int eventNumber=0;
    epoll_event events[MAX_EVENTS];
    do{
        // std::cout<<"main loop"<<std::endl;
        eventNumber= epoll_wait(this->_epollFd,events,MAX_EVENTS,EPOLL_TIMEOUT);
        std::cout<<RED<<"event number: "<<eventNumber<<RESET_COLOR<<std::endl;
        if(eventNumber>0)
        {
            if(!_handleEpollEvents(eventNumber,events)) {
                std::cout<<RED<<"Error handling epoll events"<<RESET_COLOR<<std::endl;
                return false;
            }
        }
    } while (eventNumber>=0);
    std::cout<<RED<<"out of do while loop"<<RESET_COLOR<<std::endl;
    if(eventNumber<0) {
        std::cout<<RED<<"Error epoll wait"<<RESET_COLOR<<std::endl;
        return false;
    }

}

事件处理代码 #

bool Webserver::_handleEpollEvents(int &eventNumber, epoll_event (&events)[MAX_EVENTS]) {
    std::cout<<BLUE<<"handling epoll events"<<RESET_COLOR<<std::endl;
    for (int i = 0; i < eventNumber; ++i)
    {
        std::cout<<"handling event number: "<<i<<std::endl;
        sType *type =static_cast<sType*>(events[i].data.ptr);
        if(type->socketType==SERVER_SOCK)
        {
            Server *server = static_cast<Server *>(events[i].data.ptr);
           if(!this->_acceptConnection(server))
           {
               std::cout<<"Error accepting connection"<<std::endl;
               return false;
           }
        }else if(((events[i].events & EPOLLIN) || (events[i].events & EPOLLOUT)) && _handleConnection(events[i]) )
        {
                 std::cout<<YELLOW<<"out of handling connection"<<RESET_COLOR<<std::endl;
                return true;
        }
    }
    return true;
}

当前Keep-Alive处理代码片段 #

if(client.request()->connection()=="keep-alive"&& client.response()->complete()) {
        std::cout<<GREEN<<"Connection keep-alive"<<RESET_COLOR<<std::endl;
        if(client.request()){
            delete client.request();
            client.set_request(NULL);
        }
        if(client.response()){
            delete client.response();
            client.set_response(NULL);
        }
        struct timeval tv;
        tv.tv_sec = 5;
        tv.tv_usec = 0;
        int flags = fcntl(client.getClientSock()->getFdSock(), F_GETFL, 0);
        flags |= O_NONBLOCK;
        fcntl(client.getClientSock()->getFdSock(), F_SETFL, flags);
        if (setsockopt(client.getClientSock()->getFdSock(), SOL_SOCKET, SO_KEEPALIVE,(char *) &tv, sizeof(tv)) ==SOCKET_ERROR) {
            std::cout<<RED << "cannot set socket option" << RESET_COLOR<<std::endl;
            return false;
        }

运行日志片段 #

send 3251 bytes
Response ended set to epollin
k e e p - a l i v e 
Connection keep-alive
Request : Destructor Called
Response : Destructor Called
out of handling connection
event number: 1
handling epoll events
handling event number: 0
handling connection
Request : Default Constructor Called
Response : Default Constructor Called
receiving data
client socket fd: 10
client socket service: 8090
client socket service: 127.1.0.1
client socket size: 16
Service sin family: 2
read 0 byte
Request ended set to epollout
response status code: 204client request connection: 
The socket is in non-blocking mode
response: 
�^ response size:4
send 4 bytes
Response ended set to epollin

response complete: truerequest error: false
 connection:  complete: true
event number: 1
handling epoll events
handling event number: 0
handling connection
ptr->socketType: 2
socket type: 2
Request : Default Constructor Called
Response : Default Constructor Called
receiving data
client socket fd: 10
client socket service: 8090
client socket service: 127.1.0.1
client socket size: 16
Service sin family: 2
read 0 byte
Request ended set to epollout
response status code: 204client request connection: 
The socket is in non-blocking mode
response: 
�^ response size:4

核心问题分析 #

• recv返回0的含义误解：非阻塞socket下recv返回0表示客户端已主动关闭连接（收到FIN包），此时必须关闭socket并从epoll中移除，不能继续复用。
• SO_KEEPALIVE参数设置错误：代码中用struct timeval设置SO_KEEPALIVE是错误的，该选项的参数是int类型（1开启，0关闭），心跳超时需要用TCP层的TCP_KEEPIDLE、TCP_KEEPINTVL、TCP_KEEPCNT选项配置。
• 事件处理循环逻辑漏洞：_handleEpollEvents处理完单个客户端事件就直接return true，导致同一轮epoll_wait返回的其他事件被忽略，引发异常堆积。
• 空指针访问导致无效响应：日志中的4字节乱码是因为request/response被delete后，后续处理未正确检查空指针，访问了已释放的内存。

修正方案 #

1. 正确处理recv返回0的场景 #

在_handleConnection中添加连接关闭逻辑：

ssize_t n = recv(client.getClientSock()->getFdSock(), buf, BUF_SIZE, 0);
if (n == 0) {
    // 客户端主动关闭，清理资源
    close(client.getClientSock()->getFdSock());
    epoll_ctl(_epollFd, EPOLL_CTL_DEL, client.getClientSock()->getFdSock(), nullptr);
    delete client.request();
    delete client.response();
    // 从连接管理容器中移除该客户端
    return false;
} else if (n < 0) {
    if (errno != EAGAIN && errno != EWOULDBLOCK) {
        // 发生真实错误，关闭连接
        close(client.getClientSock()->getFdSock());
        epoll_ctl(_epollFd, EPOLL_CTL_DEL, client.getClientSock()->getFdSock(), nullptr);
        delete client.request();
        delete client.response();
        return false;
    }
    // 非阻塞模式下无数据，直接返回
    return true;
}

2. 修复SO_KEEPALIVE参数设置 #

// 开启SO_KEEPALIVE
int keep_alive = 1;
if (setsockopt(client.getClientSock()->getFdSock(), SOL_SOCKET, SO_KEEPALIVE, &keep_alive, sizeof(keep_alive)) < 0) {
    perror("setsockopt SO_KEEPALIVE");
    return false;
}
// 配置心跳参数（可选）
int idle = 5;    // 5秒无数据发送第一个心跳包
int interval = 5;// 心跳包间隔时间
int cnt = 3;     // 心跳失败重试次数
setsockopt(client.getClientSock()->getFdSock(), IPPROTO_TCP, TCP_KEEPIDLE, &idle, sizeof(idle));
setsockopt(client.getClientSock()->getFdSock(), IPPROTO_TCP, TCP_KEEPINTVL, &interval, sizeof(interval));
setsockopt(client.getClientSock()->getFdSock(), IPPROTO_TCP, TCP_KEEPCNT, &cnt, sizeof(cnt));

3. 修复事件处理循环的return逻辑 #

处理完单个事件后不要直接返回，需遍历所有事件：

bool Webserver::_handleEpollEvents(int &eventNumber, epoll_event (&events)[MAX_EVENTS]) {
    std::cout<<BLUE<<"handling epoll events"<<RESET_COLOR<<std::endl;
    bool result = true;
    for (int i = 0; i < eventNumber; ++i)
    {
        std::cout<<"handling event number: "<<i<<std::endl;
        sType *type =static_cast<sType*>(events[i].data.ptr);
        if(type->socketType==SERVER_SOCK)
        {
            Server *server = static_cast<Server *>(events[i].data.ptr);
            if(!this->_acceptConnection(server))
            {
                std::cout<<"Error accepting connection"<<std::endl;
                result = false;
            }
        }else if((events[i].events & EPOLLIN) || (events[i].events & EPOLLOUT))
        {
            if(!_handleConnection(events[i])) {
                std::cout<<"Error handling connection"<<std::endl;
                result = false;
            }
        }
    }
    return result;
}

4. 完善Keep-Alive复用后的状态管理 #

处理完Keep-Alive请求后，除了释放request/response，还要确保后续处理时检查空指针：

• 在_handleConnection开头判断request是否为空，为空则仅当有有效数据时才初始化
• 避免在无合法请求的情况下生成无效响应

内容的提问来源于stack exchange，提问作者Trimia