为何之前正常运行的Boost Beast代码在1.75.0版本突然报错？是否为版本回归？

我近期将Boost版本从1.67.0升级到1.75.0后，原本用于与REST API交互的Boost Beast代码立即出现异常。原有代码运行正常，但升级后服务端收到的请求内容疑似异常，暂时未定位到问题根源。
以下代码功能为向Kubernetes API提交JSON字符串以定义自定义资源，Kubernetes API服务器目前无法识别该Boost发送的POST请求，REST API的具体细节不影响问题排查：

#include <string>
#include <iostream>
#include <sstream>
#include <fstream> 
#include <boost/beast/core.hpp>
#include <boost/beast/version.hpp>
#include <boost/beast/http.hpp>
#include <boost/asio/connect.hpp>
#include <boost/asio/ip/tcp.hpp>
#include <boost/asio/ssl/stream.hpp> 
#include <boost/asio/ssl/error.hpp>
#include <boost/property_tree/ptree.hpp>
#include <boost/property_tree/json_parser.hpp>


namespace bip = boost::asio::ip;
namespace bhttp = boost::beast::http;
namespace ssl = boost::asio::ssl;

void postServiceEndpoint(std::string topicName, std::string url, 
                         std::string host, std::string port, std::string discoveryNamespace)
{
  boost::asio::io_context context;
  boost::asio::ip::tcp::resolver resolver(context);  
  ssl::context sslCtx({ssl::context::sslv23_client});
  boost::asio::ssl::stream<boost::beast::tcp_stream> sslStream(context, sslCtx);
  
 
  auto const results = resolver.resolve(host, port);
  SSL_set_tlsext_host_name(sslStream.native_handle(), host.c_str());
  boost::beast::get_lowest_layer(sslStream).connect(results);
  sslStream.handshake(ssl::stream_base::client);


  //Load the bearer token for authenticating with K8s... 
  std::ifstream t("/var/run/secrets/kubernetes.io/serviceaccount/token");
  std::string str((std::istreambuf_iterator<char>(t)), 
                  std::istreambuf_iterator<char>());
  std::string bearerToken = str; 

  std::string target = "/apis/sdsendpoints.net/v1/namespaces/" + discoveryNamespace + "sdsendpoints"; 
  //Because the endpoint hasn't been created yet, we cant use it in the target
  //string, but if we want to retrieve the endpoint later, we have to use its name
  //in the target string... Kubernetes's REST API be weird like that. 
  bhttp::request<bhttp::string_body> request(bhttp::verb::post, target, HTTPV1DOT1);
  request.set(bhttp::field::host, host);
  request.set("Content-Type", "application/json");
  request.set("Authorization", "Bearer " + bearerToken);
   
  boost::property_tree::ptree requestTree;
  requestTree.put("apiVersion", "sdsendpoints.net/v1");
  requestTree.put("kind", "SdsEndpoint");
  requestTree.put("metadata.name", topicName);
  requestTree.put("spec.endpointURL", url);
      
  std::stringstream jsonStream;
  boost::property_tree::write_json(jsonStream, requestTree);
  request.body() = jsonStream.str();
  request.prepare_payload();
  std::cout << "REQUEST: \n" << request << std::endl;
  bhttp::write(sslStream, request);
  boost::beast::flat_buffer buffer;
  bhttp::response<bhttp::string_body> response;
  bhttp::read(sslStream, buffer, response);         
  if(response.result_int() >= 400)
  {
       std::cout << "Got failure on post endpoint: " << response.result_int() << ": " << response.result() << " : " << response.body() << std::endl;
  }
  //Cleanup the SSL socket...
  boost::system::error_code ec;
  sslStream.shutdown(ec);
  if(ec == boost::asio::error::eof)
  {
    //This is fine. I am okay with the events that are unfolding currently.
    ec.assign(0, ec.category());
  }
  if(ec)
  {
    std::cout << "Got error code: " << ec << " on socket cleanup in SSL shutdown" << std::endl;
  }
  sslStream.lowest_layer().shutdown(boost::asio::ip::tcp::socket::shutdown_both, ec);
  if(ec)
  { 
    std::cout << "Got error code: " << ec << " on socket cleanup in TCP socket shutdown." << std::endl;
  }

}

现象说明 #

运行代码输出的Boost HTTP请求如下，看起来是完全合法的HTTP请求：

POST apis/sdsendpoints.net/v1/namespaces/sds-test/sdsendpoints HTTP/1.1
Host: kubernetes
Authorization: Bearer <REDACTED> 

Content-Type: application/json
Content-length: 133

{"apiVersion" : "sdsendpoints.net/v1", "kind":"SdsEndpoint","metadata":{"name":"sds-tester"},"spec":{"endpointURL":"tcp://test-host:31337"}}

1.75.0版本下服务端返回如下报错，1.67.0版本不会出现该问题：

Got failure on post endpoint: 400: Bad Request : {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"the object provided is unrecognized (must be of type SdsEndpoint): couldn't get version/kind; json parse error: unexpected end of JSON input (\u003cempty\u003e)","reason":"BadRequest","code":400}

已排查验证点 #

• 切换回1.67版本的TCP socket套SSL流写法，问题依然存在
• 替换Boost Property Tree为Boost JSON生成请求体，问题依然存在
• 用curl携带相同请求头、请求体发起请求，请求成功
• 测试两种编译器GCC 4.8.5、Intel icpc 19.1.0.166 20191121，运行环境为RHEL 7.9，问题均复现

问题出在K8s ServiceAccount Token的读取逻辑，以及Boost.Beast 1.70+版本的HTTP头处理逻辑变更：

1. 从/var/run/secrets/kubernetes.io/serviceaccount/token读取的token末尾默认带有换行符，读取后没有做trim处理直接放入了Authorization请求头的值中
2. Boost 1.67版本的Beast库会自动过滤请求头值中的换行、回车等无效字符，不会破坏HTTP报文结构；而1.70之后的版本为了提升性能，不再自动处理头值中的无效字符，严格按照输入内容发送
3. 最终发送的报文里Authorization头末尾多了一个换行符，直接导致服务端把该换行后的空行识别为请求头和请求体的分隔符，后续的Content-Type、Content-Length头都被当成了请求体的一部分，服务端拿到的有效请求体为空，因此报JSON解析意外结束的错误
4. 打印请求时看到的Authorization头后的空行就是该换行导致的，属于报文结构被破坏的直接表现

读取token后对内容做trim处理，去掉首尾的空白字符（换行、空格、制表符、回车等）即可，修改token读取部分的代码：

std::ifstream t("/var/run/secrets/kubernetes.io/serviceaccount/token");
std::string str((std::istreambuf_iterator<char>(t)), 
                std::istreambuf_iterator<char>());
// 新增trim逻辑，去掉首尾空白符
boost::trim(str);
std::string bearerToken = str;

如果不想引入Boost的trim工具，也可以自己实现字符串trim逻辑，效果完全一致。

SSL shutdown返回的asio.ssl:2错误是服务端收到非法请求后主动断开连接导致的，属于异常请求的连锁反应，修复请求问题后该错误也会自动消失。

内容的提问来源于stack exchange，提问作者stix