使用Logback无代理向DataDog发日志遇连接失败问题求助

想要通过Datadog无代理方式直接发送日志，不使用代理、Logstash等中间件，但按官方教程配置后出现连接失败报错，具体日志如下：

20:39:26,617 |-WARN in net.logstash.logback.appender.LogstashTcpSocketAppender[JSON_TCP] - Log destination intake.logs.datadoghq.com/<unresolved>:10514: connection failed. java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@576302b[Not completed, task = net.logstash.logback.appender.AbstractLogstashTcpSocketAppender$TcpSendingEventHandler$ReaderCallable@622eaf6d] rejected from java.util.concurrent.ScheduledThreadPoolExecutor@3403e2ac[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0]
    at java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@576302b[Not completed, task = net.logstash.logback.appender.AbstractLogstashTcpSocketAppender$TcpSendingEventHandler$ReaderCallable@622eaf6d] rejected from java.util.concurrent.ScheduledThreadPoolExecutor@3403e2ac[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0]
    at  at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2081)
    at  at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:841)
    at  at java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:340)
    at  at java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:579)
    at  at java.util.concurrent.ScheduledThreadPoolExecutor.submit(ScheduledThreadPoolExecutor.java:731)
    at  at net.logstash.logback.appender.AbstractLogstashTcpSocketAppender.scheduleReaderCallable(AbstractLogstashTcpSocketAppender.java:1109)
    at  at net.logstash.logback.appender.AbstractLogstashTcpSocketAppender$TcpSendingEventHandler.openSocket(AbstractLogstashTcpSocketAppender.java:799)
    at  at net.logstash.logback.appender.AbstractLogstashTcpSocketAppender$TcpSendingEventHandler.onStart(AbstractLogstashTcpSocketAppender.java:691)
    at  at net.logstash.logback.appender.AsyncDisruptorAppender$EventClearingEventHandler.onStart(AsyncDisruptorAppender.java:382)
    at  at net.logstash.logback.encoder.com.lmax.disruptor.BatchEventProcessor.notifyStart(BatchEventProcessor.java:224)
    at  at net.logstash.logback.encoder.com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:120)
    at  at java.base/java.lang.Thread.run(Thread.java:1583)

项目Maven依赖配置：

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>3.2.3</version>
    <relativePath/> <!-- lookup parent from repository -->
</parent>

<dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-to-slf4j</artifactId>
</dependency>
<dependency>
    <groupId>ch.qos.logback</groupId>
    <artifactId>logback-classic</artifactId>
</dependency>
<dependency>
    <groupId>net.logstash.logback</groupId>
    <artifactId>logstash-logback-encoder</artifactId>
    <version>7.4</version>
</dependency>

Logback配置文件：

<configuration>
    <appender name="JSON_TCP" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
        <destination>intake.logs.datadoghq.com:10514</destination>
        <keepAliveDuration>20 seconds</keepAliveDuration>
        <encoder class="net.logstash.logback.encoder.LogstashEncoder">
            <prefix class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
                <layout class="ch.qos.logback.classic.PatternLayout">
                    <pattern>5aa43880a6a7aa59a48286a5a2a6a548 %mdc</pattern>
                </layout>
            </prefix>
        </encoder>
    </appender>

    <root level="DEBUG">
        <appender-ref ref="JSON_TCP" />
    </root>
</configuration>

结合报错信息和配置细节，按以下步骤逐一排查修复：

• 升级logstash-logback-encoder版本
  当前使用的7.4版本过于老旧，与Spring Boot 3.2.3（基于Java 17）存在兼容性问题，会导致线程池管理异常。建议升级到兼容Java 17的稳定版本，比如7.14.0：

  <dependency>
      <groupId>net.logstash.logback</groupId>
      <artifactId>logstash-logback-encoder</artifactId>
      <version>7.14.0</version>
  </dependency>
  

• 启用SSL/TLS加密连接
  Datadog的10514端口要求SSL加密传输，未配置SSL会直接导致连接被拒绝。在Logback的TCP appender中添加SSL配置：

  <appender name="JSON_TCP" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
      <destination>intake.logs.datadoghq.com:10514</destination>
      <keepAliveDuration>20 seconds</keepAliveDuration>
      <!-- 新增SSL配置 -->
      <ssl>
          <enabled>true</enabled>
      </ssl>
      <encoder class="net.logstash.logback.encoder.LogstashEncoder">
          <prefix class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
              <layout class="ch.qos.logback.classic.PatternLayout">
                  <pattern>5aa43880a6a7aa59a48286a5a2a6a548 %mdc</pattern>
              </layout>
          </prefix>
      </encoder>
  </appender>
  

• 排查DNS解析与网络连通性
  报错中的<unresolved>表明域名intake.logs.datadoghq.com无法解析，可在应用所在服务器执行nslookup intake.logs.datadoghq.com验证DNS是否正常。同时检查防火墙、安全组是否允许出站10514端口的TCP流量。

• 验证API密钥正确性
  确认prefix中的API密钥完整且格式正确，无多余空格或字符。密钥错误会导致Datadog拒绝连接，间接触发此类报错。

• 手动配置线程池（可选）
  若升级版本后仍存在线程池终止问题，可手动指定线程池参数：

  <appender name="JSON_TCP" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
      <!-- 其他配置 -->
      <executorService class="java.util.concurrent.ScheduledThreadPoolExecutor">
          <corePoolSize>2</corePoolSize>
      </executorService>
  </appender>
  

内容的提问来源于stack exchange，提问作者kafkaobservability