测试容器向LocalStack S3上传文件时触发Broken pipe错误

我用Docker容器部署LocalStack模拟AWS S3服务，通过以下Clojure代码上传文件：

(ns com.my.service.resources.s3
  (:require [clojure.tools.logging :refer [error info infof]]
            [clojure.java.io :as io])
  (:import
   [java.io File FileInputStream FileOutputStream]
   java.util.UUID
   [com.amazonaws ClientConfiguration]
   [com.amazonaws Protocol]
   [com.amazonaws.services.s3 AmazonS3Client AmazonS3URI]
   [com.amazonaws.services.s3.model DeleteObjectRequest
    GetObjectRequest ObjectMetadata PutObjectRequest]
   [com.amazonaws.services.s3.model AmazonS3Exception]))


(defn- s3-client
  []
  (let [client-configuration (ClientConfiguration.)
    s3-client (AmazonS3Client. client-configuration)]
(.setEndpoint s3-client "http://localstack:4566")
s3-client))

(defn upload
  "Upload a given file to s3 and return the path"
  [id user file filename size content-type]
  (metrics/time
   @registry
   {"operation" "upload" "service" "s3"}
   (let [key (str BASE "/" user "/" id ".tmp")
         size (.length file)
         content-type "application/shapefile"
         in (FileInputStream. file)
         metadata (doto (ObjectMetadata.)
                    (.setContentDisposition filename)
                    (.setContentLength size)
                    (.setContentType content-type))
         request (PutObjectRequest. BUCKET key in metadata)]
     (try
       (infof "uploading to s3://%s/%s" BUCKET key)
       (.putObject (s3-client) request)
       (format "s3://%s/%s" BUCKET key)
       (finally
         (try
           (.close in)
           (catch Exception ex nil)))))))

从测试容器向已创建好存储桶的LocalStack上传8k文件时，出现错误：

com.amazonaws.SdkClientException: Unable to execute HTTP request: Broken pipe (Write failed)\n\tat com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleRetryableException(AmazonHttpClient.java:1219)\n\tat

运行代码前执行aws --endpoint-url=http://localstack:4566 s3 ls能正常查看存储桶；报错后再执行该命令返回：

An error occurred (503) when calling the ListBuckets operation (reached max retries: 4): Service Unavailable

退出测试容器后重新进入，又能正常访问LocalStack。怀疑是代码问题导致管道中断，进而使测试容器异常。

容器通过docker-compose up启动，yaml配置如下：

localstack:
  image: xxx.ecr.us-east-1.amazonaws.com/hub/localstack/localstack:latest
  container_name: localstack
  ports:
    - "4566:4566"
  environment:
    - SERVICES=s3
    - DEBUG=1
    - DATA_DIR=
    - AWS_REGION=us-east-1
    - AWS_DEFAULT_REGION=us-east-1
    - AWS_ACCESS_KEY_ID=fake
    - AWS_SECRET_ACCESS_KEY=fake
    - HOSTNAME=localstack
    - HOSTNAME_EXTERNAL=localstack
  volumes:
    - "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack"
    - "/var/run/docker.sock:/var/run/docker.sock"

test:
  image: xxx.ecr.us-east-1.amazonaws.com/build-images/build-image:0.12.0
  container_name: test
  environment:
    - AWS_REGION=us-east-1
    - AWS_ACCESS_KEY_ID=myAccessKeyId
    - AWS_SECRET_ACCESS_KEY=mySecretAccessKey
    - JAVA_OPTS=-Dwebdriver.chrome.whitelistedIps=
    - ARTIFACTORY_SVC_ACCT_PWD=${ARTIFACTORY_SVC_ACCT_PWD}
    - ARTIFACTORY_SVC_ACCT=${ARTIFACTORY_SVC_ACCT}
  ports:
    - "46050:46050"
  volumes:
    - .:/tmp
  depends_on:
    - db
  entrypoint:
    - tail
    - -f
    - /dev/null

1. S3客户端配置缺失关键参数 #

你使用的是AWS SDK V1的AmazonS3Client，创建客户端时未设置区域和凭证，虽然LocalStack允许无凭证访问，但旧SDK可能因配置不完整导致请求异常，触发连接中断。

修改s3-client函数，补充凭证和区域配置：

(defn- s3-client
  []
  (let [client-configuration (ClientConfiguration.)
        credentials (com.amazonaws.auth.BasicAWSCredentials. "fake" "fake")
        s3-client (AmazonS3Client. credentials client-configuration)]
    (.setEndpoint s3-client "http://localstack:4566")
    (.setRegion s3-client (com.amazonaws.regions.RegionUtils/getRegion "us-east-1"))
    s3-client))

2. 文件流处理与元数据隐患 #

代码中size参数被重复赋值（函数入参的size被(.length file)覆盖），可能引发元数据不一致；手动管理FileInputStream存在流未正确处理的风险。改用SDK原生的文件上传方式，自动处理流：

(defn upload
  "Upload a given file to s3 and return the path"
  [id user file filename size content-type]
  (metrics/time
   @registry
   {"operation" "upload" "service" "s3"}
   (let [key (str BASE "/" user "/" id ".tmp")
         actual-size (or size (.length file))
         actual-content-type (or content-type "application/shapefile")
         metadata (doto (ObjectMetadata.)
                    (.setContentDisposition filename)
                    (.setContentLength actual-size)
                    (.setContentType actual-content-type))]
     (try
       (infof "uploading to s3://%s/%s" BUCKET key)
       ; 直接传入File对象，SDK自动处理流
       (.putObject (s3-client) (PutObjectRequest. BUCKET key file metadata))
       (format "s3://%s/%s" BUCKET key)
       (catch Exception ex
         (error ex "Failed to upload file to S3"))))))

3. LocalStack容器稳定性优化 #

报错后LocalStack返回503，可能是异常请求导致服务进程崩溃，调整以下配置：

• 改用固定版本的LocalStack镜像：latest标签可能包含不稳定特性，替换为localstack/localstack:2.3.2这类稳定版本
• 启用数据持久化：将DATA_DIR=改为DATA_DIR=/var/lib/localstack，避免状态丢失引发服务异常
• 添加服务就绪等待：测试容器的depends_on仅保证启动顺序，不确保服务就绪，在测试代码中添加等待逻辑：

(defn wait-for-s3-ready
  []
  (loop [attempts 0]
    (if (> attempts 10)
      (throw (Exception. "LocalStack S3 service not ready"))
      (try
        (.listBuckets (s3-client))
        (info "S3 service ready")
        (catch Exception ex
          (info "Waiting for S3 service...")
          (Thread/sleep 1000)
          (recur (inc attempts)))))))

4. Docker网络与凭证统一 #

• 统一容器间的AWS凭证：测试容器的AWS_ACCESS_KEY_ID和AWS_SECRET_ACCESS_KEY需与LocalStack配置一致，改为fake/fake，避免认证不匹配
• 检查文件权限：确保测试容器挂载的/tmp目录下的文件有可读权限，避免流读取失败导致连接中断

优先修复S3客户端的凭证和区域配置，优化文件上传的流处理逻辑，同时调整LocalStack的镜像版本和配置，统一容器间的AWS凭证，即可解决管道中断和服务不可用的问题。

内容的提问来源于stack exchange，提问作者Laodao