Digital Ocean Docker化环境下S3上传超时问题求助

AI 小北

2026-6-14

问题：Docker容器内无法连接DigitalOcean Spaces（S3兼容存储）

问题背景

在DigitalOcean拥有Droplet服务器和S3兼容的Spaces存储，运行Docker化Django应用时，尝试将静态资源同步到Spaces失败。本地环境和Droplet宿主机（容器外）直接测试上传均正常，推测问题出在Docker配置。

测试脚本

编写了简单测试脚本s3upload.py（同目录存在foobar.txt）：

from boto3.s3.transfer import S3Transfer
import boto3
import logging

logging.getLogger().setLevel(logging.DEBUG)

client = boto3.client('s3', 
                      aws_access_key_id="…", 
                      aws_secret_access_key="…", 
                      region_name="ams3", 
                      endpoint_url="https://ams3.digitaloceanspaces.com")
transfer = S3Transfer(client)
bucket_name = "…"
transfer.upload_file("foobar.txt", bucket_name, "foobar.txt")

容器内错误日志

执行脚本时出现超时错误：

Traceback (most recent call last):
  File "/usr/local/lib/python3.13/site-packages/boto3/s3/transfer.py", line 372, in upload_file
    future.result()
    ~~~~~~~~~~~~~^^
  File "/usr/local/lib/python3.13/site-packages/s3transfer/futures.py", line 103, in result
    return self._coordinator.result()
           ~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/usr/local/lib/python3.13/site-packages/s3transfer/futures.py", line 264, in result
    raise self._exception
  File "/usr/local/lib/python3.13/site-packages/s3transfer/tasks.py", line 135, in __call__
    return self._execute_main(kwargs)
           ~~~~~~~~~~~~~~~~~~^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/s3transfer/tasks.py", line 158, in _execute_main
    return_value = self._main(**kwargs)
  File "/usr/local/lib/python3.13/site-packages/s3transfer/upload.py", line 796, in _main
    client.put_object(Bucket=bucket, Key=key, Body=body, **extra_args)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/botocore/client.py", line 569, in _api_call
    return self._make_api_call(operation_name, kwargs)
           ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/botocore/client.py", line 1023, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (RequestTimeout) when calling the PutObject operation (reached max retries: 4): None

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/app/s3upload.py", line 14, in <module>
    transfer.upload_file("foobar.txt", bucket_name, "foobar.txt")
    ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/boto3/s3/transfer.py", line 378, in upload_file
    raise S3UploadFailedError(
    ...<3 lines>...
    )
boto3.exceptions.S3UploadFailedError: Failed to upload foobar.txt to [bucketname]/foobar.txt: An error occurred (RequestTimeout) when calling the PutObject operation (reached max retries: 4): None

Docker配置

compose.yml

services:
  db:
    image: postgres:17
    environment:
      POSTGRES_DB: ${POSTGRES_DB}
      POSTGRES_USER: ${POSTGRES_USER}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
    ports:
      - "5432:5432"
    volumes:
      - postgres_data:/var/lib/postgresql/data
    env_file:
      - .env

  django-web:
    build: .
    container_name: django-docker
    ports:
      - "80:80"
    expose:
      - "80"
    depends_on:
      - db
    environment:
      SECRET_KEY: ${SECRET_KEY}
      DEBUG: ${DEBUG}

      POSTGRES_DB: ${POSTGRES_DB}
      POSTGRES_USER: ${POSTGRES_USER}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}

      DATABASE_HOST: ${DATABASE_HOST}
      DATABASE_PORT: ${DATABASE_PORT}
    env_file:
      - .env

volumes:
  postgres_data:

Dockerfile

# Stage 1: Build

# Use the official Python runtime image
FROM python:3.13-slim AS builder 
 
# Create the app directory
RUN mkdir /app
 
# Set the working directory inside the container
WORKDIR /app
 
# Set environment variables 
# Prevents Python from writing pyc files to disk
ENV PYTHONDONTWRITEBYTECODE=1
#Prevents Python from buffering stdout and stderr
ENV PYTHONUNBUFFERED=1 
 
# Upgrade pip
RUN pip install --upgrade pip 
 
# Copy the Django project  and install dependencies
COPY requirements.txt  /app/
 
# run this command to install all dependencies 
RUN pip install --no-cache-dir -r requirements.txt

# Stage 2: Run production code

FROM python:3.13-slim

RUN useradd -m -r appuser && mkdir /app && chown -R appuser /app

# Copy the Python dependencies from the builder stage
COPY --from=builder /usr/local/lib/python3.13/site-packages/ /usr/local/lib/python3.13/site-packages/
COPY --from=builder /usr/local/bin/ /usr/local/bin/

# Set the working directory
WORKDIR /app

# Copy application code
COPY --chown=appuser:appuser . .
 
# Set environment variables to optimize Python
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1 
 
# Switch to non-root user
USER appuser

# Expose the Django port
EXPOSE 80

# # Migrate the DB
# RUN ["python", "manage.py", "migrate"]

# Gather static assets

# ---> This is the s3 command not working
# RUN ["python", "manage.py", "collectstatic", "--no-input"]

# Run the app via the gunicorn server
CMD ["gunicorn", "--bind", "0.0.0.0:80", "--workers", "3", "fonts.wsgi"]
# python manage.py migrate && python manage.py collectstatic --no-input && gunicorn myapp.wsgi

测试结果对比

本地测试环境：
- python s3upload.py 执行成功
- docker compose exec -T django-web python s3upload.py 执行成功
DigitalOcean Droplet服务器：
- python s3upload.py 执行成功
- docker compose exec -T django-web python s3upload.py 超时并出现上述错误

排查思路与可能原因

Docker网络配置检查：
- 在compose.yml的django-web服务中添加公共DNS服务器配置，解决可能的DNS解析问题：
```
django-web:
  # ...其他配置
  dns: ["8.8.8.8", "1.1.1.1"]
```
- 临时将容器网络模式改为host（生产环境不推荐），验证是否为网络隔离导致：
```
django-web:
  # ...其他配置
  network_mode: host
```
容器内网络连通性测试：
- 进入容器执行curl -v https://ams3.digitaloceanspaces.com，检查DNS解析和连接建立情况
- 执行nc -zv ams3.digitaloceanspaces.com 443，测试443端口是否能正常连通

boto3配置优化：

增加超时时间和重试次数，修改client初始化代码：

import botocore.config
config = botocore.config.Config(
    read_timeout=30,
    connect_timeout=30,
    retries={'max_attempts': 5}
)
client = boto3.client('s3', 
                      # ...其他参数
                      config=config)

临时禁用SSL验证（仅测试），排查证书问题：

client = boto3.client('s3', 
                      # ...其他参数
                      verify=False)

资源与权限检查：
- 检查compose.yml是否设置了mem_limit、cpus等资源限制，导致网络请求无法及时处理
- 临时切换到root用户执行测试脚本，排查非root用户的网络权限问题：
```
docker compose exec -u root django-web python s3upload.py
```
DigitalOcean内网优化：
- 如果Droplet和Spaces在同一区域，尝试使用内网Endpoint（http://ams3.digitaloceanspaces.com），避免公网延迟或限制

内容的提问来源于stack exchange，提问作者kontur

Digital Ocean Docker化环境下S3上传超时问题求助

问题背景 #

测试脚本 #

容器内错误日志 #

Docker配置 #

compose.yml #

Dockerfile #

测试结果对比 #

排查思路与可能原因 #