Great question—running non-root containers while still being able to write to cloud volumes like AWS EBS (via cloudstor) is a super common pain point, especially with Docker Swarm’s constraints around single-mount volumes. Let’s break down the best practices that fit your current setup, plus how this will get easier when you move to Kubernetes later.

Best Practices for Docker Swarm + Cloudstor/AWS-EBS #

1. Standardize on a Fixed UID/GID in Your Images #

First, avoid arbitrary UIDs in your containers. Define a consistent non-root user with a known UID/GID in your Dockerfile. This makes aligning volume permissions across environments straightforward:

# Create a non-root user with fixed UID/GID (adjust numbers to your needs)
RUN useradd -m -u 1001 -U appuser
USER appuser

This ensures your container runs as appuser (UID 1001) by default. Now you just need to make sure your cloud volume has permissions that let this user write to it.

2. Provision Volumes with Preconfigured Permissions #

For cloudstor and AWS EBS, you can set volume ownership at provisioning time to match your container’s UID/GID:

• Cloudstor: When creating the volume via CLI, use the uid and gid options to set ownership directly:

  docker volume create --driver cloudstor:aws \
    --opt size=10 \
    --opt uid=1001 \
    --opt gid=1001 \
    my-production-volume
  

  This ensures the volume’s root directory is owned by UID 1001/GID 1001, so your non-root container can write to it without permission errors.

• AWS EBS: If you provision volumes via CloudFormation or Terraform, add a temporary post-attachment step to set permissions. Use a short-lived Swarm task constrained to the node where the volume is attached:

  docker service create --name volume-perms-fixer \
    --restart-condition none \
    --constraint node.hostname==<target-node-hostname> \
    --mount type=volume,source=my-ebs-volume,target=/mnt \
    alpine \
    sh -c "chown -R 1001:1001 /mnt && exit 0"
  

  Run this task once before starting your main service. Since EBS volumes can only mount to one node, this guarantees permissions are fixed before your non-root container uses the volume.

3. Use a Short-Lived Helper Container (No Root for Main App) #

If you can’t pre-provision permissions (e.g., for existing volumes), use a helper container that runs as root briefly to fix permissions, then exits. This is way safer than running your main container as root even temporarily:
In your Docker Compose file, define a helper service that runs once:

version: "3.8"
services:
  volume-fixer:
    image: alpine
    command: sh -c "chown -R 1001:1001 /data"
    volumes:
      - my-volume:/data
    deploy:
      restart_policy:
        condition: none
      placement:
        constraints:
          - node.hostname==<target-node> # Match your main service's node
  my-app:
    image: my-non-root-app:latest
    user: "1001:1001"
    volumes:
      - my-volume:/data
    deploy:
      placement:
        constraints:
          - node.hostname==<target-node>
      # Add a healthcheck or wait script to ensure permissions are set before starting

Note: Docker Swarm’s depends_on doesn’t wait for the helper to finish, so add a small script in your main container to check if /data is writable before starting the app (e.g., until [ -w /data ]; do sleep 1; done).

Transitioning to Kubernetes #

Since you plan to migrate to Kubernetes, rest assured it has native support for solving this problem cleanly:

• Security Context: Define a securityContext in your Pod spec to run as a fixed UID/GID, and use fsGroup to auto-adjust volume permissions:

  securityContext:
    runAsUser: 1001
    runAsGroup: 1001
    fsGroup: 1001
  

  Kubernetes will automatically set the volume’s ownership to the fsGroup, so your non-root pod can write to it without manual fixes.
• EBS CSI Driver: When using AWS EBS with Kubernetes, the CSI driver handles volume provisioning, and the fsGroup setting ensures permissions are correct out of the box.

Why Avoid Other Evaluated Solutions? #

Let’s quickly recap why some of your tested approaches aren’t ideal:

• Running main container as root: Violates core security best practices, exposing your workload to unnecessary risks.
• Pre-creating volumes in the image: Only works for new, empty volumes—not existing cloud volumes you need to reuse.
• Volumes-provisioner: Adds unnecessary complexity and doesn’t handle EBS’s single-mount constraint reliably.
• Fixing permissions after container start: Risks race conditions and conflicts with EBS’s single-mount rule.

内容的提问来源于stack exchange，提问作者Chris Maes