如何在K8s Pod中无特权升级运行Nginx并绑定80端口？

我正尝试以最小权限运行Nginx，使其作为代理绑定80端口，当前使用的容器securityContext配置如下：

securityContext:
  allowPrivilegeEscalation: false
  readOnlyRootFilesystem: true
  runAsNonRoot: true
  runAsUser: 12321
  runAsGroup: 12321
  privileged: false
  capabilities:
    drop:
    - all
    add: ["NET_BIND_SERVICE"]

我发现相关问题大多会将allowPrivilegeEscalation设为true，仅看到一篇博客设置为false但无法复现其示例。同时尝试使用nginxinc/nginx-unprivileged基础镜像，同样失败。

自定义Dockerfile配置 #

我编写了包含大量权限调整操作的Dockerfile，安装libcap2为Nginx二进制文件设置NET_BIND_SERVICE权限，并通过自定义entrypoint调整配置文件位置：

FROM nginx

ENV NGINX_USER="proxy-user" \
    NGINXR_UID="12321" \
    NGINX_GROUP="proxy-group" \
    NGINX_GID="12321"  

RUN set -ex; \
  groupadd -r --gid "$NGINX_GID" "$NGINX_GROUP"; \
  useradd -r --uid "$NGINXR_UID" --gid "$NGINX_GID" "$NGINX_USER" 


# 创建空pid文件并赋予代理用户权限
RUN touch /var/run/nginx.pid && \
    chown -R proxy-user:proxy-group /var/run/nginx.pid

# 修改其他Nginx文件权限
RUN chmod +x /usr/sbin/nginx && \
    chown -R proxy-user:proxy-group /usr/share/nginx/html && \
    chown -R proxy-user:proxy-group /var/cache/nginx && \
    chown -R proxy-user:proxy-group /var/log/nginx && \
    chown -R proxy-user:proxy-group /etc/nginx/conf.d

# 安装libcap并为Nginx二进制文件设置NET_BIND_SERVICE权限
RUN apt-get -qq update && \
    apt-get -qq install --no-install-recommends libcap2-bin -y && \
    setcap CAP_NET_BIND_SERVICE=ep /usr/sbin/nginx

COPY nginx.conf /etc/nginx/nginx.conf
COPY nginx.conf.template /etc/nginx/templates/

# 将所有Nginx文件移动到其他目录，后续会挂载可写目录到/etc/nginx/并将文件复制回去再启动Nginx
RUN mv /etc/nginx /etc/nginx-defaults

# 该脚本负责复制文件并通过docker-entrypoint.sh启动nginx
COPY entrypoint.sh .
RUN chmod +x ./entrypoint.sh
USER proxy-user

ENTRYPOINT ./entrypoint.sh

卷挂载配置 #

我挂载了多个可写emptyDir卷供Nginx使用：

volumeMounts:
  - name: nginx-etc
    mountPath: /etc/nginx
  - name: nginx-cache
    mountPath: /var/cache/nginx
  - name: nginx-tmp
    mountPath: /tmp/nginx
  - name: nginx-pid
    mountPath: /var/run

同时在nginx.conf中指定了临时文件路径：

http {
    proxy_temp_path /tmp/nginx/proxy_temp;
    client_body_temp_path /tmp/nginx/client_temp;
    fastcgi_temp_path /tmp/nginx/fastcgi_temp;
    uwsgi_temp_path /tmp/nginx/uwsgi_temp;
    scgi_temp_path /tmp/nginx/scgi_temp;

    ...
}

自定义entrypoint脚本 #

自定义的entrypoint脚本负责复制配置文件并启动Nginx：

#!/bin/sh

echo "Running from custom entrypoint"

# 将nginx文件复制回默认目录
cp -a /etc/nginx-defaults/. /etc/nginx/

# 通过docker entrypoint启动nginx
./docker-entrypoint.sh nginx

报错信息 #

运行时遇到端口绑定权限拒绝错误：

2023/03/31 12:31:03 [emerg] 11#11: bind() to 0.0.0.0:80 failed (13: Permission denied)
nginx: [emerg] bind() to 0.0.0.0:80 failed (13: Permission denied)

将allowPrivilegeEscalation设为true或绑定1024以上的非特权端口可以解决问题，但我想知道是否有办法在不绑定高位端口的情况下绕过特权升级限制？

补充测试 #

使用未修改的nginx镜像进行测试，同样出现绑定错误，完整Pod YAML如下：

apiVersion: v1
kind: Pod
metadata:
  name: example
spec:
  containers:
    - name: proxy
      image: nginx
      imagePullPolicy: Always
      volumeMounts:
        - name: nginx-cache
          mountPath: /var/cache/nginx
        - name: nginx-tmp
          mountPath: /tmp/nginx
        - name: nginx-pid
          mountPath: /var/run
      securityContext:
        allowPrivilegeEscalation: false
        readOnlyRootFilesystem: true
        runAsNonRoot: true
        runAsUser: 12321
        runAsGroup: 12321
        privileged: false
        capabilities:
          drop:
          - all
          add: ["NET_BIND_SERVICE"]
  volumes:
  - name: nginx-cache
    emptyDir:
      sizeLimit: 1Mi
  - name: nginx-tmp
    emptyDir:
      sizeLimit: 1Mi
  - name: nginx-pid
    emptyDir:
      sizeLimit: 1Mi

内容的提问来源于stack exchange，提问作者Jurgy