使用AWS Glue镜像glue_libs_4.0.0_image_01时遇权限拒绝错误求助

问题根源 #

绑定挂载的./test和./libs目录在容器内继承了宿主机的用户权限，而容器默认以glue_user运行，该用户对挂载目录没有写入权限；同时在/home/glue_user/workspace下创建deps目录时，也可能因glue_user对该目录的权限不足导致失败。

解决方案 #

1. 匹配宿主机用户UID/GID运行容器 #

在docker run命令中添加--user $(id -u):$(id -g)参数，让容器内进程使用与宿主机当前用户相同的UID和GID，匹配挂载目录的权限：

docker run \
    --user $(id -u):$(id -g) \
    --mount=type=bind,source=./test,target=/home/glue_user/workspace/test \
    --mount=type=bind,source=./libs,target=/home/glue_user/workspace/libs \
    -w /home/glue_user/workspace \
    -e DISABLE_SSL=true \
    -e "PYTHONPATH=$PYTHONPATH:/home/glue_user/workspace/deps" \
    --rm -p 4040:4040 \
    -p 18080:18080 \
    --name glue_unit_tests docker-default-virtual.${{ vars.ARTIFACTORY_HOST }}/amazon/aws-glue-libs:glue_libs_4.0.0_image_01 \
    -c "mkdir -p deps/ && pip install -r test/requirements.txt -r libs/requirements.txt -t deps/; cd test && pytest || exit 1"

注意：CI流水线中运行时，需确保环境支持获取当前用户UID/GID，或提前配置对应变量。

2. 切换到root用户执行操作（不推荐生产环境） #

以root用户启动容器，完成依赖安装后将权限转回glue_user：

docker run \
    --user root \
    --mount=type=bind,source=./test,target=/home/glue_user/workspace/test \
    --mount=type=bind,source=./libs,target=/home/glue_user/workspace/libs \
    -w /home/glue_user/workspace \
    -e DISABLE_SSL=true \
    -e "PYTHONPATH=$PYTHONPATH:/home/glue_user/workspace/deps" \
    --rm -p 4040:4040 \
    -p 18080:18080 \
    --name glue_unit_tests docker-default-virtual.${{ vars.ARTIFACTORY_HOST }}/amazon/aws-glue-libs:glue_libs_4.0.0_image_01 \
    -c "mkdir -p deps/ && pip install -r test/requirements.txt -r libs/requirements.txt -t deps/; chown -R glue_user:glue_user /home/glue_user/workspace/deps; cd test && su glue_user -c 'pytest' || exit 1"

3. 修改pytest缓存目录到非挂载路径 #

指定--cache-dir参数将缓存写入容器内的临时路径，避开挂载的test目录：

docker run \
    --mount=type=bind,source=./test,target=/home/glue_user/workspace/test \
    --mount=type=bind,source=./libs,target=/home/glue_user/workspace/libs \
    -w /home/glue_user/workspace \
    -e DISABLE_SSL=true \
    -e "PYTHONPATH=$PYTHONPATH:/home/glue_user/workspace/deps" \
    --rm -p 4040:4040 \
    -p 18080:18080 \
    --name glue_unit_tests docker-default-virtual.${{ vars.ARTIFACTORY_HOST }}/amazon/aws-glue-libs:glue_libs_4.0.0_image_01 \
    -c "sudo mkdir -p deps/ && sudo chown glue_user:glue_user deps/ && pip install -r test/requirements.txt -r libs/requirements.txt -t deps/; cd test && pytest --cache-dir=/tmp/.pytest_cache || exit 1"

4. 使用虚拟环境替代直接安装到deps目录 #

创建虚拟环境隔离依赖，无需手动设置PYTHONPATH：

docker run \
    --mount=type=bind,source=./test,target=/home/glue_user/workspace/test \
    --mount=type=bind,source=./libs,target=/home/glue_user/workspace/libs \
    -w /home/glue_user/workspace \
    -e DISABLE_SSL=true \
    --rm -p 4040:4040 \
    -p 18080:18080 \
    --name glue_unit_tests docker-default-virtual.${{ vars.ARTIFACTORY_HOST }}/amazon/aws-glue-libs:glue_libs_4.0.0_image_01 \
    -c "python -m venv venv && source venv/bin/activate && pip install -r test/requirements.txt -r libs/requirements.txt pytest; cd test && pytest || exit 1"

内容的提问来源于stack exchange，提问作者Iulian