Airflow环境下gcsfuse挂载GCS Bucket失败求助

环境信息 #

• Cloud Composer 2，初始版本为composer-2.1.14-airflow-2.5.1，后升级至composer-2.4.2-airflow-2.5.3
• 需求：将GCS Bucket挂载到Airflow环境，实现对Bucket的读写操作

已执行操作 #

通过Airflow DAG运行的bash脚本 #

#!/bin/bash

BUCKET="my-bucket"
MOUNT_DIR="/home/airflow/gcs/data/my-bucket"

# 创建挂载目录并设置权限
mkdir -p $MOUNT_DIR
sudo chmod g+w $MOUNT_DIR

# 挂载GCS Bucket
gcsfuse --foreground --debug_fuse --debug_fs --debug_gcs --debug_http -o nonempty $BUCKET $MOUNT_DIR

额外验证操作 #

• 执行gsutil ls gs://$BUCKET，可正常列出Bucket内文件，确认Airflow具备Bucket访问权限
• 尝试sudo mount -t gcsfuse -o rw,user $BUCKET $MOUNT_DIR，仍触发相同错误

错误日志 #

初始版本环境日志 #

[2023-09-20, 11:46:39 PDT] {subprocess.py:93} INFO - Start gcsfuse/0.42.3 (Go version go1.19.5) for app "" using mount point: /home/airflow/gcs/data/my-bucket
[2023-09-20, 11:46:39 PDT] {subprocess.py:93} INFO - Opening GCS connection...
[2023-09-20, 11:46:39 PDT] {subprocess.py:93} INFO - Creating a mount at "/home/airflow/gcs/data/my-bucket"
[2023-09-20, 11:46:39 PDT] {subprocess.py:93} INFO - Creating a new server...
[2023-09-20, 11:46:39 PDT] {subprocess.py:93} INFO - Set up root directory for bucket my-bucket
[2023-09-20, 11:46:39 PDT] {subprocess.py:93} INFO - gcs: Req              0x0: <- ListObjects("")
[2023-09-20, 11:46:39 PDT] {subprocess.py:93} INFO - gcs: Req              0x0: -> ListObjects("") (131.395831ms): OK
[2023-09-20, 11:46:39 PDT] {subprocess.py:93} INFO - Mounting file system "my-bucket"...
[2023-09-20, 11:46:39 PDT] {subprocess.py:93} INFO - fuse_debug: Beginning the mounting kickoff process
[2023-09-20, 11:46:39 PDT] {subprocess.py:93} INFO - fuse_debug: Parsing fuse file descriptor
[2023-09-20, 11:46:39 PDT] {subprocess.py:93} INFO - fuse_debug: Preparing for direct mounting
[2023-09-20, 11:46:39 PDT] {subprocess.py:93} INFO - fuse_debug: Directmount failed. Trying fallback.
[2023-09-20, 11:46:39 PDT] {subprocess.py:93} INFO - fuse_debug: Creating a socket pair
[2023-09-20, 11:46:39 PDT] {subprocess.py:93} INFO - fuse_debug: Creating files to wrap the sockets
[2023-09-20, 11:46:39 PDT] {subprocess.py:93} INFO - fuse_debug: Starting fusermount/os mount
[2023-09-20, 11:46:39 PDT] {subprocess.py:93} INFO - /usr/bin/fusermount: fuse device not found, try 'modprobe fuse' first
[2023-09-20, 11:46:39 PDT] {subprocess.py:93} INFO - Error while mounting gcsfuse: mountWithConn: Mount: mount: running /usr/bin/fusermount: exit status 1
[2023-09-20, 11:46:39 PDT] {subprocess.py:93} INFO - mountWithArgs: mountWithConn: Mount: mount: running /usr/bin/fusermount: exit status 1

升级版本后环境日志 #

[2023-09-20, 16:55:38 PDT] {subprocess.py:93} INFO - {"name":"root","levelname":"INFO","severity":"INFO","message":"Start gcsfuse/1.0.1 (Go version go1.20.5) for app \"\" using mount point:/home/airflow/gcs/data/my-bucket\n","timestampSeconds":1695254138,"timestampNanos":83062812}
[2023-09-20, 16:55:38 PDT] {subprocess.py:93} INFO - {"name":"root","levelname":"INFO","severity":"INFO","message":"Opening GCS connection...\n","timestampSeconds":1695254138,"timestampNanos":83799366}
[2023-09-20, 16:55:38 PDT] {subprocess.py:93} INFO - {"name":"root","levelname":"INFO","severity":"INFO","message":"Creating a mount at \"/home/airflow/gcs/data/datavant/my-bucket\"\n","timestampSeconds":1695254138,"timestampNanos":87562370}
[2023-09-20, 16:55:38 PDT] {subprocess.py:93} INFO - {"name":"root","levelname":"INFO","severity":"INFO","message":"Creating a new server...\n","timestampSeconds":1695254138,"timestampNanos":87589651}
[2023-09-20, 16:55:38 PDT] {subprocess.py:93} INFO - {"name":"root","levelname":"INFO","severity":"INFO","message":"Set up root directory for bucket my-bucket\n","timestampSeconds":1695254138,"timestampNanos":87599362}
[2023-09-20, 16:55:38 PDT] {subprocess.py:93} INFO - {"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"gcs: Req              0x0: <- ListObjects(\"\")\n","timestampSeconds":1695254138,"timestampNanos":87612220}
[2023-09-20, 16:55:38 PDT] {subprocess.py:93} INFO - {"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"gcs: Req              0x0: -> ListObjects(\"\") (106.665835ms): OK\n","timestampSeconds":1695254138,"timestampNanos":194287578}
[2023-09-20, 16:55:38 PDT] {subprocess.py:93} INFO - {"name":"root","levelname":"INFO","severity":"INFO","message":"Mounting file system \"my-bucket\"...\n","timestampSeconds":1695254138,"timestampNanos":194342795}
[2023-09-20, 16:55:38 PDT] {subprocess.py:93} INFO - {"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Beginning the mounting kickoff process\n","timestampSeconds":1695254138,"timestampNanos":194916407}
[2023-09-20, 16:55:38 PDT] {subprocess.py:93} INFO - {"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Parsing fuse file descriptor\n","timestampSeconds":1695254138,"timestampNanos":194977401}
[2023-09-20, 16:55:38 PDT] {subprocess.py:93} INFO - {"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Preparing for direct mounting\n","timestampSeconds":1695254138,"timestampNanos":194984093}
[2023-09-20, 16:55:38 PDT] {subprocess.py:93} INFO - {"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Directmount failed. Trying fallback.\n","timestampSeconds":1695254138,"timestampNanos":195003380}
[2023-09-20, 16:55:38 PDT] {subprocess.py:93} INFO - {"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Creating a socket pair\n","timestampSeconds":1695254138,"timestampNanos":195238613}
[2023-09-20, 16:55:38 PDT] {subprocess.py:93} INFO - {"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Creating files to wrap the sockets\n","timestampSeconds":1695254138,"timestampNanos":195260643}
[2023-09-20, 16:55:38 PDT] {subprocess.py:93} INFO - {"name":"root","levelname":"DEBUG","severity":"DEBUG","message":"fuse_debug: Starting fusermount/os mount\n","timestampSeconds":1695254138,"timestampNanos":195270306}
[2023-09-20, 16:55:38 PDT] {subprocess.py:93} INFO - /usr/bin/fusermount: fuse device not found, try 'modprobe fuse' first
[2023-09-20, 16:55:38 PDT] {subprocess.py:93} INFO - {"name":"root","levelname":"INFO","severity":"INFO","message":"Error while mounting gcsfuse: mountWithConn: Mount: mount: running /usr/bin/fusermount: exit status 1\n","timestampSeconds":1695254138,"timestampNanos":198067902}
[2023-09-20, 16:55:38 PDT] {subprocess.py:93} INFO - mountWithArgs: mountWithConn: Mount: mount: running /usr/bin/fusermount: exit status 1

核心问题 #

无论是否升级Composer版本，挂载操作均触发相同错误：/usr/bin/fusermount: fuse device not found, try 'modprobe fuse' first，导致挂载失败。

内容的提问来源于stack exchange，提问作者Virendra