GCP日志未捕获Vertex Workbench执行BQ查询的用户信息咨询

现状说明 #

这种情况是预期行为。因为Vertex Workbench默认使用底层Compute实例的服务账号发起BigQuery请求，而非用户本人的身份，所以BigQuery的审计日志中只会记录该服务账号（即日志中principalEmail显示的123456789-compute@developer.gserviceaccount.com），不会自动关联实例所有者、Workbench名称或实例的自定义标签。

解决方案 #

1. 使用用户身份发起查询 #

在Vertex Workbench中，改用用户自身的身份认证提交BigQuery请求，而非依赖实例的默认服务账号：

• 如果是在Notebook中执行代码，可通过google-auth库加载用户OAuth2凭据；如果是终端操作，执行gcloud auth login切换到用户身份。这样BigQuery日志中的principalEmail会显示为用户的邮箱，直接标识查询发起人。

2. 给BigQuery作业添加自定义标签 #

提交查询时，显式将实例名称、所有者信息等作为自定义标签附加到BigQuery作业中，这些标签会被日志捕获：
示例Python代码：

from google.cloud import bigquery

client = bigquery.Client()
# 配置作业标签，填入你的实例信息
job_config = bigquery.QueryJobConfig(
    labels={
        "workbench_instance": "你的VM实例名称",
        "instance_owner": "用户邮箱@example.com"
    }
)
query = "SELECT * FROM `steady-tracer-12345.test_dataset.top_words` LIMIT 1000;"
query_job = client.query(query, job_config=job_config)

之后在BigQuery审计日志的serviceData.job.jobConfiguration.labels字段中就能看到这些自定义信息。

3. 通过Cloud Logging关联日志 #

利用BigQuery日志中的callerIp字段，关联到对应的Compute实例日志，从而获取实例的标签、所有者信息：

• 在Cloud Logging中编写查询，先定位目标BigQuery查询日志的callerIp，再用该IP过滤Compute实例的日志，提取实例的标签和所有者属性。

为什么实例自定义标签未被捕获？ #

Compute实例的自定义标签属于Compute资源的属性，BigQuery的审计日志不会自动继承这些标签，必须通过上述显式传递或日志关联的方式才能获取。

用户提供的日志示例 #

{"protoPayload": {"@type": "type.googleapis.com/google.cloud.audit.AuditLog","status": {},"authenticationInfo": {"principalEmail": "123456789-compute@developer.gserviceaccount.com","serviceAccountDelegationInfo": [{"firstPartyPrincipal": {"principalEmail": "service-123456789@compute-system.iam.gserviceaccount.com"}}]},"requestMetadata": {"callerIp": "*.*.*.*","callerSuppliedUserAgent": "ipython-8.21.0 gl-python/3.10.14 grpc/1.62.1 gax/1.34.1 gapic/3.20.1 gccl/3.20.1,gzip(gfe)","callerNetwork": "//compute.googleapis.com/projects/steady-tracer-12345/global/networks/__unknown__","requestAttributes": {},"destinationAttributes": {}},"serviceName": "bigquery.googleapis.com","methodName": "jobservice.getqueryresults","authorizationInfo": [{"resource": "projects/steady-tracer-12345","permission": "bigquery.jobs.create","granted": true,"resourceAttributes": {}},{"resource": "projects/steady-tracer-12345","permission": "bigquery.jobs.create","granted": true,"resourceAttributes": {}}],"resourceName": "projects/steady-tracer-12345/queries/5c37d6f7-919a-4d0c-9fe0-6ae21f25dbd5","serviceData": {"@type": "type.googleapis.com/google.cloud.bigquery.logging.v1.AuditData","jobGetQueryResultsRequest": {},"jobGetQueryResultsResponse": {"totalResults": "42","job": {"jobName": {"projectId": "steady-tracer-12345","jobId": "5c37d6f7-919a-4d0c-9fe0-6ae21f25dbd5","location": "US"},"jobConfiguration": {"query": {"query": "SELECT * FROM `steady-tracer-12345.test_dataset.top_words` LIMIT 1000;","destinationTable": {"projectId": "steady-tracer-12345","datasetId": "_a3ec690a6c8f1d9369fdee628f0f7aba9bd7ce24","tableId": "anonc3a5f202530cb1c670058090a9e19fd45c74fd2de785f7e562bc5a2b37fa0641"},"createDisposition": "CREATE_IF_NEEDED","writeDisposition": "WRITE_TRUNCATE","defaultDataset": {},"queryPriority": "QUERY_INTERACTIVE","statementType": "SELECT"}},"jobStatus": {"state": "DONE","error": {}},"jobStatistics": {"createTime": "2024-05-05T19:10:14.011Z","startTime": "2024-05-05T19:10:14.161Z","endTime": "2024-05-05T19:10:14.243Z","reservation": "unreserved"}}}}},"insertId": "-vc7iqse2w6df","resource": {"type": "bigquery_resource","labels": {"project_id": "steady-tracer-12345"}},"timestamp": "2024-05-05T19:10:14.465276Z","severity": "INFO","logName": "projects/steady-tracer-12345/logs/cloudaudit.googleapis.com%2Fdata_access","receiveTimestamp": "2024-05-05T19:10:14.816987674Z"}

内容的提问来源于stack exchange，提问作者saikrishnakudelli