Cloud Run Job及自定义包日志Severity异常与重复日志问题求助

我们的尝试 #

我们尝试运行一个Cloud Run Job执行计算任务，该任务会调用自定义包完成计算。此Cloud Run Job使用google-cloud-logging和Python默认logging包，配置方式参考官方文档。自定义Python包也会输出日志（仅按Python日志最佳实践定义logger）。

初始代码示例 #

from google.cloud import logging as gcp_logging
import logging
import os
import google.auth
from our_package import do_something

def log_test_function():
    SCOPES = ["https://www.googleapis.com/auth/cloud-platform"]
    credentials, project_id = google.auth.default(scopes=SCOPES)
    try:
        function_logger_name = os.getenv("FUNCTION_LOGGER_NAME")

        logging_client = gcp_logging.Client(credentials=credentials, project=project_id)
        logging.basicConfig()
        logger = logging.getLogger(function_logger_name)
        logger.setLevel(logging.INFO)
        logging_client.setup_logging(log_level=logging.INFO)

        logger.critical("Critical Log TEST")
        logger.error("Error Log TEST")
        logger.info("Info Log TEST")
        logger.debug("Debug Log TEST")

        result = do_something()
        logger.info(result)
    except Exception as e:
        print(e)    # just to test how print works

    return "Returned"

if __name__ == "__main__":
    result = log_test_function()
    print(result)

Cloud Run Job日志情况 #

蓝色框为自定义包输出的日志
黑色框为Cloud Run Job自身输出的日志

Cloud Logging无法识别日志的Severity，所有日志条目均被解析为默认级别。

Cloud Function日志情况 #

相同代码在Cloud Function中运行时，日志Severity可被正确识别（包括Cloud Function自身和自定义包的日志）。

核心疑问 #

二者均为无服务器架构，为何Cloud Function可正常工作而Cloud Run不行？

需求 #

我们希望Cloud Run Job及自定义包的所有日志均可带正确Severity输出至Cloud Logging。

更新1 #

参考Google Cloud Python库维护者的解决方案，问题接近解决。修改后的代码如下：

from google.cloud import logging as gcp_logging
import logging
import os
import google.auth
from our_package import do_something
from google.cloud.logging.handlers import CloudLoggingHandler
from google.cloud.logging_v2.handlers import setup_logging
from google.cloud.logging_v2.resource import Resource
from google.cloud.logging_v2.handlers._monitored_resources import retrieve_metadata_server, _REGION_ID, _PROJECT_NAME

def log_test_function():
    SCOPES = ["https://www.googleapis.com/auth/cloud-platform"]
    region = retrieve_metadata_server(_REGION_ID)
    project = retrieve_metadata_server(_PROJECT_NAME)
    try:
        function_logger_name = os.getenv("FUNCTION_LOGGER_NAME")

        # build a manual resource object
        cr_job_resource = Resource(
            type="cloud_run_job",
            labels={
                "job_name": os.environ.get('CLOUD_RUN_JOB', 'unknownJobId'),
                "location": region.split("/")[-1] if region else "",
                "project_id": project
            }
        )

        logging_client = gcp_logging.Client()
        gcloud_logging_handler = CloudLoggingHandler(logging_client, resource=cr_job_resource)
        setup_logging(gcloud_logging_handler, log_level=logging.INFO)

        logging.basicConfig()
        logger = logging.getLogger(function_logger_name)
        logger.setLevel(logging.INFO)

        logger.critical("Critical Log TEST")
        logger.error("Error Log TEST")
        logger.warning("Warning Log TEST")
        logger.info("Info Log TEST")
        logger.debug("Debug Log TEST")

        result = do_something()
        logger.info(result)
    except Exception as e:
        print(e)    # just to test how print works

    return "Returned"

if __name__ == "__main__":
    result = log_test_function()
    print(result)

现在每条日志会被重复输出：一条带正确Severity，另一条为默认级别，如下方截图所示。

内容的提问来源于stack exchange，提问作者Devesh Poojari