缓存大文本语料时遇Gemini Status 500错误，求排查方案

尝试缓存约180万token的大文本语料时，触发了Status 500内部错误。以下是相关信息：

脚本代码 #

import vertexai
from vertexai.preview import caching 
from vertexai.generative_models import Part
from google.oauth2.service_account import Credentials
import logging
import datetime

def create_new_gemini_cache(display_name, content, time_to_live):
    try:
        cached_content = caching.CachedContent.create(
            model_name='gemini-1.5-pro-001',
            contents=content,
            ttl=time_to_live
        )
        logging.info(f'#### Content cached')
        return cached_content

    except Exception as e:  
        handle_exception(e) # 假设handle_exception是已实现的异常处理函数

# 获取凭证
gemini_cred_json = {} # 实际服务账号JSON内容
gcp_credentials = Credentials.from_service_account_info(gemini_cred_json, scopes=['https://www.googleapis.com/auth/cloud-platform'])

# 初始化Vertex AI项目
GCP_PROJECT_ID = "你的项目ID"
GCP_REGION = "你的区域"
vertexai.init(project=GCP_PROJECT_ID, location=GCP_REGION, credentials=gcp_credentials)  

# 准备待缓存内容（约1.8M token的文本）
content = [Part.from_text("<1.8M token text>")] 

# 设置缓存有效期
time_to_live = datetime.timedelta(minutes=60)

# 请求创建缓存
cache_content = create_new_gemini_cache("测试缓存", content, time_to_live)

错误信息 #

#### Error details: { "message": "500 Internal error encountered.", "status_code": null, "traceback": [ "Traceback (most recent call last):", " File \"/home/site/wwwroot/.python_packages/lib/site-packages/google/api_core/grpc_helpers.py\", line 76, in error_remapped_callable", " return callable_(*args, **kwargs)", " ^^^^^^^^^^^^^^^^^^^^^^^^^^", " File \"/home/site/wwwroot/.python_packages/lib/site-packages/grpc/_channel.py\", line 1181, in __call__", " return _end_unary_response_blocking(state, call, False, None)", " ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^", " File \"/home/site/wwwroot/.python_packages/lib/site-packages/grpc/_channel.py\", line 1006, in _end_unary_response_blocking", " raise _InactiveRpcError(state) # pytype: disable=not-instantiable", " ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^", "grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:", "\tstatus = StatusCode.INTERNAL", "\tdetails = \"Internal error encountered.\"", "\tdebug_error_string = \"UNKNOWN:Error received from peer ipv4:142.251.33.106:443 {grpc_message:\"Internal error encountered.\", grpc_status:13, created_time:\"2024-09-05T03:51:37.113071157+00:00\"}\"", "\">", "", "The above exception was the direct cause of the following exception:", "", "Traceback (most recent call last):", " File \"/home/site/wwwroot/shared/gemini_utils.py\", line 94, in create_new_gemini_cache", " cached_content = caching.CachedContent.create(", " ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^", " File \"/home/site/wwwroot/.python_packages/lib/site-packages/vertexai/caching/_caching.py\", line 228, in create", " cached_content_resource = client.create_cached_content(request)", " ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^", " File \"/home/site/wwwroot/.python_packages/lib/site-packages/google/cloud/aiplatform_v1beta1/services/gen_ai_cache_service/client.py\", line 825, in create_cached_content", " response = rpc(", " ^^^^", " File \"/home/site/wwwroot/.python_packages/lib/site-packages/google/api_core/gapic_v1/method.py\", line 131, in __call__", " return wrapped_func(*args, **kwargs)", " ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^", " File \"/home/site/wwwroot/.python_packages/lib/site-packages/google/api_core/grpc_helpers.py\", line 78, in error_remapped_callable", " raise exceptions.from_grpc_error(exc) from exc", "google.api_core.exceptions.InternalServerError: 500 Internal error encountered." ] }

想确认是请求配置有误还是服务故障？后续该如何排查？

补充测试信息 #

编辑1 #

在Google Colab运行官方上下文缓存示例，仍触发Status 500错误。已测试多种环境（本地、Azure Function、Google Colab）、不同文本长度（远低于缓存下限、达标、超100万token等），均出现相同的InternalServerError: 500 Internal error encountered.错误，错误栈如下：

_InactiveRpcError                         Traceback (most recent call last)

/usr/local/lib/python3.10/dist-packages/google/api_core/grpc_helpers.py in error_remapped_callable(*args, **kwargs)
     75         try:
---> 76             return callable_(*args, **kwargs)
     77         except grpc.RpcError as exc:

6 frames

_InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
    status = StatusCode.INTERNAL
    details = "Internal error encountered."
    debug_error_string = "UNKNOWN:Error received from peer ipv4:172.217.7.42:443 {created_time:"2024-09-05T16:06:16.178297904+00:00", grpc_status:13, grpc_message:"Internal error encountered."}"
>


The above exception was the direct cause of the following exception:

InternalServerError                       Traceback (most recent call last)

/usr/local/lib/python3.10/dist-packages/google/api_core/grpc_helpers.py in error_remapped_callable(*args, **kwargs)
     76             return callable_(*args, **kwargs)
     77         except grpc.RpcError as exc:
---&gt; 78             raise exceptions.from_grpc_error(exc) from exc
     79 
     80     return error_remapped_callable

InternalServerError: 500 Internal error encountered.

编辑2 #

截至目前，已尝试所有以下方案，但均失败：

• 不同运行环境：Google Colab、Azure Function、本地机器
• 不同上下文长度：100万token、180万token文本，截取前10万字符，仅用测试文本"this is a test"
• 内容格式：带/不带[]包裹
• 不同模型：gemini-1.5-flash-001和gemini-1.5-pro-001
• 不同版本库：GenerativeModels的preview版与正式版
• 不同文本类型：自有文本、公共项目文本
• 文本来源：Azure Blob Storage、Google存储桶、本地文件
• 文本预处理：原始文本、清理后文本（移除非ASCII字符、换行符等）

请问是否有人能确认Vertex AI的上下文缓存功能当前是否可以正常使用？

内容的提问来源于stack exchange，提问作者SeaDude