如何向Google Cloud Vertex AI端点发起异步请求？

问题描述 #

我已在Vertex AI的模型注册表中部署了模型，也添加了端点，能够进行推理。以下是我编写的Python 3.9.12代码：

from google.cloud import aiplatform
from google.oauth2 import service_account

# settings is a Pydantic BaseSettings subclass object
credentials_json = json.loads(settings.GCP_VERTEX_SERVICE_ACC)
credentials = service_account.Credentials.from_service_account_info(
    info=credentials_json
)
aiplatform.init(project=settings.GCLOUD_PROJECT_NUMBER,
                location=settings.GCLOUD_LOCATION,
                credentials=credentials)
endpoint = aiplatform.Endpoint(settings.GCLOUD_SBERT_ENDPOINT_ID)

...

async def do_inference(list_strs: List[str]):

    result = endpoint.predict(instances=list_strs)
    return result.predictions

目前我无法发起异步请求，请问有解决办法吗？比如使用aiplatform_v1beta1.PredictionServiceAsyncClient库是否可行？

最终实现代码 #

以下是成功实现异步推理的代码，供遇到同样问题的开发者参考：

import asyncio
from google.cloud import aiplatform_v1beta1
from google.oauth2 import service_account
from google.protobuf import json_format
from google.protobuf.struct_pb2 import Value

# settings is a Pydantic BaseSettings subclass object
credentials_json = json.loads(settings.GCP_VERTEX_SERVICE_ACC)
credentials = service_account.Credentials.from_service_account_info(
    info=credentials_json
)

client_options = {"api_endpoint": f"{settings.GCLOUD_LOCATION}-aiplatform.googleapis.com"}
client = aiplatform_v1beta1.PredictionServiceAsyncClient(credentials=credentials, client_options=client_options)

...

async def do_inference(list_strs: List[str]):

    request = aiplatform_v1beta1.PredictRequest(endpoint=endpoint)
    request.instances.extend(list_strs)
    response = await client.predict(request)
    predictions = response.predictions
    return predictions

asyncio.get_event_loop().run_until_complete(do_inference())

此代码的实现很大程度上归功于@milad_raesi的回答！

内容的提问来源于stack exchange，提问作者mr_faulty