API调用指南

前提条件

1. 用户可以通过火山引擎 SDK 来使用我们提供的服务，目前有 Python, Golang 和 Java 版本。

2. 用户前往火山方舟的模型接入页面建立Endpoint，Endpoint为请求接入的入口，绑定接入方身份，限流，计费以及模型编排等信息。

3. 调用前需要获取 AccessKey ID 和 Secret Access Key（AK/SK），用于 API 请求认证和鉴权，如何获取可参考密钥管理-帮助文档。

4. 如果用户需要自己实现 client 调用代码，需要用 AK/SK 进行旁路鉴权，火山鉴权逻辑可以参考：签名方法

构造签名时使用的服务元信息如下：

region: cn-beijing
service: ml_maas
schema: https
endpoint: maas-api.ml-platform-cn-beijing.volces.com
path: /api/v2/endpoint/${endpoint_id}/chat
content-type: application/json
AccessKey ID, Secret Access Key 用你火山帐号的 AK 和 SK
action 和 version 留空

API Host

Host：maas-api.ml-platform-cn-beijing.volces.com
Region: cn-beijing

API SDK

提供统一 SDK 的接入形式（需要用 AK/SK 进行旁路鉴权，火山鉴权逻辑可以参考）

• Golang SDK: https://github.com/volcengine/volc-sdk-golang

• Python SDK: https://github.com/volcengine/volc-sdk-python

• Java SDK: https://github.com/volcengine/volc-sdk-java

Model Inference

Parameters 记录可选控制参数，具体哪些参数可用依赖具体模型的配置。

Chat

Input

[
  {
    "role": "user",
    "content": "你好"
  }
]

{
    "role": "user",
    "content": [
        {
          "type": "image_url", //object类型，目前仅支持image_url
          "image_url": {
            "url": "",
            "image_bytes": "",
            "detail": "",
          }
        }
    ]
  }

{
    "properties": {
        "query": {
            "description": "表示搜索query，如果你希望借助搜索引擎回答下面的问题，你会如何搜索",
            "type": "string"
        }
    },
    "required": [
        "query"
    ],
    "type": "object"
}

Output

{
    "message": {
        "role": "assistant",
        "content": "Learning Python can be a fun and rewarding experience, and there are many resources available to help you get started."
    },
    "finish_reason": "stop",
}

{
    "prompt_tokens": 18,
    "completion_tokens": 317,
    "total_tokens": 335
}

{
    "code_n": 1709802,
    "code": "RequestTimeout",
    "message": "请求超时"
}

Code Sample

POST /api/v2/endpoint/${endpoint_id}/chat

Curl (Non-Stream)

// Request
curl --request POST \
  --url http://maas-api.ml-platform-cn-beijing.volces.com/api/v2/endpoint/${YOUR_ENDPOINT_ID}/chat \
  --header 'Accept: application/json' \
  --header 'Content-Type: application/json' \
  --data '{
        "stream": false,
        "parameters": {
                "max_new_tokens": 1024,
                "temperature": 0.9
        },
        "messages": [
                {
                        "role": "user",
                        "content": "你好"
                },
                {
                        "role": "assistant",
                        "content": "你好，有什么可以帮助您？"
                },
                {
                        "role": "user",
                        "content": "你可以做些什么？"
                }
        ]
}'

// Response
{
        "choices": [
            {
                    "message": {
                            "role": "assistant",
                            "content": "我可以回答各种问题，例如历史、科学、技术、文化、娱乐等方面的问题。我还可以生成文本，例如摘要、文章、故事等。您需要我做什么呢？"
                    },
                    "finish_reason": "stop"
            }
        ],
        "usage": {
                "prompt_tokens": 20,
                "completion_tokens": 43,
                "total_tokens": 63
        }
}

Curl (Stream)

在 stream 模式下，基于 SSE （Server-Sent Events） 协议返回生成内容：

• 生成的内容形如data:{JSON_OF_OUTPUT_STRUCT}

data:{"choices":[{"message":{"content":"Hello!"}}]}

• 最后一个有效生成内容，可以夹带usage信息，用来进行 token 统计。

• 以data:[DONE]作为成功响应后的结束标识。

如果有error的话，不返回此标识。

// Request
curl --request POST \
  --url http://maas-api.ml-platform-cn-beijing.volces.com/api/v2/endpoint/${YOUR_ENDPOINT_ID}/chat \
  --header 'Accept: application/json' \
  --header 'Content-Type: application/json' \
  --data '{
        "stream": true,
        "parameters": {
                "max_new_tokens": 1024,
                "temperature": 0.9
        },
        "messages": [
                {
                        "role": "user",
                        "content": "你好"
                },
                {
                        "role": "assistant",
                        "content": "你好，有什么可以帮助您？"
                },
                {
                        "role": "user",
                        "content": "你可以做些什么？"
                }
        ]
}'

// Response
data:{"choices":[{"message":{"content":"我"}}]}

data:{"choices":[{"message":{"content":"可以"}}]}

data:{"choices":[{"message":{"content":"帮"}}]}

data:{"choices":[{"message":{"content":"您"}}]}

data:{"choices":[{"message":{"content":"回答"}}]}

data:{"choices":[{"message":{"content":"问题"}}]}

data:{"choices":[{"message":{"role":"assistant"},"finish_reason":"stop"}],"usage":{"prompt_tokens":20,"completion_tokens":13,"total_tokens":33}}

data:[DONE]

Tokenization

Input

"text": "天空为什么这么蓝？"

Output

"total_tokens": 5

"tokens": [
   "天空",
   "为什么",
   "这么",
   "蓝",
   "？"
]

Code Sample

POST /api/v2/endpoint/${endpoint_id}/tokenization

Curl

// Request
curl --request POST \
  --url http://maas-api.ml-platform-cn-beijing.volces.com/api/v2/endpoint/${YOUR_ENDPOINT_ID}/tokenization \
  --header 'Accept: application/json' \
  --header 'Content-Type: application/json' \
  --data '{
    "text": "天空为什么这么蓝？"
}'

// Response
{
    "total_tokens": 5,
    "tokens": [
        "天空",
        "为什么",
        "这么",
        "蓝",
        "？"
     ]
}

Classification

Input

"query": "中国的第一个经济特区是？"

"labels": [
    "北京",
    "珠海",
    "深圳",
    "厦门",
    "上海"
]

Output

"label_logprobos": {
       "上海": {
         "tokens": [ "上海" ],
         "token_logprobs": [ -8.4375 ]
       },
       "北京": {
         "tokens": [ "北京" ],
         "token_logprobs": [ -10.625 ]
       },
       "厦门": {
         "tokens": [ "厦门" ],
         "token_logprobs": [ -6.5625 ]
       },
       "深圳": {
         "tokens": [ "深圳" ],
         "token_logprobs": [ -1.8828125 ]
       },
       "珠海": {
         "tokens": [ "珠海" ],
         "token_logprobs": [ -7.9375 ]
       }
  }

Code Sample

POST /api/v2/endpoint/${endpoint_id}/classification

Curl

// Request
curl --request POST \
  --url http://maas-api.ml-platform-cn-beijing.volces.com/api/v2/endpoint/${YOUR_ENDPOINT_ID}/classification \
  --header 'Accept: application/json' \
  --header 'Content-Type: application/json' \
  --data '{
    "query": "中国的第一个经济特区是？",
    "labels": [
        "北京",
        "珠海",
        "深圳",
        "厦门",
        "上海"
    ]
}'

// Response
{
    "label": "深圳",
    "label_logprobos": {
        "北京": {
            "tokens": [
                "北京"
            ],
            "token_logprobs": [
                -12.4375
            ]
        },
        "珠海": {
            "tokens": [
                "珠海"
            ],
            "token_logprobs": [
                -10.375
            ]
        },
        "上海": {
            "tokens": [
                "上海"
            ],
            "token_logprobs": [
                -10.25
            ]
        },
        "厦门": {
            "tokens": [
                "厦门"
            ],
            "token_logprobs": [
                -8.125
            ]
        },
        "深圳": {
            "tokens": [
                "深圳"
            ],
            "token_logprobs": [
                -3.75
            ]
        }
    },
    "usage": {
        "prompt_tokens": 15,
        "completion_tokens": 5,
        "total_tokens": 20
    }
}

Embeddings

Input

"input": [
    "天很蓝",
    "海很深"
]

Output

{
    "data": [
        {
            "embedding": [
                0.01131823007017374,
                0.009313009679317474,
                ...
                -0.03317729011178017,
                0.00028884236235171556,
                -0.022333964705467224
            ],
            "object": "embedding"
        },
        {
            "embedding": [
                -0.0019542998634278774,
                ...
                -0.03738011419773102,
                -0.025939496234059334,
                -0.037807632237672806
            ],
            "index": 1,
            "object": "embedding"
        }
    ],
    "object": "list",
    "usage": {
        "prompt_tokens": 6,
        "total_tokens": 6
    }
}

Code Sample

POST /api/v2/endpoint/${endpoint_id}/embeddings

Curl

// Request
curl --request POST \
  --url http://maas-api.ml-platform-cn-beijing.volces.com/api/v2/endpoint/${YOUR_ENDPOINT_ID}/embeddings \
  --header 'Accept: application/json' \
  --header 'Content-Type: application/json' \
  --data '{
    "input": [
        "天很蓝",
        "海很深"
    ]
}'

// Response
{
    "usage": {
        "prompt_tokens": 6,
        "total_tokens": 6
    },
    "object": "list",
    "data": [
        {
            "index": 0,
            "embedding": [
                0.011312266811728477,
                0.009318970143795013,
                ...
                -0.022353211417794228
            ],
            "object": "embedding"
        },
        {
            "index": 1,
            "embedding": [
                -0.001947728800587356,
                0.015627330169081688,
                0.018870584666728973,
                ...
                -0.03777957335114479
            ],
            "object": "embedding"
        }
    ]
}