If your search and recommendation content consists of video files with metadata, you can use Video Datasets for uploading and storage. A video dataset allows you to upload video files along with their metadata (such as titles, authors, release dates, and other custom fields). The system automatically invokes video pre\-processing and understanding capabilities to analyse the metadata and extract, understand, and summarise the audio, visual, and textual components of the video files. This enables the construction of a video search engine and allows intelligent assistants to provide Q&A services based on the video content. Video datasets are suitable for the following types of content: * **Film and TV Content:** Film and TV content on video platforms includes metadata (title, summary, genre, cast, etc.) and video files. The video dataset uses JSON objects to store the metadata and associated video files for a film or TV show, supporting a hierarchical series\-episode structure to represent the seasons and episodes with parent\-child relationships. * **Video Courses:** JSON objects are used to store metadata and video files for educational content. The Series\-Episode hierarchical structure can be used to represent the parent\-child relationship between a course and its sequence of individual videos. * **Short Videos:** JSON objects are used to store short video metadata (authors, dates, keywords, etc.) and the corresponding video files. The system automatically understands the short video content and supplements it with additional metadata. # Video Dataset Format Unlike Image & Text Datasets, before uploading a Video Dataset, you must prepare your media data in accordance with specific upload requirements. The unique identifier for media content, content type, video download link, unique identifier for parent content, and sequence number **must** **be uploaded using** **fixed field names**. Other metadata can be uploaded using flexible, custom fields. #### Fixed Fields To ensure your video data is processed and indexed accurately and efficiently by the system, please pay particular attention to preparing your media data according to our specific requirements before uploading. You **must** **use our pre\-defined field names ** for storage. Fields marked as "Mandatory" must be included in each data record, including `content_id` (representing the unique ID of the media data) and `content_type` (indicating whether the media data is a series object or an individual video object). |**Field Name** |**Mandatory?** |**Field Description** | |---|---|---| |content_id|Mandatory |`*String*`|\ | | |unique identifier of the media content|\ | | |**Each data record must have a unique ID.** | |content_type |Mandatory |`*String*`|\ | | |content type, supports uploading the following enumeration values:|\ | | ||\ | | |* "**collection**": Media series object (e.g., a season of a TV drama). No video file to be uploaded for "collection" data (it will be ignored if you uploaded). Only metadata for the series is required.|\ | | |* "**video**": Video object (e.g., an individual episode of a TV drama, or a movie). Video files and their corresponding metadata to be uploaded for "video" type data. | |video_url |**Mandatory** for type **"video"** ; |`*Array* `|\ | | |A list of video download links accessible to Volcano Engine, used for video understanding and processing. If multiple `video_url` links are uploaded under the same `content_id`, system will splice them into a single video according to the list order.|\ | |Must be **empty** for type **"collection"** |:::tip \*Note|\ | | |This field supports a list of publicly accessible video download links. Crawling videos from general web links is not supported.|\ | | |Before uploading, **ensure the total size of all video files in the list does not exceed 10GB and the total duration does not exceed 3 hours.** |\ | | |Ensure these video URLs are accessible to Volcengine’s official interface domain (volcengine.com).|\ | | |**The following video file formats are supported for upload:** |\ | | ||\ | | |* Common playback formats: mp4, mkv, avi, mov, wmv, asf, rmvb|\ | | |* Streaming formats: flv, f4v, ts, mpegts, m4s, webm, m3u8|\ | | ||\ | | |:::| |parent_content_id |Must be **empty** for type **"collection"** ;|`*String*`|\ | | |If your media information has a hierarchical structure (e.g., TV dramas), use this field to identify the parent content of the video. For "video" type data, you can pass the `content_id` of the series which the video belongs to. *The * `*parent_content_id*` * of the child content is the * `*content_id*` * of the parent.* |\ | |Optional for type "video"; |**Note**: Hierarchical relationships beyond 2 levels are currently not supported. | |sequence_index |Optional for type "video" |`*Integer*`|\ | | |Indicates the sequence of multiple child content within the parent content (e.g., episode numbers of a TV drama), enabling cross\-video understanding within the same media series.|\ | | |**Note**: Cross\-video understanding will not be performed if this is not provided. | #### Custom Metadata Fields In addition to the fixed fields listed above, **you may also use custom fields to upload other metadata** such as content title, video duration, video language, etc. You can assign any names to these fields, for example, by directly using existing field names from your database to simplify integration. Custom fields must follow the format requirements below: * **Field name**: must start with a letter and can only contain letters, numbers, and underscores * **Field type**: supports the following data types defined in the JSON schema * String (and Array) * Integer (Array): supports both Int64 and Int32 * Float (and Array) * Boolean * Object (and Array): supports fields containing nested JSON objects, with a maximum of three levels of object nesting. ## About Data Hierarchy To assist you in managing and distinguishing between series and individual content more clearly, we support the uploading of media data in hierarchies. It supports two layers of media relationships, defined using the `content_id` and `parent_content_id` fields. Media content at each level can have its own metadata. **Using hierarchical relationships can meet the following business requirements:** * **Sequential Video Understanding**: Video content under the same parent can undergo **cross\-video understanding**. You must specify the sequential order of the child content under the parent (using the integer field `sequence_index`). During video understanding, all child videos under the parent will be analysed to achieve content linking and contextual understanding. * **Focusing on Core Content Display**: In film and television scenarios, you may prefer that **only parent\-level content is displayed** in search and recommendation result lists (for example, finding a TV series rather than individual episodes). You can configure the video search result return strategy in the console to ensure that only parent\-level content is displayed during retrieval. The following provides detailed explanations on content hierarchy: |**Content Type** |**Type Description** |**Sample** | |---|---|---| |Series |The top level of the hierarchy; this content usually refers to a series of works, the most typical example being a TV series.|```JSON|\ | | |{|\ | |Its structural features are as follows:| "content_id": "tv_stranger_things_s01",|\ | |`*content_id*`: unique content identifier, used as the `*parent_content_id*` of its child content.| "content_type":"collection",|\ | |`*content_type*`："collection"| "video_url": null,|\ | |`*video_url*`: must be empty| "parent_content_id": null,|\ | |`*parent_content_id*`: must be empty| "sequence_index": null,|\ | |`*sequence_index*`: must be empty | "title": "Stranger Things (Season 1)",|\ | | | "director": "The Duffer Brothers",|\ | | | "actors": ["Winona Ryder", "David Harbour", "Finn Wolfhard"],|\ | | | "first_air_date": "2016-07-15",|\ | | | "douban_rating": 9.3,|\ | | | "genres": ["Sci-Fi", "Horror", "Drama"],|\ | | | "plot_summary": "When a young boy vanishes, a small town uncovers a mystery involving secret experiments, terrifying supernatural forces, and one strange little girl.", |\ | | | "actor_list": [|\ | | | {"actor_name": "Winona Ryder", "role_name": "Joyce Byers"}, |\ | | | {"actor_name": "David Harbour", "role_name": "Jim Hopper"}, |\ | | | {"actor_name": "Finn Wolfhard", "role_name": "Mike Wheeler"}|\ | | | ],|\ | | | "language": "en-GB",|\ | | | "media_link": "https://example.com/tv_series/67890/"|\ | | |}|\ | | |```|\ | | | | |Video in the series |The second level in the hierarchy, representing a part of the series, such as an episode of a TV show. |```JSON|\ | |Each sub\-content must belong to one parent content.|{|\ | |its structural features are as follows:| "content_id": "tv_stranger_things_s01_e01",|\ | |`*content_id*`: unique identifier for the content| "content_type":"video",|\ | |`*content_type*`："video"| "video_url": ["https://example.com/tv_series/67890/episode_01.mp4"],|\ | |`*video_url*`: required, video file URL| "parent_content_id": "tv_stranger_things_s01",|\ | |`*parent_content_id*` *: * the `*content_id*` of the parent content| "sequence_index": 1,|\ | |`*sequence_index*`: indicates the order of multiple sub\-contents within the parent content | "title": "Chapter One: The Vanishing of Will Byers",|\ | | | "director": "The Duffer Brothers",|\ | | | "actors": ["Winona Ryder", "David Harbour", "Finn Wolfhard"],|\ | | | "first_air_date": "2016-07-15",|\ | | | "douban_rating": null,|\ | | | "genres": ["Sci-Fi", "Horror", "Drama"],|\ | | | "episode_summary": "On his way home from a friend's house, young Will sees something terrifying. Nearby, a sinister secret lurks in the depths of a government lab.", |\ | | | "actor_list": [|\ | | | {"actor_name": "Winona Ryder", "role_name": "Joyce Byers"}, |\ | | | {"actor_name": "David Harbour", "role_name": "Jim Hopper"}, |\ | | | {"actor_name": "Finn Wolfhard", "role_name": "Mike Wheeler"}|\ | | | ],|\ | | | "duration": 2940,|\ | | | "language": "en-GB",|\ | | | "media_link": "https://example.com/tv_series/67890/episode_1/"|\ | | |}|\ | | |```|\ | | | | |Independent video content |Content without hierarchical relationships, such as a movie.|```JSON|\ | |Its structural features are as follows:|{|\ | |`*content_id*`: unique content identifier| "content_id": "movie_12345",|\ | |`*content_type*`："video"| "content_type":"video",|\ | |`*video_url*`: required, video file URL| "video_url": ["https://example.com/movies/12345.mp4"],|\ | |`*parent_content_id*`: must be empty| "parent_content_id": null,|\ | |`*sequence_index*`: must be empty | "sequence_index": null,|\ | | | "title": "Interstellar",|\ | | | "director": "Christopher Nolan",|\ | | | "actors": ["Matthew McConaughey", "Anne Hathaway", "Jessica Chastain"],|\ | | | "release_date": "2014-11-07",|\ | | | "douban_rating": 8.7,|\ | | | "genres": ["Sci-Fi", "Drama", "Adventure"],|\ | | | "plot_summary": "In a future where Earth is becoming uninhabitable, a team of ex-pilots and scientists travel through a wormhole in search of a new home for mankind, exploring the relationship between time, gravity, and love across the cosmos.",|\ | | | "actor_list": [|\ | | | {"actor_name": "Matthew McConaughey", "role_name": "Cooper"}, |\ | | | {"actor_name": "Anne Hathaway", "role_name": "Brand"}, |\ | | | {"actor_name": "Jessica Chastain", "role_name": "Murph"}|\ | | | ],|\ | | | "duration": 10140,|\ | | | "language": "en-GB",|\ | | | "media_link": "https://example.com/movies/12345/details"|\ | | |}|\ | | |```|\ | | | | Using the TV series "*Stranger Things (Season 1)* ", episode 1 of "*Stranger Things (Season 1)* ", and the movie "Interstellar" as examples: |**Content Sample** |**Expected ** **`content_type`** |**Expected ** **`video_url`** |**Expected`parent_content_id`** | |---|---|---|---| |TV series "*Stranger Things (Season 1)* " |"collection" |must be empty |must be empty | |"*Stranger Things (Season 1)* " episode 1 |"video" |required, upload the video file link of episode 1 of "*Stranger Things (Season 1)* " in the form of ["{video_download_url}"] |required, upload the content_id of "*Stranger Things (Season 1)* " | |Movie "Interstellar" |"video" |Upload the video file link of "Interstellar" in the form of ["{video_download_url}"] if required |Must be empty | # Review Reserved Field Attributes The "attributes" of a field are a set of reserved field configurations which you will be guided to set up during dataset creation. Fields configured with specific attributes will undergo specific processing and will be used in content understanding and ranking strategies during search, recommendation and conversation. When preparing your data, you can pre\-determine whether your database contains the following information. We recommend passing these attributes in the specified format to maximise utility of Viking AI Search's intelligent data processing and online strategies. The video dataset has the following reserved attributes: |**Attribute** |**Mandatory?** |**Supported Field Type(s)** |**Attribute Description** | |---|---|---|---| |content identifier |**Required & Fixed** |`*String*` |corresponding to the `*content_id*` field | |video download URL |**Conditional & Fixed** |`*Array* ` |corresponding to the `*video_url*` field | |parent content identifier |**Conditional & Fixed** |`*String*` |corresponding to the `*parent_content_id*` field | |content index |**Conditional & Fixed** |`*Integer*` |corresponding to the `*sequence_index*` field | |content title|Optional (strongly recommended to upload) |`*String*` |media content title, **strongly recommended to upload.** |\ | | | |only one field per data can be used to store the title attribute. | |media cover image url |Optional |`*String*` |media content cover image download link | |media page link |Optional |`*String*` |media content redirect link, for search list preview only | |video duration (seconds) |Optional |`*Int64*` |video length (in seconds) | |video language (BCP 47)|Optional |`*Array* ` |language tags compliant with BCP 47|\ | | | |for example, Chinese: zh; English: en | |publish time (Date/Time Format) |Optional |`*String*` |stores the publish or listing time of the item in date\-time format, used for sorting by newness in search and recommendation. |\ | | | |supports formates of:|\ | | | ||\ | | | |* Standard ISO date\-time strings (`YYYY-MM-DDThh:mm:ss+08:00`).|\ | | | |* Non\-standard date\-time strings (`YYYY-MM-DD hh:mm:ss` or date\-only strings). The system will automatically convert these to standard format. | |publish time (UNIX Timestamp, Seconds) |Optional |`*Int64*` |item publish time represented by a standard 10\-digit UNIX timestamp, used for freshness ranking in search and recommendation. | |publish time (UNIX Timestamp, Milliseconds) |Optional |`*Int64*` |item listing time represented by a standard 13\-digit UNIX timestamp, used for freshness ranking in search and recommendation. | # Sample Data The following is an example of a video data entry (a TV episode): ```JSON { "content_id": "tv_stranger_things_s01_e01", "content_type":"video", "video_url": ["https://example.com/tv_series/67890/episode_01.mp4"], "parent_content_id": "tv_stranger_things_s01", "sequence_index": 1, "title": "Chapter One: The Vanishing of Will Byers", "director": "The Duffer Brothers", "actors": ["Winona Ryder", "David Harbour", "Finn Wolfhard"], "first_air_date": "2016-07-15", "rating": null, "genres": ["Sci-Fi", "Horror", "Drama"], "episode_summary": "On his way home from a friend's house, young Will sees something terrifying. Nearby, a sinister secret lurks in the depths of a government lab.", "actor_list": [ {"actor_name": "Winona Ryder", "role_name": "Joyce Byers"}, {"actor_name": "David Harbour", "role_name": "Jim Hopper"}, {"actor_name": "Finn Wolfhard", "role_name": "Mike Wheeler"} ], "duration": 2940, "language": "en-GB", "media_link": "https://example.com/tv_series/67890/episode_1/" } ``` |**Sample Field** |**Field Type** |**Field Attribute** |**Field Meaning** | |---|---|---|---| |content_id (fixed field) |`*String*` |content identifier |unique identifier of media content | |video_url (fixed field) |*Array* |video download URL |download link of the video under media content | |parent_content_id (fixed field) |*String* |parent content identifier |unique identifier of the parent content to which the media content belongs | |sequence_index (fixed field) |*Integer* |content sequence number |order information of media content within the parent content | |title |*String* |content title |display title of media content | |director |*String* | |director of the media content | |actors |*Array* | |cast list of media content | |first_air_date |*String* |release date (date\-time format) |release date of media content | |rating |*Float* | |rating of media content | |genres |*Array* | |type of media content | |episode_summary |*String* | |summary of media content | |actor_list |Array | |cast and character list of media content | |* actor_list.actor_name |*String* | |actor name | |* actors_list.role_name |*String* | |role played by the actor | |duration |*Int64* |video duration (seconds) |length of media content | |language |*String* |video language (BCP 47) |media content language | |media_link |*String* |media page link |media content redirect link |