使用Envelope加载Kudu表时报错，Timestamp能否作为主键列？

Great question—let's clear this up right away: Timestamp columns are absolutely supported as part of a Kudu primary key, so that's not the root cause of your IllegalArgumentException. The issue is almost certainly related to how Envelope is handling your timestamp data, or a small oversight in your setup.

First, let's confirm your Kudu table DDL is valid (I spotted a tiny syntax typo in your snippet—missing comma after client_id int—that might be a copy-paste error, but worth fixing if present in your actual code):

CREATE TABLE BAL (
  client_id int,
  bal_id int,
  effective_time timestamp,
  prsn_id int,
  bal_amount double,
  prsn_name string,
  PRIMARY KEY (client_id, bal_id, effective_time)
) PARTITION BY HASH(client_id) PARTITIONS 8 STORED AS KUDU;

This schema is fully compliant with Kudu's rules—timestamp columns in primary keys are explicitly allowed and commonly used for use cases like versioned records.

Now let's break down the most likely reasons you're hitting that error with Envelope:

• Timestamp format mismatch
  Kudu expects timestamps in one of two formats: Unix epoch milliseconds (as a long numeric type) or ISO 8601 strings (e.g., 2024-05-20T14:30:00Z). If your source data uses a different format (like a human-readable string without timezone info, or a float representing seconds since epoch), Envelope might be passing data that Kudu can't parse as a valid timestamp. Double-check your source data format, and add a transformation step in Envelope if needed to convert the timestamp to a Kudu-compatible format.

• Null values in the timestamp column
  Kudu primary key columns cannot be null. If any of your source records have a null value for effective_time, Envelope will throw an error when attempting to write to Kudu. Validate that all records have a non-null timestamp value before they reach the Kudu sink.

• Envelope configuration or version issues
  Make sure your Envelope job's schema definition correctly maps effective_time as a timestamp type. Additionally, older versions of Envelope had occasional bugs with timestamp handling in primary keys—if you're not running the latest stable version, upgrading might resolve the issue.

To narrow things down quickly:

1. Fix the DDL typo if present.
2. Try inserting a test record directly into Kudu (bypassing Envelope) using Impala or the Kudu API:

   INSERT INTO BAL VALUES (1, 100, now(), 5, 150.50, 'John Doe');
   

If this works, you know the Kudu table is functioning correctly, and the problem lies with Envelope's data handling or configuration.

内容的提问来源于stack exchange，提问作者Abhinav Singh