在Data Lake Store运行含R脚本的U-SQL任务遇类型转换错误求助

I’ve run into this exact issue before—it’s all about how U-SQL handles nullable data types when passing data to the R extension. Let’s break down what’s happening and fix it:

Root Cause #

Looking at your script, the total_bytes column in @imsi_traffic_data comes from a SUM aggregate. In U-SQL, if the SUM operation has no matching rows (or if input nulls propagate through the calculation), the result becomes a Nullable<long> (System.Nullable`1[System.Int64]). The R Reducer component can’t convert nullable arrays directly into R vectors, which triggers the exception you’re seeing.

Solution Options #

Option 1: Replace Nulls with a Default Value (e.g., 0)

If your business logic allows treating missing traffic data as 0 bytes, use ISNULL to convert the nullable total_bytes to a non-nullable long:

@imsi_traffic_data = 
SELECT 
    [H_IMSI_BK], 
    -- Use ISNULL to replace null SUM results with 0 (0L specifies a long integer)
    ISNULL(SUM(([sum_session_bytes_in] + [sum_session_bytes_out]) * [row_count]), 0L) AS [total_bytes] 
FROM @input 
GROUP BY [H_IMSI_BK];

Option 2: Filter Out Rows with Null total_bytes

If null total_bytes values are invalid for your analysis, exclude them entirely before passing data to the R script:

@imsi_traffic_data = 
SELECT 
    [H_IMSI_BK], 
    SUM(([sum_session_bytes_in] + [sum_session_bytes_out]) * [row_count]) AS [total_bytes] 
FROM @input 
GROUP BY [H_IMSI_BK]
-- Drop rows where SUM returned null
WHERE SUM(([sum_session_bytes_in] + [sum_session_bytes_out]) * [row_count]) IS NOT NULL;

Why This Works #

By converting total_bytes to a non-nullable long (either via default value or filtering), the data passed to the R Reducer becomes a regular long[] array instead of a Nullable<long>[]. The R extension can then properly convert this array to an R numeric vector, which your quantile function can process without issues.

Quick Additional Check #

The Par column in @ExtendedData maps directly to total_bytes, so fixing total_bytes will automatically make Par non-nullable too—this ensures the REDUCE operation runs without type conflicts.

内容的提问来源于stack exchange，提问作者Lorenzo Benassi