Azure Data Factory中SQL Server非空日期字符串转DateTime报错
问题详情
将CSV数据导入SQL Server时,源端EXPORT_DATE列是dd/MM/yyyy格式的字符串,要转换为目标端date类型,该列无空值,但触发如下报错:
ErrorCode=SqlOperationFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=A database operation failed. Please search error to get more details.,Source=Microsoft.DataTransfer.ClientLibrary,''Type=Microsoft.Azure.Data.Governance.Plugins.Core.TypeConversionException,Message=Exception occurred when converting value '24/01/2024' for column name 'EXPORT_DATE' from type 'String' (precision:, scale:) to type 'DateTime' (precision:255, scale:255). Additional info: String was not recognized as a valid DateTime.,Source=Microsoft.DataTransfer.ClientLibrary,''Type=System.FormatException,Message=String was not recognized as a valid DateTime.,Source=mscorlib,'
输入文件示例:
已尝试在映射环节设置dateformat为dd/MM/yyyy、ddMMyyyy、dd-MM-yyyy,均未解决问题。
SQL Server目标表结构:
ID (varchar,null) TYPE(varchar,null) TO_DATE(date,null) FROM_DATE(date,null) Work(float,null) EXPORT_DATE(date,null)
可行解决方案
- 添加派生列手动转换:在导入流程中新增派生列步骤,用表达式强制解析日期格式
若分隔符识别有问题,可先替换再转换:TO_DATE(EXPORT_DATE, 'dd/MM/yyyy')TO_DATE(REPLACE(EXPORT_DATE, '/', '-'), 'dd-MM-yyyy') - 临时修改SQL Server会话格式:在导入前执行SQL语句,设置会话级日期解析规则
SET DATEFORMAT dmy; - 预处理CSV文件:直接将CSV中日期格式改为SQL Server默认识别的
yyyy-MM-dd,再执行导入 - 自定义转换脚本:如果使用Azure Data Factory等工具,在映射中加入自定义解析代码(C#示例)
DateTime.ParseExact(row.EXPORT_DATE, "dd/MM/yyyy", System.Globalization.CultureInfo.InvariantCulture)
内容的提问来源于stack exchange,提问作者SheCodes

