Exasol数据库导入CSV报错:日期列与首列问题排查解决
问题场景
有4个各含500,000行的CSV文件,需导入Exasol数据库,需忽略文件中的首列,同时解决日期列转换报错问题。
CSV示例:
unnamed:0 , time, lat, lon, nobs_cloud_day 0, 2006-03-30, 24.125, -119.375, 22.0 1, 2006-03-30, 24.125, -119.125, 25.0
错误尝试过程
第一次尝试:包含首列的表结构
创建包含首列的表:
CREATE TABLE cloud_coverage_CONUS ( index_cloud DECIMAL(10,0) ,"time" DATE -- PRIMARY KEY ,lat DECIMAL(10,6) ,lon DECIMAL(10,6) ,nobs_cloud_day DECIMAL (3,1) )
执行导入命令:
IMPORT INTO cloud_coverage_CONUS FROM LOCAL CSV FILE 'D:\uni\BI\project 1\AOL_DB_ANALYSIS_TASK1\datasets\cloud\cfc_us_part0.csv';
报错信息:
SQL Error [42636]: java.sql.SQLException: ETL-3050: [Column=0 Row=0] [Transformation of value='Unnamed: 0' failed - invalid character value for cast; Value: 'Unnamed: 0'] (Session: 1750854753345597339) while executing '/* add path to the 4 csv files, that are in the cloud database folder*/ IMPORT INTO cloud_coverage_CONUS FROM CSV AT 'https://27.1.0.10:59205' FILE 'e12a96a6-a98f-4c0a-963a-e5dad7319fd5' ;'; 04509 java.sql.SQLException: java.net.SocketException: Connection reset by peer: socket write error
原因:CSV首行的列名Unnamed: 0无法转换为DECIMAL类型,且实际不需要导入该列。
第二次尝试:不含首列的表结构
重新创建不含首列的表:
CREATE TABLE cloud_coverage_CONUS ( "time" DATE -- PRIMARY KEY ,lat DECIMAL(10,6) ,lon DECIMAL(10,6) ,nobs_cloud_day DECIMAL (3,1) )
执行导入命令:
IMPORT INTO cloud_coverage_CONUS FROM LOCAL CSV FILE 'D:\uni\BI\project 1\AOL_DB_ANALYSIS_TASK1\datasets\cloud\cfc_us_part0.csv'(2 FORMAT='YYYY-MM-DD', 3 .. 5);
报错信息:
SQL Error [42636]: java.sql.SQLException: ETL-3052: [Column=0 Row=0] [Transformation of value='time' failed - invalid value for YYYY format token; Value: 'time' Format: 'YYYY-MM-DD'] (Session: 1750854753345597339) while executing '/* add path to the 4 csv files, that are in the cloud database folder*/ IMPORT INTO cloud_coverage_CONUS FROM CSV AT 'https://27.1.0.10:60350' FILE '22c64219-cd10-4c35-9e81-018d20146222' (2 FORMAT='YYYY-MM-DD', 3 .. 5);'; 04509 java.sql.SQLException: java.net.SocketException: Connection reset by peer: socket write error
原因:未跳过CSV的首行列名,导致把列名time当作日期值进行转换,触发格式错误。
最终解决方案
使用以下导入命令,同时实现忽略首列和跳过首行列名:
IMPORT INTO cloud_coverage_CONUS FROM LOCAL CSV FILE 'D:\uni\BI\project 1\AOL_DB_ANALYSIS_TASK1\datasets\cloud\cfc_us_part0.csv' (2 .. 5) ROW SEPARATOR = 'CRLF' COLUMN SEPARATOR = ',' SKIP = 1;
关键参数说明:
(2 .. 5):指定导入CSV的第2到第5列,跳过第1列(不需要的unnamed:0列)SKIP = 1:跳过CSV的首行列名,从第2行开始导入数据ROW SEPARATOR = 'CRLF'和COLUMN SEPARATOR = ',':明确指定行分隔符和列分隔符,避免解析歧义
注意事项
该问题主要源于MySQL与Exasol的导入语法差异,Exasol的IMPORT命令需要显式指定跳过列名行、选择导入列等参数,与MySQL的导入逻辑有所不同。
内容的提问来源于stack exchange,提问作者Shahin

