Julia中把逗号为小数点的String31列转为Float类型的方法

问题背景 #

我有一个包含190列的DataFrame，这些列的格式为String31，使用逗号作为小数点分隔符。需要将这些逗号替换为句号，然后转换为数值格式。

尝试了以下代码，但转换后的结果仍以类似2.0361331369e8的形式显示（误以为是字符串，实际是浮点数的科学计数法）：

function custom_parse(x::String, default::Float64)
    try
        return parse(Float64, x)
    catch
        return default
    end
end

for col in names(csv_coop_completo_1993a2022)[4:end]
    csv_coop_completo_1993a2022[!, col] = replace.(string.(csv_coop_completo_1993a2022[!, col]), "," => ".")
    csv_coop_completo_1993a2022[!, col] = map(x -> custom_parse(x, 0.0), csv_coop_completo_1993a2022[!, col])
end

CSV示例数据：

cnpj,ano,razao_social,10000007,11000006,11200002,39999993,60000002,61000001,61100004
171973,199312,COOPSCET,"5651,44","5651,44","5651,44","5651,44","5651,44","5651,44","8994,50"
379891,199312,CECMF CIMENTO TOCANTINS LTDA,"7429594,11","2759951,05","2759951,05","7429594,11","7161085,08","7161085,08","12550184,27"
617290,199312,"CECME SENAI-DF, LTDA","1500909,37","86432,56","86432,31","2632612,27","1573059,16","1573059,16","157922,83"
694877,199312,CECM SERV EXEC FED SECSA�DE DF,"22638658,49","306499,15","304001,15","25747440,55","24006473,31","24006473,31","20193139,66"
968602,199312,CCLA VALE S�O PATRICIO LTDA,"205124653,22","160604715,50","160469198,11","247012935,13","127063766,45","127063766,45","85617744,60"

问题分析 #

你看到的2.0361331369e8其实是浮点数的科学计数法显示，并非字符串。但原代码存在冗余步骤：先转换为字符串再替换，再单独解析，效率较低，且可能因中间步骤导致类型推断异常。

解决方案 #

方案1：简化循环逻辑，合并替换与解析 #

将逗号替换和浮点数解析合并到一个函数中，减少中间变量，确保类型正确转换：

function custom_parse(x::AbstractString, default::Float64=0.0)
    try
        return parse(Float64, replace(x, "," => "."))
    catch
        return default
    end
end

# 遍历目标列批量转换
for col in names(csv_coop_completo_1993a2022)[4:end]
    csv_coop_completo_1993a2022[!, col] = custom_parse.(csv_coop_completo_1993a2022[!, col])
end

方案2：用transform!批量处理（更简洁高效） #

利用DataFrames.jl的transform!结合ByRow，实现更简洁的批量列转换：

using DataFrames

# 定义转换函数：替换逗号为句号，解析为浮点数，失败返回0.0
parse_comma_float(x) = tryparse(Float64, replace(x, "," => ".")) |> something(0.0)

# 批量转换第4列及以后的所有列，保留原列名
transform!(csv_coop_completo_1993a2022, names(csv_coop_completo_1993a2022)[4:end] .=> ByRow(parse_comma_float); renamecols=false)

方案3：读取CSV时直接处理（根源解决） #

如果还未读取CSV文件，建议在读取阶段就指定小数点分隔符，直接生成数值类型列：

using CSV, DataFrames

# 读取时指定逗号为小数点分隔符，自动推断数值类型
df = CSV.read("your_file.csv", DataFrame; decimal=',')

# 若需强制指定列类型，可补充types参数
df = CSV.read("your_file.csv", DataFrame; decimal=',', types=Dict(zip(names(df)[4:end], repeat([Float64], 190))))

验证转换结果 #

转换后可通过以下代码确认列类型是否为Float64：

# 查看某一列的类型
println(eltype(csv_coop_completo_1993a2022[!, "10000007"]))

内容的提问来源于stack exchange，提问作者RxT