data.table中为何:=比`:=`()更快？性能差异原因解析

Great catch! Your test results are spot-on, and while function call overhead does play a role, there’s more nuance to why the functional :=() form lags behind repeated single-column assignments. Let’s break this down, including why the speed gap stays a relative multiple instead of a fixed constant.

It’s not just function call overhead #

You’re right that invoking := as a function adds some baseline overhead, but data.table’s internal handling of multi-column assignments amplifies this:

• Expression parsing & setup: When you use dt[, :=(d = a + b, e = b + c, f = a + c)], data.table has to parse all three expressions in one go, set up evaluation contexts for each, and coordinate their execution. Each separate := call only deals with one expression, cutting down on cumulative setup work.
• Memory management: Assigning one column at a time lets data.table optimize memory allocation for individual vectors. The functional form, by contrast, needs to handle multiple column allocations in a single call—this can introduce extra overhead in memory bookkeeping and less efficient batch allocation.
• Internal checks: data.table runs quick checks (like validating column names, ensuring the table isn’t locked) for each assignment. While these are fast, the functional form bundles multiple assignments into one call, leading to redundant setup steps that separate calls avoid.

Why the speed gap is a multiple, not a fixed value #

Your observation about the 1.5-2x difference holding across data sizes makes perfect sense:

• The core work (vectorized operations like a + b) scales linearly with your dataset size for both methods.
• The extra overhead of the functional form also scales linearly. Parsing three expressions, managing three column allocations—all that extra work grows proportionally with the number of rows, just like the core calculations.
• Since both the core work and the extra overhead scale at the same rate, their ratio (the speed gap) stays consistent as n increases, rather than being a fixed time difference.

Practical takeaway #

While repeated := calls are faster in these tests, the functional :=() form is usually better for readability and concise code. For most everyday tasks, this performance gap is negligible—it only becomes noticeable when working with extremely large datasets (like your 20M-row example).

Your test results for reference #

Test 1: 5M rows

library(data.table)
n <- 5000000
dt <- data.table(a = rnorm(n), b = rnorm(n), c = rnorm(n))
dt_a <- copy(dt)
system.time({
 dt_a[, d := a + b]
 dt_a[, e := b + c]
 dt_a[, f := a + c]
})
#> user  system elapsed 
#> 0.076  0.060  0.136 

dt_b <- copy(dt)
system.time({
 dt_b[, `:=`(d = a + b, e = b + c, f = a + c)]
})
#> user  system elapsed 
#> 0.096  0.116  0.211 

Test 2: 20M rows

library(data.table)
n <- 20000000
dt <- data.table(a = rnorm(n), b = rnorm(n), c = rnorm(n))
dt_a <- copy(dt)
system.time({
 dt_a[, d := a + b]
 dt_a[, e := b + c]
 dt_a[, f := a + c]
})
#> user  system elapsed 
#> 0.163  0.208  0.371 

dt_b <- copy(dt)
system.time({
 dt_b[, `:=`(d = a + b, e = b + c, f = a + c)]
})
#> user  system elapsed 
#> 0.284  0.404  0.688

内容的提问来源于stack exchange，提问作者petrovski