如何将长度不一致的嵌套列表转换为R DataFrame？

问题场景 #

我有一个如下所示的嵌套列表，其中部分子列表的字段不全（比如第三个子列表缺少name字段）：

sam <- list(list(symbol = "COUNCODOS.NS", name = "Country Condo's Limited", 
          price = 4.8, changesPercentage = 0, change = 0, dayLow = 4.6, 
          dayHigh = 4.9, yearHigh = 7.05, yearLow = 3.4, marketCap = 372467040, 
          priceAvg50 = 5.383, priceAvg200 = 4.77275, exchange = "NSE", 
          volume = 72558, avgVolume = 111060, open = 4.8, previousClose = 4.8, 
          eps = 0.23, pe = 20.87, earningsAnnouncement = "2023-03-31T04:00:00.000+0000", 
          sharesOutstanding = 77597300, timestamp = 1711619840), list(
              symbol = "SPENCERS.NS", name = "Spencer's Retail Limited", 
              price = 91.3, changesPercentage = 0.44, change = 0.4, dayLow = 90.6, 
              dayHigh = 94.55, yearHigh = 139.3, yearLow = 51.5, marketCap = 8229051600, 
              priceAvg50 = 110.369, priceAvg200 = 83.59675, exchange = "NSE", 
              volume = 500311, avgVolume = 814721, open = 92.25, previousClose = 90.9, 
              eps = -27.41, pe = -3.33, earningsAnnouncement = "2024-05-20T00:00:00.000+0000", 
              sharesOutstanding = 90132000, timestamp = 1711619998),list(symbol = "SRPL-RE.NS", price = 0.05, changesPercentage = -50, 
                                                                         change = -0.05, dayLow = 0.05, dayHigh = 0.1, yearHigh = 0.1, 
                                                                         yearLow = 0.05, marketCap = 0, priceAvg50 = 0, priceAvg200 = 0, 
                                                                         exchange = "NSE", volume = 1503842, avgVolume = 0, open = 0.1, 
                                                                         previousClose = 0.1, eps = NA, pe = 0, earningsAnnouncement = NA, 
                                                                         sharesOutstanding = 0, timestamp = 1691575172))

尝试用基础R的方法转换为DataFrame：

do.call(rbind.data.frame, sam)

运行后出现错误：

Error in (function (..., deparse.level = 1, make.row.names = TRUE, stringsAsFactors = default.stringsAsFactors(),  : 
  numbers of columns of arguments do not match

解决方案 #

方法1：使用dplyr的bind_rows() #

bind_rows()会自动识别所有子列表的字段，缺失的字段填充为NA，代码简洁：

library(dplyr)
df <- bind_rows(sam)

方法2：使用data.table的rbindlist() #

如果处理大数据量，data.table的效率更高，同样支持自动补全缺失列：

library(data.table)
df <- rbindlist(sam, fill = TRUE)

方法3：基础R手动补全字段 #

不需要加载额外包，先统一所有子列表的字段，再转换：

# 获取所有子列表的唯一字段名
all_columns <- unique(unlist(lapply(sam, names)))

# 给每个子列表补全缺失字段，值设为NA
sam_filled <- lapply(sam, function(sub_list) {
  missing_cols <- setdiff(all_columns, names(sub_list))
  sub_list[missing_cols] <- NA
  sub_list
})

# 转换为DataFrame
df <- do.call(rbind.data.frame, sam_filled)

内容的提问来源于stack exchange，提问作者rakshu vin