R语言高效处理多同格式CSV：合并Data.frame及基线差值计算

Hey there! Since your CSV outputs are already in tidy format with identical structure, we can solve both problems efficiently with some straightforward R workflows. Let's break this down step by step.

1. Efficiently Merging Variable Numbers of CSV Data Frames #

Manually calling rbind() for each file is tedious (and error-prone) when the number of files changes. Instead, use batch reading + automatic row-binding. Here are two reliable approaches:

Option 1: Tidyverse (Clean & Readable) #

The purrr::map_dfr() function is perfect here—it reads each CSV and automatically binds the results into a single data frame. We can even add a column to track which scenario each row comes from:

library(tidyverse)

# Get paths to all CSV files in your target folder
csv_paths <- list.files(
  path = "path/to/your/csvs", 
  pattern = "\\.csv$",  # Match only .csv files
  full.names = TRUE     # Return full file paths
)

# Batch read and merge, with a scenario identifier column
combined_data <- map_dfr(
  csv_paths, 
  read.csv, 
  .id = "scenario"  # Adds a column with the file index
)

# Update scenario column to use actual filenames instead of indices
combined_data <- combined_data %>%
  mutate(scenario = basename(csv_paths[as.integer(scenario)]))

Option 2: Data.Table (Faster for Large Datasets) #

If you're working with huge time series files, data.table's fread() and rbindlist() are significantly faster than base/tidyverse methods:

library(data.table)

csv_paths <- list.files("path/to/your/csvs", pattern = "\\.csv$", full.names = TRUE)

# Batch read and merge, with a scenario column
combined_data <- rbindlist(
  lapply(csv_paths, fread), 
  idcol = "scenario"
)
combined_data[, scenario := basename(csv_paths[scenario])]

2. Calculating Differences Between Baseline and Alternative Scenarios #

Since all your data has the same structure, we can easily join the baseline data to other scenarios and compute differences. Here's a tidyverse workflow that keeps things organized:

Step 1: Isolate the Baseline Data #

First, extract the baseline scenario from your combined data frame:

baseline_data <- combined_data %>%
  filter(scenario == "your_baseline_filename.csv") %>%  # Replace with your actual baseline filename
  select(date, baseline_value = value)  # Assume 'date' is your time column, 'value' is the hydrological metric

Step 2: Compute Differences for All Other Scenarios #

Join the baseline data to the rest of the scenarios, then calculate the difference (alternative - baseline):

scenario_differences <- combined_data %>%
  filter(scenario != "your_baseline_filename.csv") %>%
  left_join(baseline_data, by = "date") %>%  # Join on the time column to align values
  mutate(difference = value - baseline_value) %>%
  select(date, scenario, difference)  # Keep only relevant columns

Alternative: Wide Format Workflow #

If you prefer working with wide-format data first, you can pivot your data, compute differences, then pivot back to long format:

wide_data <- combined_data %>%
  pivot_wider(names_from = scenario, values_from = value)

scenario_differences <- wide_data %>%
  pivot_longer(
    cols = -c(date, your_baseline_filename.csv),  # Exclude date and baseline columns
    names_to = "scenario",
    values_to = "value"
  ) %>%
  mutate(difference = value - .data[["your_baseline_filename.csv"]]) %>%
  select(date, scenario, difference)

Key Notes #

• Ensure your time column (date in the examples) is formatted consistently across all files (e.g., as a Date type) to avoid join errors.
• If you have multiple hydrological metrics (e.g., discharge, rainfall), add a metric column to your data and use group_by(metric) before calculating differences to handle each metric separately.

内容的提问来源于stack exchange，提问作者dbo