如何高效替换pandas中Date2列的日期部分为Date1对应值？

Great question—regex + replace is notoriously slow for large pandas DataFrames because it's doing row-by-row Python-level processing. Let's fix this with vectorized operations (pandas' superpower for speed) that'll handle your 150k rows in seconds, not minutes.

1. Use Datetime Types (Most Robust) #

The cleanest approach is to convert both columns to datetime objects, then merge the date from Date1 with the time from Date2. Pandas' datetime tools are optimized to work on entire columns at once:

import pandas as pd

# Convert columns to datetime (skip if they're already datetime)
df['Date1'] = pd.to_datetime(df['Date1'])
df['Date2'] = pd.to_datetime(df['Date2'])

# Combine Date1's date with Date2's time
df['Date2'] = pd.to_datetime(
    df['Date1'].dt.date.astype(str) + ' ' + df['Date2'].dt.time.astype(str)
)

# Optional: Convert back to string format if required
df['Date2'] = df['Date2'].dt.strftime('%Y-%m-%d %H:%M:%S')

2. String Concatenation (Faster for Fixed Patterns) #

Since all your Date2 entries start with 1970-01-01, you can directly extract the time part and concatenate it with Date1. Pandas' string methods are vectorized, so this is blazingly fast:

# Extract the time segment from Date2 (split on first space, keep the second part)
time_segment = df['Date2'].str.split(' ', n=1).str[1]

# Merge Date1 with the time segment
df['Date2'] = df['Date1'] + ' ' + time_segment

3. Using np.where (Conditional Replacement) #

If you only need to replace rows where Date2 has the 1970-01-01 date (instead of all rows), np.where works well with the string approach:

import numpy as np

time_segment = df['Date2'].str.split(' ', n=1).str[1]
df['Date2'] = np.where(
    df['Date2'].str.startswith('1970-01-01'),
    df['Date1'] + ' ' + time_segment,
    df['Date2']  # Leave other rows unchanged
)

Why Your Original Method Was Slow #

Your regex replace runs a Python regex function on every single row individually. This has huge overhead for large datasets. Vectorized operations (like the ones above) process entire columns at once using optimized C code—this is why they're so much faster for 150k rows.

内容的提问来源于stack exchange，提问作者PyRsquared