优化代码：实现向DataFrame动态追加列表内容

Hey there! Glad your current code is up and running—let’s work out a more dynamic approach to append your trading pair time-volume data to a DataFrame. Here are a few practical, efficient methods you can use:

1. Collect all data first, then create the DataFrame (most efficient) #

Instead of appending incrementally (which can be slow for large datasets), gather all your data entries into a list first, then build the DataFrame in one go. This avoids repeated DataFrame object creation, which is way more performant.

import pandas as pd

# Initialize an empty list to store all your time-volume records
all_trade_records = []

# Example loop: iterate through your trading pairs and fetch data
for trading_pair in ["BTC/USD", "ETH/USD", "SOL/USD"]:
    # Simulate fetching time-volume data for the pair
    pair_data = {
        "timestamp": ["2024-05-20 14:00", "2024-05-20 14:01"],
        "trading_pair": trading_pair,
        "volume": [15.2, 9.8]
    }
    # Convert the pair's data to a list of dictionaries and add to our main list
    all_trade_records.extend(pd.DataFrame(pair_data).to_dict("records"))

# Create the final DataFrame once all data is collected
final_df = pd.DataFrame(all_trade_records)
print(final_df)

2. Use pd.concat for dynamic appends (replaces deprecated append) #

If you need to append data in chunks (like real-time streaming), pd.concat is the recommended replacement for the old DataFrame.append() method (which is no longer supported in newer pandas versions).

import pandas as pd

# Start with an empty DataFrame with your target columns
final_df = pd.DataFrame(columns=["timestamp", "trading_pair", "volume"])

# Example loop to process each chunk of data
for trading_pair in ["BTC/USD", "ETH/USD", "SOL/USD"]:
    # Create a temporary DataFrame for the current pair's data
    temp_df = pd.DataFrame({
        "timestamp": ["2024-05-20 14:00", "2024-05-20 14:01"],
        "trading_pair": trading_pair,
        "volume": [15.2, 9.8]
    })
    # Append the temp DataFrame to the main one with concat
    final_df = pd.concat([final_df, temp_df], ignore_index=True)

print(final_df)

3. Append single records directly (for one-off entries) #

If you’re adding individual rows one at a time, you can use loc to append to the end of the DataFrame:

# Start with your base DataFrame
final_df = pd.DataFrame(columns=["timestamp", "trading_pair", "volume"])

# Single record as a list (matches column order)
new_record = ["2024-05-20 14:02", "ADA/USD", 6.3]
# Append to the end of the DataFrame
final_df.loc[len(final_df)] = new_record

print(final_df)

Quick tip for large datasets #

If you’re dealing with a lot of data, stick with method 1 whenever possible—bulk creation is way faster than incremental appends. For real-time use cases, try batching your data (e.g., collect 100 records at a time) before appending with pd.concat to minimize overhead.

内容的提问来源于stack exchange，提问作者Dicast