如何批量读取NC文件、筛选指定时段并导出为指定格式单TXT？

拥有多个按月划分的NC文件，每个文件包含t2mj（气温）、td2mj（露点温度）、u10mj（u风场）、v10mj（v风场）四个变量。需要批量读取这些文件，筛选指定时段（示例为2022年1-10月，即时间早于2022-12-01），并将四个变量按气温、露点温度、u风场、v风场的顺序并排生成单份TXT表格。现有代码输出格式不符合预期，需优化代码以达到目标格式。

原代码 #

from netCDF4 import MFDataset
import pandas as pd
import xarray as xr
import csv
import tempfile

ds=xr.open_mfdataset('/home/milena/Documentos/dados_obs_haroldo/media_horaria/MEDIA_HORARIA_*.nc')

lat = ds.variables['lat'][:]
lon = ds.variables['lon'][:]
t2mj = ds.variables['t2mj'][:]
td2mj = ds.variables['td2mj'][:]
u10mj = ds.variables['u10mj'][:]
v10mj = ds.variables['v10mj'][:]

# Brasilia
t2mj_txt=ds.t2mj.isel(lat=153, lon=117).to_dataframe().to_csv('/home/milena/Documentos/dados_obs_haroldo/media_horaria/csv/t2mj.csv')
td2mj_txt=ds.td2mj.isel(lat=153, lon=117).to_dataframe().to_csv('/home/milena/Documentos/dados_obs_haroldo/media_horaria/csv/td2mj.csv')
u10mj_txt=ds.u10mj.isel(lat=153, lon=117).to_dataframe().to_csv('/home/milena/Documentos/dados_obs_haroldo/media_horaria/csv/u10mj.csv')
v10mj_txt=ds.v10mj.isel(lat=153, lon=117).to_dataframe().to_csv('/home/milena/Documentos/dados_obs_haroldo/media_horaria/csv/v10mj.csv')

# open csv
t2mj_csv = pd.read_csv('/home/milena/Documentos/dados_obs_haroldo/media_horaria/csv/t2mj.csv', skipinitialspace=True)
td2mj_csv = pd.read_csv('/home/milena/Documentos/dados_obs_haroldo/media_horaria/csv/td2mj.csv', skipinitialspace=True)
u10mj_csv = pd.read_csv('/home/milena/Documentos/dados_obs_haroldo/media_horaria/csv/u10mj.csv', skipinitialspace=True)
v10mj_csv = pd.read_csv('/home/milena/Documentos/dados_obs_haroldo/media_horaria/csv/v10mj.csv', skipinitialspace=True)

# filter desired period
t2mj_date=t2mj_csv[(t2mj_csv['time'])<"2022-12-01"]
td2mj_date=td2mj_csv[(td2mj_csv['time'])<"2022-12-01"]
u10mj_date=u10mj_csv[(u10mj_csv['time'])<"2022-12-01"]
v10mj_date=v10mj_csv[(v10mj_csv['time'])<"2022-12-01"]

arquivo = open("/home/milena/Documentos/dados_obs_haroldo/media_horaria/csv/t2mj_filter.txt", "w")
arquivo.write(t2mj_date['t2mj'].to_string())
arquivo.close()

arquivo2 = open("/home/milena/Documentos/dados_obs_haroldo/media_horaria/csv/td2mj_filter.txt", "w")
arquivo2.write(td2mj_date['td2mj'].to_string())
arquivo2.close()

arquivo3 = open("/home/milena/Documentos/dados_obs_haroldo/media_horaria/csv/u10mj_filter.txt", "w")
arquivo3.write(u10mj_date['u10mj'].to_string())
arquivo3.close()

arquivo4 = open("/home/milena/Documentos/dados_obs_haroldo/media_horaria/csv/v10mj_filter.txt", "w")
arquivo4.write(v10mj_date['v10mj'].to_string())
arquivo4.close()

file_list=['/home/milena/Documentos/dados_obs_haroldo/media_horaria/csv/t2mj_filter.txt', '/home/milena/Documentos/dados_obs_haroldo/media_horaria/csv/td2mj_filter.txt', '/home/milena/Documentos/dados_obs_haroldo/media_horaria/csv/u10mj_filter.txt', '/home/milena/Documentos/dados_obs_haroldo/media_horaria/csv/v10mj_filter.txt']
dfe = pd.DataFrame()
for file in file_list:
    temp_dfe = pd.read_csv(file, header=None, names=[file[:-4]])
    dfe = pd.concat([dfe, temp_dfe], axis=1)

arquivo5 = open("/home/milena/Documentos/dados_obs_haroldo/media_horaria/teste.txt", "w")
arquivo5.write(dfe.to_string())
arquivo5.close()

格式问题说明 #

• 当前输出：生成的TXT包含多余的索引列，列名是截取的文件路径片段，数据列未对齐，且丢失了时间信息，格式杂乱
• 期望输出：第一列为时间，后续依次为气温、露点温度、u风场、v风场的数值，各列对齐，表头清晰，仅保留指定时段的数据

优化代码 #

import xarray as xr
import pandas as pd

# 1. 批量读取所有NC文件
ds = xr.open_mfdataset('/home/milena/Documentos/dados_obs_haroldo/media_horaria/MEDIA_HORARIA_*.nc')

# 2. 提取指定经纬度（Brasilia）的目标变量，转换为DataFrame
# 选择需要的变量：t2mj(气温), td2mj(露点), u10mj(u风), v10mj(v风)
target_vars = ['t2mj', 'td2mj', 'u10mj', 'v10mj']
df = ds[target_vars].isel(lat=153, lon=117).to_dataframe().reset_index()

# 3. 筛选指定时段：2022年1-10月（时间早于2022-12-01）
df_filtered = df[df['time'] < '2022-12-01']

# 4. 重命名列名，让表头更清晰
df_filtered = df_filtered.rename(columns={
    't2mj': 'temperature',
    'td2mj': 'dew_point',
    'u10mj': 'u',
    'v10mj': 'v'
})

# 5. 生成符合要求的TXT表格，设置对齐格式
output_path = '/home/milena/Documentos/dados_obs_haroldo/media_horaria/result.txt'
# 使用to_string设置列对齐，保留索引，或者用to_csv指定分隔符为制表符实现对齐
df_filtered.to_string(output_path, index=False, col_space=15, float_format='%.2f')
# 若需要制表符分隔的格式，可改用：
# df_filtered.to_csv(output_path, sep='\t', index=False, float_format='%.2f')

优化点说明 #

1. 简化流程：直接在xarray层面完成变量提取和时间筛选，避免生成大量中间CSV/TXT文件，提升运行效率
2. 统一处理：一次性提取所有目标变量并合并为单DataFrame，确保时间轴完全对齐
3. 清晰表头：重命名列名为直观的英文名称，符合需求中的变量顺序
4. 格式控制：通过to_string的col_space和float_format参数控制列宽和数值精度，保证输出表格对齐整洁；也可选择制表符分隔的CSV格式（本质也是TXT），兼容更多读取工具

内容的提问来源于stack exchange，提问作者Milena Dantas