使用erddapy访问ncdcOisst21Agg_LonPM180数据集遇404错误求助

在Jupyter Notebook中使用erddapy从NOAA ERDDAP服务器获取OISST数据，指定的数据集ID ncdcOisst21Agg_LonPM180在网页搜索可找到，但代码执行后返回404错误，提示数据集ID不存在。设备为2020款Intel i5 MacBook Air。

执行代码 #

from erddapy import ERDDAP
import pandas as pd
import xarray as xr
import matplotlib.pyplot as plt
import numpy as np

def download_oisst_data(start_date, end_date, min_lat, max_lat, min_lon, max_lon):
    e = ERDDAP(
        server="https://coastwatch.pfeg.noaa.gov/erddap/",
        protocol="tabledap",
    )
    e.dataset_id = "ncdcOisst21Agg_LonPM180"  # Correctly set the dataset ID
    e.variables = ["time", "latitude", "longitude", "sst"]
    e.constraints = {
        "time>=": start_date,
        "time<=": end_date,
        "latitude>=": min_lat,
        "latitude<=": max_lat,
        "longitude>=": min_lon,
        "longitude<=": max_lon,
    }

    # Fetch the data and convert it to a pandas DataFrame
    df = e.to_pandas(
        index_col="time (UTC)",
        parse_dates=True,
        skiprows=(1,)  # Skip the units row
    ).dropna()

    # Convert the DataFrame to an xarray Dataset
    ds = df.to_xarray()  
    
    return ds

ds = download_oisst_data(start_date, end_date, min_lat, max_lat, min_lon, max_lon)

返回错误信息 #

---------------------------------------------------------------------------
HTTPStatusError                           Traceback (most recent call last)
File ~/miniconda3/lib/python3.10/site-packages/erddapy/core/url.py:24, in _urlopen(url, auth, **kwargs)
     23 try:
---> 24     response.raise_for_status()
     25 except httpx.HTTPError as err:

File ~/miniconda3/lib/python3.10/site-packages/httpx/_models.py:761, in Response.raise_for_status(self)
    760 message = message.format(self, error_type=error_type)
--> 761 raise HTTPStatusError(message, request=request, response=self)

HTTPStatusError: Client error '404 ' for url 'https://coastwatch.pfeg.noaa.gov/erddap/tabledap/ncdcOisst21Agg_LonPM180.csvp?time,latitude,longitude,sst&time%3E=368150400.0&time%3C=1704067199.0&latitude%3E=-40&latitude%3C=30&longitude%3E=30&longitude%3C=100'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/404

The above exception was the direct cause of the following exception:

HTTPError                                 Traceback (most recent call last)
Cell In[24], line 1
----> 1 ds = download_oisst_data(start_date, end_date, min_lat, max_lat, min_lon, max_lon)

Cell In[22], line 18, in download_oisst_data(start_date, end_date, min_lat, max_lat, min_lon, max_lon)
      8 e.constraints = {
      9     "time>=": start_date,
     10     "time<=": end_date,
     11     "latitude>=": min_lat,
     12     "latitude<=": max_lat,
     13     "longitude>=": min_lon,
     14     "longitude<=": max_lon,
     15 }
     17 # Fetch the data and convert it to a pandas DataFrame
---&gt; 18 df = e.to_pandas(
     19     index_col="time (UTC)",
     20     parse_dates=True,
     21     skiprows=(1,)  # Skip the units row
     22 ).dropna()
     24 # Convert the DataFrame to an xarray Dataset, if needed
     25 # This step requires importing xarray and possibly additional processing depending on the data structure
     26 ds = df.to_xarray()  # Uncomment this line if you have the necessary setup for converting DataFrame to xarray Dataset

File ~/miniconda3/lib/python3.10/site-packages/erddapy/erddapy.py:361, in ERDDAP.to_pandas(self, requests_kwargs, **kw)
    359 distinct = kw.pop("distinct", False)
    360 url = self.get_download_url(response=response, distinct=distinct)
---&gt; 361 return to_pandas(url, requests_kwargs=requests_kwargs, pandas_kwargs=dict(**kw))

File ~/miniconda3/lib/python3.10/site-packages/erddapy/core/interfaces.py:31, in to_pandas(url, requests_kwargs, pandas_kwargs)
     19 def to_pandas(
     20     url: str,
     21     requests_kwargs: Optional[Dict] = None,
     22     pandas_kwargs: Optional[Dict] = None,
     23 ) -&gt; "pd.DataFrame":
     24     """
     25     Convert a URL to Pandas DataFrame.
     26 
     27     Parameters
     28     ----------
     29     **pandas_kwargs: kwargs to be passed to third-party library (pandas).
     30     """
---&gt; 31     data = urlopen(url, requests_kwargs or {})
     32     try:
     33         return pd.read_csv(data, **(pandas_kwargs or {}))

File ~/miniconda3/lib/python3.10/site-packages/erddapy/core/url.py:42, in urlopen(url, requests_kwargs)
     40 if requests_kwargs is None:
     41     requests_kwargs = {}
---&gt; 42 data = _urlopen(url, **requests_kwargs)  # type: ignore
     43 data.seek(0)
     44 return data

File ~/miniconda3/lib/python3.10/site-packages/erddapy/core/url.py:26, in _urlopen(url, auth, **kwargs)
     24     response.raise_for_status()
     25 except httpx.HTTPError as err:
---&gt; 26     raise httpx.HTTPError(f"{response.content.decode()}") from err
     27 return io.BytesIO(response.content)

HTTPError: Error {
    code=404;
    message="Not Found: Currently unknown datasetID=ncdcOisst21Agg_LonPM180";
}

解决方法 #

1. 确认数据集的服务类型 #

ERDDAP数据集分为tabledap（表格数据）和griddap（网格数据）两种服务，部分OISST数据集仅支持griddap。访问数据集详情页，查看顶部的服务入口，若只有griddap选项，需修改代码：

• 将protocol="tabledap"改为protocol="griddap"
• 改用to_xarray()获取数据（网格数据更适合xarray格式）

修改后的示例代码：

from erddapy import ERDDAP
import xarray as xr

def download_oisst_data(start_date, end_date, min_lat, max_lat, min_lon, max_lon):
    e = ERDDAP(
        server="https://coastwatch.pfeg.noaa.gov/erddap/",
        protocol="griddap",
    )
    e.dataset_id = "ncdcOisst21Agg_LonPM180"
    e.variables = ["sst"]
    e.constraints = {
        "time>=": start_date,
        "time<=": end_date,
        "latitude>=": min_lat,
        "latitude<=": max_lat,
        "longitude>=": min_lon,
        "longitude<=": max_lon,
    }

    ds = e.to_xarray()  
    return ds

# 注意：日期需传入字符串格式（如"2020-01-01"），而非时间戳
ds = download_oisst_data("2020-01-01", "2020-01-02", -40, 30, 30, 100)

2. 验证数据集ID的准确性 #

网页显示的数据集名称和API使用的ID可能存在差异（如大小写、下划线）。打开数据集详情页，地址栏末尾的字符串即为准确的数据集ID，确认是否与代码中一致。

3. 切换ERDDAP服务器 #

部分NOAA ERDDAP服务器可能存在数据集迁移或维护情况，可尝试切换到其他服务器，比如改用https://upwell.pfeg.noaa.gov/erddap/，重新搜索并确认数据集ID。

4. 直接测试API请求 #

将错误信息中的URL复制到浏览器或使用curl命令测试，查看服务器返回的详细错误，判断是否为服务器维护、权限限制或数据集下线导致的问题。

内容的提问来源于stack exchange，提问作者54m4gr4