如何从Materials Project API提取formula、mpid及adsorptionEnergy数据？

问题描述 #

从Materials Project数据库提取数据时，已成功获取返回数据，但无法提取formula、mpid和adsorptionEnergy的具体数值，生成的CSV文件中嵌套数据未正确展开。

原代码如下：

import pandas as pd
from flatten_dict import flatten
from mpcontribs.client import Client

client = Client(apikey="MyAPIKey",
                project="open_catalyst_project")
#  print(client.available_query_params())
query = {"formula__contains": "Fe"}
fields = ["id", "identifier", "formula", "data.mpid", "data.adsorptionEnergy"]
docs = client.query_contributions(query=query, fields=fields, paginate=False)
print(docs)
for doc in docs:
  print(type(doc))
  print(doc)
df = pd.DataFrame.from_records(docs)
#  print(df.head())
df.to_csv('catalyst.csv', float_format="%.3f")

返回的docs结构示例：

{'data': [
    {'id': '6101e40e20c7bc66345c6d03', 'identifier': 'random698361', 'formula': 'Fe24Si24NO2', 
     'data': {'mpid': 'mp-871', 'adsorptionEnergy': {'display': '4.3102', 'value': 4.3102, 'error': None, 'unit': ''}}},
    {'id': '6101e41220c7bc66345c6d27', 'identifier': 'random542151', 'formula': 'Zr36Fe6Sb12H2C2O', 
     'data': {'mpid': 'mp-12962', 'adsorptionEnergy': {'display': '-2.29104', 'value': -2.29104, 'error': None, 'unit': ''}}},
    # 其余数据项省略
]}

解决方案 #

核心问题是docs为包含data键的字典，实际有效数据在docs['data']列表中，且部分字段为嵌套结构。以下两种方法可解决：

方法1：手动提取目标字段 #

直接遍历数据列表，提取所需嵌套字段的具体值：

import pandas as pd
from mpcontribs.client import Client

client = Client(apikey="MyAPIKey", project="open_catalyst_project")
query = {"formula__contains": "Fe"}
fields = ["id", "identifier", "formula", "data.mpid", "data.adsorptionEnergy"]
docs = client.query_contributions(query=query, fields=fields, paginate=False)

# 处理数据，提取目标字段
processed_data = []
for item in docs['data']:
    processed_item = {
        'id': item['id'],
        'identifier': item['identifier'],
        'formula': item['formula'],
        'mpid': item['data']['mpid'],
        'adsorptionEnergy': item['data']['adsorptionEnergy']['value']  # 提取数值
    }
    processed_data.append(processed_item)

# 生成DataFrame并保存
df = pd.DataFrame(processed_data)
df.to_csv('catalyst.csv', float_format="%.3f", index=False)

方法2：用flatten_dict自动展开嵌套结构 #

利用已导入的flatten_dict工具展开所有嵌套字段，再筛选目标列：

import pandas as pd
from flatten_dict import flatten
from mpcontribs.client import Client

client = Client(apikey="MyAPIKey", project="open_catalyst_project")
query = {"formula__contains": "Fe"}
fields = ["id", "identifier", "formula", "data.mpid", "data.adsorptionEnergy"]
docs = client.query_contributions(query=query, fields=fields, paginate=False)

# 展开所有嵌套字段
flattened_data = [flatten(item, reducer='dot') for item in docs['data']]
df = pd.DataFrame(flattened_data)

# 重命名列并筛选目标字段
df = df.rename(columns={
    'data.mpid': 'mpid',
    'data.adsorptionEnergy.value': 'adsorptionEnergy'
})
df = df[['id', 'identifier', 'formula', 'mpid', 'adsorptionEnergy']]

# 保存CSV
df.to_csv('catalyst.csv', float_format="%.3f", index=False)

关键说明 #

• 注意docs的结构：返回结果是字典，实际数据在docs['data']列表中，直接用docs创建DataFrame会导致结构错误。
• adsorptionEnergy的数值存储在value子字段中，需从嵌套字典提取。
• 保存CSV时添加index=False可避免生成不必要的索引列。

内容的提问来源于stack exchange，提问作者BaRud