使用NDB通过PITR读取数据时出现KeyError: 'read_time'问题

问题描述 #

使用NDB通过PITR（时间点恢复）读取历史实体时触发KeyError: 'read_time'错误，已确认：

• 数据库已启用PITR功能
• 目标读取日期处于7天的有效恢复窗口内

错误日志 #

File "/layers/google.python.pip/pip/lib/python3.9/site-packages/google/appengine/datastore/datastore_rpc.py", line 390, in __new__
    config_option = obj._options[key]
KeyError: 'read_time'"

"Exception on /admin/fix [GET]
Traceback (most recent call last):
  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/google/appengine/datastore/datastore_rpc.py", line 390, in __new__
    config_option = obj._options[key]
KeyError: 'read_time'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py", line 2190, in wsgi_app
    response = self.full_dispatch_request()
  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py", line 1486, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py", line 1484, in full_dispatch_request
    rv = self.dispatch_request()
  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/flask/app.py", line 1469, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "/srv/main.py", line 36, in __call__
    return self.view(*args, **kwargs)
  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/google/appengine/ext/ndb/tasklets.py", line 1138, in add_context_wrapper
    return synctaskletfunc(*args, **kwds)
  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/google/appengine/ext/ndb/tasklets.py", line 1117, in synctasklet_wrapper
    return taskletfunc(*args, **kwds).get_result()
  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/google/appengine/ext/ndb/tasklets.py", line 1087, in tasklet_wrapper
    result = func(*args, **kwds)
  File "/srv/admin/fix.py", line 803, in fix_handler
    assets = ee_item.Asset.query(ee_item.Asset.estate_id==6359851389157376).fetch(100,read_time=want_time)
  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/google/appengine/ext/ndb/utils.py", line 182, in positional_wrapper
    return wrapped(*args, **kwds)
  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/google/appengine/ext/ndb/query.py", line 1266, in fetch
    return self.fetch_async(limit, **q_options).get_result()
  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/google/appengine/ext/ndb/utils.py", line 182, in positional_wrapper
    return wrapped(*args, **kwds)
  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/google/appengine/ext/ndb/query.py", line 1285, in fetch_async
    options = self._make_options(q_options)
  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/google/appengine/ext/ndb/query.py", line 1468, in _make_options
    options = QueryOptions(**q_options)
  File "/layers/google.python.pip/pip/lib/python3.9/site-packages/google/appengine/datastore/datastore_rpc.py", line 392, in __new__
    raise TypeError('Unknown configuration option (%s)' % err)
TypeError: Unknown configuration option ('read_time')"
timestamp: "2024-09-17T13:44:07.645924Z"

调用代码 #

不带read_time参数时运行正常，出错的调用代码如下：

want_time = datetime.datetime(2024, 9, 13, 4, 30, 5)
entities = MyClass.query(Myclass.my_propname==123).fetch(read_time=want_time)

环境配置 #

使用遗留捆绑服务，app.yaml设置：

runtime: python39
app_engine_apis: true

解决方案 #

这个问题的核心是：你使用的旧版NDB库（google.appengine.ext.ndb）不支持read_time参数——该参数是新版Cloud NDB（google.cloud.ndb）才引入的特性，而你通过app_engine_apis: true启用的是App Engine旧版NDB。

方案一：切换到Cloud NDB（推荐） #

1. 替换依赖库：卸载旧版NDB，安装Cloud NDB

   pip uninstall google-appengine-ext-ndb
   pip install google-cloud-ndb
   

2. 修改代码导入：将原有NDB导入替换为Cloud NDB

   # 旧导入
   # from google.appengine.ext import ndb
   # 新导入
   from google.cloud import ndb
   

3. 添加上下文初始化：Cloud NDB需要显式初始化客户端上下文，以Flask为例：

   from google.cloud.ndb import Client
   
   client = Client()
   with client.context():
       want_time = datetime.datetime(2024, 9, 13, 4, 30, 5)
       entities = MyClass.query(Myclass.my_propname==123).fetch(read_time=want_time)
   

4. 保留原有配置：app.yaml中的app_engine_apis: true可继续保留，不影响Cloud NDB使用。

方案二：不换库，使用旧版API实现PITR #

如果暂时不想切换到Cloud NDB，可以用旧版datastore_v3接口手动指定读取时间：

from google.appengine.api import datastore_v3
from google.appengine.ext.ndb import model

# 设置目标读取时间
read_time = datetime.datetime(2024, 9, 13, 4, 30, 5)
datastore_v3.SetReadTime(read_time)

# 执行查询
entities = MyClass.query(Myclass.my_propname==123).fetch()

# 可选：恢复默认读取时间
datastore_v3.SetReadTime(None)

内容的提问来源于stack exchange，提问作者Dev93