使用petastorm加载Spark DataFrame时遇‘无匹配字段’错误

我按照Petastorm Spark Converter TensorFlow相关教程，通过Petastorm 0.12加载Spark DataFrame，核心代码如下。标题中的错误在with语句中触发，但直接执行train_context_manager = converter_train.make_tf_dataset(BATCH_SIZE)创建TFDatasetContextManager时不会出现该问题。

from petastorm import TransformSpec
from petastorm.spark import make_spark_converter

spark.conf.set(SparkDatasetConverter.PARENT_CACHE_DIR_URL_CONF, "file:///dbfs/tmp/petastorm/cache")

converter_train = make_spark_converter(DF_TRAIN)
with converter_train.make_tf_dataset(BATCH_SIZE) as X_train:
  pass

数据集肯定不为空，我还尝试显式使用TransformSpec选择目标字段：

with converter_train.make_tf_dataset(
    BATCH_SIZE,
    transform_spec=TransformSpec(selected_fields=[TRAIN_COL])
) as X_train:

另外，converter_train.make_torch_dataloader也出现相同问题。

内容的提问来源于stack exchange，提问作者ascripter