CNN模型拟合自定义数据集时首批数据后报KeyError求助

将数据拟合到CNN模型时，使用自定义数据集，拟合函数处理完第一批数据后立即报错，但相同代码在另一个数据集上运行正常。

代码片段 #

model.compile(loss='categorical_crossentropy', optimizer='adam')
model.optimizer.lr = 0.0001
epochs = 10
number_pics_per_batch = 5
steps = len(train_descriptions)//number_pics_per_bath
for i in range(epochs):
    generator = data_generator(train_descriptions, train_features, wordtoix, max_length, number_pics_per_bath)
    model.fit_generator(generator, epochs=1, steps_per_epoch=steps, verbose=1)

错误信息 #

This is separate from the ipykernel package so we can avoid doing imports until
 12/441 [..............................] - ETA: 1:27 - loss: 5.7915
---------------------------------------------------------------------------
UnknownError                              Traceback (most recent call last)
<ipython-input-73-afd8541ef3c8> in <module>
      1 for i in range(epochs):
      2     generator = data_generator(train_descriptions, train_features, wordtoix, max_length, number_pics_per_bath)
----> 3     model.fit_generator(generator, epochs=1, steps_per_epoch=steps, verbose=1)
/usr/local/lib/python3.7/dist-packages/keras/engine/training.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, validation_freq, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
   2221         use_multiprocessing=use_multiprocessing,
   2222         shuffle=shuffle,
-> 2223         initial_epoch=initial_epoch)
   2224 
   2225   @doc_controls.do_not_generate_docs
/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py in error_handler(*args, **kwargs)
     65     except Exception as e:  # pylint: disable=broad-except
     66       filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67       raise e.with_traceback(filtered_tb) from None
     68     finally:
     69       del filtered_tb
/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
     53     ctx.ensure_initialized()
     54     tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
---> 55                                         inputs, attrs, num_outputs)
     56   except core._NotOkStatusException as e:
     57     if name is not None:
UnknownError: Graph execution error:
2 root error(s) found.
  (0) UNKNOWN:  KeyError: 'eb808f61-81d5-441b-9480-ddaa24746115.jpg'
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/script_ops.py", line 271, in __call__
    ret = func(*args)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/autograph/impl/api.py", line 642, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/data/ops/dataset_ops.py", line 1004, in generator_py_func
    values = next(generator_state.get_iterator(iterator_id))
  File "/usr/local/lib/python3.7/dist-packages/keras/engine/data_adapter.py", line 830, in wrapped_generator
    for data in generator_fn():
  File "<ipython-input-61-381145846d3a>", line 12, in data_generator
    photo = photos[key+'.jpg']
KeyError: eb808f61-81d5-441b-9480-ddaa24746115.jpg
     [[{{node PyFunc}}]]
     [[IteratorGetNext]]
     [[IteratorGetNext/_12]]

核心问题是data_generator中尝试访问的图片键在train_features中不存在，或键的格式不匹配，结合代码细节，可按以下步骤修复：

• 修正拼写错误：代码中number_pics_per_bath是笔误，正确应为number_pics_per_batch，该错误会导致steps计算偏差，让generator尝试访问超出数据集范围的键。
• 核对键格式匹配：检查train_descriptions的键是否已包含.jpg后缀，如果是，key+'.jpg'会生成重复后缀（如xxx.jpg.jpg），自然找不到对应键，需修改data_generator中的拼接逻辑，统一键的格式。
• 排查缺失的特征键：用代码找出train_descriptions和train_features的键集合差异：

  desc_keys = set(train_descriptions.keys())
  feat_keys = set(train_features.keys())
  missing_keys = desc_keys - feat_keys
  print("特征集中缺失的键：", missing_keys)
  

  如果报错的键出现在missing_keys中，说明该图片的特征未被正确加载，需重新生成或补充对应特征数据。

内容的提问来源于stack exchange，提问作者user7704035