使用numpy_input_fn时TensorFlow出现挂起问题，寻求技术帮助

Hey there! Sorry to hear you're hitting a hang issue with TensorFlow's numpy input function—let's walk through common causes and fixes tailored to your setup (TF 1.6.0, Python 3.6.4, MacOS 10.13):

1. Missing Queue Cleanup or Termination Logic #

In TensorFlow 1.x, numpy_input_fn relies on internal queues to batch data. If you don't properly handle the end of your dataset or close session resources, the program will hang waiting for more data indefinitely.

Fix Example: #

import tensorflow as tf
import numpy as np

# Sample data
x_data = np.random.rand(100, 2)
y_data = np.random.randint(0, 2, size=(100,))

input_fn = tf.estimator.inputs.numpy_input_fn(
    x={"x": x_data},
    y=y_data,
    batch_size=32,
    num_epochs=1,  # Critical: Set finite number of epochs
    shuffle=True
)

# Use a session context manager to auto-cleanup resources
with tf.Session() as sess:
    iterator = input_fn().make_one_shot_iterator()
    next_batch = iterator.get_next()
    
    try:
        while True:
            batch_x, batch_y = sess.run(next_batch)
            print(f"Batch received: {batch_x.shape}, {batch_y.shape}")
    except tf.errors.OutOfRangeError:
        # This exception triggers when all epochs are processed
        print("All data batches completed successfully!")

Key note: Always catch tf.errors.OutOfRangeError to signal the end of your dataset—without this, the session will wait forever for the next batch.

2. Accidental Infinite Epochs #

If you set num_epochs=None in numpy_input_fn, the input function will loop through your data infinitely. This makes the program appear to hang, when it's actually still generating batches.

Fix: #

Set num_epochs to a specific number (like 1 for a single pass) or add manual termination logic if you need infinite iteration.

3. MacOS Thread Scheduling Bugs (TF 1.x Specific) #

Older TensorFlow versions (like 1.6.0) had known thread scheduling issues on MacOS that could cause queue hangs. Try explicitly managing queue threads:

Fix Example: #

with tf.Session() as sess:
    coord = tf.train.Coordinator()
    # Explicitly start queue runner threads
    threads = tf.train.start_queue_runners(coord=coord)
    
    iterator = input_fn().make_one_shot_iterator()
    next_batch = iterator.get_next()
    
    try:
        while not coord.should_stop():
            batch = sess.run(next_batch)
            print(batch)
    except tf.errors.OutOfRangeError:
        coord.request_stop()
    finally:
        # Wait for all threads to finish
        coord.join(threads)

You can also try setting the environment variable OMP_NUM_THREADS=1 before running your script to limit thread count and avoid conflicts on MacOS.

4. Mismatched Data Shapes #

In rare cases, a shape mismatch between your input data and the model's expected inputs can cause the queue to hang silently (a bug in older TF versions). Double-check that:

• The shape of x matches your model's input layer
• The shape of y matches your model's output layer requirements

If none of these fixes resolve the issue, sharing your full code snippet would help narrow down the exact problem!

内容的提问来源于stack exchange，提问作者samsamoa