使用tfdbg排查NAN错误时遇AttributeError问题求助

Hey there, I’ve run into this exact issue back when working with older TensorFlow versions and tfdbg, so let’s break down why this happens and how to fix it.

What’s causing the error? #

In TensorFlow 1.2.1, the LocalCLIDebugWrapperSession (the session wrapper tfdbg uses) doesn’t implement the make_callable method. When you call tf.train.start_queue_runners(sess=sess) with the debug-wrapped session, the queue runner code tries to call sess.make_callable() under the hood, which throws the AttributeError you’re seeing.

Solution 1: Manually start queue runners with the raw session #

Instead of relying on tf.train.start_queue_runners to handle the session, you can grab the queue runners directly and start them using the raw, unwrapped TensorFlow Session that’s wrapped by tfdbg. Here’s how to adjust your code:

# After wrapping your session with tfdbg
sess = tfdbg.LocalCLIDebugWrapperSession(raw_sess)

# Replace tf.train.start_queue_runners(sess=sess) with this:
queue_runners = tf.get_collection(tf.GraphKeys.QUEUE_RUNNERS)
for runner in queue_runners:
    # Use sess.raw_session() to access the original TF session
    runner.create_threads(sess.raw_session(), start=True)

The raw_session() method gives you access to the underlying regular Session object, which does have the make_callable method the queue runners need.

Solution 2: Upgrade TensorFlow (if possible) #

This bug was fixed in TensorFlow 1.3 and later versions, where tfdbg wrapper sessions were updated to include the make_callable method. If your environment allows it, upgrading to a newer TF 1.x version (like 1.15, the last 1.x release) will let you use tf.train.start_queue_runners(sess=sess) normally with tfdbg.

Just a quick note: if you can’t upgrade your environment for compatibility reasons, Solution 1 is your reliable go-to fix.

内容的提问来源于stack exchange，提问作者Bro.Zeng