TensorFlow 1.8中自定义日志处理器的实现问题

Let's tackle this logging consistency issue between TF 1.7 and 1.8. Here's what you need to know to get your standardized logs working again:

The Problem Recap #

Back in TF 1.7, you could directly tweak tf.logging._logger to hook into TensorFlow's logging system, aligning it with your app's custom setup:

• A unified log format across all your code
• INFO logs sent to stdout, DEBUG logs written to files
• GCP-specific logs pushed to Stackdriver with a searchable |mrtx prefix

Your working TF 1.7 code looked like this (abbreviated):

import tensorflow as tf
import logging as _logging

tf.logging.set_verbosity(tf.logging.INFO)
tf_logging = tf.logging._logger
tf_logging.handlers = [_std_out_handler()]  # Your custom handler
tf_logging.propagate = False

_logging.info("EXPECTED LOG FORMAT")
tf.logging.info("TF LOG FORMAT")

And it produced clean, consistent output:

[INFO |mrtx] 2018-05-18 15:10:07,613 /merantix_core/common/util/logging.py:189 --- EXPECTED LOG FORMAT
[INFO |mrtx] 2018-05-18 15:10:07,614 /usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/tf_logging.py:116 --- TF LOG FORMAT

But in TF 1.8, tf.logging._logger is no longer accessible. When you tried swapping it with tf_logging = _logging.getLogger('tensorflow'), you ended up with mismatched logs:

[INFO |mrtx] 2018-05-18 15:20:34,043 /merantix_core/common/util/logging.py:190 --- EXPECTED LOG FORMAT
INFO:tensorflow:TF LOG FORMAT

You checked tf_logging.py and found no changes between versions, so you suspect strict tf_export enforcement is blocking direct access. Let's fix this.

Working Solutions for TF 1.8 #

1. Take Full Control of the TensorFlow Logger #

The issue with your initial TF 1.8 attempt was that you didn't clear TF's default handlers first. Here's the corrected approach using the Python logging module:

import tensorflow as tf
import logging as _logging

# Set TF's verbosity level first
tf.logging.set_verbosity(tf.logging.INFO)

# Grab the official TensorFlow logger from the logging system
tf_logger = _logging.getLogger('tensorflow')

# Remove all default handlers TF sets up (this was missing before!)
for handler in tf_logger.handlers[:]:
    tf_logger.removeHandler(handler)

# Add your custom handler(s)
tf_logger.addHandler(_std_out_handler())
# Ensure logs don't propagate to the root logger (prevents duplicate logs)
tf_logger.propagate = False

# Test the setup
_logging.info("EXPECTED LOG FORMAT")
tf.logging.info("TF LOG FORMAT")

This will produce the same standardized output as your TF 1.7 setup, since we're fully replacing TF's default logging pipeline with your custom one.

2. Try the Hidden set_logger Method (If Needed) #

Some TF 1.x builds include an undocumented tf.logging.set_logger method that lets you swap out TF's logger entirely with your own pre-configured logger. Here's how to use it:

import tensorflow as tf
import logging as _logging

# Create and configure your custom logger
custom_logger = _logging.getLogger("custom_tf_logger")
custom_logger.setLevel(_logging.INFO)
custom_logger.addHandler(_std_out_handler())
custom_logger.propagate = False

# Replace TF's logger with your custom one
tf.logging.set_logger(custom_logger)

# Test it out
tf.logging.info("TF LOG FORMAT")

Just note that since this isn't officially documented, it might not work across all minor TF 1.x versions, but it's worth trying if the first approach doesn't cover your use case.

3. GCP Stackdriver Tweaks #

For your GCP environment, make sure your custom handler is set up to send logs to Stackdriver with your |mrtx prefix. You can extend the official Stackdriver logging handler (from the google-cloud-logging package) to inject this prefix into every TF log message before it's sent, keeping all your logs searchable and consistent.

Why Your Initial TF 1.8 Attempt Failed #

TF 1.8 locked down direct access to tf.logging._logger (likely via stricter tf_export rules), but the underlying logger is still registered under the name tensorflow in Python's logging system. Your first try added your custom handler but left TF's default stream handler active—so both were writing logs, resulting in the mismatched output you saw.

内容的提问来源于stack exchange，提问作者johnmcs