Airflow源码中LoggingMixin工作机制及使用问题咨询

Great question! Let's break down how Airflow's LoggingMixin works and why your code isn't producing logs right now.

LoggingMixin is a convenience mixin in Airflow that attaches a standard Python logging.Logger instance to your class via the log property. Here's the core logic under the hood:

• It uses a @property getter to create/retrieve a logger named after your class (using self.__class__.__name__). This means each subclass gets its own named logger, making it easy to filter logs by component later.
• No explicit initialization is required in your subclass—just inherit from LoggingMixin, and the log attribute becomes available automatically.
• All standard logging methods (info, debug, warning, etc.) behave exactly like Python's built-in logging module, since it’s just wrapping a regular logger instance.

The most likely culprit here (especially when using LoggingMixin outside an Airflow environment) is Python's default logging behavior:

1. Default Log Level: Python's logging module defaults to the WARNING level. Your code uses self.log.info(), which is a lower severity level—so these logs get filtered out by default.
2. Missing Log Configuration: When running outside Airflow, you don’t get Airflow’s pre-configured logging setup. Airflow normally handles setting log levels, output handlers, and destinations, but in a standalone script, you have to set this up yourself.

Quick side note: You have a typo in your log message (remvoed instead of removed)—it won’t break logging, but fixing it will make your logs clearer!

Here's how to get your logs to appear:

1. Add Basic Logging Configuration #

At the start of your script, configure the logging module to output INFO level logs (or lower):

import logging

# Set up basic logging to output to your console
logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s - %(name)s - %(levelname)s - %(message)s"
)

This tells Python to log all messages at INFO level and above, and formats them with useful context like the logger name (which will be Node for your class) and timestamp.

2. Corrected Working Code #

Here's the full, fixed version of your code:

import logging
from airflow.utils.log.logging_mixin import LoggingMixin

# Configure logging first
logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s - %(name)s - %(levelname)s - %(message)s"
)

class Node(LoggingMixin):
    """Linked list node."""
    def remove(self):
        # Fixed typo in log message
        self.log.info('removed node')

# Test the code
my_node = Node()
my_node.remove()

When you run this, you’ll see a log line like:
2024-05-20 12:34:56,789 - Node - INFO - removed node

3. If Using Inside Airflow #

If you’re running this code as part of an Airflow task, you don’t need the basicConfig call—Airflow’s logging system will automatically handle outputting logs to the task’s log files (and the Airflow UI). Just ensure your task’s log level is set to INFO or lower (Airflow’s default is usually sufficient).

• LoggingMixin is just a lightweight wrapper around Python's standard logging module, giving your class a named logger with minimal setup.
• When using it outside Airflow, you must configure logging yourself to ensure lower-severity logs (like INFO) aren’t filtered out.
• Always double-check log levels and typos when troubleshooting missing logs!

内容的提问来源于stack exchange，提问作者Chris Lan