Apache NiFi自定义处理器测试时Debug日志不显示的解决方法

问题简述 #

开发Apache NiFi自定义处理器，功能正常，但测试时仅能显示INFO级别日志，DEBUG级别日志无法输出。

当前日志输出 #

[pool-1-thread-1] INFO com.fet.processors.mdms.SampleProcessor - SampleProcessor[id=12c89263-c3a3-4758-977b-64a7593e103c] BEGIN to TEST XXXXXXXXXXXXXXXXXXXXX

测试运行方式 #

相关代码与配置 #

SampleProcessor处理器代码 #

import org.helper.FlowFileHelper;
import org.apache.nifi.annotation.behavior.*;
import org.apache.nifi.annotation.documentation.*;
import org.apache.nifi.components.PropertyDescriptor;
import org.apache.nifi.flowfile.FlowFile;
import org.apache.nifi.logging.ComponentLog;
import org.apache.nifi.processor.*;

@Tags({"Test logger message"})
@CapabilityDescription("Sample processor for testing log message")
@SeeAlso([])
@ReadsAttributes([@ReadsAttribute(attribute="", description="")])
@WritesAttributes([@WritesAttribute(attribute="", description="")])
public class SampleProcessor extends AbstractProcessor {
    public static final Relationship REL_SUCCESS = new Relationship.Builder()
            .name("success")
            .description("Test logger message")
            .build();
    private List<PropertyDescriptor> descriptors = new ArrayList<>();

    private Set<Relationship> relationships = new HashSet<>();
    private ComponentLog log;

    @Override
    protected void init(final ProcessorInitializationContext context) {
        log = getLogger();
        relationships.add(REL_SUCCESS); // 注：原代码中MY_RELATIONSHIP为笔误，需修正为REL_SUCCESS
        relationships = Collections.unmodifiableSet(relationships);
    }

    @Override
    public Set<Relationship> getRelationships() {
        return this.relationships;
    }

    @Override
    public void onTrigger(final ProcessContext context, final ProcessSession session) {
        FlowFile flowFile = session.get();
        if (flowFile == null) return;
        String content   = FlowFileHelper.readFlowfileContent(flowFile, session);
        log.info("BEGIN to TEST XXXXXXXXXXXXXXXXXXXXX");
        log.debug("TEST =================================");
        log.debug("content is {}", content);
        session.transfer(flowFile, REL_SUCCESS);
    }
}

src/main/resources/logback.xml配置 #

<configuration>
    <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
        <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
            <pattern>%-4relative [%thread] %-5level %logger{5}.%method:%line - %message%n</pattern>
        </encoder>
    </appender>

    <logger name="org.apache.nifi" level="WARN">
        <appender-ref ref="CONSOLE" />
    </logger>

    <logger name="com.fet.processors" level="TRACE" additivity="false">
        <appender-ref ref="CONSOLE" />
    </logger>

    <root level="WARN">
        <appender-ref ref="CONSOLE" />
    </root>
</configuration>

测试代码 #

import org.apache.nifi.util.TestRunner
import org.apache.nifi.util.TestRunners
import org.junit.jupiter.api.BeforeEach
import org.junit.jupiter.api.Test

class TestSampleProcessor {
    private TestRunner testRunner;
    @BeforeEach
    public void init() {
        testRunner = TestRunners.newTestRunner(SampleProcessor.class);
    }

    @Test
    public void testLogDebugMessage() {
        testRunner.enqueue("This is data of flowfile");
        testRunner.run();
    }
}

src/test/resources/logback.xml配置 #

<configuration>
    <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
        <encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
            <pattern>%-4relative [%thread] %-5level %logger{5}.%method:%line - %message%n</pattern>
        </encoder>
    </appender>

    <logger name="org.apache.nifi" level="INFO">
        <appender-ref ref="CONSOLE" />
    </logger>

    <logger name="com.fet.processors" level="TRACE" additivity="false">
        <appender-ref ref="CONSOLE" />
    </logger>

    <root level="INFO">
        <appender-ref ref="CONSOLE" />
    </root>
</configuration>

解决方案 #

核心问题是NiFi的TestRunner默认会强制将处理器的日志级别设置为INFO，覆盖logback的配置。需要在测试代码中手动调整日志级别：

方法1：通过TestRunner直接设置 #

修改测试类的init方法，添加日志级别设置：

@BeforeEach
public void init() {
    testRunner = TestRunners.newTestRunner(SampleProcessor.class);
    // 将处理器日志级别设置为DEBUG
    testRunner.getLogger().setLevel(org.apache.nifi.logging.LogLevel.DEBUG);
}

方法2：直接操作SLF4J Logger #

如果需要更细粒度的控制，可直接获取处理器对应的Logger并调整级别：

import org.slf4j.LoggerFactory;
import ch.qos.logback.classic.Logger;
import ch.qos.logback.classic.Level;

@BeforeEach
public void init() {
    testRunner = TestRunners.newTestRunner(SampleProcessor.class);
    // 获取自定义处理器的Logger并设置为DEBUG级别
    Logger logger = (Logger) LoggerFactory.getLogger(SampleProcessor.class);
    logger.setLevel(Level.DEBUG);
}

另外需修正处理器代码中的笔误：原init方法里的relationships.add(MY_RELATIONSHIP)应改为relationships.add(REL_SUCCESS)，否则运行时会抛出未定义变量异常。

内容的提问来源于stack exchange，提问作者wureka