如何在Boost Log初始化时为日志文件添加表头以支持Pandas分析

需要在日志文件初始化时添加唯一表头，用于描述日志数据格式，方便后续使用Python Pandas进行事后分析。期望的表头及日志示例如下：

YYYYMMDDHHMMSS.MS,TYPE,SIMULATION_TIME,PLATFORM_ID,LATITUDE,LONGITUDE,ELEVATION
20231008091747.442690,[info],0,drone[0],21.281,-157.796,41.7117
20231008091747.448856,[info],0,drone[2],21.2576,-157.799,6.23664
20231008091747.448907,[info],0,drone[3],21.2576,-157.799,6.23664
20231008091747.453667,[info],0,server,21.3,-157.8,196.413
20231008091747.454533,[info],0.033,drone[0],21.281,-157.796,41.6859
20231008091747.454550,[info],0.033,drone[2],21.2576,-157.799,6.24225

当前实现代码 #

日志调用代码

LOG_POSITION << simTime() << "," << getFullName() << "," << latitude << "," << longitude << "," << test;

logger.h 文件

#pragma once

#define BOOST_LOG_DYN_LINK 1
#include <boost/log/expressions.hpp>
#include <boost/log/sources/global_logger_storage.hpp>
#include <boost/log/support/date_time.hpp>
#include <boost/log/trivial.hpp>
#include <boost/log/utility/setup.hpp>

#define LOG_POSITION        BOOST_LOG_SEV(my_logger::get(), boost::log::trivial::info)
#define WARN1               BOOST_LOG_SEV(my_logger::get(), boost::log::trivial::warning)
#define ERROR1              BOOST_LOG_SEV(my_logger::get(), boost::log::trivial::error)

#define SYS_LOGFILE             "../resources/logs/droneMovements.log"

typedef boost::log::sources::severity_logger_mt<boost::log::trivial::severity_level> logger_t;

BOOST_LOG_GLOBAL_LOGGER(my_logger, logger_t)

logger.cc 文件

#include "logger.h"

namespace attrs   = boost::log::attributes;
namespace expr    = boost::log::expressions;
namespace logging = boost::log;

BOOST_LOG_GLOBAL_LOGGER_INIT(my_logger, logger_t)
{
    logger_t lg;

    logging::add_common_attributes();

    logging::add_file_log(
            boost::log::keywords::file_name = SYS_LOGFILE,
            boost::log::keywords::format = (expr::stream
                                            << expr::format_date_time<boost::posix_time::ptime>("TimeStamp", "%Y%m%d%H%M%S.%f,")
                                            << expr::smessage)
    );

    logging::core::get()->set_filter(logging::trivial::severity >= logging::trivial::info);

    return lg;
}

解决方案 #

需要完成两个核心修改：自动写入唯一表头和调整日志格式匹配表头字段。

1. 调整日志格式，补充TYPE字段

当前日志格式缺少表头中的TYPE字段（即日志级别，如[info]），需要在format中添加该字段：

修改logger.cc中的add_file_log格式部分：

logging::add_file_log(
        boost::log::keywords::file_name = SYS_LOGFILE,
        boost::log::keywords::format = (expr::stream
                                        << expr::format_date_time<boost::posix_time::ptime>("TimeStamp", "%Y%m%d%H%M%S.%f,")
                                        << "[" << logging::trivial::severity << "],"
                                        << expr::smessage)
);

这样日志会自动追加[info]/[warning]等级别标识，匹配表头的TYPE字段。

2. 初始化时自动写入表头（仅一次）

通过检查日志文件是否为空，在logger初始化时写入表头，确保表头只出现一次：

首先在logger.cc中添加文件系统相关头文件（需C++17及以上，或改用Boost Filesystem）：

#include <filesystem>
namespace fs = std::filesystem;

然后在BOOST_LOG_GLOBAL_LOGGER_INIT函数中添加表头写入逻辑：

BOOST_LOG_GLOBAL_LOGGER_INIT(my_logger, logger_t)
{
    logger_t lg;

    // 检查日志文件是否不存在或为空
    if (!fs::exists(SYS_LOGFILE) || fs::file_size(SYS_LOGFILE) == 0) {
        std::ofstream log_file(SYS_LOGFILE, std::ios_base::out | std::ios_base::app);
        if (log_file.is_open()) {
            log_file << "YYYYMMDDHHMMSS.MS,TYPE,SIMULATION_TIME,PLATFORM_ID,LATITUDE,LONGITUDE,ELEVATION\n";
            log_file.close();
        }
    }

    logging::add_common_attributes();

    logging::add_file_log(
            boost::log::keywords::file_name = SYS_LOGFILE,
            boost::log::keywords::format = (expr::stream
                                            << expr::format_date_time<boost::posix_time::ptime>("TimeStamp", "%Y%m%d%H%M%S.%f,")
                                            << "[" << logging::trivial::severity << "],"
                                            << expr::smessage)
    );

    logging::core::get()->set_filter(logging::trivial::severity >= logging::trivial::info);

    return lg;
}

3. 清理无用宏定义

删除logger.h中未使用的LOG_POSITION_HEADER宏：

// 移除该行：#define LOG_POSITION_HEADER BOOST_LOG_SEV(my_logger::get(), boost::log::trivial::info)

修改完成后，日志文件初始化时会自动写入表头，后续日志格式完全匹配需求，可直接用Pandas的read_csv函数读取分析：

import pandas as pd
df = pd.read_csv("../resources/logs/droneMovements.log")

内容的提问来源于stack exchange，提问作者abc def