Docker容器日志未显示全部输出消息技术求助

问题说明 #

开发了一款基于TLAPI的量化数据同步脚本，本地运行时自定义print语句正常输出，但部署到Docker容器后，docker logs仅能看到TLAPI库的日志，自定义打印内容完全缺失。已排除镜像缓存问题，纯print测试脚本在Docker中可正常输出，怀疑是TLAPI库修改了stdout导致。

脚本代码 #

import pandas as pd
import requests
import os
from dotenv import load_dotenv
from tradelocker import TLAPI
import schedule
import datetime
import logging
import time
import pytz
import sys

data_path = './data'
est = pytz.timezone('US/Eastern')
time_now = datetime.datetime.now(est)
load_dotenv()

tl = TLAPI(environment = "https://demo.tradelocker.com", 
           username = os.getenv("TL_USER"), 
           password = os.getenv("TL_PASS"), 
           server = os.getenv("TL_SERVER"))

def read_tickers(file_name):
  try:
    with open(file_name, 'r') as file:
      tickers = [line.strip() for line in file.readlines()]
      return tickers
  except FileNotFoundError:
    print(time_now, f"Ticker file {file_name} not found.")
    return []

tickers = read_tickers('tl_tickers.txt')

def initial_download_check():
  if not os.path.isdir(data_path):
    print(time_now, "Data folder absent")
    os.mkdir(data_path)
    return True
  if not os.listdir(data_path):
    print(time_now, "Data path is empty")
    return True

def initial_download(symbol, resolution, length):
  instrument_id = tl.get_instrument_id_from_symbol_name(symbol)
  df = tl.get_price_history(instrument_id, 
                            resolution=resolution, 
                            start_timestamp=0, 
                            end_timestamp=0,
                            lookback_period=length)
  df.set_index("t", inplace=True)
  return df

def latest_bar(symbol, resolution, timestamp):
  instrument_id = tl.get_instrument_id_from_symbol_name(symbol)
  df = tl.get_price_history(instrument_id, 
                            resolution=resolution, 
                            start_timestamp=timestamp, 
                            end_timestamp=0)
  df.set_index("t", inplace=True)
  return df

def is_market_open():
  five_pm = time(17, 0)
  if time.weekday() == 4 and time.time() >= five_pm:
    return False    
  if time.weekday() == 5:
    return False
  if time.weekday() == 6 and time.time() < five_pm:
    return False
  return True

def update_data():
  if is_market_open():
    for ticker in tickers:
      data = pd.read_feather(f"data/{ticker}_1H_10D_tl.feather")
      last_candle = data.tail(1)
      last_timestamp = data.index[-1]
      new_candle = latest_bar(ticker, "1H", int(last_timestamp))
      new_timestamp = new_candle.index[-1]
    if last_candle.equals(new_candle):
      print(time_now, f"No new data found for {ticker}")
    else:
      print(time_now, f"New data found for {ticker}, updating...")
      data = data.append(new_candle)
      data.drop(df.index[0], inplace=True)

print(time_now, "Starting...")

if initial_download_check():
  print(time_now, "Starting initial download...")
  for ticker in tickers:
    print(time_now, f"Downloading initial data for {ticker}...")
    df = initial_download(ticker, '1H', "10D")
    df.to_feather(f"data/{ticker}_1H_10D_tl.feather")
else:
  print(time_now, "Skipping initial download.")

print(time_now, "Scheduling update every 1 hour...")
schedule.every(1).hour.do(update_data)

while True:
  schedule.run_pending()
  time.sleep(1)

本地运行日志 #

[INFO 2024-08-26 12:56:29,617 tradelocker_api.__init__:143]: Initialized TLAPI with log_level='debug' self._LOGGING_FORMAT='[%(levelname)s %(asctime)s %(module)s.%(funcName)s:%(lineno)d]: %(message)s'
[INFO 2024-08-26 12:56:31,108 tradelocker_api._auth_with_password:779]: Successfully fetched authentication tokens
[DEBUG 2024-08-26 12:56:31,108 utils.wrapper:83]: **** CALLING get_all_accounts(<tradelocker.tradelocker_api.TLAPI object at 0x7f043872a1b0>)
[DEBUG 2024-08-26 12:56:32,980 utils.wrapper:94]: **** RETURN from get_all_accounts(<tradelocker.tradelocker_api.TLAPI object at 0x7f043872a1b0>):
[DEBUG 2024-08-26 12:56:32,980 tradelocker_api._set_account_id_and_acc_num:253]: Neither account_id nor acc_num specified, using the first account
[DEBUG 2024-08-26 12:56:32,980 tradelocker_api._set_account_id_and_acc_num:261]: Logging in using the first account, account_id: 000000, acc_num: 1
2024-08-26 14:56:29.616411-04:00 Starting...
2024-08-26 14:56:29.616411-04:00 Skipping initial download.
2024-08-26 14:56:29.616411-04:00 Scheduling update every 1 hour...

Docker容器运行日志 #

[INFO 2024-08-26 18:44:45,995 tradelocker_api.__init__:143]: Initialized TLAPI with log_level='debug' self._LOGGING_FORMAT='[%(levelname)s %(asctime)s %(module)s.%(funcName)s:%(lineno)d]: %(message)s'
[INFO 2024-08-26 18:44:46,287 tradelocker_api._auth_with_password:779]: Successfully fetched authentication tokens
[DEBUG 2024-08-26 18:44:46,287 utils.wrapper:83]: **** CALLING get_all_accounts(<tradelocker.tradelocker_api.TLAPI object at 0x7ee72d9b2330>)
[DEBUG 2024-08-26 18:44:47,901 utils.wrapper:94]: **** RETURN from 
[1 rows x 5 columns]
[DEBUG 2024-08-26 18:44:47,901 tradelocker_api._set_account_id_and_acc_num:253]: Neither account_id nor acc_num specified, using the first account
[DEBUG 2024-08-26 18:44:47,901 tradelocker_api._set_account_id_and_acc_num:261]: Logging in using the first account, account_id: 000000, acc_num: 1

Dockerfile内容 #

FROM python:3

WORKDIR /usr/src/app

COPY reqs_docker_prod.txt ./
RUN pip install --no-cache-dir -r reqs_docker_prod.txt

COPY . .

CMD [ "python", "main.py"]

解决方法 #

1. 禁用Python输出缓冲 #

启动容器时添加环境变量，强制Python实时输出内容：

docker run -e PYTHONUNBUFFERED=1 myusername/app:1.1

2. 替换print为logging模块 #

使用Python标准日志库替代print，确保和TLAPI日志流一致：

# 初始化logging
logging.basicConfig(level=logging.INFO, format='%(asctime)s %(message)s')

# 替换所有print语句
logging.info(f"{time_now} Starting...")
logging.info(f"{time_now} Skipping initial download.")

3. 强制刷新stdout #

在每个print语句后手动刷新输出流：

print(time_now, "Starting...")
sys.stdout.flush()

4. 调整TLAPI日志级别 #

初始化TLAPI时降低日志级别，减少日志干扰：

tl = TLAPI(environment = "https://demo.tradelocker.com", 
           username = os.getenv("TL_USER"), 
           password = os.getenv("TL_PASS"), 
           server = os.getenv("TL_SERVER"),
           log_level='info')

内容的提问来源于stack exchange，提问作者so1120