在Kubernetes Python Pod中，如何实现PVC新增文件时仅发一次邮件通知？

问题描述 #

在Kubernetes中监控Persistent Volume Claim（PVC）内的新增文件并发送邮件通知，现有Python代码存在两个问题：

• 同一新增文件会收到重复邮件通知
• Pod故障重启后，内存中的已处理文件列表被清空，所有文件会被当作新文件再次发送通知

原代码如下：

import os
import time
import smtplib
from email.mime.text import MIMEText

# Email configuration
smtp_server = 'V'
smtp_port = 25
sender_email = 'X'
receiver_email = 'Y'

# Set up the directory to monitor
dir_to_monitor = '/mnt/'

# Set up a list to store processed files
processed_files = []

# Function to recursively explore directories and detect new files
def explore_directory(directory):
    for root, dirs, files in os.walk(directory):
        for file in files:
            file_path = os.path.join(root, file)
            if file_path not in processed_files:
                # New file detected
                processed_files.append(file_path)
                print("New file:", file_path)
                send_notification_email(file_path)

# Function to send email notification
def send_notification_email(file_path):
    subject = 'New file arrived'
    message = f"A new file has arrived:\n\nDirectory: {os.path.dirname(file_path)}\nFile: {file_path}"

    msg = MIMEText(message)
    msg['Subject'] = subject
    msg['From'] = sender_email
    msg['To'] = receiver_email

    with smtplib.SMTP(smtp_server, smtp_port) as server:
        server.starttls()
        server.sendmail(sender_email, receiver_email, msg.as_string())

# Continuously monitor the directory for new files
while True:
    explore_directory(dir_to_monitor)
    time.sleep(5)

解决方案 #

核心思路是将已处理文件的记录持久化到PVC中，避免内存存储导致的重启丢失，同时优化检测逻辑避免重复通知。

1. 持久化已处理文件记录 #

在PVC挂载的目录下创建一个隐藏文件（比如/mnt/.processed_files），用来存储所有已发送过通知的文件路径。Pod重启时，先读取该文件恢复已处理记录。

2. 修改代码逻辑 #

• 使用集合存储已处理文件，提升查询效率（集合的in操作是O(1)，远快于列表的O(n)）
• 启动时加载持久化的记录文件
• 每次处理新文件后，将路径追加写入记录文件
• 排除记录文件本身，避免监控到自身的修改

修改后的代码：

import os
import time
import smtplib
from email.mime.text import MIMEText

# Email configuration
smtp_server = 'V'
smtp_port = 25
sender_email = 'X'
receiver_email = 'Y'

# Set up the directory to monitor
dir_to_monitor = '/mnt/'
# 持久化记录文件路径，存储在PVC中
processed_record_file = os.path.join(dir_to_monitor, '.processed_files')

# 初始化已处理文件集合
processed_files = set()

def load_processed_files():
    """从持久化文件加载已处理记录"""
    if os.path.exists(processed_record_file):
        with open(processed_record_file, 'r') as f:
            for line in f:
                file_path = line.strip()
                if file_path:
                    processed_files.add(file_path)

def save_processed_file(file_path):
    """将新处理的文件路径追加到记录文件"""
    with open(processed_record_file, 'a') as f:
        f.write(f"{file_path}\n")

# Function to recursively explore directories and detect new files
def explore_directory(directory):
    for root, dirs, files in os.walk(directory):
        for file in files:
            file_path = os.path.join(root, file)
            # 跳过记录文件本身
            if file_path == processed_record_file:
                continue
            if file_path not in processed_files:
                # New file detected
                processed_files.add(file_path)
                save_processed_file(file_path)
                print("New file:", file_path)
                send_notification_email(file_path)

# Function to send email notification
def send_notification_email(file_path):
    subject = 'New file arrived'
    message = f"A new file has arrived:\n\nDirectory: {os.path.dirname(file_path)}\nFile: {file_path}"

    msg = MIMEText(message)
    msg['Subject'] = subject
    msg['From'] = sender_email
    msg['To'] = receiver_email

    with smtplib.SMTP(smtp_server, smtp_port) as server:
        server.starttls()
        server.sendmail(sender_email, receiver_email, msg.as_string())

# 启动时加载已处理记录
load_processed_files()

# Continuously monitor the directory for new files
while True:
    explore_directory(dir_to_monitor)
    time.sleep(5)

3. Kubernetes部署注意事项 #

• 确保Pod挂载的PVC拥有读写权限，避免无法创建或写入记录文件
• 确认Pod的运行用户对/mnt/目录有写入权限（可通过securityContext配置用户ID或权限）
• 避免PVC被其他Pod或外部操作误删除记录文件.processed_files

内容的提问来源于stack exchange，提问作者jos97