Windows下Firefox监测下载文件致重复的Python解决求助
问题描述
我正在编写脚本,需通过访问URL下载文件。因需登录网站,放弃带登录态的curl请求方案,改为在浏览器打开URL并监测下载文件夹。
此前已用bash实现可用版本,现改用Python重写时遇到问题:使用Firefox时,监测文件大小会导致下载文件夹生成两个文件——空的原文件与带(1)后缀的实际文件,使脚本陷入死循环。
我的bash脚本通过stat规避了问题,但Python中使用os.path.getsize或os.stat均无法解决,尝试文件排他锁也因跨平台问题失败。Edge无此问题,且该现象在Windows下复现无差别。
附原bash及Python代码:
原Bash代码
#! /bin/bash get_latest_csv_in_downloads() { find ~/Downloads -maxdepth 1 -iname '*.csv' -printf '%B@ %f\0' | sort -znr | head -zn 1 | cut -zd ' ' -f 2- | head -c -1 } url="$1" initial_file="$(get_latest_csv_in_downloads)" # Open the file in the browser which will initiate a download. python -m webbrowser "$url" > /dev/null # We'll try to obtain the most recent file in the downloads folder, # until it's a different one than before we started downloading. while latest_file="$(get_latest_csv_in_downloads)"; [[ "$initial_file" == "$latest_file" ]]; do :; done # When the file is created it's sometimes empty for a bit. # At some point it jumps to being fully written, without any inbetween. # So this waits for the file size to not be zero. # NOTE: [[ ! -s "..." ]] would be a lot nicer, # but for some reason it sometimes creates a copy of the file and messes everything up. while (( "$(stat --format="%s" ~/Downloads/"$latest_file")" == 0 )); do :; done echo "got the file $latest_file"
原Python代码
#! python import os import webbrowser import sys import glob def get_latest_csv_in_downloads(): files_in_downloads = glob.glob(os.path.expanduser(os.path.join(os.path.expanduser('~'), 'Downloads', '*.csv'))) latest_file = max(files_in_downloads, key=os.path.getctime, default=None) return latest_file url = sys.argv[1] initial_file = get_latest_csv_in_downloads() webbrowser.open(url) while True: latest_file = get_latest_csv_in_downloads() if initial_file != latest_file: break while os.path.getsize(latest_file) == 0: pass print(f'got the file {latest_file}')
原因分析
Firefox的下载机制是先创建一个空的占位文件,待下载完成后再写入完整内容。如果这个占位文件被外部进程(比如你的Python脚本)以极高频率访问(比如无休眠的死循环调用os.path.getsize),Firefox会判定原文件被占用,无法正常写入,于是自动重命名原空文件,创建带(1)后缀的新文件继续下载。
而你的Bash脚本中,虽然也检查文件大小,但Bash的循环执行效率远低于Python的死循环,访问频率更低,没有触发Firefox的文件占用检测逻辑,所以不会出现重复文件的问题。
解决方案
核心思路是降低文件访问频率,同时确认文件下载完成(而非仅大小非零),避免触发Firefox的文件占用保护机制。具体实现如下:
- 给轮询逻辑添加短暂休眠(比如0.5秒),降低对文件的访问频率
- 监测文件的修改时间和大小,连续多次检查无变化时,判定下载完成
- 处理可能出现的带
(1)后缀的文件,确保能识别到实际下载的文件
修改后的Python代码
#! python import os import webbrowser import sys import glob import time def get_latest_csv_in_downloads(): downloads_dir = os.path.expanduser(os.path.join('~', 'Downloads')) # 匹配所有csv文件,包括带(1)后缀的 files_in_downloads = glob.glob(os.path.join(downloads_dir, '*.csv')) if not files_in_downloads: return None # 按修改时间排序,取最新的 latest_file = max(files_in_downloads, key=lambda x: os.path.getmtime(x)) return latest_file url = sys.argv[1] initial_file = get_latest_csv_in_downloads() webbrowser.open(url) # 等待新文件出现 while True: latest_file = get_latest_csv_in_downloads() if latest_file != initial_file: break time.sleep(0.5) # 降低轮询频率 # 等待文件下载完成:连续2次检查大小和修改时间都不变 prev_size = -1 prev_mtime = -1 stable_count = 0 while stable_count < 2: current_size = os.path.getsize(latest_file) current_mtime = os.path.getmtime(latest_file) if current_size == prev_size and current_mtime == prev_mtime: stable_count += 1 else: stable_count = 0 prev_size = current_size prev_mtime = current_mtime time.sleep(0.5) # 检查是否有带(1)的实际文件(如果原文件是空的) downloads_dir = os.path.dirname(latest_file) filename = os.path.basename(latest_file) name, ext = os.path.splitext(filename) possible_actual_file = os.path.join(downloads_dir, f"{name}(1){ext}") if os.path.exists(possible_actual_file) and os.path.getsize(latest_file) == 0: latest_file = possible_actual_file print(f'got the file {os.path.basename(latest_file)}')
补充说明
- 加入
time.sleep(0.5)后,脚本对文件的访问频率大幅降低,不会触发Firefox的文件占用保护 - 通过连续两次检查文件大小和修改时间不变,能更准确判断下载完成(避免文件大小跳变时的误判)
- 增加了对带
(1)后缀文件的处理逻辑,即使出现重命名也能正确识别实际下载的文件
内容的提问来源于stack exchange,提问作者Verpous
相关产品推荐
相关产品推荐

