Airflow设备统计DAG失败,时间格式相关异常求助排查方案
问题求助:Airflow流水线失败导致无法生成固件报告
错误日志1:时间格式不匹配ValueError
Traceback (most recent call last): File "/usr/local/lib/python3.7/runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "/usr/local/lib/python3.7/runpy.py", line 85, in _run_code exec(code, run_globals) File "/app/fw_daily/refresh_cache.py", line 849, in <module> sys.exit(main()) File "/app/fw_daily/refresh_cache.py", line 817, in main use_dd=options.use_dd, File "/app/fw_daily/refresh_cache.py", line 661, in refresh_fw query_data(refresh_cache, from_dt, to_dt, table_list) File "/app/fw_daily/refresh_cache.py", line 491, in query_data query_table(refresh_cache, query_args, table, from_dt, to_dt) File "/app/fw_daily/refresh_cache.py", line 384, in query_table last_query_time = get_last_query_time(refresh_cache, last_query_fname) File "/app/fw_daily/refresh_cache.py", line 217, in get_last_query_time last_query_time = get_time_from_file(refresh_cache, last_query_fname) File "/app/fw_daily/refresh_cache.py", line 209, in get_time_from_file "%Y%m%d", File "/usr/local/lib/python3.7/_strptime.py", line 577, in _strptime_datetime tt, fraction, gmtoff_fraction = _strptime(data_string, format) File "/usr/local/lib/python3.7/_strptime.py", line 359, in _strptime (data_string, format)) ValueError: time data ' ' does not match format '%Y%m%d'
错误日志2:类型比较TypeError
Traceback (most recent call last): File "/usr/local/lib/python3.7/runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "/usr/local/lib/python3.7/runpy.py", line 85, in _run_code exec(code, run_globals) File "/app/fw_daily/refresh_cache.py", line 858, in <module> sys.exit(main()) File "/app/fw_daily/refresh_cache.py", line 826, in main use_dd=options.use_dd, File "/app/fw_daily/refresh_cache.py", line 670, in refresh_fw query_data(refresh_cache, from_dt, to_dt, table_list) File "/app/fw_daily/refresh_cache.py", line 500, in query_data query_table(refresh_cache, query_args, table, from_dt, to_dt) File "/app/fw_daily/refresh_cache.py", line 399, in query_table last_query_time = from_dt if from_dt > last_query_time else last_query_time TypeError: can't compare datetime.datetime to datetime.date
我运行的用于5类设备统计的Airflow流水线已失败,相关截图为Airflow DAG失败截图。由此导致我无法生成聚合设备统计所需的固件报告。通过查看query_fw DAG的回溯错误日志,我发现两个核心问题:
- 时间数据与格式不匹配的ValueError
- datetime类型与date类型无法比较的TypeError
我已尝试修改时间戳格式并重跑Python脚本,但问题仍未解决,恳请提供可行的解决方案建议,谢谢!
内容的提问来源于stack exchange,提问作者nandos96
相关产品推荐
相关产品推荐

