如何阻止itunes-app-scraper的get_app_details方法输出URL到控制台?
问题:itunes-app-scraper-dmi爬取应用详情时自动输出请求URL到控制台
测试代码
!pip install itunes-app-scraper-dmi from itunes_app_scraper.scraper import AppStoreScraper print(" ") bundleID = "com.marmalade.monopoly" appDetails = AppStoreScraper().get_app_details(bundleID, country="us") version = appDetails['version'] url = appDetails['trackViewUrl'][:-5] print("\nVersion:", version) print("App URL:", url)
控制台输出
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/ Requirement already satisfied: itunes-app-scraper-dmi in /usr/local/lib/python3.7/dist-packages (0.9.4) https://itunes.apple.com/lookup?bundleId=com.marmalade.monopoly&country=us&entity=software Version: 1.7.15 App URL: https://apps.apple.com/us/app/monopoly-classic-board-game/id1477966166
问题说明:将该逻辑整合到批量爬取代码时,自动输出的URL会刷屏控制台,仅需提取应用信息存入变量,需阻止该URL输出。查看get_app_details源码未发现print语句,怀疑是requests.get相关操作导致输出。
解决方法
方法1:调整requests库日志级别
该URL大概率是requests库的调试日志输出,通过调整日志级别可屏蔽:
import logging # 禁用requests及urllib3的调试日志 logging.getLogger("requests").setLevel(logging.WARNING) logging.getLogger("urllib3").setLevel(logging.WARNING) from itunes_app_scraper.scraper import AppStoreScraper # 后续爬取代码不变 print(" ") bundleID = "com.marmalade.monopoly" appDetails = AppStoreScraper().get_app_details(bundleID, country="us") version = appDetails['version'] url = appDetails['trackViewUrl'][:-5] print("\nVersion:", version) print("App URL:", url)
将日志调整代码放在导入AppStoreScraper之前执行,即可屏蔽请求URL的日志输出。
方法2:临时重定向标准输出
若方法1无效,可临时将标准输出重定向到空设备,完成爬取后再恢复:
import sys from contextlib import redirect_stdout from itunes_app_scraper.scraper import AppStoreScraper print(" ") bundleID = "com.marmalade.monopoly" # 临时屏蔽标准输出(Linux/macOS用/dev/null,Windows用NUL) with redirect_stdout(open('/dev/null', 'w')): appDetails = AppStoreScraper().get_app_details(bundleID, country="us") version = appDetails['version'] url = appDetails['trackViewUrl'][:-5] # 恢复输出后打印需要的内容 print("\nVersion:", version) print("App URL:", url)
方法3:修改库源码(不推荐)
若上述方法均无效,可直接修改库的scraper.py文件,查找是否有print(url)或logging.debug(url)这类输出代码并注释。但该方法会影响库的全局使用,且库更新后修改会失效,仅作为临时应急方案。
内容的提问来源于stack exchange,提问作者RCube123
相关产品推荐
相关产品推荐

