使用requests爬取bandit.camp页面标题与浏览器显示不一致的问题
问题
使用requests和BeautifulSoup4爬取网页标题时出现不一致:浏览器显示的标题为"Crate Battles :: bandit.camp - The true immersive Rust gaming experience",但通过requests.get()获取到的标题却是"bandit.camp - The true immersive Rust gaming experience"。
首次尝试
import requests from bs4 import BeautifulSoup url = 'https://bandit.camp/crate-battles' s = requests.Session() html = s.get(url) soup = BeautifulSoup(html.text, 'html.parser') elements = soup.find_all('title') for element in elements: print(element) # <title>bandit.camp - The true immersive Rust gaming experience</title>
第二次尝试
尝试在第一次请求后等待5秒,再次发起请求,但返回的标题依旧没有变化。
import requests from bs4 import BeautifulSoup from time import sleep url = 'https://bandit.camp/crate-battles' s = requests.Session() s.get(url) sleep(5) html = s.get(url) soup = BeautifulSoup(html.text, 'html.parser') elements = soup.find_all('title') for element in elements: print(element) # <title>bandit.camp - The true immersive Rust gaming experience</title>
内容的提问来源于stack exchange,提问作者Flewitt
相关产品推荐
相关产品推荐

