如何在R或Python中获取原始HTML元素？用BeautifulSoup/rvest实现

开发者阿泽

2026-8-14

获取目标元素完整HTML代码的实现方案

Python + BeautifulSoup

先确保安装所需库：

pip install requests beautifulsoup4

代码示例：

import requests
from bs4 import BeautifulSoup

url = "https://www.amazon.com/Lexani-LXUHP-207-All-Season-Radial-Tire-245/dp/B07FFH8F9V/"

# 模拟浏览器请求头，规避基础反爬
headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36"
}

response = requests.get(url, headers=headers)
response.encoding = response.apparent_encoding

soup = BeautifulSoup(response.text, "html.parser")
target_element = soup.find("span", id="productTitle")

if target_element:
    # prettify()输出格式化后的HTML，str()输出紧凑原始格式
    full_html = target_element.prettify()
    print(full_html)
else:
    print("未找到目标元素")

R + rvest

先安装所需库：

install.packages(c("httr", "rvest"))

代码示例：

library(httr)
library(rvest)

url <- "https://www.amazon.com/Lexani-LXUHP-207-All-Season-Radial-Tire-245/dp/B07FFH8F9V/"

headers <- add_headers(
  "User-Agent" = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36"
)

response <- GET(url, headers)
page <- read_html(response)

target_element <- html_element(page, css = "#productTitle")

if (!is.na(target_element)) {
  full_html <- as.character(target_element)
  cat(full_html)
} else {
  cat("未找到目标元素")
}