如何用Python从指定网站URL提取特定图片源并解决403问题？

核心思路 #

目标站点会检测请求头，直接发起请求会被拦截，所以要模拟真实浏览器的请求参数；商品主图结构统一，可通过HTML解析定位，或直接匹配图片CDN域名应急提取。

完整提取代码 #

import requests
from bs4 import BeautifulSoup

def get_product_main_image(url):
    # 模拟浏览器请求头，User-Agent是规避403的关键
    headers = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36'
    }
    
    try:
        response = requests.get(url, headers=headers)
        response.raise_for_status()  # 捕获HTTP状态码错误
        
        soup = BeautifulSoup(response.text, 'html.parser')
        # 定位主图容器，适配全站商品页的统一结构
        main_img_box = soup.find('div', class_='product__main-image')
        if main_img_box:
            img_tag = main_img_box.find('img')
            if img_tag and 'src' in img_tag.attrs:
                return img_tag['src']
            return "未找到图片的src属性"
        return "未找到主图容器节点"
    except requests.exceptions.HTTPError as e:
        return f"HTTP请求错误: {e}"
    except Exception as e:
        return f"其他异常: {e}"

# 测试示例
test_urls = [
    "https://store.creality.com/ca/products/halot-mage-8k-resin-3d-printer",
    "https://store.creality.com/ca/products/uw-02-washing-curing-machine"
]

for url in test_urls:
    print(f"URL: {url}\n图片地址: {get_product_main_image(url)}\n")

临时应急方案：提取图片所在代码行 #

如果HTML结构临时变动，可直接匹配图片CDN域名快速定位：

import requests

def find_img_source_line(url):
    headers = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36'
    }
    try:
        response = requests.get(url, headers=headers)
        response.raise_for_status()
        # 匹配包含目标CDN域名的代码行
        for line in response.text.splitlines():
            if 'img.fantaskycdn.com' in line:
                return line.strip()
        return "未找到包含目标CDN的代码行"
    except Exception as e:
        return f"错误: {e}"

# 测试
print(find_img_source_line(test_urls[0]))

关键注意事项 #

• 403错误的本质是请求头被识别为非浏览器请求，必须携带真实的User-Agent，必要时可补充Referer、Accept等字段强化模拟；
• 目标站点商品页的主图结构统一，product__main-image类选择器可适配全站所有商品页面；
• 临时方案通过CDN域名匹配，适合HTML结构临时调整时的快速应急提取。

内容的提问来源于stack exchange，提问作者Brayden Baines