请求协助解决访问legislation.gov.au时的ssl.SSLError握手失败问题

我尝试了多种配置组合来实现功能，但毫无成效。以下是我的最新代码，恳请协助。
目标网站：https://www.legislation.gov.au/
持续遇到的错误：
ssl.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:1007)

我运行了命令openssl s_client -connect www.legislation.gov.au:443 -tls1_2检查SSL设置，但问题依旧。请问有人能成功对该网站发起基础请求而不触发这个异常吗？

我的代码： #

import httpx
from bs4 import BeautifulSoup
import asyncio
import ssl
import socket
import sys

async def fetch_data():
    sitemap_url = "https://www.legislation.gov.au/Browse/ByTitle/LegislativeInstruments/InForce/0/0/All/"
    headers = {
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.61 Safari/537.36"
    }
    
    ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2)
    ssl_context.options |= ssl.OP_NO_TLSv1_1  # Only TLS 1.1 allowed

    # Wrap the socket with an SSL context
    socketHandler = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    socketWraped = ssl_context.wrap_socket(socketHandler, server_hostname='www.legislation.gov.au')
    socketWraped.connect(('www.legislation.gov.au', 443))

    async with httpx.AsyncClient() as client:
        response = await client.get(sitemap_url, headers=headers, timeout=10, socket=socketWraped)
        response.raise_for_status()
        html_content = response.text

    soup = BeautifulSoup(html_content, "html.parser")
    
    # Your parsing and processing code here
    print("Successfully fetched and parsed data.")

if __name__ == "__main__":
    asyncio.run(fetch_data())

解决方案 #

你的代码存在两个核心问题：

1. 同步socket与异步客户端不兼容：手动创建的同步socket无法适配httpx的异步客户端，这会导致SSL握手流程异常。httpx异步客户端需要使用异步socket，手动传入同步socket会破坏其内部连接管理逻辑。

2. SSL上下文配置矛盾：代码注释写着“Only TLS 1.1 allowed”，但实际使用了PROTOCOL_TLSv1_2（指定TLS1.2），还添加了OP_NO_TLSv1_1禁用TLS1.1，这就导致没有可用的协议版本，直接触发握手失败。

修正后的代码（方式一：正确配置SSL上下文）

import httpx
from bs4 import BeautifulSoup
import asyncio
import ssl

async def fetch_data():
    sitemap_url = "https://www.legislation.gov.au/Browse/ByTitle/LegislativeInstruments/InForce/0/0/All/"
    headers = {
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.61 Safari/537.36"
    }
    
    # 配置仅允许TLS1.2，禁用更低版本协议
    ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2)
    ssl_context.options |= ssl.OP_NO_TLSv1 | ssl.OP_NO_TLSv1_1

    # 让httpx异步客户端自行管理连接，传入配置好的SSL上下文
    async with httpx.AsyncClient(verify=ssl_context) as client:
        response = await client.get(sitemap_url, headers=headers, timeout=10)
        response.raise_for_status()
        html_content = response.text

    soup = BeautifulSoup(html_content, "html.parser")
    
    # 这里添加你的解析处理逻辑
    print("Successfully fetched and parsed data.")

if __name__ == "__main__":
    asyncio.run(fetch_data())

简化方案（方式二：使用httpx默认配置）

现代版本的httpx默认支持TLS1.2及以上协议，可能已经满足目标网站的要求，直接去掉手动socket和SSL上下文配置即可：

import httpx
from bs4 import BeautifulSoup
import asyncio

async def fetch_data():
    sitemap_url = "https://www.legislation.gov.au/Browse/ByTitle/LegislativeInstruments/InForce/0/0/All/"
    headers = {
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.61 Safari/537.36"
    }

    async with httpx.AsyncClient() as client:
        response = await client.get(sitemap_url, headers=headers, timeout=10)
        response.raise_for_status()
        html_content = response.text

    soup = BeautifulSoup(html_content, "html.parser")
    
    print("Successfully fetched and parsed data.")

if __name__ == "__main__":
    asyncio.run(fetch_data())

内容的提问来源于stack exchange，提问作者user2090437