本地正常的Python爬虫部署Azure Linux后遇SSL协议EOF错误求助

以下Python爬虫代码在本地运行正常，但部署至Azure Linux系统时出现异常：

from bs4 import BeautifulSoup
from urllib.request import Request, urlopen
from urllib.parse import quote

url = (f"https://www.wikipedia.org/")
req = Request(url, headers={'User-Agent': 'Mozilla/5.0'})
webpage = urlopen(req).read()
soup = BeautifulSoup(webpage, "html.parser")

panel_data = soup.find('div')

print(panel_data)

在Azure Linux系统中运行时收到如下错误：

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 2190, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1486, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1484, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1469, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "/flask-doker/app.py", line 41, in scrap
    content = scrappy()
  File "/flask-doker/scrapper.py", line 10, in scrappy
    webpage = urlopen(req).read()
  File "/usr/local/lib/python3.10/urllib/request.py", line 216, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/local/lib/python3.10/urllib/request.py", line 519, in open
  File "/usr/local/lib/python3.10/urllib/request.py", line 536, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/usr/local/lib/python3.10/urllib/request.py", line 496, in _call_chain
    result = func(*args)
  File "/usr/local/lib/python3.10/urllib/request.py", line 1391, in https_open
    return self.do_open(http.client.HTTPSConnection, req,
  File "/usr/local/lib/python3.10/urllib/request.py", line 1351, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error EOF occurred in violation of protocol (_ssl.c:1007)>

该错误由SSL协议握手异常导致，常见于Azure环境的SSL配置或默认库协议支持问题，可通过以下方式解决：

• 指定SSL协议版本
  修改urlopen调用，手动指定兼容的SSL协议，示例代码：

  import ssl
  # 其余原有代码不变
  ctx = ssl.create_default_context()
  ctx.set_ciphers('DEFAULT@SECLEVEL=1')
  ctx.options |= ssl.OP_NO_TLSv1 | ssl.OP_NO_TLSv1_1  # 禁用老旧协议
  webpage = urlopen(req, context=ctx).read()
  

• 改用requests库替代urllib
  requests库对SSL处理更友好，默认兼容更多场景，安装requests后替换代码：

  from bs4 import BeautifulSoup
  import requests
  
  url = "https://www.wikipedia.org/"
  headers = {'User-Agent': 'Mozilla/5.0'}
  response = requests.get(url, headers=headers)
  soup = BeautifulSoup(response.text, "html.parser")
  
  panel_data = soup.find('div')
  print(panel_data)
  

• 检查Azure网络与证书配置

  • 确认Azure Linux虚拟机/容器的出站规则未限制443端口的HTTPS流量
  • 若使用Azure App Service，检查VNet集成或防火墙规则，确保允许访问外部HTTPS站点
  • 更新系统CA证书，执行命令：

    sudo apt-get update && sudo apt-get install -y ca-certificates
    

内容的提问来源于stack exchange，提问作者akshay soni