Google App Engine/Cloud Shell中Python Requests库HTTPS请求失败问题

我在项目中开发HTML爬取功能，本地运行完全正常，但部署到Google App Engine后出现请求超时问题。已在Google Cloud Shell中使用dig命令解析目标域名，能获取到正确的IP地址，但curl或trace操作仍无法成功。账号已启用计费，未修改过GAE的DNS设置，目标站点采用分块响应（chunked reply）。

相关代码 #

from flask import Flask, render_template, request, send_file, redirect, url_for, Response, send_from_directory
import flask
import requests
import bs4
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
import logging
import os

s = requests.Session()
r = s.get("https://kdseodb.smportkolkata.in/ccuPomsPosWeb/index.jsp")
id1 = r.cookies['JSESSIONID']
r2 = s.get("https://kdseodb.smportkolkata.in/ccuPomsPosWeb/apps/dos/PosCTS.xhtml")
id2 = r2.cookies['oam.Flash.RENDERMAP.TOKEN']
headers = {'Accept': 'application/xml, text/xml, */*; q=0.01', 'Accept-Language': 'en-US,en;q=0.9', 'Connection': 'keep-alive', 'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8', 'Faces-Request': 'partial/ajax', 'Sec-Fetch-Dest': 'empty', 'Sec-Fetch-Mode': 'cors', 'Sec-Fetch-Site': 'same-origin', 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36', 'X-Requested-With': 'XMLHttpRequest', 'Cookie': 'JSESSIONID='+id1+'; oam.Flash.RENDERMAP.TOKEN=-'+id2+''}

payload = 'javax.faces.partial.ajax=true&javax.faces.source=form1%3AbtnSearch&javax.faces.partial.execute=form1&javax.faces.partial.render=form1&form1%3AbtnSearch=form1%3AbtnSearch&form1%3AcomMloCd_focus=&form1%3AcomMloCd_input=NONE&form1%3AtxtCntNo=ABCD1234567&form1%3AtblData_scrollState=0%2C0&form1_SUBMIT=1&javax.faces.ViewState=PCsvYkkMUQAmP0zfu2tj7G5AqM%2BXrDH8%2BaG%2FROvEtWIHJyXX'

r3 = s.post("https://kdseodb.smportkolkata.in/ccuPomsPosWeb/apps/dos/PosCTS.xhtml", cookies=r.cookies, headers=headers, data=payload)
soup = bs4.BeautifulSoup(r3.text)

Google Cloud Shell中dig命令输出 #

解析目标域名正常，但IP反向查询出现NXDOMAIN错误：

dig kdseodb.smportkolkata.in

; <<>> DiG 9.16.33-Debian <<>> kdseodb.smportkolkata.in
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 414
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;kdseodb.smportkolkata.in.      IN      A

;; ANSWER SECTION:
kdseodb.smportkolkata.in. 3600  IN      A       117.242.152.158
kdseodb.smportkolkata.in. 3600  IN      A       203.153.37.38

;; Query time: 6 msec
;; SERVER: 169.254.169.254#53(169.254.169.254)
;; WHEN: Thu Dec 29 01:44:36 UTC 2022
;; MSG SIZE  rcvd: 85


dig 117.242.152.158

; <<>> DiG 9.16.33-Debian <<>> 117.242.152.158
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 29831
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;117.242.152.158.               IN      A

;; AUTHORITY SECTION:
.                       86400   IN      SOA     a.root-servers.net. nstld.verisign-grs.com. 2022122801 1800 900 604800 86400

;; Query time: 5 msec
;; SERVER: 169.254.169.254#53(169.254.169.254)
;; WHEN: Thu Dec 29 01:45:39 UTC 2022
;; MSG SIZE  rcvd: 119


dig 203.153.37.38

; <<>> DiG 9.16.33-Debian <<>> 203.153.37.38
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 32307
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;203.153.37.38.                 IN      A

;; AUTHORITY SECTION:
.                       86400   IN      SOA     a.root-servers.net. nstld.verisign-grs.com. 2022122801 1800 900 604800 86400

;; Query time: 7 msec
;; SERVER: 169.254.169.254#53(169.254.169.254)
;; WHEN: Thu Dec 29 01:46:13 UTC 2022
;; MSG SIZE  rcvd: 117

内容的提问来源于stack exchange，提问作者BejoyM