Google Cloud Composer Airflow运行Python调用API报Errno 101连接超时如何解决

我在使用Google Cloud Composer提供的Airflow运行Python代码调用API时，持续出现[Errno 101] Connection timed out报错。

关联代码 #

r1 = requests.post(
    url1,
    params=request1_params,
    headers=request1_headers
)

a_token = r1.json().get('access_token')

触发DAG后的错误日志 #

[2021-10-04 01:21:33,056] {taskinstance.py:1152} ERROR - HTTPSConnectionPool(host='xxx.com', port=443): Max retries exceeded with url: /sf/xxx/token?grant_type=password&username=???%40???&password=???%2???&client_id=????&client_secret=???? (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f43a2958860>: Failed to establish a new connection: [Errno 110] Connection timed out',))
Traceback (most recent call last):
File "/opt/python3.6/lib/python3.6/site-packages/urllib3/connection.py", line 170, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw
File "/opt/python3.6/lib/python3.6/site-packages/urllib3/util/connection.py", line 96, in create_connection
raise err
File "/opt/python3.6/lib/python3.6/site-packages/urllib3/util/connection.py", line 86, in create_connection
sock.connect(sa)
TimeoutError: [Errno 110] Connection timed out

问题定位与解决 #

最初排查时猜测是请求需要配置代理，最终定位问题根源为当前Airflow使用的私有网络缺少NAT配置，完成NAT配置并安装对应依赖库后，API请求已可正常发送。

内容的提问来源于stack exchange，提问作者Yohandy