如何解决asyncio.StreamReader抛出OpenSSL.SSL.WantReadError问题

环境与初始实现 #

通过以下代码包装已创建的TLS套接字，生成asyncio.StreamReader和StreamWriter：

self.reader, self.writer = await asyncio.open_connection(sock=self.sock)

最初使用如下函数从套接字读取指定长度的数据：

async def _receive_data(reader: asyncio.StreamReader, sz):
    pos = 0
    data = None
    while pos < sz:
        chunk = await reader.read(sz - pos)
        if pos == 0:
            data = chunk
        else:
            data += chunk
        pos += len(chunk)

    return data

问题现象 #

当尝试读取服务器返回的较大响应（约300KB）时，触发OpenSSL.SSL.WantReadError，推测是服务器生成响应需要时间导致，但无法从该错误中恢复。

修复尝试与结果 #

将_receive_data函数修改为捕获WantReadError并尝试重试：

async def _receive_data(reader: asyncio.StreamReader, sz):
    pos = 0
    data = b""

    while pos < sz:
        try:
            chunk = await reader.read(sz - pos)
        except SSL.WantReadError:
            print(
                f"SSL.WantReadError: Waiting for more data. Received {pos} bytes of {sz} bytes."
            )
            print("reader._exception: ", repr(reader))
            reader.set_exception(None)
            await asyncio.sleep(0.1)
            continue

        if pos == 0:
            data = chunk
        else:
            data += chunk

        pos += len(chunk)

    return data

执行后得到如下输出：

SSL.WantReadError: Waiting for more data. Received 0 bytes of 8 bytes.
reader._exception: <StreamReader exception=WantReadError() transport=<_SelectorSocketTransport closed fd=15>>
SSL.WantReadError: Waiting for more data. Received 163832 bytes of 303525 bytes.
reader._exception: <StreamReader exception=WantReadError() transport=<_SelectorSocketTransport closed fd=17>>

程序打印两次日志后挂起，推测原因是reader关联的套接字已关闭（如输出中closed fd=15/closed fd=17所示）。

OpenSSL.SSL.WantReadError文档说明 #

The operation did not complete; the same I/O method should be called again later, with the same arguments. Any I/O method can lead to this since new handshakes can occur at any time.

The wanted read is for dirty data sent over the network, not the clean data inside the tunnel. For a socket based SSL connection, read means data coming at us over the network. Until that read succeeds, the attempted OpenSSL.SSL.Connection.recv(), OpenSSL.SSL.Connection.send(), or OpenSSL.SSL.Connection.do_handshake() is prevented or incomplete. You probably want to select() on the socket before trying again.

当前困境 #

不清楚如何将上述文档说明应用到asyncio.Streams场景中——理想情况是在包装的套接字上重新执行相同请求，但Streams封装了底层细节。曾尝试保存未完成的读取长度（即sz - pos），使用同一套接字重新创建reader/writer，但调用asyncio.open_connection()时出现[Errno 9] Bad file descriptor错误。目前仅能考虑使用新套接字重新发起请求，但服务器生成响应耗时较长，重复请求并非最优方案，寻求更优解决办法。

内容的提问来源于stack exchange，提问作者JDogMcSteezy