Python AsyncIO Stream Reader读取异常问题求助

Hey, let's break down why your server is acting up and fix it!

The Root Cause #

TCP is a stream-based protocol, not message-based. Your current code uses reader.read(1024), which only reads up to 1024 bytes at a time. The first read worked by luck—your full message (header + body) fit into that 1024-byte chunk and arrived in one go. The second time, though, TCP might have split the data, or the server stopped reading before getting the full body because it didn't follow your custom message structure properly.

The Fix: Follow Your Custom Protocol #

Your client sends messages in a specific format: 4-byte little-endian length header + JSON body. To reliably read this, your server needs to:

1. First read exactly 4 bytes to get the body length.
2. Then read exactly that number of bytes to get the full body.
3. (Optional) If your client sends multiple messages over the same connection, loop to keep reading until the connection closes.

Here's the updated server code:

import asyncio
import json
from _socket import AF_INET

async def read_exactly(reader, num_bytes):
    # Helper to make sure we get exactly the number of bytes we need
    data = b""
    while len(data) < num_bytes:
        chunk = await reader.read(num_bytes - len(data))
        if not chunk:
            # Connection closed before we got all data
            raise asyncio.IncompleteReadError(data, num_bytes)
        data += chunk
    return data

async def handle_echo(reader, writer):
    try:
        # Keep reading messages if the client stays connected
        while True:
            # Step 1: Read the 4-byte length header
            header = await read_exactly(reader, 4)
            body_length = int.from_bytes(header, "little")
            
            # Step 2: Read the full body based on the header length
            body_data = await read_exactly(reader, body_length)
            message = json.loads(body_data.decode("utf-8"))
            
            print(f"Full received message: {message}")
            print(f"Raw data: {header + body_data}")
    except asyncio.IncompleteReadError:
        print("Client disconnected")
    finally:
        # Clean up the connection properly
        writer.close()
        await writer.wait_closed()

async def main():
    server = await asyncio.start_server(
        handle_echo, '127.0.0.1', 449, family=AF_INET)
    addr = server.sockets[0].getsockname()
    print(f'Serving on {addr}')
    async with server:
        await server.serve_forever()

asyncio.run(main())

Key Changes Explained #

• read_exactly helper: This ensures we don't stop reading until we have the exact number of bytes we need—critical for handling TCP's streaming behavior.
• Protocol-compliant reading: We strictly follow your client's message format, first grabbing the header to know how big the body is, then fetching the full body.
• Loop for multiple messages: If your client sends multiple messages over the same connection, the loop will keep processing them until the client disconnects.
• Clean connection handling: We properly close the writer and handle cases where the client drops the connection early.

Quick Note #

If your client sends one message per connection (and closes after each send), you can remove the while True loop from handle_echo—just keep the single read sequence. Either way, this approach will fix the partial read issue.

内容的提问来源于stack exchange，提问作者DINESH S