客户端-服务器网络中列表序列化发送失败，如何正确序列化列表为字节？

Hey there, let's break down what's going wrong and fix this step by step!

The Root Causes of Your Error #

1. TCP is a stream protocol (no message boundaries)
   Your client calls sock.send() twice in a row: first for the string message, then for the pickled list. TCP doesn't treat these as separate "packets"—it often merges them into a single stream of bytes. When your server calls conn.recv(8092) the first time, it might receive a mix of the string bytes and the pickled list data. Trying to run pickle.loads() on this mixed buffer (which starts with 'H' from "Hello") causes the _pickle.UnpicklingError.

2. Server logic mismatch
   You're attempting to unpickle the buffer that holds your string message (buf) instead of the buffer that should hold the pickled list. Even without粘包, this would fail immediately because the string bytes aren't valid pickle data.

Fix: Use a Length-Prefixed Protocol #

To safely send multiple pieces of data over TCP, we need to define a simple protocol: send the length of each data chunk first (as a fixed-size binary value), then send the actual data. This lets the server know exactly how many bytes to read for each message.

Modified Client Code

import socket
import pickle
import struct

portnum = 5000
hostname = 'localhost'

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_addr = (hostname, portnum)
sock.connect(server_addr)

try:
    # Send the string message
    msg = "Hello There!"
    msg_bytes = msg.encode()
    # Pack the message length into 4 bytes (unsigned int)
    msg_length = struct.pack('I', len(msg_bytes))
    sock.sendall(msg_length)
    sock.sendall(msg_bytes)
    
    # Send the pickled list
    numbers = [1,2,3,4,5]
    serialized_list = pickle.dumps(numbers)
    list_length = struct.pack('I', len(serialized_list))
    sock.sendall(list_length)
    sock.sendall(serialized_list)
    
    print("Messages sent successfully")
    print(f"Sending message: {msg}")
    print(f"Sending list: {numbers}")
finally:
    sock.close()
    print("Connection closed")

Modified Server Code

import socket
import pickle
import struct

port = 5000
hostname = 'localhost'

def recv_exact(conn, total_bytes):
    """Helper function to receive exactly the specified number of bytes"""
    received_data = b''
    while len(received_data) < total_bytes:
        chunk = conn.recv(total_bytes - len(received_data))
        if not chunk:
            raise ConnectionError("Connection closed before all data was received")
        received_data += chunk
    return received_data

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind((hostname, port))
sock.listen(2)
print("Ready. Listening to socket...")

try:
    conn, addr = sock.accept()
    print(f"Connected from {addr}")
    
    # Receive the string message
    # First read 4 bytes to get the message length
    msg_len_data = recv_exact(conn, 4)
    msg_length = struct.unpack('I', msg_len_data)[0]
    msg_bytes = recv_exact(conn, msg_length)
    msg = msg_bytes.decode()
    print(f"We've got a message: {msg}")
    
    # Receive the pickled list
    list_len_data = recv_exact(conn, 4)
    list_length = struct.unpack('I', list_len_data)[0]
    list_bytes = recv_exact(conn, list_length)
    numbers = pickle.loads(list_bytes)
    print(f"We've got the list: {numbers}")
finally:
    conn.close()
    sock.close()
    print("Server closed")

Key Improvements Explained #

• sendall() instead of send(): send() might only transmit part of your data; sendall() guarantees all bytes are sent.
• Length prefixes: Using struct.pack/struct.unpack to send/receive a fixed 4-byte length value ensures the server knows exactly how many bytes to read for each message, eliminating粘包 issues.
• recv_exact() helper: Ensures we receive the full amount of data requested, even if recv() returns partial chunks (common in TCP).

内容的提问来源于stack exchange，提问作者Alexander Paul