Great question—switching from string-based TCP communication to structured binary data is exactly what you need for efficient sensor/state transfer between Unity and Python, especially for your autonomous vehicle AI training loop. String encoding adds unnecessary overhead and parsing complexity, while binary serialization is faster and more reliable for numerical data like sensor readings and 3D poses.

Let’s adapt your existing code to implement structured data encoding/decoding, with clear examples for both C# (Unity) and Python.

C# (Unity) Implementation #

First, we’ll define a serializable vehicle state struct, implement a binary encoder, and update your TCP manager to handle byte arrays instead of strings.

Step 1: Define the Vehicle State Struct #

Create a struct to hold all your vehicle data fields. We’ll keep it simple with sensor values and 3D pose data:

[System.Serializable]
public struct VehicleState
{
    // 4 proximity sensors
    public float sensor1;
    public float sensor2;
    public float sensor3;
    public float sensor4;
    // 3D position (x/y/z)
    public float posX;
    public float posY;
    public float posZ;
    // 3D rotation (euler angles x/y/z)
    public float rotX;
    public float rotY;
    public float rotZ;
}

Step 2: Implement Struct-to-Bytes Encoding #

We’ll use BitConverter to convert each float to a 4-byte array (IEEE 754 format) and concatenate them. We also add explicit endianness handling to ensure compatibility with Python:

public static byte[] EncodeStructuredData(VehicleState state)
{
    var byteList = new List<byte>();

    // Helper method to ensure little-endian byte order (matches Python's default struct format)
    byte[] GetLittleEndianBytes(float value)
    {
        var bytes = BitConverter.GetBytes(value);
        if (!BitConverter.IsLittleEndian) Array.Reverse(bytes);
        return bytes;
    }

    // Add all fields to the byte list
    byteList.AddRange(GetLittleEndianBytes(state.sensor1));
    byteList.AddRange(GetLittleEndianBytes(state.sensor2));
    byteList.AddRange(GetLittleEndianBytes(state.sensor3));
    byteList.AddRange(GetLittleEndianBytes(state.sensor4));
    
    byteList.AddRange(GetLittleEndianBytes(state.posX));
    byteList.AddRange(GetLittleEndianBytes(state.posY));
    byteList.AddRange(GetLittleEndianBytes(state.posZ));
    
    byteList.AddRange(GetLittleEndianBytes(state.rotX));
    byteList.AddRange(GetLittleEndianBytes(state.rotY));
    byteList.AddRange(GetLittleEndianBytes(state.rotZ));

    return byteList.ToArray();
}

Step 3: Update TCP Manager for Binary Data #

Modify your TCPManager to send/receive byte arrays instead of strings, and add packet length prefixing to avoid "sticky packet" issues (critical for reliable TCP communication):

public class TCPManager 
{ 
    private int PORT_NO; 
    private string SERVER_IP; 
    private TcpClient tcpClient; 
    private NetworkStream networkStream;

    public TCPManager(string ip, int port) 
    { 
        SERVER_IP = ip; 
        PORT_NO = port; 
        tcpClient = new TcpClient(SERVER_IP, PORT_NO);
        networkStream = tcpClient.GetStream();
    } 

    // Send structured byte data and receive action response
    public byte[] SendStructuredData(byte[] dataToSend)
    {
        // First send the length of the data (4-byte unsigned int)
        var lengthBytes = BitConverter.GetBytes((uint)dataToSend.Length);
        if (!BitConverter.IsLittleEndian) Array.Reverse(lengthBytes);
        networkStream.Write(lengthBytes, 0, lengthBytes.Length);
        
        // Send the actual structured data
        networkStream.Write(dataToSend, 0, dataToSend.Length);
        
        // Receive 1-byte action code from Python
        var responseBuffer = new byte[1];
        var bytesRead = networkStream.Read(responseBuffer, 0, responseBuffer.Length);
        return bytesRead > 0 ? responseBuffer : null;
    } 

    ~TCPManager() 
    { 
        networkStream?.Close();
        tcpClient?.Close(); 
    } 
}

Step 4: Update Game Manager Logic #

Replace your string-based data handling with the structured state encoding:

public class GameManager: MonoBehaviour
{ 
    public CarManager carManager; 
    private double trialNumber; 
    private double totalTrials; 
    private TCPManager tcpManager; 
    private VehicleState currentVehicleState;

    void Start () 
    { 
        trialNumber = 0; 
        totalTrials = 1000; 
        tcpManager = new TCPManager("127.0.0.1", 10000); 
        StartCoroutine(GameLoop()); 
    } 

    IEnumerator GameLoop() 
    { 
        yield return StartCoroutine(TrialStart()); 
        yield return StartCoroutine(TrialRunning()); 
        if(trialNumber <= totalTrials) 
        { 
            StartCoroutine(GameLoop()); 
        } 
    } 

    private IEnumerator TrialStart() 
    { 
        carManager.Reset(); 
        trialNumber++; 
        yield return null; 
    } 

    private IEnumerator TrialRunning() 
    { 
        while (!carManager.HasFailed()) // Assume your CarManager has this method
        { 
            // Populate current vehicle state
            currentVehicleState.sensor1 = carManager.Sensor1Value;
            currentVehicleState.sensor2 = carManager.Sensor2Value;
            currentVehicleState.sensor3 = carManager.Sensor3Value;
            currentVehicleState.sensor4 = carManager.Sensor4Value;
            
            var carTransform = carManager.transform;
            currentVehicleState.posX = carTransform.position.x;
            currentVehicleState.posY = carTransform.position.y;
            currentVehicleState.posZ = carTransform.position.z;
            
            currentVehicleState.rotX = carTransform.eulerAngles.x;
            currentVehicleState.rotY = carTransform.eulerAngles.y;
            currentVehicleState.rotZ = carTransform.eulerAngles.z;
            
            // Encode and send data
            var encodedData = EncodeStructuredData(currentVehicleState);
            var response = tcpManager.SendStructuredData(encodedData);
            
            // Parse action response
            if(response != null)
            {
                switch(response[0])
                {
                    case 0: carManager.MoveForward(); break;
                    case 1: carManager.MoveBackward(); break;
                    case 2: carManager.TurnRight(); break;
                    case 3: carManager.TurnLeft(); break;
                    default: carManager.Stop(); break;
                }
            }
            else
            {
                carManager.Stop();
            }

            yield return null; 
        } 
    } 
}

Python Implementation #

On the Python side, we’ll decode the received byte array into usable numerical values, process it (with your neural network), and send back a compact action code.

Step 1: Decode Bytes to Structured Data #

Use Python’s built-in struct module to unpack the byte array into floats. We’ll use little-endian encoding to match the C# implementation:

import socket
import struct
from random import randint

# Action mapping: code -> name (for logging)
ACTION_CODES = {0: "FD", 1: "BK", 2: "RT", 3: "LT", 4: "ST"}
# Struct format: < = little-endian, 10f = 10 32-bit floats (4 sensors + 3 pos + 3 rot)
STRUCT_FORMAT = '<10f'
STRUCT_SIZE = struct.calcsize(STRUCT_FORMAT)

def decode_structured_data(byte_data):
    """Decode byte array into sensor data, position, and rotation tuples"""
    data = struct.unpack(STRUCT_FORMAT, byte_data)
    sensors = data[:4]
    position = data[4:7]
    rotation = data[7:10]
    return sensors, position, rotation

Step 2: Update TCP Server Logic #

Modify your server to read the data length first, then the structured data, and send back an action code:

def main():
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    server_address = ('localhost', 10000)
    print(f'Starting up on {server_address[0]} port {server_address[1]}')
    sock.bind(server_address)
    sock.listen(1)

    while True:
        print('Waiting for connection...')
        connection, client_address = sock.accept()
        try:
            print(f'Connected to {client_address}')
            
            while True:
                # Read 4-byte data length first
                length_data = connection.recv(4)
                if not length_data:
                    print('No data length received, closing connection')
                    break
                
                # Convert length to unsigned int
                data_length = struct.unpack('<I', length_data)[0]
                
                # Read full structured data
                byte_data = b''
                while len(byte_data) < data_length:
                    chunk = connection.recv(data_length - len(byte_data))
                    if not chunk:
                        break
                    byte_data += chunk
                
                if not byte_data:
                    break
                
                # Decode and process data
                sensors, pos, rot = decode_structured_data(byte_data)
                print(f'Received: Sensors={sensors}, Position={pos}, Rotation={rot}')
                
                # Replace this with your neural network inference logic
                # For now, send a random action
                chosen_code = randint(0, 4)
                print(f'Sending action: {ACTION_CODES[chosen_code]}')
                
                # Send 1-byte action code
                connection.sendall(struct.pack('B', chosen_code))
                
        finally:
            connection.close()

if __name__ == '__main__':
    main()

Critical Notes for Reliability #

1. Byte Order Consistency: We explicitly use little-endian encoding on both sides to avoid issues across different operating systems (most modern systems use little-endian, but explicit handling ensures compatibility).
2. Sticky Packet Prevention: Sending a data length prefix before the actual data ensures we always read exactly the right number of bytes, even if TCP merges multiple packets.
3. Data Type Matching: C# float = Python f (32-bit float), C# uint = Python I (32-bit unsigned int). Never mix data types (e.g., don’t use C# double with Python f).
4. Error Handling: Add try/catch blocks in C# and error checking in Python to handle disconnections or corrupted data gracefully (critical for long-running training loops).

Alternative: Protocol Buffers (Protobuf) #

If you anticipate expanding your data structure later, Google Protocol Buffers is an excellent cross-language alternative. It’s more efficient than manual encoding and supports schema evolution. You’d define a .proto file describing your vehicle state, generate C# and Python code, and use the Protobuf libraries to serialize/deserialize data. This is overkill for simple structures but perfect for complex, evolving data.

内容的提问来源于stack exchange，提问作者MyStackOverflowAccount