如何加速Python中TCP Socket非固定大小数据结构的读取?
优化TCP游戏通信的批量读取性能
我需要通过TCP连接和一款无法修改代码的游戏通信,开发者提供的Python通信代码速度太慢,根源在于它逐个读取基础数据类型:
bool <- socket_stream.read(1) int <- socket_stream.read(4) double <- socket_stream.read(8)
我希望改成更高效的批量读取方式:
Structure <- socket_stream.read(N)
但遇到一个问题:所需的缓冲区大小不是固定值,比如Game数据结构的读取逻辑如下(包含动态长度的子结构数组):
def read_from(stream: StreamWrapper) -> "Game": """Read Game from input stream """ my_id = stream.read_int() players = [] for _ in range(stream.read_int()): players_element = Player.read_from(stream) players.append(players_element) current_tick = stream.read_int() units = [] for _ in range(stream.read_int()): units_element = Unit.read_from(stream) units.append(units_element) loot = [] for _ in range(stream.read_int()): loot_element = Loot.read_from(stream) loot.append(loot_element) projectiles = [] for _ in range(stream.read_int()): projectiles_element = Projectile.read_from(stream) projectiles.append(projectiles_element) zone = Zone.read_from(stream) sounds = [] for _ in range(stream.read_int()): sounds_element = Sound.read_from(stream) sounds.append(sounds_element)
当前使用的StreamWrapper类实现如下:
class StreamWrapper: BOOL_FORMAT_STRUCT_PACK = struct.Struct("?").pack INT_FORMAT_STRUCT_PACK = struct.Struct("<i").pack LONG_FORMAT_STRUCT_PACK = struct.Struct("<q").pack FLOAT_FORMAT_STRUCT_PACK = struct.Struct("<f").pack DOUBLE_FORMAT_STRUCT_PACK = struct.Struct("<d").pack BOOL_FORMAT_STRUCT_UNPACK = struct.Struct("?").unpack INT_FORMAT_STRUCT_UNPACK = struct.Struct("<i").unpack LONG_FORMAT_STRUCT_UNPACK = struct.Struct("<q").unpack FLOAT_FORMAT_STRUCT_UNPACK = struct.Struct("<f").unpack DOUBLE_FORMAT_STRUCT_UNPACK = struct.Struct("<d").unpack # Reading primitives def read_bool(self) -> bool: return self.BOOL_FORMAT_STRUCT_UNPACK(self.stream.read(1))[0] def read_int(self) -> int: return self.INT_FORMAT_STRUCT_UNPACK(self.stream.read(4))[0] def read_long(self) -> int: return self.LONG_FORMAT_STRUCT_UNPACK(self.stream.read(8))[0] def read_float(self) -> float: return self.FLOAT_FORMAT_STRUCT_UNPACK(self.stream.read(4))[0] def read_double(self) -> float: return self.DOUBLE_FORMAT_STRUCT_UNPACK(self.stream.read(8))[0]
所有read_from(...)方法的逻辑都和上面的Game示例类似,write_to()我会自行处理。主运行函数代码如下:
def run(self): strategy = None debug_interface = DebugInterface(self.reader, self.writer) while True: message = ServerMessage.read_from(self.reader) if isinstance(message, ServerMessage.GetOrder): order = strategy.get_order(message.player_view, debug_interface if message.debug_available else None) ClientMessage.OrderMessage(order).write_to(self.writer) self.writer.flush() elif isinstance(message, ServerMessage.UpdateConstants): strategy = TrackingStrategy(message.constants) elif isinstance(message, ServerMessage.Finish): strategy.finish() break elif isinstance(message, ServerMessage.DebugUpdate): strategy.debug_update(message.displayed_tick, debug_interface) ClientMessage.DebugUpdateDone().write_to(self.writer) self.writer.flush() else: raise Exception("Unexpected server message")
我想通过np.ndarray(shape, dtype=some_structured_dtype, buffer=stream.read(length), offset=offset...)或者numpy.frombuffer(stream.read(length), dtype=some_structured_dtype, count=count, offset=offset)来实现批量读取以提升速度,但不知道如何优雅地处理动态长度结构的问题,真正实现性能提升。
内容的提问来源于stack exchange,提问作者DanteTemplar
相关产品推荐
相关产品推荐

