使用Twisted传输大文件失败求助：仅小文件可正常接收

问题根源 #

1. 客户端文件写入逻辑错误：每次收到数据都以wb模式重新打开文件，会覆盖之前写入的内容。大文件会被拆分成多个分片传输，最终只有最后一块数据被保留，导致文件损坏。
2. 服务端资源未正确释放：文件传输完成后未关闭文件句柄，存在资源泄漏风险。
3. 缺乏传输结束标识：客户端无法判断文件是否传输完成，可能提前终止写入。

修复后的服务端代码 #

from twisted.internet.protocol import Protocol, connectionDone
from twisted.python import failure
from twisted.protocols.basic import FileSender
from twisted.internet.protocol import Factory
from twisted.internet.endpoints import TCP4ServerEndpoint
from twisted.internet import reactor


class TestServer(Protocol):

    def connectionMade(self):
        self.file = open('00000.jpg', 'rb')
        filesender = FileSender()
        # 绑定传输完成后的回调，用于关闭文件和连接
        transfer_deferred = filesender.beginFileTransfer(self.file, self.transport)
        transfer_deferred.addCallback(self._onTransferComplete)

    def _onTransferComplete(self, result):
        self.file.close()
        # 主动断开连接，通知客户端传输结束
        self.transport.loseConnection()

    def dataReceived(self, data: bytes):
        data = data.decode('UTF-8')
        print(data)

    def connectionLost(self, reason: failure.Failure = connectionDone):
        print("Server lost Connection")
        # 兜底关闭文件，防止异常场景下资源泄漏
        if hasattr(self, 'file'):
            self.file.close()

class QOTDFactory(Factory):
    def buildProtocol(self, addr):
        return TestServer()

endpoint = TCP4ServerEndpoint(reactor, 8007, interface="127.0.0.1")
endpoint.listen(QOTDFactory())
reactor.run()

修复后的客户端代码 #

from twisted.internet.protocol import Protocol, ClientFactory, connectionDone
from twisted.python import failure


class TestClient(Protocol):
    def connectionMade(self):
        print("Client did connection")
        # 初始化时打开文件并保留句柄，避免每次写入重新打开覆盖内容
        self.file = open('13.jpg', 'wb')
        self.sent_ack = False

    def dataReceived(self, data):
        # 直接写入当前收到的数据分片
        self.file.write(data)
        # 仅发送一次确认消息，避免重复发送
        if not self.sent_ack:
            self.transport.write("Client take connection".encode('UTF-8'))
            self.sent_ack = True

    def connectionLost(self, reason: failure.Failure = connectionDone):
        print("Client lost Connection Protocol")
        # 连接关闭时关闭文件，确保所有数据写入完成
        self.file.close()

class EchoClientFactory(ClientFactory):
    def startedConnecting(self, connector):
        print('Started to connect.')

    def buildProtocol(self, addr):
        print('Connected.')
        return TestClient()

    def clientConnectionLost(self, connector, reason):
        print('Lost connection factory.  Reason:', reason)

    def clientConnectionFailed(self, connector, reason):
        print('Connection failed factory. Reason:', reason)

from twisted.internet import reactor
reactor.connectTCP('127.0.0.1', 8007, EchoClientFactory())
reactor.run()

关键修改说明 #

• 客户端：在connectionMade中一次性打开文件并保存句柄，dataReceived直接写入分片数据；connectionLost中关闭文件，确保所有传输的数据都被写入。
• 服务端：通过beginFileTransfer返回的Deferred对象绑定回调，传输完成后关闭文件并主动断开连接，给客户端明确的传输结束信号；同时在connectionLost中做文件关闭的兜底处理。
• 优化确认消息逻辑，避免重复发送无用数据。

内容的提问来源于stack exchange，提问作者Игорь Платонов