Ngrok隧道远程视频流延迟过高问题排查与优化求助

问题背景 #

我尝试通过互联网将搭载Raspberry Pi的机器人视频流传输到本地机器，已通过Ngrok隧道建立Socket连接，但出现长达数十秒的严重延迟。

测试发现的规律 #

• 本地WiFi环境下延迟极低，公网传输时延迟剧增
• Ngrok隧道空闲时延迟约50ms，启动视频流后最高可达10000ms
• 画面运动较多时延迟会上升

客户端（机器人）代码 #

import socket, cv2
from struct import pack, unpack, calcsize
import threading
from time import sleep
from base64 import b64encode
from imutils import resize

client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client_socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
client_socket.connect(('4.tcp.eu.ngrok.io', 13114))

def capture_vid(sock):
    vid = cv2.VideoCapture(0)
    vid.set(cv2.CAP_PROP_BUFFERSIZE, 1)
    while (vid.isOpened()):
        try:
            _, frame = vid.read()
            frame = resize(frame, width=480)
            encoded, buffer = cv2.imencode('.jpg', frame, [cv2.IMWRITE_JPEG_QUALITY, 60])
            msg = b64encode(buffer)
            message = pack("Q", len(msg)) + msg
            sock.sendall(message)
        except:
            print('SERVER DISCONNECTED')
            break
        sleep(0.06)


stream_thread = threading.Thread(target=capture_vid, args=(client_socket,))
stream_thread.start()

服务端代码 #

import socket, cv2
import threading
from base64 import b64decode
from struct import pack, unpack, calcsize
import numpy as np

class Server:
    def __init__(self):
        self.server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        self.server_socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
        host_name = socket.gethostname()
        host_ip = socket.gethostbyname(host_name)
        print('HOST IP:', host_ip)
        port = 9998
        socket_address = ('localhost', port)

        # Socket Bind
        self.server_socket.bind(socket_address)
        self.client_socket = None

    def listen(self):
        self.server_socket.listen(5)
        client_socket, addr = self.server_socket.accept()
        self.client_socket = client_socket
        print('GOT CONNECTION FROM:', addr)

        data = b""
        payload_size = calcsize("Q")
        while True:
            try:
                while len(data) < payload_size:
                    packet = self.client_socket.recv(1024)
                    if not packet: break
                    data += packet

                packed_msg_size = data[:payload_size]
                data = data[payload_size:]
                msg_size = unpack("Q", packed_msg_size)[0]
                print(msg_size)
                while len(data) < msg_size:
                    data += self.client_socket.recv(64 * 1024)
                frame_data = data[:msg_size]
                data = data[msg_size:]
                #frame = pickle.loads(frame_data)
                frame = b64decode(frame_data, ' /')
                npdata = np.frombuffer(frame, dtype=np.uint8)
                frame = cv2.imdecode(npdata, 1)
                cv2.imshow("RECEIVING VIDEO", frame)
                key = cv2.waitKey(1) & 0xFF
                if key == ord('q'):
                    break
            except:
                print("client disconnected")
                break
        client_socket.close()

server = Server()
server.listen()

已尝试操作与补充测试 #

• 调整缓存、优化cv2.VideoCapture参数，无效果
• 本地WiFi传输流畅无延迟
• Messenger视频通话流畅度远高于自制流，且两者平均上传速度相近
• 无缩放时单帧约170KB，缩至480p约30KB，但仍有延迟
• 优质网络下传输速率达800kB/s，流畅度仍远不及商业视频通话

疑问 #

1. 上传速度波动是否是延迟的诱因？
2. 问题的核心成因是什么？
3. 有哪些优化方案或替代方案？

内容的提问来源于stack exchange，提问作者Dinh Trung Che